Performance Testing

[ymifdal]

Greetings my friends.
I wish you a good ending of the week.

I have this particular question on my mind it's been a while now.
On my laptop, I run a Python code that takes about 30 seconds to compile and run.
My objective is to run the same code on a cluster of 3 nodes and determine how much time it would take.
By doing that, I can have a general understanding of the added value.

I am willing to listen to any suggestions
Thanks in advance.

 

[LnxBil]

My objective is to run the same code on a cluster of 3 nodes and determine how much time it would take.

Most Python code is not easily parallelizable even on a single machine. You can use numba for some operation, yet running it on a cluster is totally different beast. Best to have a divide-and-conquer approach and split the work you have to do in smaller units and parallize them with a work queue.

 

[ymifdal]

Most Python code is not easily parallelizable even on a single machine. You can use numba for some operation, yet running it on a cluster is totally different beast. Best to have a divide-and-conquer approach and split the work you have to do in smaller units and parallize them with a work queue.

It's not a problem for me to change the code and its language. The approach can be modified as well

Lemme rephrase my thought. Is there any way to prove the following using statistics and numbers? Working with a cluster of three nodes is much more effective than using a single laptop, when it comes to speed, performance etc

Although the question may seem silly to you, I need to record some statistics

 

[LnxBil]

Yes, if the problem and the code fits, you will scale linearly, yet this is not a PVE problem. You can do this without any cluster software, because you need to solve this in your code.