The clusters use different schedulers but they operate on the same principles – the larger the computational requirements, the longer wait time in the queue.

You can determine the amount of resources your program needs in several ways.

If you have access to the source code, you can estimate the amount of memory a particular run would require. Create a routine that goes through the initial setup without allocating the memory, so you can keep track of the allocated memory. While this does require some work initially, it will allow you to quickly calculate the required amount of memory and will save you time in the end. Some commercial packages might also give you estimates based on the input configuration.

If this is not an option, you can run trails of your program on your computer with small input parameters.

For example, if you are running a simulation requiring calculations with N=500 particles, complete runs for N=10, 20, 30, 40. Check how the memory usage increases with each increment. In Linux, you can check the memory usage with pmap.

Find the PID of your program:PID.txt. 

From the total in the last line, we can see that here the program used about 2 GB. Scaling tends to be either polynomial or logarithmic. By plotting the results in a graph and possibly fitting with a polynomial or logarithm, you can estimate the memory usage for N=500.

After submitting a job to a cluster, you will see in the automatically generated log file the memory usage of your program so you can adjust the next submissions accordingly. Our recommendation is to choose 10% more so your job will be completed when memory usage fluctuates a bit.

Estimating run-time is more difficult since this depends on the CPU speed and on the program scaling when running on multiple CPU cores. You will need to make an estimated guess and, when submitting the job, to allow for a liberal margin of error. Use the cluster reports to determine the real run time.