Threaded Jobs on the Cluster

From DISI
Jump to navigation Jump to search

It is possible to submit jobs to the cluster that require more than one core to execute.

To make a job reserve more than one slot use the following directive in the script:

   #$ -pe threaded ## -R y

Where ## is the number of slots your job needs. It is important to include the "-R y" as it will ensure your job starts reserving slots. Without this, it may not be executed for a very long time.

It is possible to request a range of slots and then reference the number allocated during execution. Simply provide a range for the number of slots in the parallel environment (pe) directive. The number of slots is then available in the $pe_slots environmental variable.

   #$ pe threaded 4-8 -R y
   
   ...
   /run/some/threaded command --threaded --cores=$pe_slots

Note that even with reserved slots, multi-slot jobs often take a long time to leave the queue. It is worth your while to try breaking your job up into a number of smaller, serial jobs if possible.