Welcome to Breakit’s documentation!¶
- breakit - KSL many jobs automated submitter
- breakit –job=any-Slurm-Job
- –range=Total-number-of-jobs-to-submit [–chunk=Maximum-number-of-jobs-to-be-seen-in-the-queue]
breakit assists the user in submitting Slurm job array of important size on shaheen II. It has been designed, developped and officially maintained by the Kaust Supercomputing Laboratory.
By policy, on shaheen II, the maximum number of jobs authorized in the queue is limited per user. breakit seamlessly allows the submission of job arrays whose total number can be higher than this limit.
Not a single modification needs to be made to the Slurm job it self. Let assume 1000 occcurences of “my_job” needs to be submitted, the com- mand:breakit –job=my-job –range=1000 –chunk=20
will submit the 1000 jobs to shaheen II in several chunks of 20 jobs. At a time, no more than 20 jobs will appear in the queue. In reality, breakit will use job dependency to automate the ongoing submission of the remaining jobs. These batches of 20 jobs will be seen as successive job arrays in the scheduling queue.
If no option –chunk is mentionned, default number of jobs queued simultaneously is set to 8.
To cancel the full set of jobs, an option –kill is under development and testing. In the meantime, when choosing a different name for all the jobs launched through breakit the following Slurm command can be used:scancel -n=name-of-job
- Written by Samuel Kortas (samuel.kortas (at) kaust.edu.sa)
- REPORTING BUGS
- Report breakit bugs to firstname.lastname@example.org breakit home page: <https://github.com/samkos/breakit> KAUST Supercomputing Laboratory: <http://hpc.kaust.edu.sa/>
- Copyright © 2016 KAUST Supercomputing Laboratory License LGPLv3+: GNU LGPL version 3 or later <http://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
- SEE ALSO
- A comprehensive presentation of breakit has been given by Samuel Kortas during the KSL Workshop entitled ‘Boost your efficiency when dealing with multiple jobs on the Cray XC40 supercomputer Shaheen II’ held at KAUST On Sunday June 5th 2016. The workshop slides can be freely down- loaded from <https://www.hpc.kaust.edu.sa/sites/default/files/files/public/many_jobs.pdf>