Worker & Atools

Topics covered

In many scientific domains experiments are set up to explore parameter spaces. This type of workload can easily be parallelized since the individual  computations are independent. However, a considerable amount of bookkeeping may be required to perform such experiments effectively.

Two frameworks have been developed to make this type of task trivial. Worker was developed in 2010, and has been improved over the years. Atools is a recent development to accommodate use cases that worker is not optimal for.

Both frameworks have a fairly large feature set, including monitoring of running jobs to keep track of the tasks that are completed/failed/pending,  resuming interrupted tasks, merging output from the tasks, and so on.

This lunchbox session will introduce you to these tools, and illustrate the more advanced features. You will also get tips on how to use the tools efficiently.

Target audience