Worker & Atools

Topics covered

In many scientific domains experiments are set up to explore parameter spaces. This type of workload can easily be parallelized since the individual  computations are independent. However, a considerable amount of bookkeeping may be required to perform such experiments effectively.

Two frameworks have been developed to make this type of task trivial. Worker was developed in 2010, and has been improved over the years. Atools is a recent development to accommodate use cases that worker is not optimal for.

Both frameworks have a fairly large feature set, including monitoring of running jobs to keep track of the tasks that are completed/failed/pending,  resuming interrupted tasks, merging output from the tasks, and so on.

This lunchbox session will introduce you to these tools, and illustrate the more advanced features. You will also get tips on how to use the tools efficiently.

Target audience

This info session is primarily targeted at VSC-users, although other interested parties are welcome as well.

Previous knowledge

Participants have a working knowledge of the Linux command shell. Level: introductory

Result/Objectives

Participants know how to efficiently run embarrassingly parallel workloads on a cluster, and can select the best tool for the task at hand.