HPC Workflow @ENCCB

Overview

Whenever one has to deal with multiple jobs on a HPC system, the idea of automating parts or all of the job management process involves describing and implementing so-called 'workflows'. Options for managing workflows are numerous and range from using basic scheduler features such as job arrays and job dependencies, up to using a complex system managed by a central, multi-user, database. Moreover, workflows tools are also available from the software development and deployment ecosystem and the whole "devops" movement. This online workshop aims at guiding participants towards the right tool for their use and help them reduce the time they spend managing their jobs by automating what can be automated and follow best practices.

After attending the workshop, you will

• know about generally available GNU/Linux tools to manage multiple tasks and jobs

• learn about some easy-to-install scientific workflow tools for single-user usage

• learn how to automate packaging and…