top of page

HPC Workflow @ENCCB

Tue, 15 Feb

|

Online Event

HPC Workflow @ENCCB
HPC Workflow @ENCCB

Time & Location

15 Feb 2022, 09:00 – 16 Feb 2022, 17:00

Online Event

About the Event

Overview

Whenever one has to deal with multiple jobs on a HPC system, the idea of automating parts or all of the job management process involves describing and implementing so-called 'workflows'. Options for managing workflows are numerous and range from using basic scheduler features such as job arrays and job dependencies, up to using a complex system managed by a central, multi-user, database. Moreover, workflows tools are also available from the software development and deployment ecosystem and the whole "devops" movement. This online workshop aims at guiding participants towards the right tool for their use and help them reduce the time they spend managing their jobs by automating what can be automated and follow best practices.

After attending the workshop, you will

  • know about generally available GNU/Linux tools to manage multiple tasks and jobs
  • learn about some easy-to-install scientific workflow tools for single-user usage
  • learn how to automate packaging and deploying software with containers or git-based web sites
  • see how some research groups used around workflow management system tools

Prerequisites

  • General knowledge of Linux and the command line
  • Knowing how to install software as a regular user
  • Knowing about Slurm and job submission and management

Agenda

Day1: February 15th 2022
  • 9h-9h15: Introduction/Welcome (Raphaël Leplae)
  • 9h15-9h30: Introduction to Workflow (Damien François)
  • 9h30-11h + 30 min Q/A+break: Workflows with basic GNU tools (and Maestro) (Damien François)
  • 11h30-12h30 + 15min Q/A: Checkpoint/Restart (Olivier Mattelaer)
  • Lunch break
  • 14h-15h + 30 min Q/A+break: Atools (Geert Jan Bex)
  • 15h30-16h15 + 15 min Q/A+break: Makeflow (Damien François)
Day2: February 16th 2022
  • 9h-10h + 30 min Q/A+break: Singularity (Olivier Mattelaer)
  • 10h30-11h30 + 30 min Q/A+break: Gitlab (CI/CD) (Olivier Mattelaer)
  • Lunch break:
  • 14h-15h + 30 min Q/A+break: Snakemake (Orian Louant)
  • 15h30-17h: Users testimonials15h30-15h50: NextFlow for bioinformatics by Luc Cornet
  • 15h50-16h10: Fireworks for material science by Guillaume Brunin
  • 16h10-16h30: Coral, a home made workflow system tool to manage numerical climate simulations by François Klein
  • 16h30-16h50: Examples of CI/CD in research codebases using git-based websites and HPC Denis-Gabriel Caprace

Share This Event

bottom of page