From Questions to Solutions: Inside VSC User Support
- 10 minutes ago
- 8 min read
VSC Connect Blog
Interview with Alex Domingo, VSC Support Team
Q: Alex, could you start by telling us about your role at VSC and what your work at VUB entails?
I am part of the Scientific Data & Compute team at VUB, and we manage the VSC Tier-2 and Tier-1 compute clusters hosted in Brussels. My roles in the team are multiple: half of my time is invested in user support and the installation of scientific software applications for our users, and the other half is a mix of system administration, maintenance tasks, and the development of new projects for our HPC clusters.
Q: What does the VSC support team do, and how does it help researchers across Flanders use HPC systems more effectively?
The goal of the VSC support team is to make VSC users successful. To this end, we try to solve any problems or answer any questions that users have about the VSC platforms.
On the HPC compute platform, we mostly troubleshoot users’ jobs and workflows that fail to complete, as well as install scientific software for users. In recent times, due to the increasing complexity of workflows and the growing number of solutions at VSC, we are providing more and more guidance to help users sketch a plan for their projects on the HPC.
Q: VSC recently updated its documentation portal. What improvements were made, and how do
they benefit users?
The VSC documentation website contains in-depth technical information covering the requirements and specificities of all VSC sites and services. The challenge is to provide all that knowledge in an approachable way for researchers from all scientific domains.
The goal of the recent major update of the VSC documentation was to restructure it into pages and sections that are organically reachable from the front page, without the need to explicitly search for the document you are looking for. Moreover, the front page has been redesigned to show all VSC services at a glance, helping new users see what is possible within VSC.
The other side of the update was more technical. We enhanced the editing tools of the VSC documentation to standardize how contributions from all VSC sites are integrated and added new formatting elements to present information more clearly.
Q: You often emphasize the importance of reading the documentation first. Why is this so crucial for users?
Reading the documentation is very helpful not only to understand how to use the software tools available on the HPC, but also to understand their design and purpose. We frequently see users struggling with some errors or poor performance of their jobs that boil down to misuse of the software application at hand.
In those cases a quick read of the documentation shows if the tool is the appropriate one for the job, or if the way it is used is the optimal one. The VSC support team relies on documentation all the time. We cannot know how every software component works, so the first thing we do is check its documentation. Most times, that is all it takes to fix the problem.
Q: What are the most common issues or support requests you encounter from users, and what advice would you give them before reaching out for help?
Issues related to software installations are by far the most common. Either users need new software modules on the cluster, or they have trouble installing their own software. The complexity of scientific software applications has increased drastically in recent years, and nowadays it is common for computational jobs to use hundreds of software libraries and applications.
Managing this vast ecosystem is difficult, not only for users but also for the VSC teams handling software installations. Therefore, we have recently improved support for users to manage their own software on VSC compute clusters. We introduced new tooling for Python virtual environments with “vsc-venv” and for R projects with “vsc-Rproject.” Moreover, we created a new VSC training focused on software installations, covering everything from modules to manual compilation, including containers, conda, and Python.
My recommendation to any user having trouble with software installations is to check the “Getting Scientific Software Installed” training on the VSC YouTube channel.
Q: In your presentations, you mention that “support is a shared responsibility.” What does that mean in practice?
We cannot help somebody who does not want to be helped. Users who are willing to engage with support and adapt how they work on the cluster will achieve their goals much more easily. Situations that result in disappointment or frustration are commonly caused by a lack of understanding of how the systems in our clusters work and the purpose of those design choices.
A clear example is the job queue in HPC clusters. From time to time, we see users struggling with the queue because they come from single-user environments and are not used to working in a multi-user system with shared resources. In those cases, it is critical to follow the advice of the support team to use the queue effectively. Switching from launching a single job at a time and waiting for it to complete to launching multiple jobs in parallel can make a big difference between a failed and a successful research project.
Q: From your perspective, what are the top best practices every HPC user should follow to get the most out of the infrastructure?
Planning and testing. Carrying out a research project on HPC is more complex than users typically anticipate. There are three basic subjects to take into account: data, software, and hardware resources.
Data needs to be moved in and out of the system. How will that be done? What storage options provide enough capacity for your data?
Many software applications will be used at different stages of the project. Is that software already available? Does the project need very specific versions? Does the project involve developing new code?
The VSC HPC offering is rich, with multiple compute clusters featuring different hardware configurations. Is the scale of your project small enough for a Tier-2 cluster, or will you need Tier-1? Are GPUs required? Are memory requirements particularly large?
Any time invested in the early stages of a project to plan ahead for these questions is time well invested.
Q: Indeed, you frequently encourage users to plan their strategy before submitting jobs. What are some practical steps they can take to do this efficiently?
Going back to the three main subjects from the previous question:
Data management needs to answer the questions “Where is the data?” and “How does data move in and out?” It can be useful to draw a flow chart with storage locations and the actors or events responsible for data movement. It is important to know which storage options are available, which can be used from the target HPC facility, and which tools are needed to manage transfers.
Planning the collection of software applications needed throughout the project boils down to ensuring that all required software works on the target HPC cluster. Check whether it is already available in the central library as modules. If not, consider requesting installation well in advance to allow support teams time to carry it out. Alternatively, installing the software yourself is also possible, but time should be planned to test that everything works as expected.
The hardest part might be planning hardware resources for future compute jobs. Knowing how many CPU cores or how much RAM will be used is not trivial. In many cases, these parameters cannot be easily calculated beforehand and will be fine-tuned through trial and error during the project. Planning hardware resources early can be framed as a question of scale: Will you need multi-node jobs? How many nodes? Will you launch a hundred jobs using a few CPU cores, or thousands of small jobs? Will you need GPUs? Answering these questions helps define your target VSC clusters. You can then check resource availability and determine whether you need to request compute credits.
Q: VSC also offers multiple ways to interact with HPC systems, from command-line access to web interfaces like Open OnDemand and JupyterLab. How do these options support different types of users?
The introduction of web-based interfaces in VSC clusters has been a game changer for many users. They not only lowered the barrier to entry to HPC but also enable novel interactive use cases, such as JupyterLab, RStudio, VS Code, and remote desktop environments. Moreover, our web-based portals provide a terminal interface while avoiding the hassle of setting up secure keys for terminal access. They are quickly becoming the default way of using our clusters.
The terminal interface is not going anywhere, though. It remains the preferred tool for power users running heavy batch workloads. It still offers the best experience for experienced users working with code and the Linux system.
Q: The VSC documentation includes detailed guidance on using Tier-1 and Tier-2 systems. How should users decide which level of infrastructure to use?
The main criterion is the scale of your compute jobs. All users should start small on Tier-2 infrastructure to test things out. When you are ready for larger production runs, evaluate how big your jobs are. As a rule of thumb, if you regularly need more than a quarter of all nodes in your Tier-2 cluster, it is time to consider moving to Tier-1.
Q: Can you tell us more about how VSC collaborates across sites to ensure consistent and effective user support?
VSC puts significant effort into ensuring that the user experience across sites is consistent. Teams from each VSC site interact at different levels to coordinate decisions on security and user experience. Our goal is to make running workloads across VSC sites as frictionless as possible while guaranteeing system security.
For example, users moving from Tier-2 to Tier-1 should be able to transfer their experience seamlessly. Command-line tools and software modules share a common basis across clusters. Modules are loaded in the same way, certain toolchains are available everywhere, and so on. In the future, we will expand this consistency to applications on our web portals and to the behavior of job scripts.
Q: What role do training and continuous learning play in helping researchers become more confident HPC users?
The landscape of computational software and tools available to researchers is constantly evolving. Nowadays, it is uncommon to stick to a single application for the lifetime of a project. Users interact with dozens of software applications, and many require high-level coding in languages like Python and R. Therefore, it is more important than ever to stay up to date with the tools available on HPC and become proficient with those needed for your project.
To support this, VSC organizes a wide range of training sessions. I encourage all users to check our training offerings on the VSC website; participation is free.
Q: What’s one common misconception about HPC or VSC support that you’d like to clear up?
Probably the main misconception is that we are bothered when we receive support requests. In fact, it is the opposite. We appreciate when users reach out to ask questions or share the issues they are facing. Situations where someone encounters serious trouble close to project deadlines are often caused by not reaching out in time. We need user feedback to assess what is working and what is not, and to make improvements.
Q: Finally, what advice would you give to new users who are just getting started with HPC at VSC?
My main advice for new users is to take the time to learn the tools at your disposal. Becoming proficient with computational tools will not only increase your chances of success in your current project but will very likely be useful in the future as well. Most software and techniques needed to use HPC are also useful in non-HPC environments, and today computers are ubiquitous across all scientific domains.
Explore more voices from VSC
This interview is part of the VSC Connect Blog series, where VSC staff share their expertise and perspectives on HPC, AI, and innovation.
👉 Discover all VSC Connect interviews: https://www.vscentrum.be/news