Anaconda and Singularity
- Jeff Polasek
There are two tools for building an environment for using the GPU’s on Olympus, Anacona/Miniconda and Singularity.
Please install anaconda in your /mnt/shared-scratch directory. Anaconda environments can become large and fill up your home directory.
Anaconda/Miniconda
Anaconda is a distribution of the Python and R programming languages for scientific computing (data science, machine learning applications, large-scale data processing, predictive analytics, etc.), that aims to simplify package management and deployment.
Anaconda is a powerful tool that loads the correct versions of python modules to ensure compatibility in your environment. For example, you can create different virtual environments for different versions of python, tensor-flow or pytorch.
One limitation of Anaconda is there are a limited number of versions available for tensor-flow and pytorch.
Singularity
Singularity is a container system similar to Docker. Singularity containers allows users to run applications in a Linux environment of their choosing. Containers can run different different distros (Ubuntu, CentOS, Mint) on a single system. This provides more flexibility than Anaconda, but can be more difficult to configure as since there are no versioning tools to ensure compatibility.
Should I use Anaconda or Singularity?
In many cases, Anaconda (or Miniconda) is the best tool to configure your machine learning environment.
For beginning users, good reference for getting started with Anaconda is
Getting started with conda — conda 24.9.3.dev45 documentation
The following site provides instructions for installing anaconda
However, there are cases where a singularity container is the better choice. Some of these cases include needing to use a different Linux distribution/version number, being unable to configure anaconda to match the versions needed for specialty python modules, or requiring specialty applications.
More information on using singularity containers on Olympus is located at