Atlas Cluster

What is Atlas

Altas is a Linux-based high-performance computing (HPC) cluster supported by the College of Engineering.  This cluster is available to all research groups in Engineering.  

History

Originally, Atlas was purchased by the Energy Institute. As the College of Engineering consolidated into one IT group and the University enforced higher security requirements, nodes from other research group clusters have been merged into Atlas.  The latest addition was an additional 72 nodes from the ADA supercomputer.

Resources

Hardware

Atlas consists of the following resources:

• A login node - 16 Core Xeon with 64GB RAM

• Two storage nodes - 137 TB storage

• 133 Compute Nodes

Software

Atlas is currently running CentOS 7.9 on all systems and uses the same scheduling software, Slurm, as the TAMU High-Performance Research Computing.

A full list of currently available software can be found below.

Adding software to the cluster

To request additional software be installed on Atlas, send an email to linux-engr-helpdesk@tamu.edu.  Please include the name and manufacturer of the software in your request.

Usage

Research groups that purchased nodes have priority over the nodes they purchased.  All users have access to these nodes, but users in the purchasing group’s jobs will be prioritized over non-group user jobs.  These nodes may also be reserved for group users only if requested.

How to use the cluster

The cluster can run either batch or interactive jobs.  Batch jobs run in the background and require no user interaction.  Interactive jobs run in a terminal window.  Both job types require the user to specify the resources needed for the job.  Resources include the number of cores, nodes, amount of RAM, and job run time.  When you submit your job request, the Slurm scheduler will allocate the resources requested to your job.  Your job now owns these resources and the resources are not available to other jobs.  

Batch Jobs

The primary method of using the cluster is to submit a batch job to the schedule using a script file.  This script file will define the resources and software your job requires.  A basic job submission script for a single core Matlab job is shown below.

#!/bin/sh

#SBATCH --job-name=matlab_test         # Job name
#SBATCH --mail-type=ALL                # Mail events (NONE, BEGIN, END, FAIL, ALL)
#SBATCH --mail-user=NetID@tamu.edu     # Where to send mail
#SBATCH --nodes=1                      # Use one node
#SBATCH --ntasks=1                     # Run a single task
#SBATCH --cpus-per-task=1              # Number of CPU cores per task
#SBATCH --time=01:05:00                # Time limit hrs:min:sec
#SBATCH --output=parallel_%j.out       # Standard output and error log
#SBATCH --partition=normal             # Partition/Queue to run in (normal IS DEFAULT)

#Load matlab environment module
module load MATLABR2021A

# set working directory - if you do not set this your working directory will
# be the directory you submitted the script from
# cd /working/directory
# run matlab job without GUI
 matlab -nodisplay < matlab_parfor.m

To use the ada queue, you will need to specify a quality of service (qos).  This affects the prioritization of resources in the ada queue.  An example script with the qos specification is below:

#!/bin/sh
#SBATCH --job-name=matlab_test         # Job name
#SBATCH --mail-type=ALL                # Mail events (NONE, BEGIN, END, FAIL, ALL)
#SBATCH --mail-user=j-polasek@tamu.edu # Where to send mail	
#SBATCH --nodes=1                      # Use one node
#SBATCH --ntasks=1                     # Run a single task	
#SBATCH --cpus-per-task=1              # Number of CPU cores per task
#SBATCH --time=01:05:00                # Time limit hrs:min:sec
#SBATCH --output=parallel_%j.out       # Standard output and error log
#SBATCH --partition=ada    	           # Partition/Queue to run in
#SBATCH --qos=atlas-ada


#Load matlab environment module
module load MATLABR2021A

# set working directory - if you do not set this your working directory will
# be the directory you submitted the script from
# cd /working/directory

# run matlab job without GUI
 matlab -nodisplay < matlab_parfor.m

It is extremely important that jobs only request the resources required.  Requesting multiple cores for a single-threaded program does not improve the performance of the job.  Doing this will reduce the number of available resources for other users.

Interactive Jobs

Interactive jobs may be run on the compute nodes by using the srun command.  These support both terminal and X11 interfaces.  The same options that are available in the script file on the previous page work with the srun command.

An example srun command that will open a bash shell on an ada-2 compute node with X11 enabled reserving two CPUs:

srun --cpus-per-task=2 --partition=ada-2 --pty --x11=first bash

Your prompt will change to [net-id]@computeXXX

Once on the compute node you can use the module load command and run software interactively.

To run in the ada queue/partition, you will need to add the -q  and -p qualifier.

srun --cpus-per-task=2 -p ada  -q atlas-ada --pty --x11=first bash

Example Scripts

Example scripts for running batch jobs are located in the /mnt/share/scripts directory.

If you have any questions on running scripts or interactive jobs, send an email to linux-engr-helpdesk@tamu.edu.   

Software and Tools

Software List

• AdHocTrex - Instructions are in the following file:  /mnt/share/sw/README-AdHocTrex

• Autodock Vina 

• CHARMM an example script file is located at /mnt/share/scripts/charmmi.sh

• CPLEX_2010

• dligand2

• Fluent 2021R1

• Fread - login node only

• GAMS

• GROMACS an  example script file is located at /mnt/share/scripts/namd_2.14-smp-mpi.sh

• Matlab 2021a - available without module

• MGL Tools (AutoDock Tools)

• Modeller

• NAMD -  an  example script file is located at /mnt/share/scripts/namd_2.14-smp-mpi.sh

• Open Babel - available without module

• R - available without module

• Scwrl4

• Smina - Vinardo is implemented as an optional scoring function.

• VMD - only on login node

• Wordom

• Zdock

Compilers and tools

• Gcc 10.2.0

• Gcc 11.2.0b 

• Gfortran

• Gnuplot

• Intel Oneapi HPCkit

  • compiler/2021.4.0:  - tools needed for compiling

  • mkl/2021.4.0 - Intel’s Math Kernel Libraries

  • mpi/2021.4.0 - Intel version of MPI