Atlas Cluster

1 What is Atlas
2 History
3 Resources
- 3.1 Hardware
- 3.2 Software
- 3.3 Adding software to the cluster
4 Usage
5 How to use the cluster
- 5.1 Batch Jobs
- 5.2 Interactive Jobs
- 5.3 Example Scripts
6 Software and Tools
- 6.1 Software List
- 6.2 Compilers and tools

What is Atlas

Altas is a Linux-based high-performance computing (HPC) cluster supported by the Technology Services - Engineering. This cluster is available to all research groups in Engineering.

History

Originally, Atlas was purchased by the Energy Institute. As the College of Engineering consolidated into one IT group and the University enforced higher security requirements, nodes from other research group clusters have been merged into Atlas. The latest addition was an additional 72 nodes from the ADA supercomputer.

Resources

Hardware

Atlas consists of the following resources:

A login node - 16 Core Xeon with 64GB RAM
Two storage nodes - 137 TB storage
133 Compute Nodes

Partition	Nodes	Hardware	Owner
enrgy	15	2X E5-2660 v2 (20 cores), 24GB RAM	Energy Institute
Isen	10	1X. E5-2276G (6 cores), 64GB RAM	ISEN
dwc	13	2X E5-2660 v4 (28 cores), 256GB RAM	Turbomachinery Lab
bigmo	11	2X E5-2660 v3 (20 cores), 256GB RAM	Turbomachinery Lab
ada	72	2X E5-2670 v2 (20 cores), 64GB RAM	Tamamis Group/General
normal	12	2X E5-2670 v2 (20 cores), 64GB RAM	General Use/default

Software

Atlas is currently running CentOS 7.9 on all systems and uses the same scheduling software, Slurm, as the TAMU High-Performance Research Computing.

A full list of currently available software can be found below.

Adding software to the cluster

To request additional software be installed on Atlas, send an email to linux-engr-helpdesk@tamu.edu. Please include the name and manufacturer of the software in your request.

Usage

Atlas data is not backed up. Please be sure to copy all critical data to your group's network research drive.

Research groups that purchased nodes have priority over the nodes they purchased. All users have access to these nodes, but users in the purchasing group’s jobs will be prioritized over non-group user jobs. These nodes may also be reserved for group users only if requested.

How to use the cluster

The cluster can run either batch or interactive jobs. Batch jobs run in the background and require no user interaction. Interactive jobs run in a terminal window. Both job types require the user to specify the resources needed for the job. Resources include the number of cores, nodes, amount of RAM, and job run time. When you submit your job request, the Slurm scheduler will allocate the resources requested to your job. Your job now owns these resources and the resources are not available to other jobs.

Batch Jobs

The primary method of using the cluster is to submit a batch job to the schedule using a script file. This script file will define the resources and software your job requires. A basic job submission script for a single core Matlab job is shown below.

#!/bin/sh

#SBATCH --job-name=matlab_test         # Job name
#SBATCH --mail-type=ALL                # Mail events (NONE, BEGIN, END, FAIL, ALL)
#SBATCH --mail-user=NetID@tamu.edu     # Where to send mail
#SBATCH --nodes=1                      # Use one node
#SBATCH --ntasks=1                     # Run a single task
#SBATCH --cpus-per-task=1              # Number of CPU cores per task
#SBATCH --time=01:05:00                # Time limit hrs:min:sec
#SBATCH --output=parallel_%j.out       # Standard output and error log
#SBATCH --partition=normal             # Partition/Queue to run in (normal IS DEFAULT)

#Load matlab environment module
module load MATLABR2021A

# set working directory - if you do not set this your working directory will
# be the directory you submitted the script from
# cd /working/directory
# run matlab job without GUI
 matlab -nodisplay < matlab_parfor.m

To use the ada queue, you will need to specify a quality of service (qos). This affects the prioritization of resources in the ada queue. An example script with the qos specification is below:

#!/bin/sh
#SBATCH --job-name=matlab_test         # Job name
#SBATCH --mail-type=ALL                # Mail events (NONE, BEGIN, END, FAIL, ALL)
#SBATCH --mail-user=j-polasek@tamu.edu # Where to send mail	
#SBATCH --nodes=1                      # Use one node
#SBATCH --ntasks=1                     # Run a single task	
#SBATCH --cpus-per-task=1              # Number of CPU cores per task
#SBATCH --time=01:05:00                # Time limit hrs:min:sec
#SBATCH --output=parallel_%j.out       # Standard output and error log
#SBATCH --partition=ada    	           # Partition/Queue to run in
#SBATCH --qos=atlas-ada


#Load matlab environment module
module load MATLABR2021A

# set working directory - if you do not set this your working directory will
# be the directory you submitted the script from
# cd /working/directory

# run matlab job without GUI
 matlab -nodisplay < matlab_parfor.m

It is extremely important that jobs only request the resources required. Requesting multiple cores for a single-threaded program does not improve the performance of the job. Doing this will reduce the number of available resources for other users.

Interactive Jobs

Interactive jobs may be run on the compute nodes by using the srun command. These support both terminal and X11 interfaces. The same options that are available in the script file on the previous page work with the srun command.

An example srun command that will open a bash shell on an ada-2 compute node with X11 enabled reserving two CPUs:

srun --cpus-per-task=2 --partition=ada-2 --pty --x11=first bash

Your prompt will change to [net-id]@computeXXX

Once on the compute node you can use the module load command and run software interactively.

To run in the ada queue/partition, you will need to add the -q and -p qualifier.

Example Scripts

Example scripts for running batch jobs are located in the /mnt/share/scripts directory.

If you have any questions on running scripts or interactive jobs, send an email to linux-engr-helpdesk@tamu.edu.

Software and Tools

Software List

AdHocTrex - Instructions are in the following file: /mnt/share/sw/README-AdHocTrex
Autodock Vina
CHARMM an example script file is located at /mnt/share/scripts/charmmi.sh
CPLEX_2010
dligand2
Fluent 2021R1
Fread - login node only
GAMS
GROMACS an example script file is located at /mnt/share/scripts/namd_2.14-smp-mpi.sh
Matlab 2021a - available without module
MGL Tools (AutoDock Tools)
Modeller
NAMD - an example script file is located at /mnt/share/scripts/namd_2.14-smp-mpi.sh
Open Babel - available without module
R - available without module
Scwrl4
Smina - Vinardo is implemented as an optional scoring function.
VMD - only on login node
Wordom
Zdock

Compilers and tools

Gcc 10.2.0
Gcc 11.2.0b
Gfortran
Gnuplot
Intel Oneapi HPCkit
- compiler/2021.4.0: - tools needed for compiling
- mkl/2021.4.0 - Intel’s Math Kernel Libraries
- mpi/2021.4.0 - Intel version of MPI