Resources

NVIDIA DGX-A100 Compute Nodes	82 (20992 cpu cores)
Total host (compute node) memory	82 TB (82 nodes * 1 TB per node)
NVIDIA A100-40GB Tensor Core GPUs	656 (82 nodes * 8 gpus per node)
Total GPU Memory	26.24 TB (82 Nodes * 8 GPUs per node * 40 GB Per Node)
Mellanox 200G HDR InfiniBand Switch having 320Tb/s aggregate switch throughput (Compute Communication)	800 Ports (20 leafs *40 ports per leaf)
Mellanox 200G HDR InfiniBand Switches (Storage Delivery)	400 Ports (10 Switches * 40 ports per switch)
PFS based storage (Network attched) @250 GB/Sec, 4M IOPs	10.5 PiB (2 Tier Storage)

AIRAWAT-PSAI Compute Node Specification

Component	Specification
CPU	AMD EPYC 7742 64C 2.25GHz
CPU Cores	128 cores (Dual Socket, each with 64 cores) [256 cores with Hyper-threading]
L3 Cache	256 MB
System Memory (RAM)	1 TB
GPU	NVIDIA A100 - SXM4
GPU Memory	40 GB
Local Storage	14 TB
Total No. of GPUs per node	8
Networking	Mellanox ConnectX-6 VPI (Infiniband HDR), 1.6 Tb/Sec.

The AIRAWAT PoC of 200 AI Petaflops integrated with PARAM Siddhi-AI of 210 AI Petaflops gives a total peak compute of 410 AI Petaflops Mixed Precision and sustained compute capacity of 8.5 Petaflops (Rmax) Double Precision. The peak compute capacity (Double Precision, Rpeak) is 13.17 Petaflops

AIRAWAT-PSAI System Software Environment

• OS Ubuntu 20.04.2 LTS (DGXOS 5.0.5), x86_64 architecture
• Kernel 5.4.0-80-generic
• NVIDIA Driver Version 450.142.00
• NVIDIA NGC Support
• Python
• HPC-SDKs, MPIs and CUDA
• Development tools (compiler, libraries & AI Frameworks)
• Enroot Container Support

Application Frameworks - AI

AI Frameworks	PyTorch , TensorFlow, MxNet, Caffe2, MCTK, Theano, XGBoost
Deep Learning SDK	DeepStream, RAPIDS, TensorRT Inference Engine
Conversational AI(ASR, NLP, TTS)	Kaldi, NeMo
Vision AI (Computer Vision)	DeepVision, DeepCognition, YOLO, OpenCV
Workbook based Editor	Jupyter

Application Frameworks - HPC

Domain	Applications
Atomic & Molecular Sciences	Quantum Espresso, GROMACS, NAMD, LAMMPS, VMD, QMCPack, VASP *, SIESTA, NWChem, CPMD, Molden, MRCC*, DIRAC*, GAUSSIAN*, ABINIT
Computational Fluid Dynamics	OpenFOAM, STARCCM*
Computational Chemistry	GAMESS
Climate and Environmental Sciences	CESM, WRF*
Electronic Sciences	Ngspice
SDK	NVIDIA HPC-SDK, HPC-X(openMPI), intel oneAPI
Libraries	Math Kernel Library

Application Frameworks - Other Tools

Compilers	PGI, GNU, OpenACC , CUDA
Visualization Tools	Paraview , Graphana, MatplotLib,seaborn, cuGraph , wxMacMolPlt
Python Packages	NumPy, Pydicom, skimage, Pandas,Flask, LibROSA, Apache singa, Scikit-learn,Amazon Deep Java Library, Deeplearning4j, deep learning framework, Natural Language, Toolkit (NLTK), Gensim, CoreNLP, spaCy
CUDA toolkit & CUDA SDK	V10, 11, CUDA tuned Neural Network (cuDNN) Primitives, CUDA tuned BLAS, CUDA tuned Sparse, Matrix Operations (cuSPARSE)
Computation Framework	Hypre, Matlab*, Octave