Getting Started

Account request and accessing the system

Requesting for an account on GPU Cluster

• New raad2 user:
New users are required to apply for an account on raad2 by visiting this link. While submitting the account request form, you need to;
• Select "Yes" under "GPUs Required?"
• Fill in "Research decription" field with brief summary of your research work and how GPUs will be helpful.
• Existing raad2 user:
Existing raad2 users can email TAMUQ IT service desk with subject "GPU access required" and provide a brief summary of your research description and how GPUs will be helpful.

Login to the system

After your account is approved and getting access to raad2, follow these steps to access the raad2 GPU:

• Using Windows:
After accessing raad2 via: MobaXterm, you need to issue the following command using your own raad2 username:

ssh <username>@raad2-gfx.qatar.tamu.edu

Please note that the password is the same as your raad2 account.

• Using Mac:
Go to termenal and issue the following command using your own raad2 username:

ssh <username>@raad2-gfx.qatar.tamu.edu

OR: To have X11 forwarding, which is a mechanism that allows you to start up remote applications but forward the application display to your local machine.

ssh -Y <username>@raad2-gfx.qatar.tamu.edu

Please note that the password is the same as your raad2 account.

Transferring your data

Please refer to the documentation here.

Programming Environments

Available Modules

In general, modules are pieces of code that can be loaded and unloaded into the environment upon demand. You need to initialize them for every application you will referencing to during the session. To see the available modules on the GPU, issue the following command:

abtakid91@raad2-gfx:~$ module avail

The first few lines of the output should look like this:

------------------------------------------- /opt/cray/pe/craype/default/modulefiles --------------------------------------------
craype-accel-nvidia20  craype-accel-nvidia60  craype-haswell    craype-network-infiniband  craype-x86-naples   
craype-accel-nvidia35  craype-accel-nvidia70  craype-ivybridge  craype-network-opa         craype-x86-skylake  
craype-accel-nvidia52  craype-broadwell       craype-mic-knl    craype-sandybridge

Available CUDA Versions

• CUDA10
• CUDA90
• CUDA91
• CUDA92

The CUDA version can be loaded with the command below: (For example, to load CUDA91)

abtakid91@raad2-gfx:~$ module load cuda91

The current cuDNN version is 7.3.1, it can be loaded with the command below:

abtakid91@raad2-gfx:~$ module load cudnn/7.3.1

Available Python Packages

• Python 2 (Default)
• Python 3.6

The Python version can be loaded with the command below: (For example, to load Python 3.6)

abtakid91@raad2-gfx:~$ module load python36

To see the loaded list of modules:

abtakid91@raad2-gfx:~$ module list

The first few lines of the output should look like this:

Currently Loaded Modulefiles:
 1) craype-x86-skylake          3) shared          5) default-environment   7) python36               
 2) craype-network-infiniband   4) slurm/18.08.4   6) gcc/6.1.0(default)    8) cuda91/toolkit/9.1.85

Please note that if you do not load any python version, it will be python2 automatically and it will not show when you issue the module list command above.

Python Virtual Environments

A virtual environment is an isolated environment for your projects. This means that each project can have its own dependencies, regardless of what dependencies every other project has. Creating a virtual environment solves the problem of having multiple projects on the same system with conflicting package requirements.

There are two python virtual environments options, Python Virtual Environment and Conda Python Virtual Environment (virtualenv). The main difference between them is that conda is a bit more full-featured. Conda has dedicated syntax for creating environments and installing packages, and can also manage the installation of different python versions. For virtualenv, you just activate the environment and then use all the normal commands.

Python Virtual Environment

Let us create the mlproject virtual environment from the sample section: Let us load the python version we wish to use: (e.g. python 3.6)

abtakid91@raad2-gfx:~$ module load python36

To create the virtual environment:

abtakid91@raad2-gfx:~$ virtualenv mlproject

To activate the virtual environment:

abtakid91@raad2-gfx:~$ source mlproject/bin/activate

In the terminal, you will notice that (mlproject) is added, this means that you are working inside the python virtual environment.

To verify the python version of your envirnment:

(mlproject) abtakid91@raad2-gfx:~$ which python

The output should be something like this:

~/mlproject/bin/python

For the mlproject sample, we need to install the sklearn and pandas packages.

(mlproject) abtakid91@raad2-gfx:~$ pip install pandas
(mlproject) abtakid91@raad2-gfx:~$ pip install sklearn

To deactivate the virtual environment:

(mlproject) abtakid91@raad2-gfx:~$ deactivate

Conda Python Virtual Environment

Before creating the virtual envirnment, it is convenient to add the source line to the .bashrc by: (you have to do this only once)

echo "source /cm/shared/apps/anaconda3/etc/profile.d/conda.sh" >> .bashrc

Let us create the dlproject virtual environment from the sample section:

To create the virtual envirnment:

abtakid91@raad2-gfx:~$ conda create -n dlproject python=3.8

To activate the created envirnment:

conda activate dlproject

In the terminal, you will notice that (dlproject) is added, this means that you are working inside the Conda python virtual environment.

To verify the python version of your envirnment:

(dlproject) abtakid91@raad2-gfx:~$ which python

The output should be something like this:

~/.conda/envs/dlproject/bin/python

To deactivate the current envirnemnt:

(dlproject) abtakid91@raad2-gfx:~$ conda deactivate

To see the list of conda environments you have:

abtakid91@raad2-gfx:~$ conda env list