Singularity Containers

Installing Docker on Linux

Before we begin, make sure the account being used is a sudo account. To give an account sudo privileges (or create a new sudo user), follow the steps here: https://linuxize.com/post/how-to-create-a-sudo-user-on-ubuntu/

Installing Docker on Linux

1. Follow steps in the following to install Docker to your Linux Machine: https://docs.docker.com/install/linux/docker-ce/ubuntu/#install-docker-ce

2. Go to docker hub (https://hub.docker.com/) and find the container that is needed

3. In the linux terminal, type:

$ sudo docker pull 'name'

The name will be specified on the container’s information page

To list all the available Docker images on your host run the following command:

$ sudo docker images

3. Next we need to run the container:

a. Run the following command:

$ sudo docker run --name () -it repository name bash

After name (inside the brackets) name the container by your choice (DO NOT keep the brackets). The repository name is found in the first column after doing sudo docker image

The prompts should look like this (You enter the container):

user@namec: ~$ sudo docker run --name () -it repository name bash
root@123456abcdef:/#

5. Next type this command:

root@123456abcdef:/# vim samplescript.py

Now you will notice that the previous command line will not work as vim needs to be installed. To do this:

5. Next type this command:

root@123456abcdef:/# apt install vim

6. Now running the command in step 5 again will open up the file and it will look like this:

~
~
~
~
~
~
~
~
~
~
~
~
~
"samplescript.py" [New File]

7. Next, we can now write in the container. We will be writing this simple Python code and running it later:

print("Now you have installed Docker, run a container (This one being Python), and used vim to create and image!")
~
~
~
~
~
~
~
~
~
~
~
~
~
-- INSERT --recording @a

To exit the editing window, press "!wa" and press enter.

8. Now to run the image, input the following commands:

root@123456abcdef:/# chmod +x samplescript.py
root@123456abcdef:/# python3 samplescript.py

The output should run the image we made and look like this:

root@123456abcdef:/# chmod +x samplescript.py
root@123456abcdef:/# python3 samplescript.py
Now you have installed Docker, run a container (This one being Python), and used vim to create an image!
root@123456abcdef:/#

Singularity

Introduction to Singularity

Introduction to singularity: Singularity is a container platform. It allows you to create and run containers that package up pieces of software in a way that is portable and reproducible. You can build a container using Singularity on your laptop, and then run it on many of the largest HPC clusters in the world, local university or company clusters, a single server, in the cloud, or on a workstation down the hall. Your container is a single file, and you don’t have to worry about how to install all the software you need on each different operating system and system.

Why Use Singularity

Singularity was created to run complex applications on HPC clusters in a simple, portable, and reproducible way. First developed at Lawrence Berkeley National Laboratory, it quickly became popular at other HPC sites, academic sites, and beyond. Singularity is an open-source project, with a friendly community of developers and users. The user base continues to expand, with Singularity now used across industry and academia in many areas of work.

Installing Singularity

On an updated guide on installing Singularity, follow the steps here: https://sylabs.io/guides/3.5/user-guide/quick_start.html

Pulling Containers into Singularity from Docker

To pull Docker containers into Singularity, run the following command:

$ sudo singularity pull docker://repository name

For example, Singularity pull docker://python

Building a Container

Knowing about the Build command is vital to understanding containers in Singularity.

The Build command accepts a target as input and produces a container as output. The target defines the method that Build uses to create the container. It is one of the following:

A URL beginning with docker:// to build from the Docker Hub

A URL beginning with library:// to build from the Container Library

A URL beginning with shub:// to build from the Singularity Hub

A path to an existing container on your local machine

A path to a directory to build from a sandbox

A path to a Singularity definition file

Build can produce containers in two different formats:

SIF: Singularity Image File: Is a compressed read-only format

Writable (ch)root directory called a sandbox for interactive development (--sandbox)

Existing containers in these formats can be converted between either format.

Downloading an existing Container from Docker Hub

$ sudo singularity build lolcow.sif docker://godlovedc/lolcow

lolcow and the Docker directory are for example.

Creating writable Sandbox Directories

If you wanted to create a container within a writable directory (called a sandbox) you can do so with the --sandbox option. It’s possible to create a sandbox without root privileges, but to ensure proper file permissions it is recommended to do so as root.

$ sudo singularity build --sandbox lolcow/ library://sylabs-jms/testing/lolcow

The resulting directory operated just like a container in a SIF file. TO make changed within the container, use the --writable flag when you invoke your container. It is a good idea to do this as root to ensure you have permission to access the files and directories that you want to change:

sudo singularity shell --writable lolcow/

You can use commands like shell, exec , and run with this directory just as you would with a Singularity image. If you pass the --writable option when you use your container you can also write files within the sandbox directory (provided you have the permissions to do so).

$ sudo singularity exec --writable ubuntu touch/foo
$ singularity exec ubuntu/ls/foo/foo

Converting Containers from one Format to Another

If you already have a container saved locally, you can use it as a target to build a new container. This allows you to convert containers from one format to another. For example, if you had a sandbox container called development/ and you wanted to convert it to SIF container called production.sif you could:

$ sudo sungularity build production.sif development

In our case, our sandbox container name is lolcow/, so to convert it to a SIF container called production.sif, you would replace development/ with lolcow/.

Use care when converting a sandbox directory to the default SIF format. If changes were made to the writable container before conversion, there is no record of those changes in the Singularity definition file rendering your container non-reproducible. It is a best practice to build your immutable production containers directly from a Singularity definition file instead.

For more information, see: https://sylabs.io/guides/3.5/user-guide/build_a_container.html

Installing an Application in a Container in Singularity

The following section will summarize most of what is mentioned above, and is an easy way to practice and clear any confusions.

Using "R" as an example application:

1. First we will pull R from Docker Hub into Singularity:

$ sudo singularity pull docker://r-base

2. Next to build an R script into the container:

$ sudo singularity build script.R docker://r-base

2. Now run the file:

$ sudo singularity run script.R

The output should look like this:

R version 3.6.3 (2020-2-29) -- "Holding the Windsock"
Copyright (C) 2020 The R foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY. 
You are welcome to redistribute it under certain conditions.
Type 'License()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and 
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

As can be observed, R is now running in the singularity container!