diff --git a/singularity_container.ipynb b/singularity_container.ipynb index 02854af577938cb8b598aa74737b861160a216a5..2fb59ef13292d97c42d773fd29b7fd43307deb23 100644 --- a/singularity_container.ipynb +++ b/singularity_container.ipynb @@ -38,7 +38,11 @@ "source": [ "## Docker vs Singularity?\n", "\n", - "[Docker](https://www.docker.com/) is the most popular and widely used container system in the industry. But [Singularity](https://www.sylabs.io/singularity/) was built keeping HPC in mind, i.e a shared environment. Singularity is designed so that you can use it within SLURM jobs and it does not violate security constraints on the cluster. Though, since Docker is very popular and a lot of people were already using the Docker for their softwares, Singularity maintained a compatibility for Docker images. We'll be seeing this compatibility later in the notebook.\n", + "[Docker](https://www.docker.com/) is the most popular and widely used container system in the industry. But [Singularity](https://www.sylabs.io/singularity/) was built keeping HPC in mind, i.e a shared environment. Singularity is designed so that you can use it within SLURM jobs and it does not violate security constraints on the cluster. Though, since Docker is very popular and a lot of people were already using the Docker for their softwares, Singularity maintained a compatibility for Docker images. We'll be seeing this compatibility later in the notebook. Both Singularity and Docker maintain a hub where you can keep your docker remotely, and pull them from anywhere. Here is a link for both the hubs:\n", + "\n", + "[Singularity-Hub](https://singularity-hub.org) \n", + "[Docker Hub](https://hub.docker.com/)\n", + "\n", "\n", "Singularity is already available on Cheaha. To check the available modules for Cheaha, run the cell below:\n" ] @@ -226,6 +230,13 @@ "!singularity exec -B /data gcloud.simg gsutil" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To clean your directory of all the container images, you can run the command below" + ] + }, { "cell_type": "code", "execution_count": null, @@ -234,6 +245,86 @@ "source": [ "!rm *.simg" ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example 3:\n", + "\n", + "[NVIDIA GPU Cloud](https://www.nvidia.com/en-us/gpu-cloud/) (NGC) offers a container registry of Docker images with over 35 HPC, HPC visualization, deep learning, and data analytics containers optimized for GPUs and delivering accelerated performance (figure 3). The registry includes some of the most popular applications including GROMACS, NAMD, ParaView, VMD, and TensorFlow.\n", + "\n", + "For this example you would have to start a new Jupyter Notebook session on the 'pascalnodes' partition. Most of the settings are similar to the ones that you had in the [Git repo](https://gitlab.rc.uab.edu/rc-training-sessions/singularity_containers) for this session. you just need to add/modify the following things.\n", + "\n", + "In the Environment parameter add\n", + "```\n", + "module load cuda92/toolkit/9.2.88\n", + "module load CUDA/9.2.88-GCC-7.3.0-2.30\n", + "module load Singularity/2.5.2-GCC-5.4.0-2.26\n", + "module load Anaconda3\n", + "```\n", + "\n", + "And in the partition choose:\n", + "```\n", + "pascalnodes\n", + "```\n", + "\n", + "You will need to create a new account on Nvidia GPU Cloud, to pull down these conatiners." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!singularity pull docker://nvcr.io/hpc/vmd:cuda9-ubuntu1604-egl-1.9.4a17" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!singularity exec --nv vmd-cuda9-ubuntu1604-egl-1.9.4a17.simg /opt/vmd/bin/vmd" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Building a Singularity container recipe file\n", + "\n", + "Though building a recipe file for Singularity containers is beyond the scope of this session, we have provided a few important links below which would tell you how to create a recipe file for Singularity containers as well as build the container using them.\n", + "\n", + "When you want to create a container for production use on the cluster, you should build a container image from a definition file. Unfortunately, building containers from a definition file requires you to be a system administrator (root) on the machine you use for building. You will need to build Singularity containers on a machine that you control.\n", + "\n", + "To install Singularity on your system, follow the steps outlined here: \n", + "http://singularity.lbl.gov/install-linux \n", + "http://singularity.lbl.gov/install-mac \n", + "http://singularity.lbl.gov/install-windows \n", + "\n", + "Method of creating a Singularity recipe file: \n", + "http://singularity.lbl.gov/docs-recipes\n", + "\n", + "Method of building from a singularity recipe file: \n", + "http://singularity.lbl.gov/docs-build-container#building-containers-from-singularity-recipe-files\n", + "\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Acknowledgments / Useful Links:\n", + "\n", + "https://github.com/singularityhub/singularityhub.github.io/wiki \n", + "https://portal.biohpc.swmed.edu/content/guides/singularity-containers-biohpc/ \n", + "https://www.docker.com/ \n", + "https://devblogs.nvidia.com/docker-compatibility-singularity-hpc/ " + ] } ], "metadata": {