Skip to content
Snippets Groups Projects

Update docs for MPI conda packages

Merged Natasha Pavlovikj requested to merge conda_update into master
1 file
+ 47
0
Compare changes
  • Side-by-side
  • Inline
@@ -22,6 +22,7 @@ straightforward.
@@ -22,6 +22,7 @@ straightforward.
- [Using /common for environments](#using-common-for-environments)
- [Using /common for environments](#using-common-for-environments)
- [Adding and Removing Packages from an Existing Environment](#adding-and-removing-packages-from-an-existing-environment)
- [Adding and Removing Packages from an Existing Environment](#adding-and-removing-packages-from-an-existing-environment)
- [Creating custom GPU Anaconda Environment](#creating-custom-gpu-anaconda-environment)
- [Creating custom GPU Anaconda Environment](#creating-custom-gpu-anaconda-environment)
 
- [Creating custom MPI Anaconda Environment](#creating-custom-mpi-anaconda-environment)
- [Using an Anaconda Environment in a Jupyter Notebook](#using-an-anaconda-environment-in-a-jupyter-notebook)
- [Using an Anaconda Environment in a Jupyter Notebook](#using-an-anaconda-environment-in-a-jupyter-notebook)
### Using Anaconda
### Using Anaconda
@@ -245,6 +246,52 @@ If you have custom GPU Anaconda environment please only use the two lines from a
@@ -245,6 +246,52 @@ If you have custom GPU Anaconda environment please only use the two lines from a
Using `module load tensorflow-gpu/py36/1.14` and `conda activate tensorflow-gpu-1.14-custom` in the same script is **wrong** and may give you various errors and incorrect results.
Using `module load tensorflow-gpu/py36/1.14` and `conda activate tensorflow-gpu-1.14-custom` in the same script is **wrong** and may give you various errors and incorrect results.
{{% /notice %}}
{{% /notice %}}
 
### Creating Custom MPI Anaconda Environment
 
 
Some conda packages available on `conda-forge` and `bioconda` support MPI (via `openmpi` or `mpich`).
 
**Currently only packages that were built using `openmpi 4.1` and `mpich 4.0` are supported on HCC clusters.**
 
 
In order to be able to correctly use these MPI packages with the MPI libraries installed on our clusters, two steps need to be performed.
 
 
First, at install time, besides the package, the package `openmpi=4.1.*=external_*` or `mpich=4.0.*=external_*` needs to be installed for `openmpi` or `mpich` respectively.
 
 
Secondly, when activating the conda environment and using the package, the system-wide `openmpi/4.1` or `mpich/4.0` module needs to be loaded depending on the MPI library used.
 
 
For example, the steps for creating conda environment with `mpi4py` that supports `openmpi` are:
 
{{% panel theme="info" header="Creating Anaconda environment with openmpi" %}}
 
{{< highlight bash >}}
 
module purge
 
module load anaconda
 
conda create -n mpi4py-openmpi mpi4py openmpi=4.1.*=external_*
 
{{< /highlight >}}
 
{{% /panel %}}
 
and the steps for using this environment are:
 
{{% panel theme="info" header="Using Anaconda environment with openmpi" %}}
 
{{< highlight bash >}}
 
module purge
 
module load compiler/gcc/10 openmpi/4.1 anaconda
 
conda activate mpi4py-openmpi
 
{{< /highlight >}}
 
{{% /panel %}}
 
 
 
The steps for creating conda environment with `mpi4py` that supports `mpich` are:
 
{{% panel theme="info" header="Creating Anaconda environment with mpich" %}}
 
{{< highlight bash >}}
 
module purge
 
module load anaconda
 
conda create -n mpi4py-mpich mpi4py mpich=4.0.*=external_*
 
{{< /highlight >}}
 
{{% /panel %}}
 
and the steps for using this environment are:
 
{{% panel theme="info" header="Using Anaconda environment with mpich" %}}
 
{{< highlight bash >}}
 
module purge
 
module load compiler/gcc/10 mpich/4.0 anaconda
 
conda activate mpi4py-mpich
 
{{< /highlight >}}
 
{{% /panel %}}
 
### Using an Anaconda Environment in a Jupyter Notebook
### Using an Anaconda Environment in a Jupyter Notebook
It is not difficult to make an Anaconda environment available to a
It is not difficult to make an Anaconda environment available to a
Loading