Skip to content
Snippets Groups Projects
Commit c2b49a76 authored by Natasha Pavlovikj's avatar Natasha Pavlovikj
Browse files

Update anaconda page

This resolves #10, resolves #14
parent e9b77d48
No related branches found
No related tags found
1 merge request!181Update anaconda page
......@@ -17,8 +17,10 @@ package and environment manager to make managing these environments
straightforward.
- [Using Anaconda](#using-anaconda)
- [Installing Packages](#installing-packages)
- [Adding Packages to an Existing Environment](#adding-packages-to-an-existing-environment)
- [Searching for Packages](#searching-for-packages)
- [Creating custom Anaconda Environment](#creating-custom-anaconda-environment)
- [Adding and Removing Packages from an Existing Environment](#adding-and-removing-packages-from-an-existing-environment)
- [Creating custom GPU Anaconda Environment](#creating-custom-gpu-anaconda-environment)
- [Using an Anaconda Environment in a Jupyter Notebook on Crane](#using-an-anaconda-environment-in-a-jupyter-notebook-on-crane)
### Using Anaconda
......@@ -68,6 +70,7 @@ conda info -e
The active environment will be marked with an asterisk (\*) character.
The `list` command will show all packages installed
in the currently active environment.
......@@ -77,9 +80,9 @@ conda list
{{< /highlight >}}
{{% /panel %}}
### Installing Packages
### Searching for Packages
To find the names of packages, use the `search` subcommand.
To find packages, use the `search` subcommand.
{{% panel theme="info" header="Search for packages" %}}
{{< highlight bash >}}
......@@ -91,32 +94,33 @@ If the package is available, this will also display available package
versions and compatible Python versions the package may be installed
under.
### Creating Custom Anaconda Environment
The `create` command is used to create a new environment. It requires
at a minimum a name for the environment, and at least one package to
install. For example, suppose we wish to create a new environment, and
need version 1.8 of NumPy.
need version 1.17 of NumPy.
{{% panel theme="info" header="Create a new environment by providing a name and package specification" %}}
{{< highlight bash >}}
conda create -n mynumpy numpy=1.8
conda create -n mynumpy numpy=1.17
{{< /highlight >}}
{{% /panel %}}
This will create a new environment called 'mynumpy' and installed NumPy
version 1.8, along with any required dependencies.
version 1.17, along with any required dependencies.
To use the environment, we must first *activate* it.
{{% panel theme="info" header="Activate environment" %}}
{{< highlight bash >}}
source activate mynumpy
conda activate mynumpy
{{< /highlight >}}
{{% /panel %}}
Our new environment is now active, and we can use it. The shell prompt
will change to indicate this as well (this can be disable if desired).
Our new environment is now active, and we can use it. The shell prompt will change to indicate this as well.
### Adding Packages to an Existing Environment
### Adding and Removing Packages from an Existing Environment
To install additional packages in an environment, use the `install`
subcommand. Suppose we want to install iPython in our 'mynumpy'
......@@ -156,7 +160,7 @@ To exit an environment, we *deactivate* it.
{{% panel theme="info" header="Exit current environment" %}}
{{< highlight bash >}}
source deactivate
conda deactivate
{{< /highlight >}}
{{% /panel %}}
......@@ -169,6 +173,49 @@ conda remove -n mynumpy --all
{{< /highlight >}}
{{% /panel %}}
### Creating Custom GPU Anaconda Environment
We provide GPU versions of various frameworks such as `tensorflow`, `keras`, `theano`, via [modules](../../modules).
However, sometimes you may need additional libraries or packages that are not available as part of these modules.
In this case, you will need to create your own GPU Anaconda environment.
To do this, you need to first clone one of our GPU modules to a new Anaconda environment, and then install the desired packages in this new environment.
The reason for this is that the GPU modules we support are built using the specific CUDA drivers our GPU nodes have.
If you just create custom GPU environment without cloning the module, your code will not utilize the GPUs correctly.
For example, if you want to use `tensorflow` with additional packages, first do:
{{% panel theme="info" header="Cloning GPU module to a new Anaconda environment" %}}
{{< highlight bash >}}
module load tensorflow-gpu/py36/1.14
module load anaconda
conda create -n tensorflow-gpu-1.14-custom --clone $CONDA_DEFAULT_ENV
module purge
{{< /highlight >}}
{{% /panel %}}
This will create a new `tensorflow-gpu-1.14-custom` environment in your home directory that is a copy of the `tensorflow-gpu` module.
Then, you can install the additional packages you need in this environment.
{{% panel theme="info" header="Install new packages in the currently active environment" %}}
{{< highlight bash >}}
module load anaconda
conda activate tensorflow-gpu-1.14-custom
conda install <packages>
{{< /highlight >}}
{{% /panel %}}
Next, whenever you want to use this custom GPU Anaconda environment, you need to add these two lines in your submit script:
{{< highlight bash >}}
module load anaconda
conda activate tensorflow-gpu-1.14-custom
{{< /highlight >}}
{{% notice info %}}
If you have custom GPU Anaconda environment please only use the two lines from above and **DO NOT** load the module you have cloned earlier.
Using `module load tensorflow-gpu/py36/1.14` and `conda activate tensorflow-gpu-1.14-custom` in the same script is **wrong** and may give you various errors and incorrect results.
{{% /notice %}}
### Using an Anaconda Environment in a Jupyter Notebook on Crane
It is not difficult to make an Anaconda environment available to a
......@@ -185,7 +232,7 @@ Jupyter Notebook. To do so, follow the steps below, replacing
2. Using the command-line environment, load the target conda
environment:
{{< highlight bash >}}source activate myenv{{< /highlight >}}
{{< highlight bash >}}conda activate myenv{{< /highlight >}}
3. Install the Jupyter kernel and add the environment:
......@@ -216,7 +263,7 @@ Jupyter Notebook. To do so, follow the steps below, replacing
{{< /highlight >}}
4. Once you have the environment set up, deactivate it:
{{< highlight bash >}}source deactivate{{< /highlight >}}
{{< highlight bash >}}conda deactivate{{< /highlight >}}
5. To make your conda environments accessible from the worker nodes,
enter the following commands:
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment