Commit 65e4056c authored by Adam Caprez's avatar Adam Caprez
Browse files

Merge branch 'conda_clone' into 'master'

Add blurb about conda clone

See merge request hcc/hcc-docs!95
parents 5d4d1a16 0af13239
......@@ -19,7 +19,9 @@ must be loaded before or along with, that module.
For example, the `cdo/2.1` modules requires `compiler/pgi/13.` To load
the cdo module, doing either
`module load compiler/pgi/13module load cdo/2.1`
`module load compiler/pgi/13`
`module load cdo/2.1`
or
......@@ -34,4 +36,10 @@ Some packages list multiple compilers for prerequisites. This means that
the package has been built with each version of the compilers listed.
{{% /panel %}}
{{% panel theme="warning" header="Custom GPU Anaconda Environment" %}}
If you are using custom GPU Anaconda Environment, the only module you need to load is `anaconda`:
`module load anaconda`
{{% /panel %}}
{{< table url="http://crane-head.unl.edu:8192/lmod/spider/json" >}}
......@@ -19,7 +19,9 @@ must be loaded before or along with, that module.
For example, the `cdo/2.1` modules requires `compiler/pgi/13.` To load
the cdo module, doing either
`module load compiler/pgi/13module load cdo/2.1`
`module load compiler/pgi/13`
`module load cdo/2.1`
or
......
......@@ -16,7 +16,8 @@ package and environment manager to make managing these environments
straightforward.
- [Using Anaconda](#using-anaconda)
- [Installing Packages](#installing-packages)
- [Creating custom Anaconda Environment](#creating-custom-anaconda-environment)
- [Creating custom GPU Anaconda Environment](#creating-custom-gpu-anaconda-environment)
- [Adding Packages to an Existing Environment](#adding-packages-to-an-existing-environment)
- [Using an Anaconda Environment in a Jupyter Notebook on Crane](#using-an-anaconda-environment-in-a-jupyter-notebook-on-crane)
......@@ -76,8 +77,6 @@ conda list
{{< /highlight >}}
{{% /panel %}}
### Installing Packages
To find the names of packages, use the `search` subcommand.
{{% panel theme="info" header="Search for packages" %}}
......@@ -90,6 +89,8 @@ If the package is available, this will also display available package
versions and compatible Python versions the package may be installed
under.
### Creating Custom Anaconda Environment
The `create` command is used to create a new environment.  It requires
at a minimum a name for the environment, and at least one package to
install.  For example, suppose we wish to create a new environment, and
......@@ -115,6 +116,44 @@ source activate mynumpy
Our new environment is now active, and we can use it.  The shell prompt
will change to indicate this as well (this can be disable if desired).
### Creating Custom GPU Anaconda Environment
We provide GPU versions of various frameworks such as `tensorflow`, `keras`, `theano`, via [modules](../module_commands). However, sometimes you may need additional libraries or packages that are not available as part of these modules. In this case, you will need to create your own GPU Anaconda environment.
To do this, you need to first clone one of our GPU modules to a new Anaconda environment, and then install the desired packages in this new environment.
The reason for this is that the GPU modules we support are built using the specific CUDA drivers our GPU nodes have. If you just create custom GPU environment without cloning the module, your code will not utilize the GPUs.
For example, if you want to use `tensorflow` with additional packages, first do:
{{% panel theme="info" header="Cloning GPU module to a new Anaconda environment" %}}
{{< highlight bash >}}
module load tensorflow-gpu/py36/1.12
conda create -n tensorflow-gpu-1.12-custom --clone $CONDA_DEFAULT_ENV
module purge
{{< /highlight >}}
{{% /panel %}}
This will create a new `tensorflow-gpu-1.12-custom` environment in your home directory that is a copy of the `tensorflow-gpu` module. Then, you can install the additional packages you need in this environment.
{{% panel theme="info" header="Install new packages in the currently active environment" %}}
{{< highlight bash >}}
module load anaconda
source activate tensorflow-gpu-1.12-custom
conda install <packages>
{{< /highlight >}}
{{% /panel %}}
Next, whenever you want to use this custom GPU Anaconda environment, you need to add these two lines in your submit script:
{{< highlight bash >}}
module load anaconda
source activate tensorflow-gpu-1.12-custom
{{< /highlight >}}
{{% notice info %}}
If you have custom GPU Anaconda environment please only use the two lines from above and **DO NOT** load the module you have cloned earlier. Using `module load tensorflow-gpu/py36/1.12` and `source activate tensorflow-gpu-1.12-custom` in the same script is **wrong** and may give you various errors and incorrect results.
{{% /notice %}}
### Adding Packages to an Existing Environment
To install additional packages in an environment, use the `install`
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment