diff --git a/content/applications/user_software/compiling_an_openmp_application.md b/content/applications/user_software/compiling_an_openmp_application.md index e409a5bfdff476f808439e1fbf7778b03f8e7bd9..c4a0d71b6689fe54ef2150afb301604c5618d267 100644 --- a/content/applications/user_software/compiling_an_openmp_application.md +++ b/content/applications/user_software/compiling_an_openmp_application.md @@ -1,6 +1,7 @@ +++ title = "Compiling an OpenMP Application" description = "How to compile an OpenMP-based application on HCC resources." +weight=20 +++ Compiling an [OpenMP](https://computing.llnl.gov/tutorials/openMP) diff --git a/content/applications/user_software/installing_perl_modules.md b/content/applications/user_software/installing_perl_modules.md index 4674bce2d04985feb8284b64ca43173068cad76c..518c67310d5d460dfed4ee0f9bc095dfd2d6f7ba 100644 --- a/content/applications/user_software/installing_perl_modules.md +++ b/content/applications/user_software/installing_perl_modules.md @@ -1,6 +1,7 @@ +++ title = "Installing Perl modules" description = "How to install needed Perl modules under your account." +weight=90 +++ If you need additional Perl modules, they can be installed into your diff --git a/content/applications/user_software/using_anaconda_package_manager.md b/content/applications/user_software/using_anaconda_package_manager.md index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..13a3db0f51428289f7ef789197b7149c0f99a43c 100644 --- a/content/applications/user_software/using_anaconda_package_manager.md +++ b/content/applications/user_software/using_anaconda_package_manager.md @@ -0,0 +1,241 @@ ++++ +title = "Using Anaconda Package Manager" +description = "How to use the Anaconda Package Manager on HCC resources." +weight=10 ++++ + +[Anaconda](https://www.anaconda.com/what-is-anaconda), +from [Anaconda, Inc](https://www.anaconda.com) +is a completely free enterprise-ready distribution for large-scale data +processing, predictive analytics, and scientific computing. It includes +over 195 of the most popular Python packages for science, math, +engineering, and data analysis. **It also offers the ability to easily +create custom _environments_ by mixing and matching different versions +of Python and/or R and other packages into isolated environments that +individual users are free to create.** Anaconda includes the `conda` +package and environment manager to make managing these environments +straightforward. + +- [Using Anaconda](#using-anaconda) +- [Installing Packages](#installing-packages) +- [Adding Packages to an Existing Environment](#adding-packages-to-an-existing-environment) +- [Using an Anaconda Environment in a Jupyter Notebook on Crane](#using-an-anaconda-environment-in-a-jupyter-notebook-on-crane) + +### Using Anaconda + +While the standard methods of installing packages via `pip` +and `easy_install` work with Anaconda, the preferred method is using +the `conda` command. + +{{% notice info %}} +Full documentation on using Conda is available +at http://conda.pydata.org/docs/ + +A [cheatsheet](/attachments/11635089.pdf) is also provided. +{{% /notice %}} + +A few examples of the basic commands are provided here. For a full +explanation of all of Anaconda/Conda's capabilities, see the +documentation linked above. + +Anaconda is provided through the `anaconda` module on HCC machines. To +begin using it, load the Anaconda module. + +{{% panel theme="info" header="Load the Anaconda module to start using Conda" %}} +{{< highlight bash >}} +module load anaconda +{{< /highlight >}} +{{% /panel %}} + +To display general information about Conda/Anaconda, use the `info` subcommand. + +{{% panel theme="info" header="Display general information about Conda/Anaconda" %}} +{{< highlight bash >}} +conda info +{{< /highlight >}} +{{% /panel %}} + +Conda allows the easy creation of isolated, custom environments with +packages and versions of your choosing. To show all currently available +environments, and which is active, use the `info `subcommand with the +`-e` option. + +{{% panel theme="info" header="List available environments" %}} +{{< highlight bash >}} +conda info -e +{{< /highlight >}} +{{% /panel %}} + +The active environment will be marked with an asterisk (\*) character. + +The `list` command will show all packages installed +in the currently active environment. + +{{% panel theme="info" header="List installed packages in current environment" %}} +{{< highlight bash >}} +conda list +{{< /highlight >}} +{{% /panel %}} + +### Installing Packages + +To find the names of packages, use the `search` subcommand. + +{{% panel theme="info" header="Search for packages" %}} +{{< highlight bash >}} +conda search numpy +{{< /highlight >}} +{{% /panel %}} + +If the package is available, this will also display available package +versions and compatible Python versions the package may be installed +under. + +The `create` command is used to create a new environment. It requires +at a minimum a name for the environment, and at least one package to +install. For example, suppose we wish to create a new environment, and +need version 1.8 of NumPy. + +{{% panel theme="info" header="Create a new environment by providing a name and package specification" %}} +{{< highlight bash >}} +conda create -n mynumpy numpy=1.8 +{{< /highlight >}} +{{% /panel %}} + +This will create a new environment called 'mynumpy' and installed NumPy +version 1.8, along with any required dependencies. + +To use the environment, we must first *activate* it. + +{{% panel theme="info" header="Activate environment" %}} +{{< highlight bash >}} +source activate mynumpy +{{< /highlight >}} +{{% /panel %}} + +Our new environment is now active, and we can use it. The shell prompt +will change to indicate this as well (this can be disable if desired). + +### Adding Packages to an Existing Environment + +To install additional packages in an environment, use the `install` +subcommand. Suppose we want to install iPython in our 'mynumpy' +environment. While the environment is active, use `install `with no +additional arguments. + +{{% panel theme="info" header="Install a new package in the currently active environment" %}} +{{< highlight bash >}} +conda install ipython +{{< /highlight >}} +{{% /panel %}} + +If you aren't currently in the environment you wish to install the +package in, add the `-n `option to specify the name. + +{{% panel theme="info" header="Install new packages in a specified environment" %}} +{{< highlight bash >}} +conda install -n mynumpy ipython +{{< /highlight >}} +{{% /panel %}} + +The `remove` subcommand to uninstall a package functions similarly. + +{{% panel theme="info" header="Remove package from currently active environment" %}} +{{< highlight bash >}} +conda remove ipython +{{< /highlight >}} +{{% /panel %}} + +{{% panel theme="info" header="Remove package from environment specified by name" %}} +{{< highlight bash >}} +conda remove -n mynumpy ipython +{{< /highlight >}} +{{% /panel %}} + +To exit an environment, we *deactivate* it. + +{{% panel theme="info" header="Exit current environment" %}} +{{< highlight bash >}} +source deactivate +{{< /highlight >}} +{{% /panel %}} + +Finally, to completely remove an environment, add the `--all `option +to `remove`. + +{{% panel theme="info" header="Completely remove an environment" %}} +{{< highlight bash >}} +conda remove -n mynumpy --all +{{< /highlight >}} +{{% /panel %}} + +### Using an Anaconda Environment in a Jupyter Notebook on Crane + +It is not difficult to make an Anaconda environment available to a +Jupyter Notebook. To do so, follow the steps below, replacing +`myenv` with the name of the Python or R environment you wish to use: + +1. Stop any running Jupyter Notebooks and ensure you are logged out of + the JupyterHub instance at https://crane.unl.edu + 1. If you are not logged out, please click the Control Panel button + located in the top right corner. + 2. Click the "Stop My Server" Button to terminate the Jupyter + server. + 3. Click the logout button in the top right corner. + +2. Using the command-line environment, load the target conda + environment: + {{< highlight bash >}}source activate myenv{{< /highlight >}} + +3. Install the Jupyter kernel and add the environment: + + 1. For a **Python** conda environment, install the IPykernel + package, and then the kernel specification: + + {{< highlight bash >}} + # Install ipykernel + conda install ipykernel + + # Install the kernel specification + python -m ipykernel install --user --name "$CONDA_DEFAULT_ENV" --display-name "Python ($CONDA_DEFAULT_ENV)" + {{< /highlight >}} + + 2. For an **R** conda environment, install the jupyter\_client and + IRkernel packages, and then the kernel specification: + + {{< highlight bash >}} + # Install PNG support for R, the R kernel for Jupyter, and the Jupyter client + conda install r-png + conda install r-irkernel jupyter_client + + # Install jupyter_client 5.2.3 from anaconda channel for bug workaround + conda install -c anaconda jupyter_client + + # Install the kernel specification + R -e "IRkernel::installspec(name = '$CONDA_DEFAULT_ENV', displayname = 'R ($CONDA_DEFAULT_ENV)', user = TRUE)" + {{< /highlight >}} + +4. Once you have the environment set up, deactivate it: + {{< highlight bash >}}source deactivate{{< /highlight >}} + +5. To make your conda environments accessible from the worker nodes, + enter the following commands: + + {{< highlight bash >}} + mkdir -p $WORK/.jupyter + mv ~/.local/share/jupyter/kernels $WORK/.jupyter + ln -s $WORK/.jupyter/kernels ~/.local/share/jupyter/kernels + {{< /highlight >}} + +{{% notice note %}} +**Note**: Step 5 only needs to be done once. Any future created +environments will automatically be accessible from SLURM notebooks +once this is done. +{{% /notice %}} + +6. Login to JupyterHub at https://crane.unl.edu + and create a new notebook using the environment by selecting the + correct entry in the `New` dropdown menu in the top right + corner. + {{< figure src="/images/24151931.png" height="400" class="img-border">}} + diff --git a/content/applications/user_software/using_singularity.md b/content/applications/user_software/using_singularity.md index b04b7d1f413d29738d59840133c804d055477a14..d698904a6a18b64ab669ce803b4a747e8146e9dd 100644 --- a/content/applications/user_software/using_singularity.md +++ b/content/applications/user_software/using_singularity.md @@ -1,6 +1,7 @@ +++ title = "Using Singularity and Docker Containers" description = "How to use the Singularity containerization software on HCC resources." +weight=20 +++ ## What is Singularity