Skip to content
Snippets Groups Projects
Commit 6d559556 authored by mtanash2's avatar mtanash2
Browse files

Add documentation for running jupyterlab with slurm

parent 2007bd9f
No related branches found
No related tags found
1 merge request!416Add documentation for running jupyterlab with slurm
+++
title = "Running JupyterLab Notebooks with Slurm"
description = "Running JupyterLab Notebooks with Slurm."
+++
If you want to run JupyterLab Notebooks within Slurm job, you can use one of these methods:
## Method 1: Converting the Notebook to Python file using the Open OnDemand JupyterLab Interface
1- Open your Jupyter Notebook with JupyterLab.
2- From the "File" menu from the JupyterLab interface choose **"Save and Export Notebook As"** and select **"Executable Script"**. This step will save the Python script on your local computer.
3- Copy your exported code into HCC Swan cluster.
4- Now, you can create a SLURM submit file and load the system-wide module that corresponds to the system-wide kernel in the submit script.
5- Finally, you can run your Python script by adding **python test.py** in your SLURM submit script where **test.py** is the name of your Python file.
{{% notice info %}}
If you have used your custom kernel, then you will need to load the anaconda module using the command **module load anaconda** in the SLURM submit script and activate your custom conda environment as shown in https://hcc.unl.edu/docs/applications/user_software/using_anaconda_package_manager/#creating-custom-anaconda-environments.
{{% /notice %}}
## Method 2: Running ipynb file using ipython
In this method, **ipython** and **nbconvert** are required. The majority of system-wide kernels already have ipython and nbconvert installed. If that is not the case, please email us at hcc-support@unl.edu.
1- Create a SLURM submit file and load the system-wide module that corresponds to the system-wide kernel in the submit script.
2- Now, you can run your JupyterLab Notebook by adding **ipython -c "%run test.ipynb"** in your SLURM submit script where **test.ipynb** is the name of your Notebook.
{{% notice info %}}
If you want to use your custom kernel, please make sure ipython and nbconvert are installed in the conda environment. Then you will need to load the anaconda module using the command **module load anaconda** in the SLURM submit script and activate your custom conda environment as shown in https://hcc.unl.edu/docs/applications/user_software/using_anaconda_package_manager/#creating-custom-anaconda-environments.
{{% /notice %}}
## Method 3: Running ipynb file using jupyter-nbconvert
In this method **nbconvert** is required. The majority of system-wide kernels already have nbconvert installed. If that is not the case, please email us at hcc-support@unl.edu.
1- Create a SLURM submit file and load the system-wide module that corresponds to the system-wide kernel in the submit script.
2- Now, you can run your JupyterLab Notebook by adding **jupyter-nbconvert –execute –clear-output test.ipynb** in your SLURM submit script where **test.ipynb** is the name of your Notebook.
{{% notice info %}}
If you want to use your custom kernel, please make sure nbconvert is installed in the conda environment. Then you will need to load the anaconda module using the command **module load anaconda** in the SLURM submit script and activate your custom conda environment as shown in https://hcc.unl.edu/docs/applications/user_software/using_anaconda_package_manager/#creating-custom-anaconda-environments.
{{% /notice %}}
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment