Skip to content
Snippets Groups Projects
Commit 51924534 authored by Natasha Pavlovikj's avatar Natasha Pavlovikj
Browse files

Modify CS paths

parent ca774446
No related branches found
No related tags found
1 merge request!463Update cs docs
...@@ -29,18 +29,17 @@ The CryoSPARC projects and data files on Swan can only be accessed by you, unles ...@@ -29,18 +29,17 @@ The CryoSPARC projects and data files on Swan can only be accessed by you, unles
- This will open a form that needs to be filled with information regarding CryoSPARC and the resources requested. Below is explanation of some of the required fields to launch the CryoSPARC App: - This will open a form that needs to be filled with information regarding CryoSPARC and the resources requested. Below is explanation of some of the required fields to launch the CryoSPARC App:
- **CryoSPARC License ID**: To start CryoSPARC you will need to have a license ID. **Each user must have their own license ID**. They may not be shared, even within a lab group. Attempting to - **CryoSPARC License ID**: To start CryoSPARC you will need to have a license ID. **Each user must have their own license ID**. They may not be shared, even within a lab group. Attempting to
share a license ID will cause jobs to fail. If you have multiple CryoSPARC sessions running at the same time, then you will need different license ID for each session. CryoSPARC license IDs are free for non-profit academic use and can be obtained from [here](https://cryosparc.com/download). share a license ID will cause jobs to fail. If you have multiple CryoSPARC sessions running at the same time, then you will need different license ID for each session. CryoSPARC license IDs are free for non-profit academic use and can be obtained from [here](https://cryosparc.com/download).
- **Start a new CryoSPARC session?**: The default location of the CryoSPARC database is `$WORK/cryosparc`. If you select this checkbox, all the existing database and configuration files in the CryoSPARC session location will be erased. *Please select this checkbox only if needed.* - **Start a new CryoSPARC session?**: The default location of the CryoSPARC database is `$NRDSTOR/cryosparc`. If you select this checkbox, all the existing database and configuration files in the CryoSPARC session location will be erased. *Please select this checkbox only if needed.*
- **Number of cores**: This parameter defines the cores needed for the CryoSPARC _"master"_ process. The _"master"_ process does not require many resources, so requesting <ins>1-2 cores</ins> should be sufficient. - **Number of cores**: This parameter defines the cores needed for the CryoSPARC _"master"_ process. The _"master"_ process does not require many resources, so requesting <ins>1-2 cores</ins> should be sufficient.
- **Running time in hours**: This parameter defines the runtime of the CryoSPARC _"master"_ process. The _"master"_ process should be running while the CryoSPARC SLURM jobs are running. If you are not sure how long the submitted jobs will be running for, please <ins>select the maximum runtime of 168 hours (7 days)</ins> to avoid any issues if the CryoSPARC _"master"_ process finishes before the submitted jobs. While the CryoSPARC _"master"_ process is running and the GPU CryoSPARC jobs are submitted to the cluster, you don't need to have the CryoSPARC OOD App open. - **Running time in hours**: This parameter defines the runtime of the CryoSPARC _"master"_ process. The _"master"_ process should be running while the CryoSPARC SLURM jobs are running. If you are not sure how long the submitted jobs will be running for, please <ins>select the maximum runtime of 168 hours (7 days)</ins> to avoid any issues if the CryoSPARC _"master"_ process finishes before the submitted jobs. While the CryoSPARC _"master"_ process is running and the GPU CryoSPARC jobs are submitted to the cluster, you don't need to have the CryoSPARC OOD App open.
- **Requested RAM in GBs**: This parameter defines the RAM memory needed for the CryoSPARC _"master"_ process. The _"master"_ process does not require many resources, so requesting <ins>8GBs</ins> for example should be sufficient. - **Requested RAM in GBs**: This parameter defines the RAM memory needed for the CryoSPARC _"master"_ process. The _"master"_ process does not require many resources, so requesting <ins>8GBs</ins> for example should be sufficient.
- **Partition selection**: This parameter defines the partition used for the CryoSPARC _"master"_ process. The _"master"_ process *does not require GPUs*, so you can you use a CPU partition, such as `batch`, or any other leased partition you have access to. - **Partition selection**: This parameter defines the partition used for the CryoSPARC _"master"_ process. The _"master"_ process *does not require GPUs*, so you can you use a CPU partition, such as `batch`, or any other leased partition you have access to.
- In addition to the basic fields, there are two advanced settings you can set under the _Show advanced settings..._ checkbox in the CryoSPARC OOD Form: - In addition to the basic fields, there are two advanced settings you can set under the _Show advanced settings..._ checkbox in the CryoSPARC OOD Form:
- **CryoSPARC session path**: The default location of the CryoSPARC database and configuration files is `$WORK/cryosparc`. You can change this location using the *Select Path* button. You can select any of the [available file systems on Swan]({{< relref "../handling_data/data_storage/" >}}) aside from the `$HOME` filesystem. _We do not recommend using `$HOME` for the session folder_. Please note that each file system has its own advantages and disadvantages. **Note this location is \*not\* the same as the project path(s) where projects are stored.** This location is used for CryoSPARC's internal database and configuration. Do not change this value unless you are absolutely sure of the ramifications. - **CryoSPARC session path**: The default location of the CryoSPARC database and configuration files is `$NRDSTOR/cryosparc`. You can change this location using the *Select Path* button. You can select any of the [available file systems on Swan]({{< relref "../handling_data/data_storage/" >}}) aside from the `$HOME` filesystem. _We do not recommend using `$HOME` for the session folder_. Please note that each file system has its own advantages and disadvantages. **Note this location is \*not\* the same as the project path(s) where projects are stored.** This location is used for CryoSPARC's internal database and configuration. Do not change this value unless you are absolutely sure of the ramifications.
- **Specify partition for CryoSPARC GPU jobs**: This parameter defines the GPU partition used for the GPU CryoSPARC jobs submitted to the cluster via SLURM through the interactive Desktop. If not specified, by default, these jobs are submitted to the general `gpu` partition. If you have access to a priority access partition with GPU resources, you may specify that partition here to reduce queue time. - **Specify partition for CryoSPARC GPU jobs**: This parameter defines the GPU partition used for the GPU CryoSPARC jobs submitted to the cluster via SLURM through the interactive Desktop. If not specified, by default, these jobs are submitted to the general `gpu` partition. If you have access to a priority access partition with GPU resources, you may specify that partition here to reduce queue time.
- **Highmem factor**: This parameter defines the multiplicative factor used to increase the memory (RAM) request for the `highmem` cluster lane (described below). That is, the preset memory values within CryoSPARC for each job type will be multiplied by this factor. Certain input sets combined with particular options may require more memory than the preset CryoSPARC values, and will otherwise fail. Use this value in combination with using the `highmem` cluster lane to increase the requested memory on specified jobs to allow them to complete successfully. The valid range are integer values from 1 to 10. - **Highmem factor**: This parameter defines the multiplicative factor used to increase the memory (RAM) request for the `highmem` cluster lane (described below). That is, the preset memory values within CryoSPARC for each job type will be multiplied by this factor. Certain input sets combined with particular options may require more memory than the preset CryoSPARC values, and will otherwise fail. Use this value in combination with using the `highmem` cluster lane to increase the requested memory on specified jobs to allow them to complete successfully. The valid range are integer values from 1 to 10.
- **Batch job maximum runtime**: This parameter controls the maximum runtime of the batch jobs CryoSPARC submits to SLURM via the cluster lanes. This value must be greater than the longest CryoSPARC task submitted to the cluster lanes else those tasks will be killed by the scheduler. Smaller values will generally decrease queue time, so if you are confident all tasks for your particular datasets will complete sooner, you may lower this value to improve job throughput and decrease queue time. - **Batch job maximum runtime**: This parameter controls the maximum runtime of the batch jobs CryoSPARC submits to SLURM via the cluster lanes. This value must be greater than the longest CryoSPARC task submitted to the cluster lanes else those tasks will be killed by the scheduler. Smaller values will generally decrease queue time, so if you are confident all tasks for your particular datasets will complete sooner, you may lower this value to improve job throughput and decrease queue time.
- **Allow multiple CryoSPARC masters**: By default only one instance of the CryoSPARC app can be run at a time, as the app launches the CryoSPARC master process. Selecting this box overrides this limitation and allows multiple copies of the app (i.e. master) to run. This can be useful for testing a newer CryoSPARC version side by side with an existing version for example. **In order to avoid data corruption and loss, you must use a distinct session path for each master. Multiple masters cannot share a session path!** _HCC strongly recommends against using this option unless you know exactly what you're doing and have a secure backup of both your project(s) and CryoSPARC database(s)._
{{< figure src="/images/OOD_CS_advanced_settings.png" width="55%" class="img-border">}} {{< figure src="/images/OOD_CS_advanced_settings.png" width="55%" class="img-border">}}
...@@ -78,11 +77,13 @@ Once this is done, the set location will apply to all future newly created jobs. ...@@ -78,11 +77,13 @@ Once this is done, the set location will apply to all future newly created jobs.
#### CryoSPARC database #### CryoSPARC database
The default location of the CryoSPARC database is `$WORK/cryosparc`. The default location of the CryoSPARC database is `$NRDSTOR/cryosparc`.
- If you need to move this database elsewhere, please change the **CryoSPARC session path** using **Select Path** under the _Show advanced settings..._ checkbox. - If you need to move this database elsewhere, please change the **CryoSPARC session path** using **Select Path** under the _Show advanced settings..._ checkbox.
- If you want to store the CryoSPARC session folder (database and configuration files) elsewhere, please change the **CryoSPARC session path** before you start the CryoSPARC session. - If you want to store the CryoSPARC session folder (database and configuration files) elsewhere, please change the **CryoSPARC session path** before you start the CryoSPARC session.
When the CryoSPARC OOD App is properly shut down, backup of the database is saved in `$NRDSTOR/.cryosparc/db_backups`.
#### CryoSPARC example #### CryoSPARC example
If you want to test the CryoSPARC OOD App, you can use the [CryoSPARC Introductory Tutorial](https://guide.cryosparc.com/processing-data/get-started-with-cryosparc-introductory-tutorial). If you want to test the CryoSPARC OOD App, you can use the [CryoSPARC Introductory Tutorial](https://guide.cryosparc.com/processing-data/get-started-with-cryosparc-introductory-tutorial).
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment