Skip to content
Snippets Groups Projects
Commit ae06dfa5 authored by Natasha Pavlovikj's avatar Natasha Pavlovikj
Browse files

Update documentation

parent 8963e94a
Branches
No related tags found
2 merge requests!197Add page for Good HCC practices,!196Add Good HCC Practices page
...@@ -9,14 +9,16 @@ Sometimes, some users' activities may negatively impact the clusters and the use ...@@ -9,14 +9,16 @@ Sometimes, some users' activities may negatively impact the clusters and the use
To avoid this, we provide the following guidelines for good HCC practices. To avoid this, we provide the following guidelines for good HCC practices.
## Login Node ## Login Node
* **Do not run jobs on the login node.** The login node is shared among all users and it * **Be kind to the login node.** The login node is shared among all users and it
should be used only for light tasks, such as moving and editing files, compiling programs, should be used only for light tasks, such as moving and editing files, compiling programs,
and submitting and monitoring jobs. If a researcher runs a computationally intensive task and submitting and monitoring jobs. If a researcher runs a computationally intensive task
on the login node, that will negatively impact the performance for other users. For any CPU on the login node, that will negatively impact the performance for other users. Moreover, the
or memory intensive operations, such as testing and running applications, one should use an resources on the login node are limited, so any lengthy or intensive task will highly likely
exceed these resources and be terminated. For any CPU or memory intensive
operations, such as testing and running applications, one should use an
[interactive session]({{< relref "creating_an_interactive_job" >}}), or [interactive session]({{< relref "creating_an_interactive_job" >}}), or
[submit a job to the batch queue]({{< relref "submitting_jobs" >}}). [submit a job to the batch queue]({{< relref "submitting_jobs" >}}).
* **Do not launch multiple simultaneous processes on the login node.** This may include using * **Avoid launching multiple simultaneous processes on the login node.** This may include using
lots of threads for compiling applications, or checking the job status multiple times a minute. lots of threads for compiling applications, or checking the job status multiple times a minute.
## File Systems ## File Systems
...@@ -24,8 +26,7 @@ lots of threads for compiling applications, or checking the job status multiple ...@@ -24,8 +26,7 @@ lots of threads for compiling applications, or checking the job status multiple
file system local to each worker nodes**. The */scratch* directories are unique per job, and file system local to each worker nodes**. The */scratch* directories are unique per job, and
are deleted when the job finishes. Thus, the last step of the batch script should copy the are deleted when the job finishes. Thus, the last step of the batch script should copy the
needed output files from */scratch* to either */work* or */common*. Please see the needed output files from */scratch* to either */work* or */common*. Please see the
[Running BLAST Alignment]({{< relref "running_blast_alignment" >}}) page for an example. [Running BLAST Alignment]({{< relref "running_blast_alignment" >}}) page for an example.
Currently, we do not have quota on the */scratch* file system.
* */work* has two quotas - one for **file count** and the second one for **disk space**. * */work* has two quotas - one for **file count** and the second one for **disk space**.
Reaching these quotas may additionally stress the file system. Therefore, please make sure you Reaching these quotas may additionally stress the file system. Therefore, please make sure you
monitor these quotas regularly, and delete all the files that are not needed or copy them to more permanent location. monitor these quotas regularly, and delete all the files that are not needed or copy them to more permanent location.
...@@ -37,12 +38,12 @@ Instead, consider reading and writing large blocks of data in memory over time, ...@@ -37,12 +38,12 @@ Instead, consider reading and writing large blocks of data in memory over time,
utilizing more advanced parallel I/O libraries, such as *parallel hdf5* and *parallel netcdf*. utilizing more advanced parallel I/O libraries, such as *parallel hdf5* and *parallel netcdf*.
## Internal and External Networks ## Internal and External Networks
* **Transferring many files to/from/within the cluster can harm the file system.** If you are * **Use archives to transfer large number of files.** If you are performing file transfer of
performing file transfer of many small files, please put these files in an archive file format, many small files, please put these files in an archive file format, such that the many files are
such that the many files are replaced by a single file. We recommend using *zip* as the archive format replaced by a single file. We recommend using *zip* as the archive format as zip files keep an
as zip files keep an index of the files. Moreover, zip files can be quickly indexed by the various zip tools, and allow index of the files. Moreover, zip files can be quickly indexed by the various zip tools, and allow
extraction of all files or a subset of files. The *tar* formats are stream oriented, and a full decompression is required extraction of all files or a subset of files. The *tar* formats are stream oriented, and a full decompression
for the tools to know if the requested files have been found. is required for the tools to know if the requested files have been found.
## Running Applications ## Running Applications
* **Before you request multiple nodes and cores in your submit script, make sure that the application you are * **Before you request multiple nodes and cores in your submit script, make sure that the application you are
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment