Skip to content
Snippets Groups Projects
Commit ae06dfa5 authored by Natasha Pavlovikj's avatar Natasha Pavlovikj
Browse files

Update documentation

parent 8963e94a
No related branches found
No related tags found
2 merge requests!197Add page for Good HCC practices,!196Add Good HCC Practices page
......@@ -9,14 +9,16 @@ Sometimes, some users' activities may negatively impact the clusters and the use
To avoid this, we provide the following guidelines for good HCC practices.
## Login Node
* **Do not run jobs on the login node.** The login node is shared among all users and it
* **Be kind to the login node.** The login node is shared among all users and it
should be used only for light tasks, such as moving and editing files, compiling programs,
and submitting and monitoring jobs. If a researcher runs a computationally intensive task
on the login node, that will negatively impact the performance for other users. For any CPU
or memory intensive operations, such as testing and running applications, one should use an
on the login node, that will negatively impact the performance for other users. Moreover, the
resources on the login node are limited, so any lengthy or intensive task will highly likely
exceed these resources and be terminated. For any CPU or memory intensive
operations, such as testing and running applications, one should use an
[interactive session]({{< relref "creating_an_interactive_job" >}}), or
[submit a job to the batch queue]({{< relref "submitting_jobs" >}}).
* **Do not launch multiple simultaneous processes on the login node.** This may include using
* **Avoid launching multiple simultaneous processes on the login node.** This may include using
lots of threads for compiling applications, or checking the job status multiple times a minute.
## File Systems
......@@ -24,8 +26,7 @@ lots of threads for compiling applications, or checking the job status multiple
file system local to each worker nodes**. The */scratch* directories are unique per job, and
are deleted when the job finishes. Thus, the last step of the batch script should copy the
needed output files from */scratch* to either */work* or */common*. Please see the
[Running BLAST Alignment]({{< relref "running_blast_alignment" >}}) page for an example.
Currently, we do not have quota on the */scratch* file system.
[Running BLAST Alignment]({{< relref "running_blast_alignment" >}}) page for an example.
* */work* has two quotas - one for **file count** and the second one for **disk space**.
Reaching these quotas may additionally stress the file system. Therefore, please make sure you
monitor these quotas regularly, and delete all the files that are not needed or copy them to more permanent location.
......@@ -37,12 +38,12 @@ Instead, consider reading and writing large blocks of data in memory over time,
utilizing more advanced parallel I/O libraries, such as *parallel hdf5* and *parallel netcdf*.
## Internal and External Networks
* **Transferring many files to/from/within the cluster can harm the file system.** If you are
performing file transfer of many small files, please put these files in an archive file format,
such that the many files are replaced by a single file. We recommend using *zip* as the archive format
as zip files keep an index of the files. Moreover, zip files can be quickly indexed by the various zip tools, and allow
extraction of all files or a subset of files. The *tar* formats are stream oriented, and a full decompression is required
for the tools to know if the requested files have been found.
* **Use archives to transfer large number of files.** If you are performing file transfer of
many small files, please put these files in an archive file format, such that the many files are
replaced by a single file. We recommend using *zip* as the archive format as zip files keep an
index of the files. Moreover, zip files can be quickly indexed by the various zip tools, and allow
extraction of all files or a subset of files. The *tar* formats are stream oriented, and a full decompression
is required for the tools to know if the requested files have been found.
## Running Applications
* **Before you request multiple nodes and cores in your submit script, make sure that the application you are
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment