diff --git a/content/good_hcc_practices/_index.md b/content/good_hcc_practices/_index.md index 1ee1df04e9ecae8889490e1acf6a88cb97e4a5ec..616a5430c3834d46b9ce81329d7466831743dfde 100644 --- a/content/good_hcc_practices/_index.md +++ b/content/good_hcc_practices/_index.md @@ -9,14 +9,16 @@ Sometimes, some users' activities may negatively impact the clusters and the use To avoid this, we provide the following guidelines for good HCC practices. ## Login Node -* **Do not run jobs on the login node.** The login node is shared among all users and it +* **Be kind to the login node.** The login node is shared among all users and it should be used only for light tasks, such as moving and editing files, compiling programs, and submitting and monitoring jobs. If a researcher runs a computationally intensive task -on the login node, that will negatively impact the performance for other users. For any CPU -or memory intensive operations, such as testing and running applications, one should use an +on the login node, that will negatively impact the performance for other users. Moreover, the +resources on the login node are limited, so any lengthy or intensive task will highly likely +exceed these resources and be terminated. For any CPU or memory intensive +operations, such as testing and running applications, one should use an [interactive session]({{< relref "creating_an_interactive_job" >}}), or [submit a job to the batch queue]({{< relref "submitting_jobs" >}}). -* **Do not launch multiple simultaneous processes on the login node.** This may include using +* **Avoid launching multiple simultaneous processes on the login node.** This may include using lots of threads for compiling applications, or checking the job status multiple times a minute. ## File Systems @@ -24,8 +26,7 @@ lots of threads for compiling applications, or checking the job status multiple file system local to each worker nodes**. The */scratch* directories are unique per job, and are deleted when the job finishes. Thus, the last step of the batch script should copy the needed output files from */scratch* to either */work* or */common*. Please see the -[Running BLAST Alignment]({{< relref "running_blast_alignment" >}}) page for an example. -Currently, we do not have quota on the */scratch* file system. +[Running BLAST Alignment]({{< relref "running_blast_alignment" >}}) page for an example. * */work* has two quotas - one for **file count** and the second one for **disk space**. Reaching these quotas may additionally stress the file system. Therefore, please make sure you monitor these quotas regularly, and delete all the files that are not needed or copy them to more permanent location. @@ -37,12 +38,12 @@ Instead, consider reading and writing large blocks of data in memory over time, utilizing more advanced parallel I/O libraries, such as *parallel hdf5* and *parallel netcdf*. ## Internal and External Networks -* **Transferring many files to/from/within the cluster can harm the file system.** If you are -performing file transfer of many small files, please put these files in an archive file format, -such that the many files are replaced by a single file. We recommend using *zip* as the archive format -as zip files keep an index of the files. Moreover, zip files can be quickly indexed by the various zip tools, and allow -extraction of all files or a subset of files. The *tar* formats are stream oriented, and a full decompression is required -for the tools to know if the requested files have been found. +* **Use archives to transfer large number of files.** If you are performing file transfer of +many small files, please put these files in an archive file format, such that the many files are +replaced by a single file. We recommend using *zip* as the archive format as zip files keep an +index of the files. Moreover, zip files can be quickly indexed by the various zip tools, and allow +extraction of all files or a subset of files. The *tar* formats are stream oriented, and a full decompression +is required for the tools to know if the requested files have been found. ## Running Applications * **Before you request multiple nodes and cores in your submit script, make sure that the application you are