Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
H
HCC docs
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Deploy
Releases
Monitor
Service Desk
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Holland Computing Center
HCC docs
Commits
ae06dfa5
Commit
ae06dfa5
authored
5 years ago
by
Natasha Pavlovikj
Browse files
Options
Downloads
Patches
Plain Diff
Update documentation
parent
8963e94a
No related branches found
No related tags found
2 merge requests
!197
Add page for Good HCC practices
,
!196
Add Good HCC Practices page
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
content/good_hcc_practices/_index.md
+13
-12
13 additions, 12 deletions
content/good_hcc_practices/_index.md
with
13 additions
and
12 deletions
content/good_hcc_practices/_index.md
+
13
−
12
View file @
ae06dfa5
...
...
@@ -9,14 +9,16 @@ Sometimes, some users' activities may negatively impact the clusters and the use
To avoid this, we provide the following guidelines for good HCC practices.
## Login Node
*
**
Do not run jobs on
the login node.**
The login node is shared among all users and it
*
**
Be kind to
the login node.**
The login node is shared among all users and it
should be used only for light tasks, such as moving and editing files, compiling programs,
and submitting and monitoring jobs. If a researcher runs a computationally intensive task
on the login node, that will negatively impact the performance for other users. For any CPU
or memory intensive operations, such as testing and running applications, one should use an
on the login node, that will negatively impact the performance for other users. Moreover, the
resources on the login node are limited, so any lengthy or intensive task will highly likely
exceed these resources and be terminated. For any CPU or memory intensive
operations, such as testing and running applications, one should use an
[
interactive session
](
{{
<
relref
"
creating_an_interactive_job
"
>
}}), or
[
submit a job to the batch queue
](
{{
<
relref
"
submitting_jobs
"
>
}}).
*
**
Do not
launch multiple simultaneous processes on the login node.**
This may include using
*
**
Avoid
launch
ing
multiple simultaneous processes on the login node.**
This may include using
lots of threads for compiling applications, or checking the job status multiple times a minute.
## File Systems
...
...
@@ -24,8 +26,7 @@ lots of threads for compiling applications, or checking the job status multiple
file system local to each worker nodes
**
. The
*/scratch*
directories are unique per job, and
are deleted when the job finishes. Thus, the last step of the batch script should copy the
needed output files from
*/scratch*
to either
*/work*
or
*/common*
. Please see the
[
Running BLAST Alignment
](
{{
<
relref
"
running_blast_alignment
"
>
}}) page for an example.
Currently, we do not have quota on the
*/scratch*
file system.
[
Running BLAST Alignment
](
{{
<
relref
"
running_blast_alignment
"
>
}}) page for an example.
*
*/work*
has two quotas - one for
**file count**
and the second one for
**disk space**
.
Reaching these quotas may additionally stress the file system. Therefore, please make sure you
monitor these quotas regularly, and delete all the files that are not needed or copy them to more permanent location.
...
...
@@ -37,12 +38,12 @@ Instead, consider reading and writing large blocks of data in memory over time,
utilizing more advanced parallel I/O libraries, such as
*parallel hdf5*
and
*parallel netcdf*
.
## Internal and External Networks
*
**
Transferring many files to/from/within the cluster can harm the file system.**
If you are
performing file transfer of
many small files, please put these files in an archive file format,
such that the many files are
replaced by a single file. We recommend using
*zip*
as the archive format
as zip files keep an
index of the files. Moreover, zip files can be quickly indexed by the various zip tools, and allow
extraction of all files or a subset of files. The
*tar*
formats are stream oriented, and a full decompression
is required
for the tools to know if the requested files have been found.
*
**
Use archives to transfer large number of files.**
If you are performing file transfer of
many small files, please put these files in an archive file format,
such that the many files are
replaced by a single file. We recommend using
*zip*
as the archive format
as zip files keep an
index of the files. Moreover, zip files can be quickly indexed by the various zip tools, and allow
extraction of all files or a subset of files. The
*tar*
formats are stream oriented, and a full decompression
is required
for the tools to know if the requested files have been found.
## Running Applications
*
**
Before you request multiple nodes and cores in your submit script, make sure that the application you are
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment