diff --git a/content/FAQ/_index.md b/content/FAQ/_index.md index deb1d796ce30b00729c00555554db51c79377fdd..a130cf58ab3648ec387f8128b0711cb401f86973 100644 --- a/content/FAQ/_index.md +++ b/content/FAQ/_index.md @@ -24,11 +24,8 @@ Shell, and it allows you to securely connect to a remote computer and operate it just like you would a personal machine. Depending on your operating system, you may need to install software to -make this connection. Check out on Quick Start Guides for information on -how to install the necessary software for your operating system - -- [For Mac/Linux Users]({{< relref "for_maclinux_users" >}}) -- [For Windows Users]({{< relref "for_windows_users" >}}) +make this connection. Check out our documentation on [Connecting to HCC Clusters] +({{< relref "../connecting/" >}}). #### How do I change my password? @@ -107,7 +104,7 @@ you select. Sometimes it can help to speak with someone else who has used the software before to see if they can give you an idea of what has worked for them. -But ultimately, it comes down to trial and error; try different +Ultimately, it comes down to trial and error; try different combinations and see what works and what doesn’t. Good practice is to check the output and utilization of each job you run. This will help you determine what parameters you will need in the future. diff --git a/content/_index.md b/content/_index.md index 29338843aaa9e52eeb6830bda7d8b86a0ece9513..a2e9676fecb2a0fac4b2d8845d6bf396c374b48a 100644 --- a/content/_index.md +++ b/content/_index.md @@ -7,7 +7,6 @@ weight = "1" HCC Documentation ============================ - The Holland Computing Center supports a diverse collection of research computing hardware. Anyone in the University of Nebraska system is welcome to apply for an account on HCC machines. @@ -19,7 +18,15 @@ existing resource, acquiring 'priority access'. Finally, several machines are available via Condor for opportunistic use. This will allow users almost immediate access, but the job is subject to preemption. -#### [New Users Sign Up](http://hcc.unl.edu/new-user-request) +## Getting Started + +To begin using HCC resources: + +1. [Set up your HCC account]({{< relref "accounts/" >}}) +2. [Connect to HCC Clusters]({{< relref "connecting/" >}}) +3. [Transfer data to HCC Clusters]({{< relref "handling_data/" >}}) +4. [Check software availability]({{< relref "applications/" >}}) +5. [Submit jobs on HCC Clusters]({{< relref "submitting_jobs/" >}}) Which Cluster to Use? --------------------- @@ -34,31 +41,6 @@ Rhino has 4 AMD Interlagos CPUs (64 cores) per node, with either 192GB or 256GB node in the default partition. For extremely large RAM needs, there is also a 'highmem' partition with 2 x 512GB and 2 x 1TB nodes. -User Login ----------- - -For Windows users, please refer to this link [For Windows Users]({{< relref "for_windows_users" >}}). -For Mac or Linux users, please refer to this link [For Mac/Linux Users]({{< relref "for_maclinux_users">}}). - -**Logging into Crane or Rhino** - -{{< highlight bash >}} -ssh <username>@crane.unl.edu -{{< /highlight >}} - -or - -{{< highlight bash >}} -ssh <username>@rhino.unl.edu -{{< /highlight >}} - -Duo Security ------------- - -Duo two-factor authentication is **required** for access to HCC -resources. Registration and usage of Duo security can be found in this -section: [Setting up and using Duo]({{< relref "/accounts/setting_up_and_using_duo">}}) - **Important Notes** - The Crane and Rhino clusters are separate. But, they are diff --git a/content/applications/app_specific/fortran_c_on_hcc.md b/content/applications/app_specific/fortran_c_on_hcc.md index beec01f8505f52c39bb67e9af09aa8fc338a7ba2..3c41ebfc36f5be1d4428f2435e23c5da96971faa 100644 --- a/content/applications/app_specific/fortran_c_on_hcc.md +++ b/content/applications/app_specific/fortran_c_on_hcc.md @@ -10,8 +10,8 @@ downloaded from [serial_dir.zip](/attachments/serial_dir.zip). #### Login to a HCC Cluster -Log in to a HCC cluster through PuTTY ([For Windows Users]({{< relref "/connecting/for_windows_users">}})) or Terminal ([For Mac/Linux -Users]({{< relref "/connecting/for_maclinux_users">}})) and make a subdirectory called `serial_dir` under the `$WORK` directory. +Connect to a HCC cluster]({{< relref "../../connecting/" >}}) and make a subdirectory +called `serial_dir` under your `$WORK` directory. {{< highlight bash >}} $ cd $WORK diff --git a/content/applications/app_specific/mpi_jobs_on_hcc.md b/content/applications/app_specific/mpi_jobs_on_hcc.md index bfb9abd7f86588f33dd98b4ea2ef7e10c6d147c4..182012219d5f3e665d097ba3f366238114c06878 100644 --- a/content/applications/app_specific/mpi_jobs_on_hcc.md +++ b/content/applications/app_specific/mpi_jobs_on_hcc.md @@ -10,8 +10,8 @@ scripts can be downloaded from [mpi_dir.zip](/attachments/mpi_dir.zip). #### Login to a HCC Cluster -Log in to a HCC cluster through PuTTY ([For Windows Users]({{< relref "/connecting/for_windows_users">}})) or Terminal ([For Mac/Linux -Users]({{< relref "/connecting/for_maclinux_users">}})) and make a subdirectory called `mpi_dir` under the `$WORK` directory. +Connect to a HCC cluster]({{< relref "../../connecting/" >}}) and make a subdirectory +and make a subdirectory called `mpi_dir` under your `$WORK` directory. {{< highlight bash >}} $ cd $WORK diff --git a/content/connecting/for_maclinux_users.md b/content/connecting/for_maclinux_users.md deleted file mode 100644 index 2e1a2243604979607b334b5953fc861cdfa0f70d..0000000000000000000000000000000000000000 --- a/content/connecting/for_maclinux_users.md +++ /dev/null @@ -1,145 +0,0 @@ -+++ -title = "For Mac/Linux Users" -description = "Quickstart Guide for Mac/Linux Users" -weight = "22" -+++ - -##### Use of Duo two-factor authentication is **required** to access HCC resources. -##### Please see [Setting up and Using Duo]({{< relref "setting_up_and_using_duo" >}}). ---- -- [Access to HCC Supercomputers] (#access-to-hcc-supercomputers) -- [File Transferring with HCC Supercomputers] (#file-transfering) - - [SCP] (#using-the-scp-command) - - [CyberDuck] (#using-cyberduck) -- [Mac Tutorial Video](#mac-tutorial-video) -- [Linux Tutorial Video](#linux-tutorial-video) - -This quick start will help you configure your personal computer to work -with the HCC supercomputers. - -If you are running Windows, please use the quickstart [For Windows -Users]({{< relref "/connecting/for_windows_users" >}}). - -Access to HCC Supercomputers -------------------------------- - -For Mac/Linux users, use the system program Terminal to access to the -HCC supercomputers. In the Terminal prompt, -type `ssh <username>@crane.unl.edu` and the corresponding password -to get access to the HCC cluster **Crane**. Note that <username> -should be replaced by your HCC account username. If you do not have a -HCC account, please contact a HCC specialist -({{< icon name="envelope" >}}[hcc-support@unl.edu] (mailto:hcc-support@unl.edu)) -or go to https://hcc.unl.edu/newusers. -To use the **Rhino** cluster, replace crane.unl.edu with with rhino.unl.edu. - -{{< highlight bash >}} -$ ssh <username>@crane.unl.edu -$ <password> -{{< /highlight >}} - -File Transferring with HCC Supercomputers ------------------------------------------ - -### Using the SCP command - -For Mac/Linux users, file transferring between your personal computer -and the HCC supercomputers can be achieved through the command `scp`. -Here we use **Crane** for example. **The following commands should be -executed from your computer. ** - -**Uploading from local to remote** - -{{< highlight bash >}} -$ scp -r ./<folder name> <username>@crane.unl.edu:/work/<group name>/<username> -{{< /highlight >}} - -The above command line transfers a folder from the current directory -(`./`) of the your computer to the `$WORK` directory of the HCC -supercomputer, Crane. Note that you need to replace `<group name>` -and `<username>` with your HCC group name and username. - -**Downloading from remote to local** - -{{< highlight bash >}} -$ scp -r <username>@crane.unl.edu:/work/<group name>/<username>/<folder name> ./ -{{< /highlight >}} - -The above command line transfers a folder from the `$WORK` directory of -the HCC supercomputer, Crane, to the current directory (`./`) of the -your computer. - -### Using Cyberduck ---------------- - -If you wish to use a GUI, be aware that not all programs will function -correctly with Duo two-factor authentication. Mac users are recommended -to use [Cyberduck](https://cyberduck.io). It is compatible with Duo, but a -few settings need to be changed. - -Under **Preferences - General**, change the default protocol to SFTP: - -{{< figure src="/images/7274497.png" height="450" >}} - -Under **Preferences - Transfers**, reuse the browser connection for file -transfers. This will avoid the need to reenter your password for every -file transfer: - -{{< figure src="/images/7274498.png" height="450" >}} - -Finally, under **Preferences - SFTP**, set the file transfer method to -SCP: - -{{< figure src="/images/7274499.png" height="450" >}} - -To add an HCC machine, in the bookmarks pane click the "+" icon: - -{{< figure src="/images/7274500.png" height="450" >}} - -Ensure the type of connection is SFTP. Enter the hostname of the machine -you wish to connect to (crane.unl.edu, rhino.unl.edu) in the **Server** -field, and your HCC username in the **Username** field. The -**Nickname** field is arbitrary, so enter whatever you prefer. - -{{< figure src="/images/7274501.png" height="450" >}} - -After you add the bookmark, double-click it to connect. - -{{< figure src="/images/7274505.png" height="450" >}} - -Enter your HCC username and password in the dialog box that will appear -and click *Login*. - -{{< figure src="/images/7274508.png" height="450" >}} - -A second login dialogue will now appear. Notice the text has changed to -say Duo two-factor. - -{{< figure src="/images/7274510.png" height="450" >}} - -Clear the **Password** field in the dialogue. If you are using the Duo -Mobile app, enter '1' to have a push notification send to your phone or -tablet. If you are using a Yubikey, ensure the cursor is active in the -**Password** field, and press the button on the Yubikey. - -{{< figure src="/images/7274509.png" height="450" >}} - -The login should complete and you can simply drag and drop files to or -from the window. - -{{< figure src="/images/7274511.png" height="450" >}} - -If you run into issues with two-factor authentication, try the command below for a quick fix: -{{< highlight bash >}} -$ rm -rf ~/Library/ApplicationSupport/Cyberduck -{{< /highlight >}} - -Mac Tutorial Video ------------------- - -{{< youtube ulfcmRGfqxU >}} - -Linux Tutorial Video --------------------- - -{{< youtube K0i3swpwtdc >}} diff --git a/content/connecting/for_windows_users.md b/content/connecting/for_windows_users.md deleted file mode 100644 index 1e913b5de61d36a8c58a580dc5ce7507821c5896..0000000000000000000000000000000000000000 --- a/content/connecting/for_windows_users.md +++ /dev/null @@ -1,195 +0,0 @@ -+++ -title = "For Windows Users" -description = "Quickstart for Windows Users" -weight = "20" -+++ - -##### Use of Duo two-factor authentication is **required** to access HCC resources. -##### Please see [Setting up and Using Duo]({{< relref "setting_up_and_using_duo" >}}). ---- - - -- [Access to HCC Supercomputers] (#access-to-hcc-supercomputers) - - [For Windows 10 Users](#windows-10) - - [For Windows 7 and 8.1 Users](#windows-7-and-8-1) -- [File Transferring with HCC Supercomputers] (#file-transferring) - - [SCP - Command Line] (#scp) - - [WinSCP - GUI] (#winscp) -- [Tutorial Video](#tutorial-video) - -Access to HCC Supercomputers -------------------------------- - -{{% notice info %}} -If you are on a Mac, please use the quickstart for [For Mac/Linux -Users]({{< relref "/connecting/for_maclinux_users" >}}). -{{% /notice %}} - - -### Windows 10 --------------- -For Windows 10 users, use the Command Prompt, accessed by entering `cmd` in the start menu, to access to the -HCC supercomputers. In the Command Prompt, -type `ssh <username>@crane.unl.edu` and the corresponding password -to get access to the HCC cluster **Crane**. Note that <username> -should be replaced by your HCC account username. If you do not have a -HCC account, please contact a HCC specialist -({{< icon name="envelope" >}}[hcc-support@unl.edu] (mailto:hcc-support@unl.edu)) -or go to http://hcc.unl.edu/newusers. -To use the **Rhino** cluster, replace crane.unl.edu with rhino.unl.edu. - -{{< highlight bash >}} -C:\> ssh <username>@crane.unl.edu -C:\> <password> -{{< /highlight >}} - - - -### Windows 7 and 8.1 --------------- -This quick start will help you configure your personal computer to work -with the HCC supercomputers. Here we use the two third party application -**PuTTY** and **WinSCP** for demonstration. - -PuTTY: https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html -or [Direct Link](https://the.earth.li/~sgtatham/putty/latest/w32/putty.exe) - - -Here we use the HCC cluster **Crane** for demonstration. To use the -**Rhino** or cluster, replace `crane.unl.edu` with `rhino.unl.edu`. - -1. On the first screen, type `crane.unl.edu` for Host Name, then click - **Open**. - {{< figure src="/images/3178523.png" height="450" >}} -2. On the second screen, click on **Yes**. - {{< figure src="/images/3178524.png" height="300" >}} -3. On the third screen, enter your HCC account **username**. If you do - not have a HCC account, please contact an HCC specialist - ({{< icon name="envelope" >}}[hcc-support@unl.edu] (mailto:hcc-support@unl.edu)) - or go to http://hcc.unl.edu/newusers. - - {{% notice info %}}Replace `jingchao` with your username.{{% /notice %}} - - - {{< figure src="/images/8127261.png" height="450" >}} - -4. On the next screen, enter your HCC account **password**. - - {{% notice info %}}**Note that PuTTY will not show the characters as you type for security reasons.**{{% /notice %}} - - {{< figure src="/images/8127262.png" height="450" >}} - -5. After you input the correct - password, you will be asked to choose a Duo authentication - method. - -6. If you have a Yubikey set up by HCC, please hold the Yubikey for \~1 - second. Then you will be brought to your home directory similar as - below. - - {{< figure src="/images/8127266.png" height="450" >}} - -7. If you set up Duo via a smart phone, please type "1" in your - terminal and press "Enter". (Duo-Push is the most cost-effective way - for Duo authentication, we recommend all user use this option if - that is applicable.) - -8. Check your smart phone for Duo login request. Press "Approve" if you - can verify the request. If you find any Duo login request that is - not initiated by yourself, deny it and report this incident - immediately to {{< icon name="envelope" >}}[hcc-support@unl.edu] (mailto:hcc-support@unl.edu). - - {{< figure src="/images/8127263.png" height="450" >}} - -9. After you approve the Duo login request, you will be brought to your - home directory similar as below. - - {{< figure src="/images/8127264.png" height="450" >}} - - -File Transferring with HCC Supercomputers ------------------------------------------ - -{{% notice info%}} -For best results when transfering data to and from the clusters, refer to [Handling Data]({{< ref "/handling_data" >}}) -{{%/notice%}} - -### SCP - -For Windows users, file transferring between your personal computer -and the HCC supercomputers can be achieved through the command `scp`. -Here we use **Crane** for example. **The following commands should be -executed from your computer. ** - -**Uploading from local to remote** - -{{< highlight bash >}} -C:\> scp -r .\<folder name> <username>@crane.unl.edu:/work/<group name>/<username> -{{< /highlight >}} - -The above command line transfers a folder from the current directory -(`.\`) of the your computer to the `$WORK` directory of the HCC -supercomputer, Crane. Note that you need to replace `<group name>` -and `<username>` with your HCC group name and username. - -**Downloading from remote to local** - -{{< highlight bash >}} -C:\> scp -r <username>@crane.unl.edu:/work/<group name>/<username>/<folder name> .\ -{{< /highlight >}} - -The above command line transfers a folder from the `$WORK` directory of -the HCC supercomputer, Crane, to the current directory (`.\`) of the -your computer. - - -### WinSCP - -WinSCP: http://winscp.net/eng/download.php - -Usually it is convenient to upload and download files between your personal computer -and the HCC supercomputers through a Graphic User Interface (GUI). -Download and install the third party application **WinSCP** -to connect the file systems between your personal computer and the HCC supercomputers. -Below is a step-by-step installation guide. Here we use the HCC cluster **Crane** -for demonstration. To use the **Rhino** cluster, replace `crane.unl.edu` -with `rhino.unl.edu`. - -1. On the first screen, type `crane.unl.edu` for Host name, enter your - HCC account username and password for User name and Password. Then - click on **Login**. - - {{< figure src="/images/3178530.png" height="450" >}} - -2. On the second screen, click on **Yes**. - - {{< figure src="/images/3178531.png" >}} - -3. Choose option "1" and press "Enter". Or simply press your Yubikey if - you have one. - - {{< figure src="/images/8127268.png" >}} - -4. On the third screen, click on **Remote**. Under Remote, choose Go To - and Open Directory/Bookmark. Alternatively, you can use the keyboard - shortcut "Ctrl + O". - - {{< figure src="/images/3178532.png" height="450" >}} - -5. On the final screen, type `/work/<group name>/<username>` for Open - directory. Use your HCC group name and username to replace - `<group name>` and `<username>`. Then click on **OK**. - - {{< figure src="/images/3178533.png" height="450" >}} - -6. Now you can drop and drag the files between your personal computer - and the HCC supercomputers. - {{< figure src="/images/3178539.png" height="450" >}} - -Tutorial Video --------------- - -{{< youtube -Vh7SyC-3mA >}} - - - diff --git a/content/connecting/mobaxterm_windows.md b/content/connecting/mobaxterm.md similarity index 96% rename from content/connecting/mobaxterm_windows.md rename to content/connecting/mobaxterm.md index c3c9cac5bce87a158d8cdedf451203a317401f33..0bed8e41c98cc7d63795b7bc009d6c5adfb36c0d 100644 --- a/content/connecting/mobaxterm_windows.md +++ b/content/connecting/mobaxterm.md @@ -1,6 +1,6 @@ +++ -title = "Using MobaXterm" -description = "Quickstart Guide for MobaXterm" +title = "Connecting with MobaXterm" +description = "Connecting to and Transfering files between HCC Clusters with MobaXterm" weight = "22" +++ diff --git a/content/connecting/putty.md b/content/connecting/putty.md new file mode 100644 index 0000000000000000000000000000000000000000..966c8013dba310ba44eadfbae5041fa3826af334 --- /dev/null +++ b/content/connecting/putty.md @@ -0,0 +1,106 @@ ++++ +title = "Connecting with PuTTY (Windows)" +description = "How to connect to HCC clusters using the PuTTY SSH client" +weight = "20" ++++ + +##### Use of Duo two-factor authentication is **required** to access HCC resources. +##### Please see [Setting up and Using Duo]({{< relref "setting_up_and_using_duo" >}}). + +- [Connecting to HCC Clusters](#connecting-to-hcc-clusters) + - [For Windows 10 Users](#windows-10) + - [For Windows 7 and 8.1 Users](#windows-7-and-8-1) + +## Connecting to HCC Clusters + +{{% notice info %}} +If you have a MacOS or Linux system, please see [Connecting with Terminal] +({{< relref "/connecting/terminal.md" >}}). +{{% /notice %}} + + +### Windows 10 +-------------- + +Newer Windows users (Windows 10 build version 1803 and later), can connect using the +Command Prompt. Please see [Connecting with the Terminal]({{< relref "/connecting/terminal" >}}) +for more details. + +### Windows 7 and 8.1 +-------------- + +For users with older Windows versions, you will need to install an SSH client to connect. +We will cover the use of PuTTY here, but you are free to use any compatable client, such as +[MobaXterm]({{< relref "/connecting/mobaxterm" >}}) as well. + +To download and install PuTTY, visit the [PuTTY website] +(https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html). + +Once you have PuTTY installed, run the application and follow these steps: + +{{% notice info %}} +**Note that the example below uses the `Tusker` cluster, which is no longer active. +Replace all instances of `tusker` with either `crane` or `rhino` depending on which cluster +you want to connect to. +{{% /notice %}} + +1. On the first screen, type `tusker.unl.edu` for Host Name, then click + **Open**. + {{< figure src="/images/3178523.png" height="450" >}} + +2. On the second screen, click on **Yes**. + {{< figure src="/images/3178524.png" height="300" >}} + +3. On the third screen, enter your HCC account **username**. If you do + not have a HCC account, please contact an HCC specialist + ({{< icon name="envelope" >}}[hcc-support@unl.edu] (mailto:hcc-support@unl.edu)) + or go to http://hcc.unl.edu/newusers. + + {{% notice info %}}Replace `jingchao` with your username.{{% /notice %}} + + + {{< figure src="/images/8127261.png" height="450" >}} + +4. On the next screen, enter your HCC account **password**. + + {{% notice info %}}**Note that PuTTY will not show the characters as you type for security reasons.**{{% /notice %}} + + {{< figure src="/images/8127262.png" height="450" >}} + +5. After you input the correct + password, you will be asked to choose a Duo authentication + method. + +6. If you have a Yubikey set up by HCC, please hold the Yubikey for \~1 + second. Then you will be brought to your home directory similar as + below. + + {{< figure src="/images/8127266.png" height="450" >}} + +7. If you set up Duo via a smart phone, please type "1" in your + terminal and press "Enter". (Duo-Push is the most cost-effective way + for Duo authentication, we recommend all user use this option if + that is applicable.) + +8. Check your smart phone for Duo login request. Press "Approve" if you + can verify the request. If you find any Duo login request that is + not initiated by yourself, deny it and report this incident + immediately to {{< icon name="envelope" >}}[hcc-support@unl.edu] (mailto:hcc-support@unl.edu). + + {{< figure src="/images/8127263.png" height="450" >}} + +9. After you approve the Duo login request, you will be brought to your + home directory similar as below. + + {{< figure src="/images/8127264.png" height="450" >}} + +### Next Steps: + +Now that you are connected, check out these resources to learn more: + - [Handling Data]({{< relref "../handling_data/">}}) + - [Running Applications]({{< relref "../applications/">}}) + - [Submitting Jobs]({{< relref "../submitting_jobs/">}}) + +If you are not familiar with using command line Linux, check out these resources: + - [Basic Linux Commands]({{< relref "basic_linux_commands">}}) + - [Software Carpentry Introduction to the Unix Shell Lesson](http://swcarpentry.github.io/shell-novice/) diff --git a/content/connecting/terminal.md b/content/connecting/terminal.md new file mode 100644 index 0000000000000000000000000000000000000000..9951e798371291d9c93f1eb91a79057a666d8303 --- /dev/null +++ b/content/connecting/terminal.md @@ -0,0 +1,102 @@ ++++ +title = "Connecting with Terminal" +description = "How to connect to HCC clusters using Terminal (Mac/Linux) or Command Prompt (Windows)" +weight = "10" ++++ + +##### Use of Duo two-factor authentication is **required** to access HCC resources. +##### Please see [Setting up and Using Duo]({{< relref "setting_up_and_using_duo" >}}). +--- +- [Starting Terminal] (#starting-terminal) + - [MacOS](#macos) + - [Linux](#linux) + - [Windows](#windows) +- [Connecting to HCC Clusters](#connecting-to-hcc-clusters) + +# Starting Terminal: + +## MacOS + +To start a terminal on MacOS, launch Spotlight by pressing `Command - spacebar`. +Type "Terminal" and press `enter` to launch the application. You should see a prompt, +which is a string of text, followed by a `$` or `>` character and a blinking cursor. +Continue to connect by following the [connection directions below] +(#connecting-to-hcc-clusters). + +## Linux + +Terminal can be opened on most Linux systems by pressing the `Ctrl + Alt + T` keys simultaneously. +You should see a prompt - a string of text, followed by a `$` or `>` character and a blinking cursor. +Continue to connect by following the [connection directions below](#connecting-to-hcc-clusters). + + +## Windows + +Computers with newer versions of windows (Windows 10 build version 1803 and later), +can connect using the Command Prompt. To start the command prompt, open the start menu +and type `cmd`. Press `enter` to run. +You should see a prompt - a string of text, followed by a `$` or `>` character and a blinking cursor. +Continue to connect by following the [connection directions below] +(#connecting-to-hcc-clusters). + +If you are using an older versions of Windows, you will need to [download an SSH client]({{< relref "putty">}}) + +### Connecting To HCC Clusters + +Once you have opened your terminal and you have a prompt, you can connect using the `ssh` command. +For example, to connect to the Crane cluster type the following in your terminal or Command Prompt window: + +{{< highlight bash >}} +$ ssh <username>@crane.unl.edu +{{< /highlight >}} + +where `<username>` is replaced with your HCC account name. To use the **Rhino** cluster, +replace crane.unl.edu with rhino.unl.edu. + +The first time you connect to one of our clusters from a computer, you will be prompted to verify the connection: + +{{< highlight bash >}} +The authenticity of host 'crane.unl.edu (129.93.227.113)' can't be established. +RSA key fingerprint is SHA256:GDH3+iqSp3WJxtUE6tXNQcWRwpf0xjYgkQrYBDX3Ir0. +RSA key fingerprint is MD5:e5:c5:ac:07:ff:47:53:18:5e:8b:44:16:51:78:4a:c7. +Are you sure you want to continue connecting (yes/no)? +{{< /highlight >}} + +Type `yes` to indicate that you do intend to connect to Crane. + +You will then be prompted for your password. + +{{% notice info %}}**Note that when entering your password, the characters you type will +not be displayed on screen for security reasons.**{{% /notice %}} + +Once the account password is entered successfully, two-factor authentication prompt will appear. + +{{< highlight bash >}} +Duo two-factor login for demo01 + +Enter a passcode or select one of the following options: + + 1. Duo Push to XXX-XXX-4247 + 2. Phone call to XXX-XXX-4247 + 3. SMS passcodes to XXX-XXX-4247 + +Passcode or option (1-3): +{{< /highlight >}} + +Enter the number which corresponds to your preferred option. You will know you have connected +if your prompt changes to the following: + +{{< highlight bash >}} +[<username>@login.crane ~]$ +{{< /highlight >}} + +### Next Steps: + +Now that you are connected, check out these resources to learn more: + - [Handling Data]({{< relref "../handling_data/">}}) + - [Running Applications]({{< relref "../applications/">}}) + - [Submitting Jobs]({{< relref "../submitting_jobs/">}}) + +If you are not familiar with using command line Linux, check out these resources: + - [Basic Linux Commands]({{< relref "basic_linux_commands">}}) + - [Software Carpentry Introduction to the Unix Shell Lesson](http://swcarpentry.github.io/shell-novice/) diff --git a/content/handling_data/data_transfer/connect_to_cb3_irods.md b/content/handling_data/data_transfer/connect_to_cb3_irods.md index c123dd62309960df2f6ffa1a3c4c59b9fb2258e3..1431f6874bf323a3614f8504a98d387d76a3830a 100644 --- a/content/handling_data/data_transfer/connect_to_cb3_irods.md +++ b/content/handling_data/data_transfer/connect_to_cb3_irods.md @@ -1,7 +1,7 @@ +++ title = "Connecting to CB3 iRODS" description= "How to connect to the CB3 iRODS instance." -weight = 40 +weight = 70 +++ {{% panel theme="danger" header="Sensitive and Protected Data" %}} diff --git a/content/handling_data/data_transfer/cyberduck.md b/content/handling_data/data_transfer/cyberduck.md new file mode 100644 index 0000000000000000000000000000000000000000..0770749b75b215a9c1a847b156db6e3f4621e4bb --- /dev/null +++ b/content/handling_data/data_transfer/cyberduck.md @@ -0,0 +1,73 @@ ++++ +title = "File Transfer with CyberDuck" +description = "Transfering data to and from HCC clusters with the Cyberduck SCP Client" +weight = "30" ++++ + +--- + +## Using Cyberduck +--------------- + +If you wish to use a GUI for data transfer, be aware that not all programs will function +correctly with Duo two-factor authentication. Mac users are recommended +to use [Cyberduck](https://cyberduck.io). It is compatible with Duo, but a +few settings need to be changed. + +Under **Preferences - General**, change the default protocol to SFTP: + +{{< figure src="/images/7274497.png" height="450" >}} + +Under **Preferences - Transfers**, reuse the browser connection for file +transfers. This will avoid the need to reenter your password for every +file transfer: + +{{< figure src="/images/7274498.png" height="450" >}} + +Finally, under **Preferences - SFTP**, set the file transfer method to +SCP: + +{{< figure src="/images/7274499.png" height="450" >}} + +To add an HCC machine, in the bookmarks pane click the "+" icon: + +{{< figure src="/images/7274500.png" height="450" >}} + +Ensure the type of connection is SFTP. Enter the hostname of the machine +you wish to connect to (crane.unl.edu, rhino.unl.edu) in the **Server** +field, and your HCC username in the **Username** field. The +**Nickname** field is arbitrary, so enter whatever you prefer. + +{{< figure src="/images/7274501.png" height="450" >}} + +After you add the bookmark, double-click it to connect. + +{{< figure src="/images/7274505.png" height="450" >}} + +Enter your HCC username and password in the dialog box that will appear +and click *Login*. + +{{< figure src="/images/7274508.png" height="450" >}} + +A second login dialogue will now appear. Notice the text has changed to +say Duo two-factor. + +{{< figure src="/images/7274510.png" height="450" >}} + +Clear the **Password** field in the dialogue. If you are using the Duo +Mobile app, enter '1' to have a push notification send to your phone or +tablet. If you are using a Yubikey, ensure the cursor is active in the +**Password** field, and press the button on the Yubikey. + +{{< figure src="/images/7274509.png" height="450" >}} + +The login should complete and you can simply drag and drop files to or +from the window. + +{{< figure src="/images/7274511.png" height="450" >}} + +If you run into issues with two-factor authentication, try the command below for a quick fix: +{{< highlight bash >}} +$ rm -rf ~/Library/ApplicationSupport/Cyberduck +{{< /highlight >}} + diff --git a/content/handling_data/data_transfer/globus_connect/_index.md b/content/handling_data/data_transfer/globus_connect/_index.md index 61830c406dd9477414c72525536790a8c01ead65..8191bd1b3d37ca1ec8bdfef03f8047134e8e6887 100644 --- a/content/handling_data/data_transfer/globus_connect/_index.md +++ b/content/handling_data/data_transfer/globus_connect/_index.md @@ -1,7 +1,7 @@ +++ title = "Globus Connect" description = "Globus Connect overview" -weight = 20 +weight = 5 +++ <a href="https://www.globus.org/globus-connect" class="external-link">Globus Connect</a> is diff --git a/content/handling_data/data_transfer/high_speed_data_transfers.md b/content/handling_data/data_transfer/high_speed_data_transfers.md index 922eb4b43902030f7a09eeb1f96332fea2bbe294..07c5161af93d2b4d07b1538e37105dc6b7cd9465 100644 --- a/content/handling_data/data_transfer/high_speed_data_transfers.md +++ b/content/handling_data/data_transfer/high_speed_data_transfers.md @@ -1,7 +1,7 @@ +++ title = "High Speed Data Transfers" description = "How to transfer files directly from the transfer servers" -weight = 10 +weight = 40 +++ Crane, Rhino, and Attic each have a dedicated transfer server with diff --git a/content/handling_data/data_transfer/scp.md b/content/handling_data/data_transfer/scp.md new file mode 100644 index 0000000000000000000000000000000000000000..f69d8cabb886eeeea714c693594a5451264e8861 --- /dev/null +++ b/content/handling_data/data_transfer/scp.md @@ -0,0 +1,46 @@ ++++ +title = "File Transfer with scp" +description = "Transfering data to and from HCC clusters with the scp command" +weight = "10" ++++ + +## Using the SCP command + +For MacOS, Linux, and later Windows users, file transferring between your personal computer +and the HCC supercomputers can be achieved through the command `scp` which stands for secure copy. +This method is ideal for quick transfer of smaller files. For large volume transfers, +we recommend the using [Globus] or an SCP client such as [WinSCP for Windows]({{< relref "winscp">}}) or +[CyberDuck for Mac/Linux]({{< relref "cyberduck">}}). + +Just like the `cp` copy command, the `scp` command requires two arguments, +the path to source file(s) and the path to the target location. +Since one or more of these locations are remote, you will need to specify the username and host for those. + +{{< highlight bash >}} +$ scp <username>@<host>:<path_to_files> <username>@<host>:<path_to_files> +{{< /highlight >}} + +For the local location, you do not need to specify the username or host. **When transferring to and from +your local computer, the `scp` command should be ran on your computer, NOT from HCC clusters.** + +### Uploading a file to Crane + +Here is an example of file transfer to and from the Crane cluster. + +To upload the file `data.csv` in your current directory to your `$WORK` directory +on the Crane cluster, you would use the command: + +{{< highlight bash >}} +$ scp ./data.csv <user_name>@crane.unl.edu:/work/<group_name>/<user_name> +{{< /highlight >}} + +where `<user_name>` and `<group_name>` are replaced with your user name and your group name. + +### Downloading a file from Crane + +To download the file `data.csv` from your `$WORK` directory +on the Crane cluster to your current directory, you would use the command: + +{{< highlight bash >}} +$ scp <user_name>@crane.unl.edu:/work/<group_name>/<user_name>/data.csv ./ +{{< /highlight >}} diff --git a/content/handling_data/data_transfer/using_rclone_with_hcc.md b/content/handling_data/data_transfer/using_rclone_with_hcc.md index 02fc1c2617c7fba88dcd5772a5ec622bc5fab1ee..a81a0899f0166eae262fefed978cee24d099d60e 100644 --- a/content/handling_data/data_transfer/using_rclone_with_hcc.md +++ b/content/handling_data/data_transfer/using_rclone_with_hcc.md @@ -1,7 +1,7 @@ +++ title = "Using Rclone for File Transfer" description = "How to use Rclone with HCC" -weight =30 +weight =60 +++ Rclone is an open source file transfer tool to make transfering files to and from various cloud resources such as Box, Amazon S3, Microsoft OneDrive, and Google Cloud Storage and your local machine a simpler task. Guides on how to set up a variety of resources to transfer to and from can be found at [rclone's webpage](https://rclone.org/). diff --git a/content/handling_data/data_transfer/winscp.md b/content/handling_data/data_transfer/winscp.md new file mode 100644 index 0000000000000000000000000000000000000000..317dd8245849b378dcf78bf9ee5c9578fe253b3c --- /dev/null +++ b/content/handling_data/data_transfer/winscp.md @@ -0,0 +1,51 @@ ++++ +title = "File Transfer with WinSCP" +description = "Transfering Files to and from HCC Clusters using the WinSCP Client" +weight = "20" ++++ + +### WinSCP + +WinSCP: http://winscp.net/eng/download.php + +Usually it is convenient to upload and download files between your personal computer +and the HCC supercomputers through a Graphic User Interface (GUI). +Download and install the third party application **WinSCP** +to connect the file systems between your personal computer and the HCC supercomputers. +Below is a step-by-step installation guide. Here we use the HCC cluster **Tusker** +for demonstration. To use the **Crane** cluster, replace `tusker.unl.edu` +with `crane.unl.edu`. + +1. On the first screen, type `tusker.unl.edu` for Host name, enter your + HCC account username and password for User name and Password. Then + click on **Login**. + + {{< figure src="/images/3178530.png" height="450" >}} + +2. On the second screen, click on **Yes**. + + {{< figure src="/images/3178531.png" >}} + +3. Choose option "1" and press "Enter". Or simply press your Yubikey if + you have one. + + {{< figure src="/images/8127268.png" >}} + +4. On the third screen, click on **Remote**. Under Remote, choose Go To + and Open Directory/Bookmark. Alternatively, you can use the keyboard + shortcut "Ctrl + O". + + {{< figure src="/images/3178532.png" height="450" >}} + +5. On the final screen, type `/work/<group name>/<username>` for Open + directory. Use your HCC group name and username to replace + `<group name>` and `<username>`. Then click on **OK**. + + {{< figure src="/images/3178533.png" height="450" >}} + +6. Now you can drop and drag the files between your personal computer + and the HCC supercomputers. + {{< figure src="/images/3178539.png" height="450" >}} + + +