Skip to content
Snippets Groups Projects
Commit 0c995db6 authored by eharstad's avatar eharstad
Browse files

Updating rclone instructions to use OOD virtual desktop

parent 5851b216
No related branches found
No related tags found
1 merge request!309Updating rclone instructions to use OOD virtual desktop
......@@ -12,26 +12,31 @@ This tool can be used to transfer files between HCC clusters and outside cloud p
1. You must be able to access your [NU Office365](http://office.com/) account before beginning this process. Contact your local campus IT support if you need help with initial account setup.
2. Due to the clusters being remote machines, **Rclone will need to be installed on your [local machine](https://rclone.org/downloads/) in order to authorize OneDrive**. Some services, such as Google Drive, do not require Rclone to be installed on your local machine.
3. After logging into the cluster of your choice, load the `rclone` module by entering the command below at the prompt:
2. Open a browser on your local machine and navigate to the [On-Demand portal]({{< relref "/open_ondemand" >}}) for the cluster of your choice. We use Crane for this example: [https://crane-ood.unl.edu](https://crane-ood.unl.edu). Select `Desktop` under `Interactive Apps` in the menu at the top of the page to get a virtual desktop on the cluster.
{{< figure src="/images/rclone_select_virtual_desktop.png" width="500" class="img-border">}}
Scroll down to the bottom of the next page, and click on the blue `Launch` button. When the resource is ready, click on the blue `Launch Desktop` button that appears on the next page.
{{< figure src="/images/rclone_launch_desktop.png" width="500" class="img-border">}}
On the virtual desktop, click on the `Terminal Emulator` icon at the bottom of the window to open up a command shell.
{{< figure src="/images/rclone_open_terminal.png" width="500" class="img-border">}}
3. At the command prompt in the shell that opens, load the `rclone` module by entering the command below at the prompt:
{{% panel theme="info" header="Load the Rclone module" %}}
{{< highlight bash >}}
[demo2@login.crane ~]$ module load rclone
[demo03@c0809.crane ~]$ module load rclone
{{< /highlight >}}
{{% /panel %}}
4. We will need to start the basic configuration for OneDrive. To do this run `rclone config`:
{{% panel theme="info" header="Load the rclone config" %}}
{{< highlight bash >}}
[demo2@login.crane ~]$ rclone config
[demo03@c0809 ~]$ rclone config
{{< /highlight >}}
{{% /panel %}}
5. In a new configuration, you will see no remotes found. Enter `n` to make a new remote and name it a name you will know. In our example, we will use "myOneDrive". Select `Microsoft OneDrive` by entering in the corresponding number, in our case `23`. Hit Enter for the client_id, client_secret, and Edit advanced config. **When you are prompted for auto config, select `n`**. The terminal will stop at a `result>` prompt. Proceed to the next step.
{{% panel theme="info" header="Configure OneDrive" %}}
{{< highlight bash >}}
[demo2@login.crane ~]$ rclone config
[demo03@c0809 ~]$ rclone config
No remotes found - make a new one
n) New remote
s) Set configuration password
......@@ -75,16 +80,13 @@ result>
{{< /highlight >}}
{{% /panel %}}
6. Now **switch to a terminal on your local machine** and run `rclone authorize "onedrive"`. You will be prompted to go to a 127.0.0.1 address in your web browser if a browser doesn't open automatically. On the Microsoft Office sign-in page that opens enter in your NU e-mail address and click "Next". You will be taken to your campus' single sign-on page where you can sign in using your TrueYou/campus credentials. If login is successful, you should be redirected to a page that says "Success!" Return to your local rclone session. You should see a message instructing you to paste a line of code from your local machine to the cluster and then to confirm that the config is correct.
{{% panel theme="info" header="Authorize OneDrive on local machine" %}}
{{< highlight bash >}}
[myaccount@local.machine ~]$ rclone authorize "onedrive"
{{< /highlight >}}
{{% /panel %}}
{{< figure src="/images/MicrosoftSign-In.png" height="500" class="img-border">}}
{{% panel theme="info" header="Local Authorization after logging in with browser" %}}
6. Now **open up another terminal window on the virtual desktop** by clicking again on the `Terminal Emulator` icon at the bottom of the window.
{{< figure src="/images/rclone_open_second_terminal.png" width="500" class="img-border">}}
In the new shell that opens, load the `rclone` module like you did in the first shell with `module load rclone`, and then run `rclone authorize "onedrive"` at the command prompt. You will be prompted to go to a 127.0.0.1 address in a web browser.
{{% panel theme="info" header="Authorize OneDrive from the terminal" %}}
{{< highlight script >}}
[myaccount@local.machine ~]$ rclone authorize "onedrive"
[demo03@c0809 ~]$ module load rclone
[demo03@c0809 ~]$ rclone authorize "onedrive"
If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
Log in and authorize Rclone for access
Waiting for code...
......@@ -94,9 +96,11 @@ Paste the following into your remote machine --->
<---End paste
{{< /highlight >}}
{{% /panel %}}
For other services, please refer to the [rclone documentation](https://rclone.org/).
7. **Returning to the cluster**, now paste the code into the waiting `result` prompt. Then select option 1 for `OneDrive Personal or Business` and then 0 for `OneDrive (business)`. Press `y` at the next two prompts to confirm and `q` to exit.
Right-click on the link and select `open link` from the menu. A browser window will open on the virtual desktop. On the Microsoft Office sign-in page that opens enter in your NU e-mail address and click "Next". You will be taken to your campus' single sign-on page where you can sign in using your TrueYou/campus credentials. If login is successful, you should be redirected to a page that says "Success!"
{{< figure src="/images/rclone_authorize_success.png" width="500" class="img-border">}}
Return to the terminal window where you ran the authorize command. You should see a message instructing you to paste a line of code into your "remote machine" which is the first terminal you opened to run `rclone config`.
{{< figure src="/images/rclone_authorize_code.png" width="500" class="img-border">}}
Copy the text by highlighting it, right-clicking, and selecting `copy`. Then, return to the first terminal with the waiting `result` prompt and paste the text by right-clicking and selecting `paste`, then press `enter`. Next, select option 1 for `OneDrive Personal or Business` and then 0 for `OneDrive (business)`. Press `y` at the next two prompts to confirm and `q` to exit.
{{% panel theme="info" header="Authorize OnDemand on cluster" %}}
{{< highlight bash >}}
result> {"access_token":"XXXX","token_type":"bearer","refresh_token":"XXXX","expiry":"XXXX"}
......@@ -150,7 +154,7 @@ e/n/d/r/c/s/q> q
Now test the connection by running the `ls` command. You should see a listing of your OneDrive files.
{{% panel theme="info" header="List contents of OneDrive" %}}
{{< highlight bash >}}
[demo2@login.crane ~]$ rclone ls myOneDrive:/
[demo03@c0809 ~]$ rclone ls myOneDrive:/
{{< /highlight >}}
{{% /panel %}}
......@@ -158,28 +162,28 @@ Now test the connection by running the `ls` command. You should see a listing o
8. To upload or download files, use the `rclone copy` command. For example:
{{% panel theme="info" header="Transferring files" %}}
{{< highlight bash >}}
[demo2@login.crane ~]$ rclone copy myOneDrive:/SomeFile.txt ./
[demo2@login.crane ~]$ rclone copy ./SomeFile.txt myOneDrive:/
[demo03@c0809 ~]$ rclone copy myOneDrive:/SomeFile.txt ./
[demo03@c0809 ~]$ rclone copy ./SomeFile.txt myOneDrive:/
{{< /highlight >}}
{{% /panel %}}
9. To download directories, use the `rclone copy` command and use directory names over file. This copies the contents of the folders, so you need to specify a destination folder.
{{% panel theme="info" header="Download a directory from OneDrive" %}}
{{< highlight bash >}}
[demo2@login.crane ~]$ rclone copy myOneDrive:/my_hcc_dir ./my_hcc_dir
[demo03@c0809 ~]$ rclone copy myOneDrive:/my_hcc_dir ./my_hcc_dir
{{< /highlight >}}
{{% /panel %}}
To upload a directory named `my_hcc_dir` to OneDrive, use `rclone copy`.
{{% panel theme="info" header="Upload a directory to OneDrive" %}}
{{< highlight bash >}}
[demo2@login.crane ~]$ rclone copy ./my_hcc_dir myOneDrive:/my_hcc_dir
[demo03@c0809 ~]$ rclone copy ./my_hcc_dir myOneDrive:/my_hcc_dir
{{< /highlight >}}
{{% /panel %}}
11. Rclone also supports using sync to transfer files, similar to rsync. The syntax is similar to `rclone copy`. This would only transfer files that are updated by name, checksum, or time. The example below would sync the files of the local directory to the remote directory on OneDrive.
{{% panel theme="info" header="transfer.sh" %}}
{{< highlight bash >}}
[demo2@login.crane ~]$ rclone sync ./my_hcc_dir myOneDrive:/my_hcc_dir
[demo03@c0809 ~]$ rclone sync ./my_hcc_dir myOneDrive:/my_hcc_dir
{{< /highlight >}}
{{% /panel %}}
......
static/images/rclone_authorize_code.png

267 KiB

static/images/rclone_authorize_success.png

142 KiB

static/images/rclone_launch_desktop.png

63.9 KiB

static/images/rclone_open_second_terminal.png

133 KiB

static/images/rclone_open_terminal.png

97.2 KiB

static/images/rclone_select_virtual_desktop.png

264 KiB

0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment