Skip to content
Snippets Groups Projects

Update anvil guide 1

Merged Update anvil guide 1
update-anvil-guide-1 into master
Merged Adam Caprez requested to merge update-anvil-guide-1 into master
Compare
and
30 files
+ 218
− 590
Compare changes
  • Side-by-side
  • Inline
Files
30
content/guides/anvil/_index.md
+ 89
− 123
@@ -4,46 +4,38 @@ description = "How to use Anvil, HCC's OpenStack-based cloud resource"
@@ -4,46 +4,38 @@ description = "How to use Anvil, HCC's OpenStack-based cloud resource"
weight = "40"
weight = "40"
+++
+++
<span id="title-text"> HCC-DOCS : Anvil: HCC's Cloud </span>
- [Overview](#overview)
============================================================
- [Cloud Terms](#cloud-terms)
 
- [Steps for Access](#steps-for-access)
 
- [Backups](#backups)
Created by <span class="author"> Derek Weitzel</span>, last modified by
<span class="editor"> Adam Caprez</span> on Apr 19, 2018
- [Overview](#Anvil:HCC'sCloud-Overview)
- [Cloud Terms](#Anvil:HCC'sCloud-CloudTerms)
- [Steps for Access](#Anvil:HCC'sCloud-StepsforAccess)
- [Backups](#Anvil:HCC'sCloud-Backups)
<span
class="aui-icon aui-icon-small aui-iconfont-approve confluence-information-macro-icon"></span>
 
{{% notice tip %}}
Have your account and ready to go? Visit the Anvil OpenStack web
Have your account and ready to go? Visit the Anvil OpenStack web
interface at
interface at https://anvil.unl.edu.
<a href="https://anvil.unl.edu/" class="external-link">https://anvil.unl.edu/</a>
{{% /notice %}}
Overview
---
--------
 
### Overview
Anvil is the Holland Computing Center's cloud computing resource, based
Anvil is the Holland Computing Center's cloud computing resource, based
on
on the [OpenStack](https://www.openstack.org) software.
the <a href="https://www.openstack.org/" class="external-link">OpenStack</a>
OpenStack is a free and open-source software platform for
software. OpenStack is a free and open-source software platform for
cloud computing. Anvil was created to address the needs of NU's
cloud computing. Anvil was created to address the needs of NU's
research community that are not well served by a traditional
research community that are not well served by a traditional
batch-scheduled Linux cluster environment. Examples of use cases that
batch-scheduled Linux cluster environment. Examples of use cases that
are well suited to Anvil include:
are well suited to Anvil include:
- A highly interactive environment, especially GUI applications
- A highly interactive environment, especially GUI applications
- Require root-level access, such as kernel modification or
- Require root-level access, such as kernel modification or
virtualization work
virtualization work
- Alternate operating systems, such as Windows or other distributions
- Alternate operating systems, such as Windows or other distributions
of Linux
of Linux
- Test cluster environments for various software frameworks, such as
- Test cluster environments for various software frameworks, such as
<a href="http://hadoop.apache.org" class="external-link">Hadoop</a>
[Hadoop](http://hadoop.apache.org)
or
or [Spark](https://spark.apache.org)
<a href="https://spark.apache.org" class="external-link">Spark</a>
- Cluster applications that require a persistent resource, such as a
- Cluster applications that require a persistent resource, such as a
web or database server
web or database server
Using Anvil, one or more virtual machines (VMs) can be easily be created
Using Anvil, one or more virtual machines (VMs) can be easily be created
@@ -52,66 +44,62 @@ HCC clusters, or your own workstation once connected to the Anvil
@@ -52,66 +44,62 @@ HCC clusters, or your own workstation once connected to the Anvil
Virtual Private Network (VPN). Access is through standard means,
Virtual Private Network (VPN). Access is through standard means,
typically via SSH for Linux VMs and Remote Desktop for Windows VMs.
typically via SSH for Linux VMs and Remote Desktop for Windows VMs.
Cloud Terms
### Cloud Terms
-----------
There are a few terms used within the OpenStack interface and in the
There are a few terms used within the OpenStack interface and in the
instructions below that may be unfamiliar. The following brief
instructions below that may be unfamiliar. The following brief
definitions may be useful. More detailed information is available in
definitions may be useful. More detailed information is available in
the <a href="http://docs.openstack.org/user-guide/" class="external-link">OpenStack User Guide</a>.
the [OpenStack User Guide](http://docs.openstack.org/user-guide).
- ***Project**: *A project is the base unit of ownership in
- **Project**: A project is the base unit of ownership in
OpenStack. Resources (CPUs, RAM, storage, etc.) are allocated and
OpenStack. Resources (CPUs, RAM, storage, etc.) are allocated and
user accounts are associated with a project. Within Anvil, each HCC
user accounts are associated with a project. Within Anvil, each HCC
research group corresponds directly to a project. Similar to
research group corresponds directly to a project. Similar to
resource allocation on HCC clusters, the members of a group share
resource allocation on HCC clusters, the members of a group share
the [project's resources](13599589.html).
the [project's resources]({{< relref "what_are_the_per_group_resource_limits" >}}).
- ***Image**: * An image corresponds to everything needed to create a
- **Image**: An image corresponds to everything needed to create a
virtual machine for a specific operating system (OS), such as Linux
virtual machine for a specific operating system (OS), such as Linux
or Windows. HCC creates and maintains [basic Windows and
or Windows. HCC creates and maintains [basic Windows and Linux]({{< relref "available_images" >}})
Linux](Available-Images_13042823.html) images for convenience.
images for convenience.
Users can also create their own images that can then be uploaded to
Users can also create their own images that can then be uploaded to
OpenStack and used within the project.
OpenStack and used within the project.
- ***Flavor**:* A flavor (also known as *instance type*), defines the
- **Flavor**: A flavor (also known as *instance type*), defines the
parameters (i.e. resources) of the virtual machine. This includes
parameters (i.e. resources) of the virtual machine. This includes
things such as number of CPUs, amount of RAM, storage, etc. There
things such as number of CPUs, amount of RAM, storage, etc. There
are many instance types [available within
are many instance types [available within Anvil]({{< relref "anvil_instance_types" >}}),
Anvil](Anvil-Instance-Types_13042763.html), designed to meet a
designed to meet a variety of needs.
variety of needs.
- ***Instance**:* An instance is a running virtual machine, created
- **Instance**: An instance is a running virtual machine, created
by combining an image (the basic OS) with a flavor (resources).
by combining an image (the basic OS) with a flavor (resources).
That is, *Image + Flavor = Instance*.
That is, *Image + Flavor = Instance*.
- ***Volume**: *A volume is a means for persistent storage within
- **Volume**: A volume is a means for persistent storage within
OpenStack. When an instance is destroyed, any additional data that
OpenStack. When an instance is destroyed, any additional data that
was on the OS hard drive is lost. A volume can be thought of
was on the OS hard drive is lost. A volume can be thought of
similar to an external hard drive. It can be attached to an
similar to an external hard drive. It can be attached to an
instance and accessed as a second drive. When the instance is
instance and accessed as a second drive. When the instance is
destroyed, data on the volume is retained. It can then be attached
destroyed, data on the volume is retained. It can then be attached
and accessed from another instance later.
and accessed from another instance later.
Steps for Access
### Steps for Access
----------------
The guide below outlines the steps needed to begin using Anvil. Please
The guide below outlines the steps needed to begin using Anvil. Please
note that Anvil is currently in the *beta testing* phase. While
note that Anvil is currently in the *beta testing* phase. While
reasonable precautions are taken against data loss, **sole copies of
reasonable precautions are taken against data loss, **sole copies of
precious or irreproducible data should not be placed or left on Anvil**.
precious or irreproducible data should not be placed or left on Anvil**.
1. **Request access to Anvil**
1. **Request access to Anvil**
Access and resources are provided on a per-group basis, similar to
Access and resources are provided on a per-group basis, similar to
HCC clusters. For details, please see [What are the per group
HCC clusters. For details, please see [What are the per group
resource limits?](13599589.html) To begin using Anvil, the group
resource limits?]({{< relref "what_are_the_per_group_resource_limits" >}})
owner should fill out the short request form
To begin using Anvil, the group owner should fill out the short request form
at <a href="http://hcc.unl.edu/request-anvil-access" class="external-link">http://hcc.unl.edu/request-anvil-access</a>.
at http://hcc.unl.edu/request-anvil-access.
An automated confirmation email will be sent, and an HCC staff
An automated confirmation email will be sent, and an HCC staff
member will follow-up once access is available.
member will follow-up once access is available.
**
**
2. **Create SSH keys**
2. **Create SSH keys**
OpenStack uses SSH key pairs to identify users and control access to
OpenStack uses SSH key pairs to identify users and control access to
the VMs themselves, as opposed to the traditional username/password
the VMs themselves, as opposed to the traditional username/password
combination. SSH key pairs consist of two files, a public key and a
combination. SSH key pairs consist of two files, a public key and a
@@ -123,59 +111,41 @@ precious or irreproducible data should not be placed or left on Anvil**.
@@ -123,59 +111,41 @@ precious or irreproducible data should not be placed or left on Anvil**.
it's best practice to create a dedicated pair for use with Anvil.
it's best practice to create a dedicated pair for use with Anvil.
The process for creating key pairs is different between Windows and
The process for creating key pairs is different between Windows and
Mac. Follow the relevant guide below for your operating system.
Mac. Follow the relevant guide below for your operating system.
1. [Creating SSH key pairs on
1. [Creating SSH key pairs on Windows]({{< relref "creating_ssh_key_pairs_on_windows" >}})
Windows](Creating-SSH-key-pairs-on-Windows_13598839.html)
2. [Creating SSH key pairs on Mac]({{< relref "creating_ssh_key_pairs_on_mac" >}})
2. [Creating SSH key pairs on Mac
](Creating-SSH-key-pairs-on-Mac_13598841.html)
3. **Connect to the Anvil VPN**
**
**
3. **Connect to the Anvil VPN**
The Anvil web portal is accessible from the Internet. On the other
The Anvil web portal is accessible from the Internet. On the other
hand, for security reasons, the Anvil instances are not generally
hand, for security reasons, the Anvil instances are not generally
accessible from the Internet. In order to access the instances from
accessible from the Internet. In order to access the instances from
on and off-campus, you will need to first be connected to the Anvil
on and off-campus, you will need to first be connected to the Anvil
VPN. Follow the instructions below to connect.
VPN. Follow the instructions below to connect.
1. [Connecting to the Anvil VPN
1. [Connecting to the Anvil VPN]({{< relref "connecting_to_the_anvil_vpn" >}})
](Connecting-to-the-Anvil-VPN_13042829.html)
**
4. **Add the SSH Key Pair to your account**
**
4. **Add the SSH Key Pair to your account**
Before creating your first instance, you'll need to associate the
Before creating your first instance, you'll need to associate the
SSH key created in step 2 with your account. Follow the guide
SSH key created in step 2 with your account. Follow the guide
below to login to the web dashboard and add the key pair.
below to login to the web dashboard and add the key pair.
1. [Adding SSH Key Pairs](Adding-SSH-Key-Pairs_13599025.html)**
1. [Adding SSH Key Pairs]({{< relref "adding_ssh_key_pairs" >}})
**
**
5. **Create an instance**
**
Once the setup steps above are completed, you can create an
5. **Create an instance
**Once the setup steps above are completed, you can create an
instance within the web dashboard. Follow the guide below to create
instance within the web dashboard. Follow the guide below to create
an instance.
an instance.
1. [Creating an Instance](Creating-an-Instance_13599047.html)
1. [Creating an Instance]({{< relref "creating_an_instance" >}})
**
**
6. **Connect to your instance**
6. **Connect to your instance**
After an instance has been created, you can connect (login) and
After an instance has been created, you can connect (login) and
begin to use it. Connecting is done via SSH or X2Go for Linux
begin to use it. Connecting is done via SSH or X2Go for Linux
instances and via Remote Desktop (RDP) for Windows instances.
instances and via Remote Desktop (RDP) for Windows instances.
Follow the relevant guide below for your instance and the type of
Follow the relevant guide below for your instance and the type of
OS you're connecting from.
OS you're connecting from.
1. [Connecting to Windows
1. [Connecting to Windows Instances]({{< relref "connecting_to_windows_instances" >}})
Instances](Connecting-to-Windows-Instances_13042841.html)
2. [Connecting to Linux Instances via SSH from Mac]({{< relref "connecting_to_linux_instances_from_mac" >}})
2. [Connecting to Linux Instances via SSH from
3. [Connecting to Linux instances via SSH from Windows]({{< relref "connecting_to_linux_instances_from_windows" >}})
Mac](Connecting-to-Linux-Instances-from-Mac_13599097.html)
4. [Connecting to Linux instances using X2Go (for images with Xfce)]({{< relref "connecting_to_linux_instances_using_x2go" >}})
3. [Connecting to Linux instances via SSH from
Windows](Connecting-to-Linux-instances-from-Windows_13599107.html)
7. **Create and attach a volume to your instance (optional)**
4. [Connecting to Linux instances using X2Go (for images with
Xfce)](Connecting-to-Linux-instances-using-X2Go_16520062.html)
**
**
7. **Create and attach a volume to your instance (optional)**
Volumes are a means within OpenStack for persistent storage. When
Volumes are a means within OpenStack for persistent storage. When
an instance is destroyed, all data that was placed on the OS hard
an instance is destroyed, all data that was placed on the OS hard
drive is lost. A volume can be thought of similar to an external
drive is lost. A volume can be thought of similar to an external
@@ -187,29 +157,26 @@ precious or irreproducible data should not be placed or left on Anvil**.
@@ -187,29 +157,26 @@ precious or irreproducible data should not be placed or left on Anvil**.
Windows) of instance it will be attached to. Once the volume is
Windows) of instance it will be attached to. Once the volume is
attached, follow the corresponding guide for your instance's OS to
attached, follow the corresponding guide for your instance's OS to
format and make the volume usable within your instance.
format and make the volume usable within your instance.
1. [Creating and attaching a
1. [Creating and attaching a volume]({{< relref "creating_and_attaching_a_volume" >}})
volume](Creating-and-attaching-a-volume_13599293.html)
2. [Formatting and mounting a volume in Windows]({{< relref "formatting_and_mounting_a_volume_in_windows" >}})
2. [Formatting and mounting a volume in
3. [Formatting and mounting a volume in Linux]({{< relref "formatting_and_mounting_a_volume_in_linux" >}})
Windows](Formatting-and-mounting-a-volume-in-Windows_13599295.html)
3. [Formatting and mounting a volume in Linux
8. **Transferring files to or from your instance (optional)**
](Formatting-and-mounting-a-volume-in-Linux_13599298.html)
Transferring files to or from an instance is similar to doing so
8. **Transferring files to or from your instance (optional)
**Transferring files to or from an instance is similar to doing so
with a personal laptop or workstation. To transfer between an
with a personal laptop or workstation. To transfer between an
instance and another HCC resource, both SCP and [Globus
instance and another HCC resource, both SCP and [Globus
Connect](Globus-Connect_6357013.html) can be used. For transferring
Connect]({{< relref "guides/handling_data/globus_connect" >}}) can be used. For transferring
between an instance and a laptop/workstation or another instance,
between an instance and a laptop/workstation or another instance,
standard file sharing utilities such as Dropbox or Box can be used.
standard file sharing utilities such as Dropbox or Box can be used.
Globus may also be used, with one stipulation. In order to
Globus may also be used, with one stipulation. In order to
transfer files between two personal endpoints, a Globus Plus
transfer files between two personal endpoints, a Globus Plus
subscription is required. As part of HCC's Globus Provider Plan,
subscription is required. As part of HCC's Globus Provider Plan,
HCC can provide this on a per-user basis free of charge. If you are
HCC can provide this on a per-user basis free of charge. If you are
interested in Globus Plus, please
interested in Globus Plus, please email
email <a href="mailto:hcc-support@unl.edu" class="external-link">hcc-support@unl.edu</a> with
{{< icon name="envelope" >}}[hcc-support@unl.edu] (mailto:hcc-support@unl.edu)
your request and a brief explanation.
with your request and a brief explanation.
Backups
## Backups
-------
HCC creates daily backups of images and volume snapshots for disaster
HCC creates daily backups of images and volume snapshots for disaster
recovery. All users' images, detached volumes, and volume snapshots will
recovery. All users' images, detached volumes, and volume snapshots will
@@ -224,7 +191,6 @@ is unable to restore single files within instances.  Further, HCC's
@@ -224,7 +191,6 @@ is unable to restore single files within instances.  Further, HCC's
disaster recovery backups should not be the only source of backups for
disaster recovery backups should not be the only source of backups for
important data. The backup policies are subject to change without prior
important data. The backup policies are subject to change without prior
notice. To retrieve your backups, please contact HCC. If you have
notice. To retrieve your backups, please contact HCC. If you have
special concerns please
special concerns please contact us at
<a href="mailto:hcc-support@unl.edu" class="external-link">contact us</a>.
{{< icon name="envelope" >}}[hcc-support@unl.edu] (mailto:hcc-support@unl.edu).