Introduction
The Cloud Computing service of the SCIGNE platform offers the user the possiblity to start virtual machines (VM) easily. This virtual machines can be used independently to provide single computing server, or orchestrated to create an advanced computing infrastructure (like a computing or a kubernetes cluster). Such Cloud service is called Infrastructure-as-a-Service and permits the user to choose the operating system, the size of the server (number of cores, memory size, etc). It is based on the OpenStack software.
Thanks to all available modules (authentication, image management, storage management, container as a service, etc), OpenStack makes possible to run a complete Cloud infrastructures and to provide a large range of services to the users.
This documentation details the use of the OpenStack command-line client (CLI), named openstack. This tools interacts with the OpenStack modules through their public API and permits to use efficiently the SCIGNE Cloud Computing service. However, for user looking for a more user-friendly interface, a web-based dashboard (Horizon). is available. As its use is very intuitive, it is not covered by the following documentation.
The team managing the SCIGNE platform is available for helping you to use this service. Please, feel free to contact us if you need any further information or if you want to have a training organised on this topic!
Installation
This section details the installation process of the OpenStack command-line client.
The OpenStack client is available for several operating systems. It can be installed using the Operating System package manager, like apt and yum, or by using PIP. In any case, a recent version of Python is required. Note that you should not mix a PIP-based installation with a package-based installation, as it may lead to compatibility issues. We are recommending to use package-based installation, when it is feasible. The different installation modes are explained in the following chapters.
Installation Based on Operating System Packages
Debian and Ubuntu Systems
The version of the OpenStack client provided by default on Ubuntu may be old, compared to the OpenStack version used on the SCIGNE platform. In order to get access to all features, it is recommended to use a recent version of the client. Such versions are made available through the cloud-archive package repository. For installing a specific version of the client (i.e. Train as in this example), you should use the following commands as super-user:
$ sudo add-apt-repository cloud-archive:train
$ sudo apt-get update
$ sudo apt-get install python-openstackclientInstallation on RedHat Entreprise Linux
The installation processes on both RedHat Enterprise Linux and RedHat-like systems (CentOS, RockyLinux, etc) are different. This section details the installation on RedHat system with yum. It is done by completing the following steps. The software is only available for version 7 and 8 of RedHat. If you want to install the clients on an earlier version, it is necessary to follow the PIP installation method.
$ sudo yum install -y https://www.rdoproject.org/repos/openstack/openstack-train/rdo-release-train-1.noarch.rpm
$ sudo yum update -y
$ sudo yum install -y python-openstackclientInstallation on CentOS
The installation by using yum on CentOS system are realised with the following steps. The client is available for the CentOS 7 (up to OpenStack Train version) and CentOS 8 (starting with OpenStack Train version). If you want to use recent OpenStack version with CentOS 7, you should follow the PIP-based installation process.
$ sudo yum install -y centos-release-openstack-train
$ sudo yum update -y
$ sudo yum install -y python-openstackclientThe centos-release-openstack-train package is provided by the extras repository, enabled by default on CentOS 7.
Installation Based on PIP
First, it is required to verify that PIP is correctly installed on your system. The installation of PIP is detailed on the PIP site. The OpenStack client is installed with the following command line :
$ pip install python-openstackclientVerification of the installation
Execute the openstack command with the --version argument to verify that the client is correctly installed:
$ openstack --version
openstack 3.14.2Please note that the version returned by the openstack command can be different with the version displayed in this example.
Definition of the Environment Variables
Once the installation is done, you are nearly ready to use the SCIGNE Cloud service. The last step is to configure the client. For doing this, create the ${HOME}/.novacreds/iphc.sh file with the following content (username, password and FG_Cloud values have to be replaced with the values you received from the SCIGNE team when your account has been created):
export LANG=en_US.utf-8
export LC_ALL=en_US.utf-8
export OS_USERNAME=username
export OS_PASSWORD=password
export OS_PROJECT_NAME=FG_Cloud
export OS_USER_DOMAIN_NAME=default
export OS_PROJECT_DOMAIN_NAME=default
export OS_AUTH_URL=https://sbgcloud.in2p3.fr:5000/v3
export OS_IDENTITY_API_VERSION=3Once the file is created, it has to be sourced for loading the environment variables within your actuel shell:
$ source ${HOME}/.novacreds/iphc.shIt is necessary to perform this step each time you are running a new shell, unless you are adding the source command into you shell initialisation script (i.e., the ${HOME}/.bashrc file for bash).
You can now test your client againt the SCIGNE Cloud service:
$ openstack server list -c ID -c Name -c Status -c Image
+--------------------------------------+-------+--------+----------------------+
| ID | Name | Status | Image Name |
+--------------------------------------+-------+--------+----------------------+
| 36dd1e32-2a79-4e78-a4e1-999335de3cbd | MY_VM | ACTIVE | Image for Ubuntu ... |
+--------------------------------------+-------+--------+----------------------+The above command displays the list of virtual machines running in the project defined by the OS_PROJECT_NAME variable at the time you run it. Thus, the list displayed will differ from the one used in this example.
Password Modification
Before using the service, it is required to modify your password for security reasons. This is done with the following command:
$ openstack user password set
Current Password:
New Password:
Repeat New Password:
You should update the password you are using to authenticate to match your new passwordOnce this operation has been performed, the new password has been updated on the server side, but not on the client side:
$ openstack server list
ERROR: Invalid OpenStack Nova credentials.You have to update the corresponding environment variable:
$ export OS_PASSWORD=MY_NEW_PASSWORD
$ openstack server list
+--------------------------------------+-------+--------+---------------------------+------------------------------+
| ID | Name | Status | Networks | Image Name |
+--------------------------------------+-------+--------+---------------------------+------------------------------+
| 36dd1e32-2a79-4e78-a4e1-999335de3cbd | MY_VM | ACTIVE | fg-default-net=172.16.19.226 | Image for Ubuntu Server 2... |
+--------------------------------------+-------+--------+---------------------------+------------------------------+If the command is successfull, then you can update the ${HOME}/.novacreds/iphc.sh file with your new password.
Once the password has been updated in the ${HOME}/.novacreds/iphc.sh file, it is necessary to source this file again to update the environment variable value.
The following interactive help is available:
$ openstack help user set
Change current user password
optional arguments:
-h, --help show this help message and exit
--password <new-password>
New user password
--original-password <original-password>
Original user passwordSSH Key Management
To be able to connect to your virtal machine, it is required to have a SSH key and to register it against the OpenStack service:
$ ssh-keygen -t rsa -f ${HOME}/.novacreds/cloudkey
[...]
$ openstack keypair create --public-key=${HOME}/.novacreds/cloudkey.pub cloudkey
$ openstack keypair list
+----------+-------------------------------------------------+
| Name | Fingerprint |
+----------+-------------------------------------------------+
| cloudkey | 33:91:4b:38:52:63:1d:18:7b:e0:e3:0d:3b:63:ce:4b |
+----------+-------------------------------------------------+Virtual Machine Management
This part describes the management of virtual machines.
OpenStack Environment Discovery
A few commands permit your to see all available elements available to build your virtual machine. First, the list of operation system images can be obtained with:
$ openstack image list
+--------------------------------------+------------------------------------------------------+--------+
| ID | Name | Status |
+--------------------------------------+------------------------------------------------------+--------+
| 6368a92f-4fab-4c8b-93a8-24b80d3eccf9 | Image for CentOS Server 7 [CentOS/7/KVM] | active |
| badb9c02-779a-4da8-9c5a-9c59b6284362 | Image for CentOS Server 8 [CentOS/8/KVM] | active |
| bcac8255-a321-4349-82e6-6918b34e1df1 | Image for RancherOS [Other/1.5.6/KVM] | active |
| a8a2b0d1-ffed-4315-be00-fc14828a5203 | Image for Ubuntu Server 18.04 LTS [Ubuntu/18.04/KVM] | active |
| 979d05cb-4164-4f0b-91da-c70608285401 | Image for Ubuntu Server 20.04 LTS [Ubuntu/20.0] | active |
| d55e54e0-1d9b-494d-a529-a4a40c60eb23 | ubuntu-18.04-docker-x86_64 | active |
+--------------------------------------+------------------------------------------------------+--------+As the list of the available images is updated regularly, it is very likely that you will get a slightly different result.
Then, the list of available flavor (size of servers) is displayed with:
$ openstack flavor list
+--------------------------------------+-------------------+--------+------+-----------+-------+-----------+
| ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public |
+--------------------------------------+-------------------+--------+------+-----------+-------+-----------+
| 00b54b02-63d0-4d15-927d-6e2f5a4d4920 | g4.xlarge-4xmem | 65536 | 40 | 0 | 8 | True |
| 048fc66b-d60a-4d07-b1e4-1fae3e6ab395 | m1.small-mmem | 4096 | 20 | 0 | 1 | True |
| 1ff24edd-1283-469c-abc1-7f697873d63c | m1.2xlarge | 32768 | 20 | 0 | 16 | True |
| 2 | m1.small | 2048 | 20 | 0 | 1 | True |
| 242a578e-b314-4f33-83ea-b05f50f08960 | g2.xlarge-4xmem | 65536 | 40 | 0 | 8 | True |
| 588f4548-b8e7-48fd-8ee5-357662a4c08f | m1.2xlarge-8xmem | 131072 | 20 | 0 | 16 | True |
| 5f0a3206-5f25-4341-beba-7d0a96cdb108 | m1.8xlarge-16xmem | 262144 | 20 | 0 | 64 | True |
| 6cdf0bde-85c6-4e92-b7ab-f80c75a695b7 | m1.xlarge | 16384 | 20 | 0 | 8 | True |
| 7005975e-735f-4b08-a69d-2ce1a771be85 | m1.8xlarge | 131072 | 20 | 0 | 64 | True |
| 73730dea-ca08-47fb-ac0b-2ebd6dbe1465 | g1.xlarge-4xmem | 65536 | 40 | 0 | 8 | True |
| 895f886c-85dd-4924-9a35-4177655559c5 | m1.large-2d | 8192 | 40 | 0 | 4 | True |
| ab1fbd4c-324d-4155-bd0f-72f077f0ebce | m1.medium | 4096 | 20 | 0 | 2 | True |
| ab9784d7-e785-4f4e-a415-1d188505b5eb | m1.large | 8192 | 20 | 0 | 4 | True |
| aca4fac9-4761-4735-88b8-4102d6f4ecb3 | m1.2xlarge-4xmem | 65536 | 20 | 0 | 16 | True |
| cf7f0d1f-8ecd-4b19-b6bb-f2f89cf55bd0 | m1.4xlarge | 65536 | 20 | 0 | 32 | True |
| d6aae2a5-8f5d-40dd-b6f3-2e8d9fb7efa1 | m1.xlarge-8xmem | 131072 | 20 | 0 | 8 | True |
+--------------------------------------+-------------------+--------+------+-----------+-------+-----------+At least, to know the networks usable by the virtual machine, use :
$ openstack network list
+--------------------------------------+----------------+--------------------------------------+
| ID | Name | Subnets |
+--------------------------------------+----------------+--------------------------------------+
| 3877b236-dbca-4541-a007-7444d38acabc | fg-default-net | c8ba2397-a874-47e9-8f54-0ed433fa2240 |
| 8fa5eb12-ee72-4e77-a45b-8fde11efc75f | ext-net | 6e8ffc7a-4536-4c21-a8e5-00227ee8619a |
+--------------------------------------+----------------+--------------------------------------+In the previous table, two networks are available: fg-default-net and ext-net. In fact, only the fg-default-net network is usable directly at boot time by the virtual machine. The other network is used afterwards to provide public IP addresses.
Virtual Machine Creation
In the previous section, we have retrieved the list of all the elements that can be used to compose the virtual machine. Once you have chosen the different elements of your virtual machine, you can instantiate it using the openstack server create command. For example, if we want to run an Ubuntu image with 1 cpu, 2 GB RAM and 20 GB hard disk on the fg-default-net network and whose name will be MY_VM_NAME, we would use the following command use the following command:
$ openstack server create --key-name cloudkey --image 979d05cb-4164-4f0b-91da-c70608285401 \
--flavor m1.small --nic net-id=3877b236-dbca-4541-a007-7444d38acabc MY_VM_NAME
+-----------------------------+----------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------------+----------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | W6jJUvpRhAyU |
| config_drive | |
| created | 2021-11-24T06:24:39Z |
| flavor | m1.small (2) |
| hostId | |
| id | 367110ac-b05a-4855-884e-e3b00717ab4a |
| image | Image for Ubuntu Server 20.04 LTS [Ubuntu/20.0] (979d05cb-4164-4f0b-91da-c70608285401) |
| key_name | cloudkey |
| name | MY_VM_NAME |
| progress | 0 |
| project_id | 92d2f61945214c8a80d12d696b82426e |
| properties | |
| security_groups | name='default' |
| status | BUILD |
| updated | 2021-11-24T06:24:39Z |
| user_id | d04e19e72801e8d9d802a6178a7d5a006e3cc9383c35e82a769fc6686782b667 |
| volumes_attached | |
+-----------------------------+----------------------------------------------------------------------------------------+When launching virtual machines, you may find yourself issues with quota overruns:
$ openstack server create [...] MY_VM_NAME
ERROR: Quota exceeded for cores: Requested 1, but already used 22 of 22 cores
(HTTP 413) (Request-ID: req-6aefedba-4666-4393-b6a1-24423f3bef78)In this example, the error tells you that all available processors for your group (project) are currently occupied. You can either wait for other users to free up some, or ask the SCIGNE team to allocate you additional quota.
You can list the quota limits with the following command:
$ openstack quota show
+-----------------------+----------------------------------+
| Field | Value |
+-----------------------+----------------------------------+
| backup-gigabytes | 1000 |
| backups | 10 |
| cores | 256 |
| fixed-ips | -1 |
| floating-ips | 64 |
| gigabytes | 8192 |
| gigabytes___DEFAULT__ | -1 |
| gigabytes_ceph | -1 |
| groups | 10 |
| health_monitors | None |
| injected-file-size | 10240 |
| injected-files | 5 |
| injected-path-size | 255 |
| instances | 64 |
| key-pairs | 100 |
| l7_policies | None |
| listeners | None |
| load_balancers | None |
| location | None |
| name | None |
| networks | 100 |
| per-volume-gigabytes | -1 |
| pools | None |
| ports | 500 |
| project | 92d2f61945214c8a80d12d696b82426e |
| project_name | FG_Cloud |
| properties | 128 |
| ram | 1048576 |
| rbac_policies | 10 |
| routers | 10 |
| secgroup-rules | 256 |
| secgroups | 32 |
| server-group-members | 10 |
| server-groups | 32 |
| snapshots | 10 |
| snapshots___DEFAULT__ | -1 |
| snapshots_ceph | -1 |
| subnet_pools | -1 |
| subnets | 100 |
| volumes | 64 |
| volumes___DEFAULT__ | -1 |
| volumes_ceph | -1 |
+-----------------------+----------------------------------+Online help is available and describes all usable parameters:
$ openstack help server create
usage: openstack server create [-h] [-f {json,shell,table,value,yaml}]
[-c COLUMN] [--max-width <integer>]
[--fit-width] [--print-empty] [--noindent]
[--prefix PREFIX]
(--image <image> | --volume <volume>) --flavor
<flavor> [--security-group <security-group>]
[--key-name <key-name>]
[--property <key=value>]
[--file <dest-filename=source-filename>]
[--user-data <user-data>]
[--availability-zone <zone-name>]
[--block-device-mapping <dev-name=mapping>]
[--nic <net-id=net-uuid,v4-fixed-ip=ip-addr,v6-fixed-ip=ip-addr,port-id=port-uuid,auto,none>]
[--network <network>] [--port <port>]
[--hint <key=value>]
[--config-drive <config-drive-volume>|True]
[--min <count>] [--max <count>] [--wait]
<server-name>
Create a new server
[...]When using the openstack command, it is possible to use names (e.g. MY_VM_NAME) as well as identifiers (e.g. 36dd1e32-2a79-4e78-a4e1-999335de3cbd). It is recommended to use the IDs, as they are unique (it is possible to run two virtual machines with the same name).
With the following command, it is possible to monitor the status of the VM:
$ openstack server show MY_VM_NAME
+-----------------------------+----------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------------+----------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-STS:power_state | Running |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2021-11-24T06:26:47.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | fg-default-net=172.16.19.226 |
| config_drive | |
| created | 2021-11-24T06:24:39Z |
| flavor | m1.small (2) |
| hostId | cf41a7ccb0611d6dcc24e54c5983dbe17dda36cf954f5e4625607474 |
| id | 367110ac-b05a-4855-884e-e3b00717ab4a |
| image | Image for Ubuntu Server 20.04 LTS [Ubuntu/20.0] (979d05cb-4164-4f0b-91da-c70608285401) |
| key_name | cloudkey |
| name | MY_VM_NAME |
| progress | 0 |
| project_id | 92d2f61945214c8a80d12d696b82426e |
| properties | |
| security_groups | name='default' |
| status | ACTIVE |
| updated | 2021-11-24T06:25:49Z |
| user_id | d04e19e72801e8d9d802a6178a7d5a006e3cc9383c35e82a769fc6686782b667 |
| volumes_attached | |
+-----------------------------+----------------------------------------------------------------------------------------+The ACTIVE status tells us that the VM is ready to be used. However, it doesn’t have an interface to the external network yet, since its IP address is in the internal network. Before being able to connect to it with SSH, it is necessary to attach a public IP address to it. This IP is visible and accessible from the internet. This is detailed in the Network Management chapter.
Customizing Virtual Machines
You can customize your virtual machine during their creation by using the --user-data parameter with the openstack server create command. This parameter references a file containing instructions for the cloud-init software that is present in the virtual machine.
The following example upgrades the system and installs additional packages packages, useful for encrypting a volume and the security.
Create a file cloud_init_cfg.txt containing:
#cloud-config
# Upgrade the instance OS packages on first boot
package_upgrade: true
# Add packages for entropy gathering and encrypted volume management
packages:
- cryptsetup
- fail2ban
- havegedThe first ‘#’ character must be present in the file as it is, it tells the cloud-init software the format of the contents of the cloud_init_cfg.txt file.
And use it when creating your virtual machine:
$ openstack server create [...] --user-data cloud_init_cfg.txt MY_VM_NAMEWarning: the execution of the cloud-init software is not instantaneous. If you request a full operating system update, it may take several minutes to complete, even though you can already connect to the VM with ssh. To check that everything went well, you can check the contents of the /var/log/cloud-init.log and /var/log/cloud-init-output.log files on the virtual machine.
The cloud-init software is very sensitive to the file format of the cloud_init_cfg.txt file: spaces, tabs and other formatting elements are very important. In addition to referring to the documentation indicated below, you can use the yamlint tool to check the format. However, it is more picky than cloud-init, so use its remarks only for information. Build your contextualisation script a little at a time, checking at each step that what you have added is working correctly.
Look at the cloud-init documentation to get more information about the configuration options of this tool.
Network Management
To be able to connect to your VM, you have to add a public IP address. This section details how to add such address to you VM.
First, you need to check if a public IP address is available:
$ openstack floating ip list -c ID -c "Floating IP Address" -c "Fixed IP Address"
+--------------------------------------+---------------------+------------------+
| ID | Floating IP Address | Fixed IP Address |
+--------------------------------------+---------------------+------------------+
| 03a25383-a3ee-42d9-8dc2-7184257f55c3 | 134.158.151.32 | 172.16.19.133 |
| 07514ea0-8b5c-4eb4-83b1-791c848ae29d | 134.158.151.205 | 172.16.19.43 |
| 09918b58-9ae3-4e9f-a759-7929ab3f78f6 | 134.158.151.110 | 172.16.19.26 |
| 106cdc30-1aba-4ac0-b9ec-373321488487 | 134.158.151.142 | None |
...
+--------------------------------------+---------------------+------------------+In this list, three IP adresses are already associated with a private IP and an IP address is available (the Fixed IP Address field contains the None value). If all IP addresses are used, then it is necessary to create a new:
$ openstack floating ip create ext-net
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| description | |
| fixed_ip_address | None |
| floating_ip_address | 134.158.151.224 |
| floating_network_id | 314be651-fcaa-4dcc-ac0a-dfdcd7e58ba1 |
| headers | |
| id | f60c77d9-bae4-422d-83c3-16d155bac396 |
| port_id | None |
| project_id | 59402a7327114c2cb97f0d9e6263cdda |
| router_id | None |
| status | DOWN |
+---------------------+--------------------------------------+Once the IP address is available, it can be attached to the VM with the following command:
$ openstack server add floating ip MY_VM_NAME 134.158.151.224It is possible to verify that the IP address is correctly associated to the virtual machine:
$ openstack server list --name MY_VM_NAME -c ID -c Name -c Networks
+--------------------------------------+------------+-----------------------------------------------+
| ID | Name | Networks |
+--------------------------------------+------------+-----------------------------------------------+
| 91243eef-9168-4250-b4d8-466f02797823 | MY_VM_NAME | fg-default-net=172.16.19.226, 134.158.151.224 |
+--------------------------------------+------------+-----------------------------------------------+Now that the virtual machine has a public IP address, you can check the network connectivity with the ping command:
$ ping -c 3 134.158.151.224If it works fine, then you can connect to it with ssh:
$ ssh -i ${HOME}/.novacreds/cloudkey centos@134.158.151.224If the connection to the virtual machine fails, look at the SSH Error section.
The default username used to connect to Ubuntu and CentOS systems are ubuntu and centos respectively.
Storage Management
By default, when a virtual machine is deleted, all changes you have applied to it are lost. If you want to store data, like software or scientific data, that are reusable between two following virtual machines, you have to create a persistent storage. This type of storage is like an external drive that you plug to your VM either at boot time or later. To display the list of persistent storage, use:
$ openstack volume list
+--------------------------------------+---------------+-----------+------+--------------------------------------------+
| ID | Display Name | Status | Size | Attached to |
+--------------------------------------+---------------+-----------+------+--------------------------------------------+
| d874ed50-f4dd-11e6-b99e-8f36e05a44c6 | MONGODB | in-use | 100 | Attached to DATA_ANALYSIS on /dev/vdd |
| e3469d32-f4dd-11e6-b9be-3b36b57652e5 | DATA_1 | in-use | 200 | Attached to DATA_ANALYSIS on /dev/vdc |
| ec8de7c4-f4dd-11e6-a96b-7f9e7042f1e8 | chemistry-vol | available | 200 | |
| 0243cb42-f4de-11e6-9d72-b73ff56f8215 | VM_IPHC_START | available | 10 | |
| 09c16c44-f4de-11e6-8b45-433e4aa51e4c | SIMDAT_VOLUME | available | 8 | |
| 0e48a656-f4de-11e6-878b-eb0c0efe4cca | vmdata_disk_2 | available | 250 | |
| 13c86eae-f4de-11e6-b316-1f8b7ed3376e | DATA_DISK | in-use | 50 | Attached to DATA_ANALYSIS on /dev/vdb |
| 179a0326-f4de-11e6-920c-c7f2b28ed02b | R_Data | in-use | 8 | Attached to vm_iphc_r_analysis on /dev/vdb |
+--------------------------------------+---------------+-----------+------+--------------------------------------------+For creating a persistent storage with a size of 8 GB and named MY_VOLUME_NAME, use the following command:
$ openstack volume create --size 8 MY_VOLUME_NAME
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2017-02-16T10:08:02.493311 |
| description | None |
| encrypted | False |
| id | 58230433-086f-4ff2-99ac-8c3e0f5520c2 |
| multiattach | False |
| name | MY_VOLUME_NAME |
| properties | |
| replication_status | disabled |
| size | 8 |
| snapshot_id | None |
| source_volid | None |
| status | creating |
| type | None |
| updated_at | None |
| user_id | c13c93bf1b79447c8cd1464128d7685f |
+---------------------+--------------------------------------+Then, attach this volume to your virtual machine by using the identifier of the storage:
$ openstack server add volume --device /dev/vdb MY_VM_NAME 58230433-086f-4ff2-99ac-8c3e0f5520c2The storage will be shown to the operating system as /dev/vdb. To verifiy that the disk is correctly associated with the VM, check the content of the Status column with the following command. It the Status column has for value in-use and the Attached to column contains the identified of the VM, then it is fine.
$ openstack volume list --name MY_VOLUME_NAME
+--------------------------------------+----------------+--------+------+------------------------------------+
| ID | Display Name | Status | Size | Attached to |
+--------------------------------------+----------------+--------+------+------------------------------------+
| 58230433-086f-4ff2-99ac-8c3e0f5520c2 | MY_VOLUME_NAME | in-use | 8 | Attached to MY_VM_NAME on /dev/vdb |
+--------------------------------------+-------------------------+------+------------------------------------+At the begin, this disk does not contain any data and is not formatted. If this disk will store confidential data, it is recommended to encrypt the volume. This operation is detailed in the Disk Encryption chapter. Otherwise, connect to your VM, format the volume and mount it:
[centos@my-vm-name ~]$ sudo su
[root@my-vm-name centos]# mkfs -t ext4 /dev/vdb
[root@my-vm-name centos]# mkdir /storage1
[root@my-vm-name centos]# mount /dev/vdb /storage1
[root@my-vm-name centos]# df -h /storage1
Filesystem Size Used Avail Use% Mounted on
/dev/vdb 8.0G 3.0M 7.8G 1% /storage1The last command permits us to verify that the size of the disk corresponds to the one we have chosen (8 GB in this example) and that the disk is correctly mounted on /storage1.
The virtual disk can also be partitioned before being formated.
This disk can be moved from one virtual machine to another. For doing this, you have to umount it correctly:
[root@my-vm-name centos]# umount /storage1If you have added the reference of your persistent disk in the /etc/fstab file of your VM, and you have move the disk to another VM, do not forget to remove any reference to this disk in the /etc/fstab file, otherwise the virtual machine will not boot (it will be looking for a disk that is not available anymore).
The disk can be removed from the VM with the following command:
$ openstack server remove volume MY_VM_NAME 58230433-086f-4ff2-99ac-8c3e0f5520c2
$ openstack volume list --name MY_VOLUME_NAME
+--------------------------------------+----------------+-----------+------+-------------+
| ID | Display Name | Status | Size | Attached to |
+--------------------------------------+----------------+-----------+------+-------------+
| 58230433-086f-4ff2-99ac-8c3e0f5520c2 | MY_VOLUME_NAME | available | 8 | |
+--------------------------------------+----------------------------+------+-------------+When the virtual machine is rebooted, the disk is not automatically mounted to the filesystem. To mount the disk automatically, a reference to it must be added to the /etc/fstab file.
Virtual Machine Deletion
If the virtual machine is no longer required, it can be deleted. Deleting the virtual machine permits to free up the resources (as long it is not deleted, it will be accounted in the usage statistics and in the quota). In this case, all the modifications that have been made to the image (package installation, etc) will be lost, except for those that are on the persistent disk. Before deleting the VM, you have to connect to the VM and unmount the disk (to avoid data corruption):
[root@my-vm-name centos]# umount /storage1If you do not need anymore the data on the persistent storage, you may initialise it with random data (this task can be quite long), detache it from the server and delete it:
[root@my-vm-name centos]# dd if=/dev/urandom of=/dev/vdb bs=4k
$ openstack server remove volume MY_VM_NAME 58230433-086f-4ff2-99ac-8c3e0f5520c2
$ openstack volume delete 58230433-086f-4ff2-99ac-8c3e0f5520c2At least, the following command can be run to delete effectively the VM:
$ openstack server delete MY_VM_NAMEAdvanced Usage
This chapter describes several possibilities to get further in the usage of the SCIGNE Cloud service.
Time synchronisation
The access to NTP servers outside the SCIGNE platform is not permitted. If you are using time synchronisation tools, you need to configure them with our NTP servers:
- sbgntp1.in2p3.fr (193.48.89.16)
- sbgntp2.in2p3.fr (193.48.89.17)
ntpd
If you are using ntpd for time synchronisation, you need to have the following lines in your /etc/ntp.conf configuration file:
server 193.48.89.16 restrict 193.48.89.16 mask 255.255.255.255 nomodify notrap noquery server 193.48.89.17 restrict 193.48.89.17 mask 255.255.255.255 nomodify notrap noquery
chronyd
If you are using chronyd for time synchronisation, you need to have the following lines in your /etc/chrony.conf configuration file:
server 193.48.89.16 server 193.48.89.17
Disk Encryption
This section details the use of the dm-crypt/LUKS tool for the encryption of permanent disks. This tools is provided as standard by Linux distributions and can easily be installed with the package manager of your virtual machine. The example is based on using an Ubuntu distribution, but can easily be reproduced with a RedHat-based operating system.
ubuntu@my-vm-name:~$ sudo su -
sudo: unable to resolve host my-vm-name
root@my-vm-name:~# apt -y update
root@my-vm-name:~# apt -y install cryptsetupTo encrypt a permanent disk, it must first be properly initialized. In the example below, the disk named /dev/vdb is first filled with random data, then it is initialized using the cryptsetup luksFormat command. This first step can be quite long.
root@my-vm-name:~# dd if=/dev/urandom of=/dev/vdb bs=4k
root@my-vm-name:~# cryptsetup -v --cipher aes-xts-plain64 --key-size 512 \
--hash sha512 --iter-time 5000 --use-random luksFormat /dev/vdbIf the latter command slows down or even freezes with the following message:
System is out of entropy while generating volume key.
Please move mouse or type some text in another window to gather some random events.
Generating key (0% done).Then you can make the cryptsetup […] luksFormat command faster by installing the haveged program in your virtual machine (which is already done if you followed the procedure described in the Customizing Virtual Machines chapter).
The following command verifies that the disk is now of type LUKS:
root@my-vm-name:~# cryptsetup luksDump /dev/vdb
LUKS header information for /dev/vdb
Version: 1
Cipher name: aes
Cipher mode: xts-plain64
Hash spec: sha512
Payload offset: 4096
MK bits: 512
MK digest: c4 f7 4b 02 2a 3f 12 c1 2c ba e5 c9 d2 45 9a cd 89 20 6c 73
MK salt: 98 58 3e f3 f6 88 99 ea 2a f3 cf 71 a0 0d e5 8b
d5 76 64 cb d2 5c 9b d1 8a d3 1d 18 0e 04 7a eb
MK iterations: 81250
UUID: c216d954-199e-4eab-a167-a3587bd41cb3
Key Slot 0: ENABLED
Iterations: 323227
Salt: a0 45 3e 98 fa cf 60 74 c6 09 3d 54 97 89 be 65
5b 96 7c 1c 39 26 47 b4 8b 0e c1 3a c9 94 83 c2
Key material offset: 8
AF stripes: 4000
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: DISABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLEDThe disc is now ready for use. The first time you use it, the following steps should be taken:
- Open the encrypted disk with the cryptsetup luksOpen command. The name storage1 is only indicative, you can choose what you want:
root@my-vm-name:~# cryptsetup luksOpen /dev/vdb storage1 - Create a filesystem on the disk:
root@my-vm-name:~# mkfs.ext4 /dev/mapper/storage1 - Create a mount point for the disk:
root@my-vm-name:~# mkdir /storage1 - Mount the disk:
root@my-vm-name:~# mount -t ext4 /dev/mapper/storage1 /storage1 - Verify the available space (which can be slightly different with the value given to the openstack volume create):
root@my-vm-name:~# df -h /storage1 Filesystem Size Used Avail Use% Mounted on /dev/mapper/storage1 2.0G 6.0M 1.9G 1% /storage1
Once the disk is operational, steps 2 and 3 are no longer necessary.
You can now send files (e.g. DATA.dat) from your home computer to your virtual machine securely, for example with scp:
$ scp -i ${HOME}/.novacreds/cloudkey DONNEES.dat ubuntu@134.158.151.224:/le/chemin/destination
DONNEES.dat 100% 82 0.1KB/s 00:00When you have finished working on the disk, you can remove it cleanly with the following commands:
root@my-vm-name:~# umount /storage1
root@my-vm-name:~# cryptsetup close storage1For the next uses of this persistent virtual disk, there will be no need to perform all these operations, only do the following:
root@my-vm-name:~# cryptsetup luksOpen /dev/vdb storage1
root@my-vm-name:~# mkdir /storage1
root@my-vm-name:~# mount -t ext4 /dev/mapper/storage1 /storage1Troubleshooting
This section allows you to solve the most common problems encountered when using OpenStack.
SSH Error
Immediately after starting the virtual machine, you may find yourself with connection problems:
$ ssh -i ${HOME}/.novacreds/cloudkey ubuntu@134.158.151.224
ssh: connect to host 134.158.151.224 port 22: Connection refusedThis problem is usually due to the fact that the SSH service is not yet started on the virtual machine. You have to wait, which can take up to several minutes.
Another type of error may be encountered:
$ ssh -i ${HOME}/.novacreds/cloudkey ubuntu@134.158.151.224
[...]
Permission denied (publickey).In this case, you need to check that the SSH key you are using is the same as the one stored in the OpenStack database. To do this, check that their fingerprints are equivalent. To obtain the fingerprint of the key stored by OpenStack, run:
$ openstack keypair list
+----------+-------------------------------------------------+
| Name | Fingerprint |
+----------+-------------------------------------------------+
| cloudkey | 0d:97:13:66:49:03:44:da:69:82:3c:dd:b9:d7:25:64 |
+----------+-------------------------------------------------+And to get the key fingerprint used by the ssh command, run:
$ ssh-keygen -lf ${HOME}/.novacreds/cloudkey.pub
2048 33:91:4b:38:52:63:1d:18:7b:e0:e3:0d:3b:63:ce:4b you@home (RSA)If they are different, you must update your public key in OpenStack:
$ openstack keypair delete cloudkey
$ openstack keypair add --pub-key=${HOME}/.novacreds/cloudkey.pub cloudkey
$ openstack keypair list
+----------+-------------------------------------------------+
| Name | Fingerprint |
+----------+-------------------------------------------------+
| cloudkey | 33:91:4b:38:52:63:1d:18:7b:e0:e3:0d:3b:63:ce:4b |
+----------+-------------------------------------------------+If they are the same, OpenStack may not have been able to install them correctly when the new VM was started. The cloud-init software is responsible for installing the SSH key in your virtual machine. You can see the cloud-init log lines in the virtual machine console. To get the console connection url, use the following command:
$ openstack console url show --novnc MY_VM_NAME
+-------+----------------------------------------------------------------------------------------+
| Type | Url |
+-------+----------------------------------------------------------------------------------------+
| novnc | http://sbgcloud.in2p3.fr:6080/vnc_auto.html?token=d609176d-c397-4841-b4dc-566300209209 |
+-------+----------------------------------------------------------------------------------------+Then, after logging into the console with a web browser, look for lines that contain the elements ci-info and Authorized keys from and compare with your key fingerprints. If you see any error messages, the easiest thing to do is to contact the SCIGNE team with a copy of these error messages.
When the key is successfully transferred to the virtual machine, you get a message like:
Cloud-init v. 0.7.5 running 'module:final' at Mon, 27 Jun 2015 08:15:34 +0000. Up 65.40 seconds.
ci-info: ++++++++++++++Authorized keys from /root/.ssh/authorized_keys for user root++++++++++++++
ci-info: +---------+-------------------------------------------------+---------+-----------------+
ci-info: | Keytype | Fingerprint (md5) | Options | Comment |
ci-info: +---------+-------------------------------------------------+---------+-----------------+
ci-info: | ssh-rsa | 33:91:4b:38:52:63:1d:18:7b:e0:e3:0d:3b:63:ce:4b | - | joe@hey.com |
ci-info: +---------+-------------------------------------------------+---------+-----------------+If you only want to view the console output of the virtual machine, it is more convenient to use :
$ openstack console log show MY_VM_NAMEAdditional Documentation
The following sites can be consulted to obtain more information about the use of OpenStack: