Skip to content

Deploy an Instance in MicroStack with Terraform

Jump to bottom
Juan Caviedes edited this page Apr 9, 2021 · 7 revisions

Introduction

QUÉ ES TERRAFORM

Install Terraform

This procedure summarizes the Terraform installation. Go to this link and download the .zip file for your operating system (OS). In our case, the OS is Ubuntu 20.04 LTS. Next, unzip the file and copy the Terraform file in bin files. The steps (in our case) are next.

sudo apt update && sudo apt upgrade
sudo apt install -y zip
# At time of writing, Terraform latest stable version is 0.13.4
cd && wget https://releases.hashicorp.com/terraform/0.13.4/terraform_0.13.4_linux_amd64.zip
unzip terraform*.zip
sudo rm terraform*.zip
sudo mv terraform /usr/local/bin/

Next, for validating the installation. In CLI you should see Terraform version printed out.

capri@caprivm:~$ terraform version
Terraform v0.13.4

Now, before initialize Terraform, create the the next folders.

mkdir ~/terraform && cd ~/terraform
mkdir openstack

These folders may be used for provide OpenStack to terraform.

Reference: https://www.techrepublic.com/article/how-to-install-terraform-on-ubuntu-server/

Adding Additional Images to OpenStack

By default, MicroStack only includes the CirrOS image. While it is useful for testing that OpenStack can provision resources, it does not provide the ability to install packages. Many Linux distributions create OpenStack-ready images, a list of which can be found in: https://docs.openstack.org/image-guide/obtain-images.html.

For this experiment, we use a Debian Buster image. The next steps shows how to download the OpenStack compatible image and add to the cloud platform.

# (i) Download the image
wget https://cdimage.debian.org/cdimage/openstack/current/debian-10.6.0-openstack-amd64.qcow2
# (ii) Add the image to OpenStack
openstack image create --container-format bare --disk-format qcow2 --file debian-9-openstack-amd64.qcow2 debian-buster

Next, verify that the image is defined. Assume that you have alias openstack="microstack.openstack" configured.

capri@caprivm:~$ openstack image list
+--------------------------------------+---------------+--------+
| ID                                   | Name          | Status |
+--------------------------------------+---------------+--------+
| cf5aa2d9-1ed6-4292-975c-ff0d7ae121ec | cirros        | active |
| b4e47934-c000-467c-bced-bb7f17d31373 | debian-buster | active |
+--------------------------------------+---------------+--------+

The advantage of this image (and many others) is that it allow the use of cloud-init file, allowing you to specify first-time boot configuration, package installation and remote access credentials like SSH passwords.

Configure the Application Credential for Terraform

To generate an application credential we use the user interface (UI). In UI go to the application credential tab in the Identity page and click there. In screen you must be show the next form.

application_credential_form

You can assign an application credential role. In this case, for Terraform we choose the admin role. Please consider several additional definition for Terraform configuration:

  • application_credential_secret: As shown in the last image, a secret will be generated for you if you don't provide your own secret. Once application credential is created, the secret will be revealed once. Remember that if you lose the secret, you will have to generate a new application credential. In our case, the application credential secret is: application_credential_secret: "sJbZXO0c3lxCvRwm8Udjq96vgUC0jATceyPQrA-1r_9K_Dtovgu4ueU-XZ4_aywCr8OGx2BPSK3kwwslwkIaNg"
  • application_credential_id: When you create an application credential, OpenStack assign an ID that can be shown in the UI or typing openstack application credential list in CLI. The result is shown below. In this example, the ID corresponds to terraform_prov.
capri@caprivm:~$ openstack application credential list
+----------------------------------+----------------+----------------------------------+
| ID                               | Name           | Project ID                       |
+----------------------------------+----------------+----------------------------------+
| 880a9c2fa37d4164b36da41bec63b1da | prometheus_sd  | 86bc6ac1721143f4919d221d39079696 |
| 0c87aac399244f52800f2ffab484d3d4 | terraform_prov | 86bc6ac1721143f4919d221d39079696 |
+----------------------------------+----------------+----------------------------------+
  • user_name: By default, in MicroStack installation the username is admin. If you created another user, be sure which user you are going to use in Terraform configuration.
  • user_id: In the same way that application credential ID, OpenStack assign an ID for each user in platform. You can consult both user_name and user_id in the UI or typing openstack user list in CLI. The result is shown below.
capri@caprivm:~$ openstack user list
+----------------------------------+-----------+
| ID                               | Name      |
+----------------------------------+-----------+
| 4dfe6f20436047159e21fc8ae0efbb9a | admin     |
| 7b796b466a584806b935a5e7f9ca55c7 | placement |
| f2074b6245024a818031490e63498b5a | nova      |
| e46d986420c34955947ddd1269a795c6 | neutron   |
| 9bf81ea77a904742a4fd11f0a6b0f5ed | glance    |
| 685d6e3dd7684dd39c4c3b7cde495f12 | cinder    |
+----------------------------------+-----------+
  • tenant_id: You can consult the tenant ID in the OpenRC file that OpenStack provides or typing openstack project list in CLI. In the OpenRC file the tenant ID is the same project ID. The result is shown below. In this case, the tenant ID that corresponds to admin user is if interest to us.
capri@caprivm:~$ openstack project list
+----------------------------------+---------+
| ID                               | Name    |
+----------------------------------+---------+
| 08d668ff3bd74909a69e0d2246ac831e | service |
| 86bc6ac1721143f4919d221d39079696 | admin   |
+----------------------------------+---------+
  • region: By default in MicroStack installation the region is microstack. You can consult it typing openstack region list or in the OpenRC file. The result is shown below.
capri@caprivm:~$ openstack region list
+------------+---------------+-------------+
| Region     | Parent Region | Description |
+------------+---------------+-------------+
| microstack | None          |             |
+------------+---------------+-------------+
  • auth_url: The identity service in Keystone OpenStack, provide an endpoint for authentication and authorization. This endpoint can be consulted in OpenRC file. By default in MicroStack installation (Stein version) is http://localhost:5000/v3/. In our OpenRC file is: auth_url: "http://10.20.20.1:5000/v3/"

An example for our OpenRC file is:

#!/usr/bin/env bash
# To use an OpenStack cloud you need to authenticate against the Identity
# service named keystone, which returns a **Token** and **Service Catalog**.
# The catalog contains the endpoints for all services the user/tenant has
# access to - such as Compute, Image Service, Identity, Object Storage, Block
# Storage, and Networking (code-named nova, glance, keystone, swift,
# cinder, and neutron).
#
# *NOTE*: Using the 3 *Identity API* does not necessarily mean any other
# OpenStack API is version 3. For example, your cloud provider may implement
# Image API v1.1, Block Storage API v2, and Compute API v2.0. OS_AUTH_URL is
# only for the Identity API served through keystone.
export OS_AUTH_URL=http://10.20.20.1:5000/v3/
# With the addition of Keystone we have standardized on the term **project**
# as the entity that owns the resources.
export OS_PROJECT_ID=86bc6ac1721143f4919d221d39079696
export OS_PROJECT_NAME="admin"
export OS_USER_DOMAIN_NAME="Default"
if [ -z "$OS_USER_DOMAIN_NAME" ]; then unset OS_USER_DOMAIN_NAME; fi
export OS_PROJECT_DOMAIN_ID="default"
if [ -z "$OS_PROJECT_DOMAIN_ID" ]; then unset OS_PROJECT_DOMAIN_ID; fi
# unset v2.0 items in case set
unset OS_TENANT_ID
unset OS_TENANT_NAME
# In addition to the owning entity (tenant), OpenStack stores the entity
# performing the action as the **user**.
export OS_USERNAME="admin"
# With Keystone you pass the keystone password.
echo "Please enter your OpenStack Password for project $OS_PROJECT_NAME as user $OS_USERNAME: "
read -sr OS_PASSWORD_INPUT
export OS_PASSWORD=$OS_PASSWORD_INPUT
# If your configuration has multiple regions, we set that information here.
# OS_REGION_NAME is optional and only valid in certain environments.
export OS_REGION_NAME="microstack"
# Don't leave a blank variable, unset it if it was empty
if [ -z "$OS_REGION_NAME" ]; then unset OS_REGION_NAME; fi
export OS_INTERFACE=public
export OS_IDENTITY_API_VERSION=3

Initialize Terraform + OpenStack

In the ~/terraform/openstack folder is necessary to create a main file. The content of this file match with the application credential fields discussed in the previous section. Create a main.tf file and put into them the next code (replace the values with your MicroStack current configuration).

provider "openstack" {
  user_name                     = "admin"
  auth_url                      = "http://10.20.20.1:5000/v3/"
  region                        = "microstack"
  user_id                       = "4dfe6f20436047159e21fc8ae0efbb9a"
  application_credential_id     = "0c87aac399244f52800f2ffab484d3d4"
  application_credential_secret = "sJbZXO0c3lxCvRwm8Udjq96vgUC0jATceyPQrA-1r_9K_Dtovgu4ueU-XZ4_aywCr8OGx2BPSK3kwwslwkIaNg"
  tenant_id                     = "86bc6ac1721143f4919d221d39079696"
}

After the creation of the main file perform the next step.

terraform init

In CLI you should see a message like Terraform has been successfully initialized! in some part of log.

Define the Infrastructure

In this point, you can now create the configuration files for your first compute instance. The below is from the file instance.tf in the ~/terraform/openstack directory.

data "openstack_images_image_v2" "debian-buster" {
  name        = "debian-buster"
  most_recent = true
}

resource "openstack_compute_flavor_v2" "m1-micro" {
  name  = "m1.micro"
  ram   = "512"
  vcpus = "1"
  disk  = "5"
}


resource "openstack_compute_keypair_v2" "symphonyx" {
    name       = "symphonyx"
}

resource "openstack_compute_instance_v2" "debian-buster" {
  name            = "debian-buster"
  image_id        = data.openstack_images_image_v2.debian-buster.id
  flavor_id       = openstack_compute_flavor_v2.m1-micro.id
  key_pair        = openstack_compute_keypair_v2.symphonyx.name
  security_groups = [
    openstack_networking_secgroup_v2.buster.name
  ]
  user_data       = data.template_file.debian.template

  metadata = {
    prometheus = "true"
    node_exporter = "true"
  }

  network {
    name = "test"
  }
}

resource "openstack_networking_floatingip_v2" "debian-buster-fip" {
    pool = "external"
}

resource "openstack_compute_floatingip_associate_v2" "debian-buster-fip" {
    floating_ip = openstack_networking_floatingip_v2.debian-buster-fip.address
    instance_id = openstack_compute_instance_v2.debian-buster.id
}

resource "openstack_networking_secgroup_v2" "buster" {
    name        = "buster"
    description = "Buster Security Group"
}

resource "openstack_networking_secgroup_rule_v2" "node_exporter" {
  direction         = "ingress"
  ethertype         = "IPv4"
  protocol          = "tcp"
  port_range_min    = 9100
  port_range_max    = 9100
  remote_ip_prefix  = "0.0.0.0/0"
  security_group_id = openstack_networking_secgroup_v2.buster.id
}

resource "openstack_networking_secgroup_rule_v2" "ssh" {
  direction         = "ingress"
  ethertype         = "IPv4"
  protocol          = "tcp"
  port_range_min    = 22
  port_range_max    = 22
  remote_ip_prefix  = "0.0.0.0/0"
  security_group_id = openstack_networking_secgroup_v2.buster.id
}

resource "openstack_networking_secgroup_rule_v2" "icmp_v4" {
  direction         = "ingress"
  ethertype         = "IPv4"
  protocol          = "icmp"
  remote_ip_prefix  = "0.0.0.0/0"
  security_group_id = openstack_networking_secgroup_v2.buster.id
}

To summarize the steps performed in this file, we:

  • Use a data source to match the Debian Buster Openstack image that we define earlier.
  • Define an OpenStack flavour, i.e., size of the instance (CPU, RAM and Disk).
    • Flavor m1.micro: 1 vCPU, 512 MB of RAM, 5 GB of Disk and Debian as OS based on Debian Buster image.
  • Create a SSH key for managing the instance.
  • Create an OpenStack instance called debian-buster using the flavour, image and keypair previously defined. Also:
    • Associate a security group (i.e. firewall rules). Allow 22 and 9100 mainly.
    • Use a cloud-config template to the instance. Note that OpenStack support user_data in Terraform configuration.
    • Add metadata (i.e. tags) for prometheus and node_exporter.
    • Associate the network test which is predefined in MicroStack.
  • Create a floating IP and associate it with this instance.
    • The instance can be destroyed and recreated, but retain the same floating IP.
  • Create the security group and add rules for inbound SSH, ICMP (i.e. ping commands) and also allow the node_exporter port that is TCP9100.

You need to create an user_data.tf file in a new folder named ~/terraform/files/. You can create it with mkdir ~/terraform/files/ if before the terraform folder was created. The user_data.tf file is defined as shown.

data "template_file" "debian" {
  template = "${file("$HOME/terraform/files/debian.tpl")}"
}

data "template_cloudinit_config" "debian" {
  gzip          = false
  base64_encode = false

  part {
    filename     = "init.cfg"
    content_type = "text/cloud-config"
    content      = data.template_file.debian.rendered
  }
}

Last, the contents of the template file (in this case, located in $HOME/terraform/files/debian.tpl) are:

#cloud-config
packages:
 - prometheus-node-exporter
password: debian
chpasswd: { expire: False }
ssh_pwauth: True

Note that in cloud-config file we add some lines that define a SSH password (debian) to access the instance. In our case, in addition to the keypair, we add this password to facilitate experiments. Also, note the initial configuration for the prometheus-node-exporter package. It package is necessary for the monitoring with prometheus and grafana.

AÑADIR SECCIONES DE PROMETHEUS + GRAFANA COMO HIPERVÍNCULO

Build the Infrastructure

We can now apply the configuration and see if it builds an OpenStack instance. For this, only execute the next lines in the ~/terraform/openstack directory.

terraform init
terraform apply

When finish, the CLI shown Apply complete! Resources: 9 added, 0 changed, 0 destroyed in some part of log. For double check that Terraform is managing these resources execute the next command. You will see similar messages in your CLI.

capri@caprivm:~$ terraform state list
data.openstack_images_image_v2.debian-buster
data.template_cloudinit_config.debian
data.template_file.debian
openstack_compute_flavor_v2.m1-micro
openstack_compute_floatingip_associate_v2.debian-buster-fip
openstack_compute_instance_v2.debian-buster
openstack_compute_keypair_v2.symphonyx
openstack_networking_floatingip_v2.debian-buster-fip
openstack_networking_secgroup_rule_v2.icmp_v4
openstack_networking_secgroup_rule_v2.node_exporter
openstack_networking_secgroup_rule_v2.ssh
openstack_networking_secgroup_v2.buster

Moreover, we can see if the instance was created in the OpenStack UI. For this, click on Admin page, next on Compute and next on Instances. In your screen you must see some like this.

Instance Created

Now, let's access via SSH to the instance using the password provided in the cloud-config file. Use the floating IP configured in OpenStack. In Our case 10.20.20.167. Be sure that the instance is on.

capri@caprivm:~$ ssh debian@10.20.20.167
debian@10.20.20.167's password:
Linux debian-buster 4.19.0-11-cloud-amd64 #1 SMP Debian 4.19.146-1 (2020-09-17) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Oct  8 00:21:03 2020 from 10.20.20.1
debian@debian-buster:~$

Finally, run the next command to see if the prometheus-node-exporter package is running into instance.

debian@debian-buster:~$ ps aux | grep -i prometh
prometh+   408  0.0  2.0 335540 10364 ?        Ssl  21:10   0:00 /usr/bin/prometheus-node-exporter
debian     856  0.0  0.1   6148   884 pts/0    S+   21:10   0:00 grep -i prometh

Reference: https://yetiops.net/posts/prometheus-service-discovery-openstack/

Clone this wiki locally