Automate infrastructure provisioning¶

OpenTofu is a Open Source forked project from Terraform (hereinafter referred to as "terraform")

Those tools provide a efficient, reusable and reproductible way to provision complex IT infrastructure on cloud.

Basic requirement¶

You have to configuration your OpenStack CLI has described on dedicated section. The example are based on this configuration, if you have some specific configuration you have to adapt.

Terraform can use gitlab as infrastructure state reference. This is optional but highly recommended. You need to have access to a gitlab service as the one provided by Université Paris Saclay.

Note

gitlab-cd doesn't use clouds.yaml configuration but use OS_* variables declarations. To allow terraform to work on your client and gitlab-cd, you have to add export OS_CLOUD=virtualdata. That make virtualdata as your default cloud for your client.

Official documentation¶

OpenTofu official documentation is available here

Terraform official documentation is available here

Installation¶

On Mac OS¶

$ brew install terraform
$ mkdir -p ~/terraform/example
$ cd ~/terraform/example
$

On ubuntu 24.04¶

$ sudo snap install terraform
$

Gitlab¶

You have to create a dedicated project for your infrastructure, in this tutorial, we will use it to store infrastructure state and use gitlab CD to apply infrastructure modification.

You can find a example project here. This repository provide a modules wrote to simplify terraform usage based on image naming defined in packer documentation.

To allow you to access and store state, you need to have a gitlab token. You can create it Settings>Access Token on project web page. When created, you need to put it on $GITLAB_TOKEN_ACCESS variable

$ export GITLAB_ACCESS_TOKEN=<your-token>
$

Quick start¶

Fork & Clone example repository¶

VirtualData provide a terraform example that can help you to start. You have to modify the configuration to match your own needs.

$ git clone https://gitlab.in2p3.fr/<username>/infrastructure.git
$ cd infrastructure-example
$

Explore repository¶

$ tree -A
.
├── README.md
├── backend.tf
├── init.sh
├── main.tf
└── modules
    └── cloud
        ├── main.tf
        └── variables.tf

modules directory contains some mechanism to avoid a lot of rewriting. It's based on the naming convention we used on packer example to name images (os-flavor-timestamp).
backend.tf contains terraform configuration to use gitlab as state repository
main.tf contains information about infrastructure we want to provide, this is the file you will mostly modify

Initialise terraform state respository¶

On project web page, in Operate>Terraform state, you can find the terraform init command you need to run to initialize your terraform state. This should be something like.

$ export TF_STATE_NAME=default
$ export GITLAB_HOST=<gitlab-hostname>
$ export GITLAB_PROJECT_ID=<project-id>
$ export GITLAB_USERNAME=<gitlab-username>
$ terraform init \
    -backend-config="address=https://${GITLAB_HOST}/api/v4/projects/\
${GITLAB_PROJECT_ID}/terraform/state/$TF_STATE_NAME" \
    -backend-config="lock_address=https://${GITLAB_HOST}/api/v4/projects/\
${GITLAB_PROJECT_ID}/terraform/state/$TF_STATE_NAME/lock" \
    -backend-config="unlock_address=https://${GITLAB_HOST}/api/v4/projects/\
${GITLAB_PROJECT_ID}/terraform/state/$TF_STATE_NAME/lock" \
    -backend-config="username=$GITLAB_USERNAME" \
    -backend-config="password=$GITLAB_ACCESS_TOKEN" \
    -backend-config="lock_method=POST" \
    -backend-config="unlock_method=DELETE" \
    -backend-config="retry_wait_min=5"

main.tf in details¶

Warning

operating_system_date should be modify to match your image date.

$ cat main.tf
module "my_server_1_virtualdata_fr" {
  source = "./modules/cloud"
  hostname = "my-server-1.virtualdata.fr"
  operating_system_date = "202404281305"
}

module "my_server_2_virtualdata_fr" {
  source = "./modules/cloud"
  hostname = "my-server-2.virtualdata.fr"
  operating_system_date = "202404281305"
}

Each module describe a server with the following mandatory variables :

source : the base module for the server. On our case it modules/cloud
hostname : the name for the VM in OpenStack
operating_system_date : the timestamp part of image name

Each module also have access to optional variables that can be used to customize your deployment

operating_system_flavor : the flavor of the based image (default: core)
operating_system_name : name of operating system (default: alma-9x)
openstack_flavor_name : the OpenStack flavor (default: vd.1)
keyname : the public ssh key put on default user account (default: null).
public_address : the fixed IP we want to use for this service (default: null)
security_group_ports: list of range port open to specific IP range with format ['port_min', 'port_max', 'ip_prefix'] per example [[ "80", "80", "0.0.0.0/0"], [ "443", "443", "0.0.0.0/0"]] (default: [])
persistent_volume: list of cinder's uuid volume which already exist (see "OpenStack CLI section") (default: [])

Deploy infrastructure¶

$ terraform apply --auto-approve
$ openstack server list \
  --os-cloud virtualdata
+----------+-------------------+----+---------------------------+--------+
| ID       | Name              |[..]| Image                     | Flavor |
+----------+-------------------+----+---------------------------+--------+
| 654[...] | my-server-1.[...] |[..]| alma-9x-core-202404281305 | vd.1   |
| 767[...] | my-server-2.[...] |[..]| alma-9x-core-202404281305 | vd.1   |
+----------+-------------------+----+---------------------------+--------+

Destroy a infrastructure¶

terraform also provide a way to delete all references of a specific infrastructure with the option destroy

$ terraform destroy
$

Go futher¶

Modify a flavor of a virtual machine¶

$ git diff
diff --git a/main.tf b/main.tf
index e133a97..88e753b 100644
--- a/main.tf
+++ b/main.tf
@@ -1,7 +1,8 @@
 module "my_server_1_virtualdata_fr" {
   source = "./modules/cloud"
   hostname = "my-server-1.virtualdata.fr"
-  operating_system_date = "202404302030"
+  operating_system_flavor = "web"
+  operating_system_date = "202404302041"
 }

 module "my_server_2_virtualdata_fr" {
$ terraform validate
Success! The configuration is valid.
$ terraform apply --auto-approve
[...]
$ openstack server list \
  --os-cloud virtualdata
+----------+-------------------+----+---------------------+--------+
| ID       | Name              |[..]| Image               | Flavor |
+----------+-------------------+----+---------------------+--------+
| 654[...] | my-server-1.[...] |[..]| alma-9x-web-[...]   | vd.1   |
| 767[...] | my-server-2.[...] |[..]| alma-9x-core-[...]  | vd.1   |
+----------+-------------------+----+---------------------+--------+

Integration in gitlab CD¶

Add credential on gitlab¶

To use gitlab-cd, you need to provide OpenStack credential to gitlab. As there are sensitives information, gitlab developed a data sharing mechanism called "masked variables". Those variables can't be print on any gitlab logs and can't be used outside protected branches.

On Settings>CI/CD>Variables project menu, you will have to add some gitlab specifics secrets

GITLAB_REMOTE_STATE: https://gitlab.dsi.universite-paris-saclay.fr/api/v4/projects/project-id/terraform/state/default
GITLAB_ACCESS_TOKEN
GITLAB_PROJECT_ID
GITLAB_USERNAME

and of course some OpenStack secrets

OS_AUTH_URL https://keystone.ijclab.in2p3.fr:5000/v3
OS_AUTH_TYPE (v3applicationcredential)
OS_KEYNAME (vd-key)
OS_APPLICATION_CREDENTIAL_ID
OS_APPLICATION_CREDENTIAL_SECRET

Activate gitlab CI/CD¶

you're ready to activate gitlab-cd. On Settings>CI/CD>General Pipelines change CI/CD configuration file from .gitlab-ci.yaml to ci/.gitlab-ci.yaml.

Now, everytime you will push a new version of terraform file, the pipeline will run and apply your modification on OpenStack. Let's try !

On Build>Pipelines in gitlab you can follow your pipeline.

Test gitlab CI/CD¶

$ openstack server list --os-cloud virtualdata
+----------+-------------------+----+---------------------+--------+
| ID       | Name              |[..]| Image               | Flavor |
+----------+-------------------+----+---------------------+--------+
| 654[...] | my-server-1.[...] |[..]| alma-9x-web-[...]   | vd.1   |
| 767[...] | my-server-2.[...] |[..]| alma-9x-core-[...]  | vd.1   |
+----------+-------------------+----+---------------------+--------+
$ git diff
diff --git a/main.tf b/main.tf
index 88e753b..0cc4846 100644
--- a/main.tf
+++ b/main.tf
@@ -8,5 +8,6 @@ module "my_server_1_virtualdata_fr" {
 module "my_server_2_virtualdata_fr" {
   source = "./modules/cloud"
   hostname = "my-server-2.virtualdata.fr"
-  operating_system_date = "202404302030"
+  operating_system_flavor = "web"
+  operating_system_date = "202404302041"
 }
$ git add main.tf
$ git commit -m 'change base image'
$ git push
$ openstack server list --os-cloud virtualdata
+----------+-------------------+----+---------------------+--------+
| ID       | Name              |[..]| Image               | Flavor |
+----------+-------------------+----+---------------------+--------+
| 654[...] | my-server-1.[...] |[..]| alma-9x-web-[...]   | vd.1   |
| 8b9[...] | my-server-2.[...] |[..]| alma-9x-web-[...]   | vd.1   |
+----------+-------------------+----+---------------------+--------+

Using cloud-config¶

OpenStack provide a mechanism called cloud-init to allow user to modify a image during boot sequence and so customize a image without regenerating a new image.

That's useful is you want, per example, put a crontab on a specific web server, it's useless to create a specific image with crontab installed, but you don't want to reconfigure it everytime you update your webserver image.

With standard OpenStack CLI, it's uneasy to use cloud-init. The configuration file should be put as a one line string, but terraform provide a template provider that allow user to put a .yaml config file instead.

To use it, you just need to add cloud_config_cfg = "./cloud-config/web.yaml" on main.tf file.

$ cat ./cloud-config/web.yaml
packages:
  - mariadb

runcmd:
  - [ ls, -l, / ]
  - [ sh, -xc, "echo $(date) ': hello world!'" ]
  - [ sh, -c, echo "=========hello world=========" ]
  - ls -l /root
$ git diff
diff --git a/main.tf b/main.tf
index e113b36..adbf2a4 100644
--- a/main.tf
+++ b/main.tf
@@ -1,6 +1,7 @@
 module "my_server_1_virtualdata_fr" {
   source = "./modules/cloud"
   hostname = "my-server-1.virtualdata.fr"
+  cloud_config_cfg = "./cloud-config/web.yaml"
   operating_system_flavor = "web"
   operating_system_date = "202404302041"
 }

You can check the cloud-init output in /var/log/cloud-init-output.log and verify that mariadb is installed on my-server-1.virtualdata.fr.

$ cat /var/log/cloud-init-output.log
[...]
$ sudo rpm -qa |grep mariadb-[0-9]
mariadb-10.5.22-1.el9_2.alma.1.x86_64