Building a custom Terraform module

Introduction

Terraform modules allow you to group your infrastructure's distinct resources into a single, unified resource. You can reuse them later with possible customizations without having to repeat resource definitions every time you need them, which is useful for large projects with complex structures. You can customize module instances using input variables that you define, and also extract information from them using outputs. In addition to creating your own custom modules, you can also use pre-built modules that are publicly published in the Terraform registry. Developers can use and customize them using inputs like the modules you create, but their source code is stored in and out of the cloud. In this tutorial, you create a Terraform module that launches multiple Droplets behind a Load Balancer for redundancy. You will also use the for_each and count Looping features of the Hashicorp Configuration Language (HCL) to deploy multiple customized module instances simultaneously.

Prerequisites

• A personal DigitalOcean access code
• Terraform is installed on your system and a project is set up with the DO provider.
• Introduction to data types and HCL loops
• Introduction to Terraform outputs and their uses

Module structure and benefits

In this section, you will learn the benefits that modules bring, where they are typically placed in a project, and how they should be structured. Custom Terraform modules are created to encapsulate connected components that are often used in larger projects and deployed together. They are self-contained, bringing together only the resources, variables, and providers they need. Modules are typically stored in a central folder in the project root, each in its own subfolder. To maintain a clean separation between modules, always design them to have a single purpose and make sure they never contain submodules. Packaging a single resource as a module can be redundant and gradually remove the simplicity of the overall architecture. For small development and testing projects, combining modules is not necessary, as they do not bring much progress in those cases. Modules also offer the advantage that definitions only need to be modified in one place, which is then propagated to the rest of the infrastructure.

Next, you define, use, and customize modules in your Terraform projects.

Create a module

In this section, you define multiple Droplets and a Load Balancer as Terraform resources and package them into a module. You also configure the resulting module using configurable module inputs.

Place the module in a directory called droplet-lb, you will save under a directory called modules. Assuming you are in the terraform-modules directory that you created as part of the prerequisites, install both at the same time by running:

mkdir -p modules/droplet-lb

The -p argument instructs mkdir to create all directories in the provided path.

Go to it:

cd modules/droplet-lb

As mentioned in the previous section, modules contain the resources and variables they use. Starting with Terraform 0.13, they must also contain definitions of the providers they use. Modules do not require any special configuration to indicate that the code represents a module, because Terraform considers any directory containing HCL code to be a module, even the project root directory.

Variables defined in a module are exposed as its inputs and can be used in resource definitions to customize them. The module you create will have two inputs: the number of droplets to create and the name of their group. To edit a file called variables.tf Create and open where you will store the variables:

nano variables.tf

Add the following lines:

variable "droplet_count" {}
variable "group_name" {}

Save and close the file.

You define the Droplet in a file called droplets.tf You will save it. Create and open it for editing:

nano droplets.tf

Add the following lines:

resource "digitalocean_droplet" "droplets" {
count = var.droplet_count
image = "ubuntu-22-04-x64"
name = "${var.group_name}-${count.index}"
region = "fra1"
size = "s-1vcpu-1gb"
lifecycle {
precondition {
condition = var.droplet_count >= 2
error_message = "At least two droplets must be created."
}
}
}

For parameter countYou pass in the droplet_count variable, which specifies how many instances of a resource to create. Its value will be determined when the module is called from the main project code. The name of each deployed droplet will be different, which is achieved by appending the current droplet index to the provided group name. Deploying Droplets in the Zone fra1 It will be and will run Ubuntu 22.04.

Section Lifecycle Contains a Precondition is executed before the resources are actually deployed. Here, it verifies that at least two drops will be created – having only one defeats the purpose of the Load Balancer. Another example of validations can be found in the k8s-bootstrapper repository, which contains templates for setting up a DigitalOcean Kubernetes cluster using Terraform. There, the validations are used to ensure that the number of nodes in the cluster is within range.

When you're done, save and close the file.

With the Droplets defined, you can move on to creating the Load Balancer. Define its source in a file called lb.tf You will save it. Create and open it for editing by running the following:

nano lb.tf

Add its source definition:

resource "digitalocean_loadbalancer" "www-lb" {
name = "lb-${var.group_name}"
region = "fra1"
forwarding_rule {
entry_port = 80
entry_protocol = "http"
target_port = 80
target_protocol = "http"
}
healthcheck {
port = 22
protocol = "tcp"
}
droplet_ids = [
for droplet in digitalocean_droplet.droplets:
droplet.id
]
}

You define the Load Balancer with the group name in its name so that it is recognizable. You define it along with the drops in the zone. fra1 The next two sections specify the target and monitoring ports and protocols.

The highlighted droplet_ids block gets the IDs of the droplets that need to be managed by the Load Balancer. Since there are multiple droplets and the number is not known in advance, you use a for loop to loop through the collection of droplets (digitalocean_droplet.droplets) and get their IDs. You surround the for loop with brackets ([]) so that the resulting collection is a list.

Save and close the file.

Now that you have defined the Droplet, Load Balancer, and variables for your module, you need to define the provider requirements and specify which providers the module will use, including their version and location. As of Terraform 0.13, modules must explicitly define the non-Hashicorp provider resources they will use. This is because they will not inherit them from the parent project.

You define the provider requirements in a file called provider.tf You will save it. Make it available for editing by running the following:

nano provider.tf

Add the following lines to require the Digitalocean provider:

terraform {
required_providers {
digitalocean = {
source = "digitalocean/digitalocean"
version = "~> 2.0"
}
}
}

When finished, save and close the file. Module droplet-lb Now to the provider Digitalocean Needs.

Modules also support outputs that you can use to extract internal information about the status of their resources. You define an output that shows the IP address of the Load Balancer and put it in a file called outputs.tf Save it. Create it for editing:

nano outputs.tf

Add the following definition:

output "lb_ip" {
value = digitalocean_loadbalancer.www-lb.ip
}

This output will retrieve the IP address of the Load Balancer. Save and close the file.

Module droplet-lb It is now fully functional and ready to deploy. You will call it from the main code, which you will store in the root of the project. First, navigate to it by double-clicking up in your file directory:

cd ../..

Then a file called main.tf Create and open for editing where you will use the module:

nano main.tf

Add the following lines:

module "groups" {
source = "./modules/droplet-lb"
droplet_count = 3
group_name = "group1"
}
output "loadbalancer-ip" {
value = module.groups.lb_ip
}

In this announcement, the module droplet-lb You call the source directory, which is located in the specified directory. The input it provides is droplet_count and group_name You configure it to be set to group1 so that you can distinguish between instances later.

Since the Load Balancer IP output is defined in a module, it is not automatically shown when you deploy the project. The solution to this is to create another output by retrieving its value (loadbalancer_ip).

When finished, save and close the file.

Run the module by running:

terraform init

The output will be as follows:

OutputInitializing modules...
- groups in modules/droplet-lb
Initializing the backend...
Initializing provider plugins...
- Finding digitalocean/digitalocean versions matching "~> 2.0"...
- Installing digitalocean/digitalocean v2.34.1...
- Installed digitalocean/digitalocean v2.34.1 (signed by a HashiCorp partner, key ID F82037E524B9C0E8)
...
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

You can schedule the project to see what actions Terraform will take upon execution:

terraform plan -var "do_token=${DO_PAT}"

The output will be similar to this:

Output...
Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
+ create
Terraform will perform the following actions:
# module.groups.digitalocean_droplet.droplets[0] will be created
+ resource "digitalocean_droplet" "droplets" {
...
+ name = "group1-0"
...
}
# module.groups.digitalocean_droplet.droplets[1] will be created
+ resource "digitalocean_droplet" "droplets" {
...
+ name = "group1-1"
...
}
# module.groups.digitalocean_droplet.droplets[2] will be created
+ resource "digitalocean_droplet" "droplets" {
...
+ name = "group1-2"
...
}
# module.groups.digitalocean_loadbalancer.www-lb will be created
+ resource "digitalocean_loadbalancer" "www-lb" {
...
+ name = "lb-group1"
...
}
Plan: 4 to add, 0 to change, 0 to destroy.
...

This output explains that Terraform creates three droplets named group1-0, group1-1, and group1-2, and also creates a Load Balancer named group1-lb that handles traffic to and from the three droplets.

You can deploy the project to the cloud by running the following:

terraform apply -var "do_token=${DO_PAT}"

When prompted, enter yes. The output will show all actions and the Load Balancer IP address will also be displayed:

Outputmodule.groups.digitalocean_droplet.droplets[1]: Creating...
module.groups.digitalocean_droplet.droplets[0]: Creating...
module.groups.digitalocean_droplet.droplets[2]: Creating...
...
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
Outputs:
loadbalancer-ip = ip_address

You have created a module containing a configurable number of droplets and a load balancer that is automatically configured to manage their incoming and outgoing traffic.

Rename deployed resources

In the previous section, you deployed the module you defined and named it Group. If you want to change its name, simply renaming the module call will not produce the expected results. Renaming the call will cause Terraform to destroy and recreate the resources, causing excessive downtime.

For example, main.tf Open it for editing by running:

nano main.tf

Rename the groups module to group_renamed as specified:

module "groups_renamed" {
source = "./modules/droplet-lb"
droplet_count = 3
group_name = "group1"
}
output "loadbalancer-ip" {
value = module.groups_renamed.lb_ip
}

Save and close the file. Then initialize the project again:

terraform init

Now you can plan the project:

terraform plan -var "do_token=${DO_PAT}"

The output will be long but will look something like this:

Output...
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
- destroy
Terraform will perform the following actions:
# module.groups.digitalocean_droplet.droplets[0] will be destroyed
...
# module.groups_renamed.digitalocean_droplet.droplets[0] will be created
...

Terraform requires you to destroy existing instances and create new ones. This is destructive and unnecessary and may lead to unintended crashes.

Instead, you can instruct Terraform to migrate the old resources with the new name using the migrated block. Open main.tf for editing and add the following lines to the end of the file:

moved {
from = module.groups
to = module.groups_renamed
}

When you're done, save and close the file.

Now you can plan the project:

terraform plan -var "do_token=${DO_PAT}"

When you program with the moved block in main.tf, Terraform wants to move the resources instead of recreating them:

OutputTerraform will perform the following actions:
# module.groups.digitalocean_droplet.droplets[0] has moved to module.groups_renamed.digitalocean_droplet.droplets[0]
...
# module.groups.digitalocean_droplet.droplets[1] has moved to module.groups_renamed.digitalocean_droplet.droplets[1]
...

Mobile resources change their location in Terraform mode, meaning that the actual cloud resources are not changed, destroyed, or recreated.

Since you will be changing the configuration significantly in the next step, update the deployed resources by running:

terraform destroy -var "do_token=${DO_PAT}"

When you were asked, Yes Enter . The output ends with:

Output...
Destroy complete! Resources: 4 destroyed.

In this section, you renamed resources in your Terraform project without destroying them. Now you will deploy multiple instances of a module from the same code using for_each and count.

Deploying multiple module instances

In this section, you use count and for_each to deploy the droplet-lb module multiple times with customization.

 Using count

One way to deploy multiple instances of a module at the same time is to pass a count to the count parameter, which is automatically available for each module. Open main.tf for editing:

nano main.tf

Change it to the following by deleting the existing output definition and the moved block:

module "groups" {
source = "./modules/droplet-lb"
count = 3
droplet_count = 3
group_name = "group1-${count.index}"
}

By setting the count to 3, you instruct Terraform to deploy the module three times, each with a different group name. When you're done, save and close the file.

Schedule the deployment by running:

terraform plan -var "do_token=${DO_PAT}"

The output will be long and will look like this:

Output...
Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
+ create
Terraform will perform the following actions:
# module.groups[0].digitalocean_droplet.droplets[0] will be created
...
# module.groups[0].digitalocean_droplet.droplets[1] will be created
...
# module.groups[0].digitalocean_droplet.droplets[2] will be created
...
# module.groups[0].digitalocean_loadbalancer.www-lb will be created
...
# module.groups[1].digitalocean_droplet.droplets[0] will be created
...
# module.groups[1].digitalocean_droplet.droplets[1] will be created
...
# module.groups[1].digitalocean_droplet.droplets[2] will be created
...
# module.groups[1].digitalocean_loadbalancer.www-lb will be created
...
# module.groups[2].digitalocean_droplet.droplets[0] will be created
...
# module.groups[2].digitalocean_droplet.droplets[1] will be created
...
# module.groups[2].digitalocean_droplet.droplets[2] will be created
...
# module.groups[2].digitalocean_loadbalancer.www-lb will be created
...
Plan: 12 to add, 0 to change, 0 to destroy.
...

Terraform explains in the output that each of the three module instances has three Droplets and a Load Balancer associated with them.

Using for_each

You can use for_each for modules when you need more complex instance customization or when the number of instances depends on third-party data (often provided as a map) that is not known when writing the code.

Now you define a map that pairs the group names with the number of droplets and implements droplet-lb instances based on it. Open main.tf for editing by running:

nano main.tf

Change the file as follows:

variable "group_counts" {
type = map
default = {
"group1" = 1
"group2" = 3
}
}
module "groups" {
source = "./modules/droplet-lb"
for_each = var.group_counts
droplet_count = each.value
group_name = each.key
}

First, you define a map called group_counts that contains the number of droplets in a given group. Next, you call the droplet-lb module, but specify that the for_each loop should operate on var.group_counts, the map you defined earlier. droplet_count takes each.value, the current pair value, which is the number of droplets for the current group. group_name takes the group name.

When finished, save and close the file.

Try applying the configuration by running:

terraform plan -var "do_token=${DO_PAT}"

The output shows the details of Terraform's actions to create these two groups with Droplets and Load Balancers:

Output...
Terraform used the selected providers to generate the following execution plan. Resource actions are
indicated with the following symbols:
+ create
Terraform will perform the following actions:
# module.groups["group1"].digitalocean_droplet.droplets[0] will be created
...
# module.groups["group1"].digitalocean_loadbalancer.www-lb will be created
...
# module.groups["group2"].digitalocean_droplet.droplets[0] will be created
...
# module.groups["group2"].digitalocean_droplet.droplets[1] will be created
...
# module.groups["group2"].digitalocean_droplet.droplets[2] will be created
...
# module.groups["group2"].digitalocean_loadbalancer.www-lb will be created
...

At this point, you have used count and for_each to deploy multiple customized instances of a module from the same code.

Result

In this tutorial, you created and implemented Terraform modules. You used modules to group logically related resources together and customized them to deploy multiple different instances of a central code definition. You also used outputs to display the properties of the resources in the module.