Reduce Terraform plan time when using modules

Context

My team at OVHcloud is in the process of adopting Terraform for various tasks related to software deployment on our internal infrastructure. The Developer Platform team offers a Terraform provider, which exposes raw resources that we can manipulate to deploy software, expose APIs, manage access control, etc.

I won’t explain all the Terraform related terms in this article. You can refer to the Terraform glossary.

After some experimentation, I decided to write a couple of Terraform modules¹ to reduce the boilerplate of our use cases.

Issue

One of the modules was dedicated to producing a gateway_policy resource, and here’s how I initially implemented it:

variable "project" {
  type = string
}

variable "stack" {
  type = string
}

data "project" "this" {
  name = var.project
}

data "kubernetes_tenant" "this" {
  project = data.project.this.name
  name    = var.stack
}

data "gateway_account" "this" {
  project = data.project.this.name
  name    = var.project
}

resource "gateway_policy" "this" {
  kubernetes_tenant = data.kubernetes_tenant.this.id
  gateway_account   = data.gateway_account.this.id
  # ...
}

I’m omitting what makes it interesting as a module, it’s a bunch of variable manipulation unrelated to what I want to discuss in this article.

It was called like this, more than eighty times:

module "acl_foo" {
  source = "../../modules/custom_acl"

  project = "project_name"
  stack   = "stack_name"
  
  # ...
}

After adding these module calls to our project, our plan time was roughly 80 seconds (1min20s). The plan contained lots of the following:

module.acl_foo.data.project.this: Reading...
module.acl_foo.data.kubernetes_tenant.this: Reading...
module.acl_foo.data.gateway_account.this: Reading...

Terraform was fetching the same data over and over again, because the variables project and stack were (purposefully) given the same value in every call. Far from a useful use of resources!

Refactor

The gateway_policy resource needs access to the kubernetes_tenant and gateway_account though, so I refactored my module to take these as inputs instead.

variable "kubernetes_tenant_id" {
  type = string
}

variable "gateway_account_id" {
  type = string
}

resource "gateway_policy" "this" {
  kubernetes_tenant = var.kubernetes_tenant_id
  gateway_account   = var.gateway_account_id
  # ...
}

The calls now look like this:

module "acl_foo" {
  source = "../../modules/custom_acl"

  kubernetes_tenant_id = data.kubernetes_tenant.foo.id
  gateway_account_id   = data.gateway_account.foo.id
  
  # ...
}

Learnings

This allowed Terraform to fetch the data only once and pass it down to all module calls. The plan time went down from 80s to 40s, with the same result.

I learned a couple of things along the way:

• data calls are costly: push them up, and avoid them in modules if possible

• it’s worth properly reading the whole plan from time to time to look for optimisation opportunities, instead of skipping to the end where the infrastructure changes are listed.

1. You can think of modules as kind of functions that take inputs, produce resources, and return outputs. ↵