Home › Posts › One Terraform Module Many Resources
AWS

One Terraform Module Many Resources

Most applications don't start with five message queues. They start with one, for incoming events. Then notifications get their own queue, then a background worker needs a retry…

Nov 16, 2026 • 13 min read • maki96milosavljevic@gmail.com

Most applications don’t start with five message queues. They start with one, for incoming events. Then notifications get their own queue, then a background worker needs a retry queue, and audit messages need one more. Each queue arrives with a small change, and after a year you have a Terraform file with five almost identical blocks.

Moving the queue into a reusable module is the obvious first step, but it only solves half the problem. The resource definition now exists once. The module calls are still repeated, each one passing the same ten arguments, and the setting that actually makes a queue special is hidden somewhere in the middle.

This post shows a pattern that solves both halves. The module owns the defaults, each queue is described only by what makes it different, and a single module block creates all of them.

The idea

Three Terraform features work together here:

  • optional() attributes let a module declare sensible defaults inside the type of an input variable. A caller can leave an attribute out, and Terraform fills in the default.
  • A map in locals describes every queue in one place. Each entry contains only its differences from the defaults.
  • for_each on the module creates one module instance for every entry in that map.

The result is a configuration where reviewing a new queue means reading one line.

The project layout is small:

terraform/
├── providers.tf
├── variables.tf
├── locals.tf
├── queues.tf
├── outputs.tf
└── modules/sqs/
    ├── variables.tf
    ├── main.tf
    └── outputs.tf
Step 1: Let the provider handle common tags

Tags such as project name, environment, and owner usually apply to every resource in a configuration. Passing them through every module and merging them by hand adds noise without adding information.

The AWS provider has a built-in solution called default_tags. Every resource created through this provider receives these tags automatically, including resources inside modules:

# providers.tf
provider "aws" {
  default_tags {
    tags = {
      Project     = "event-platform"
      Environment = var.aws_stage
      ManagedBy   = "terraform"
    }
  }
}
# variables.tf
variable "aws_stage" {
  description = "Deployment environment, used for naming and tagging."
  type        = string
  default     = "dev"
}

From now on, a queue only needs to declare tags that are specific to it. The provider combines them with the default tags, and if the same key appears in both, the resource-level value wins.

Step 2: Put the defaults where the resource lives

The module receives one object describing a single queue. The important part is the type of that object:

# modules/sqs/variables.tf
variable "name" {
  description = "Queue name, without the .fifo suffix."
  type        = string
}

variable "config" {
  description = "Settings for the queue. Every attribute is optional."

  type = object({
    visibility_timeout_seconds  = optional(number, 30)
    message_retention_seconds   = optional(number, 345600) # 4 days
    delay_seconds               = optional(number, 0)
    fifo_queue                  = optional(bool, false)
    content_based_deduplication = optional(bool, false)
    tags                        = optional(map(string), {})
  })

  default = {}

  validation {
    condition     = var.config.fifo_queue || !var.config.content_based_deduplication
    error_message = "content_based_deduplication can only be enabled on FIFO queues."
  }
}

Every setting is wrapped in optional(type, default). When the caller leaves an attribute out, or passes null, Terraform uses the default. This happens while Terraform checks the input’s type, so the code inside the module always sees a complete object and never has to check whether a value exists.

Keeping the defaults in the module has a practical benefit. Anyone who opens the module can see exactly what a queue looks like when nothing is overridden. Every configuration that uses the module also gets the same baseline without copying a defaults map from project to project.

The validation block catches an invalid combination before anything reaches AWS. Content-based deduplication exists only for FIFO queues, so a standard queue with deduplication enabled fails during plan with a clear message. Terraform applies the defaults before it runs the validation, so both values are always set here.

The resource maps the inputs directly:

# modules/sqs/main.tf
resource "aws_sqs_queue" "this" {
  # SQS requires FIFO queue names to end in .fifo
  name = var.config.fifo_queue ? format("%s.fifo", var.name) : var.name

  visibility_timeout_seconds  = var.config.visibility_timeout_seconds
  message_retention_seconds   = var.config.message_retention_seconds
  delay_seconds               = var.config.delay_seconds
  fifo_queue                  = var.config.fifo_queue
  content_based_deduplication = var.config.content_based_deduplication
  sqs_managed_sse_enabled     = true

  tags = var.config.tags
}

The .fifo suffix is added here, not by the caller. AWS rejects FIFO queues whose names don’t end in .fifo, so this is a rule of the service, not a naming preference. Because the module handles it, nobody can forget it.

Encryption with SQS-managed keys is always on, so it isn’t exposed as an option. Not every setting has to be configurable, and leaving one out is also a design decision.

The module returns the two values other resources usually need:

# modules/sqs/outputs.tf
output "arn" {
  value = aws_sqs_queue.this.arn
}

output "url" {
  value = aws_sqs_queue.this.url
}
Step 3: Describe each queue by its differences

With the defaults handled by the module, the root configuration only needs to say what makes each queue special:

# locals.tf
locals {
  queues = {
    # Consumers may take up to 3 minutes per message
    ingestion = { visibility_timeout_seconds = 180 }

    notifications = { visibility_timeout_seconds = 60 }

    # Wait a minute before retrying, keep failed messages for a week
    retry = {
      delay_seconds             = 60
      message_retention_seconds = 604800
    }

    # Security needs 14 days of audit history
    audit = {
      message_retention_seconds = 1209600
      tags                      = { Owner = "security" }
    }

    # Events must be processed in order
    ordered_events = {
      fifo_queue                  = true
      content_based_deduplication = true
    }
  }
}

This map is the main thing reviewers will read, so it should read almost like documentation. You can see at a glance which queues hold messages longer, which one delays delivery, and which one is ordered. A queue that needs only default values would simply be name = {}.

Step 4: One module block for all queues
# queues.tf
module "sqs" {
  source   = "./modules/sqs"
  for_each = local.queues

  name   = format("%s-%s", each.key, var.aws_stage)
  config = each.value
}

for_each creates one module instance for every entry in local.queues. Inside the block, each.key is the queue’s key, such as ingestion, and each.value holds its overrides.

The name combines the key and the environment. In dev, the ingestion queue becomes ingestion-dev, and the ordered queue becomes ordered_events-dev.fifo after the module adds the suffix.

The overrides go to the module as they are. Terraform fills in everything that’s missing, so the ingestion queue’s config arrives in the module as:

{
  visibility_timeout_seconds  = 180
  message_retention_seconds   = 345600
  delay_seconds               = 0
  fifo_queue                  = false
  content_based_deduplication = false
  tags                        = {}
}

In the state, each queue gets its own address based on its key: module.sqs["ingestion"], module.sqs["audit"], and so on. These keys are what Terraform uses to track the resources, which becomes important later.

Using the queues elsewhere

The same keys are used to reference individual queues from the rest of the configuration:

module.sqs["ingestion"].url   # passed to a consumer as an environment variable
module.sqs["audit"].arn       # used in an IAM policy

If another configuration needs all queues, expose them as one map:

# outputs.tf
output "queues" {
  description = "ARN and URL of every queue, keyed by queue name."
  value = {
    for key, queue in module.sqs : key => {
      arn = queue.arn
      url = queue.url
    }
  }
}

The ingestion key now means the same thing everywhere: in the queue definition, in the state, and in the outputs.

Adding a new queue

When the billing service needs a queue, the change is one entry:

billing = {
  visibility_timeout_seconds = 120
  tags = {
    Owner = "finance"
  }
}

The new queue automatically gets the naming convention, the default settings, encryption, the common tags, and the validation. The pull request contains one line, and the reviewer only has to check whether a 120-second timeout makes sense for billing.

When defaults depend on the environment

Module defaults are the same everywhere, but sometimes a value should differ between environments. Production might keep audit messages longer than development, for example.

For a single queue, a conditional in the map is enough:

locals {
  is_prod = var.aws_stage == "prod"

  queues = {
    audit = {
      message_retention_seconds = local.is_prod ? 1209600 : 345600
    }
  }
}

If many queues share the same environment-specific settings, define those settings once and merge them into each queue. Put them first, so each queue’s own values still win:

config = merge(local.stage_defaults, each.value)

Keep in mind that merge() is shallow. If both objects contain a nested map, such as tags, the second map replaces the first one entirely. The two maps are not combined. That is one more reason to leave common tags to the provider.

*** Typos are ignored, not reported. Attributes that don’t exist in the module’s config type are silently dropped, so a misspelled setting quietly falls back to its default. Check the plan to confirm each queue got the values you intended.

An alternative: separate variables and module calls

The pattern in this post isn’t the only option. Many teams prefer a more traditional module, where every setting is its own variable with a default value:

# modules/sqs/variables.tf
variable "name" {
  type = string
}

variable "visibility_timeout_seconds" {
  type    = number
  default = 30
}

variable "message_retention_seconds" {
  type    = number
  default = 345600
}

variable "fifo_queue" {
  type    = bool
  default = false
}

Each queue then gets its own module block and passes only what differs from the defaults:

# queues.tf
module "ingestion_queue" {
  source = "./modules/sqs"

  name                       = format("ingestion-%s", var.aws_stage)
  visibility_timeout_seconds = 180
}

module "audit_queue" {
  source = "./modules/sqs"

  name                      = format("audit-%s", var.aws_stage)
  message_retention_seconds = 1209600
}

This approach is easy to read, even for people who are new to Terraform. Each queue is a separate block, and you can find it by searching for its name. For two or three queues, or for queues that differ a lot from each other, it is often the more practical choice.

The cost appears as the number of queues grows. Every new queue adds a whole module block, and the outputs have to be listed one by one. Combining this module with for_each is also less convenient: you have to pass every argument explicitly. To make a missing value fall back to its default, you have to set nullable = false on each variable. The optional() approach handles that automatically.

Technically, the optional() approach is stronger because the whole queue configuration is one typed object. Terraform validates it in one place and fills in the defaults during type conversion. That object can also be passed straight from a map to for_each without listing arguments one by one, so adding a new setting to the module never requires changes to the module call.

Conslusion

One Terraform module for many resources comes down to three decisions. The module defines what a normal resource looks like through optional() defaults and enforces the rules the service imposes. The root configuration lists each resource and describes only what makes it different. A single for_each module block creates them all, while the provider takes care of the tags every resource shares.

We used SQS queues as the example, but nothing here is specific to SQS. The same approach works for SNS topics, S3 buckets, Lambda functions, DynamoDB tables, CloudWatch alarms, or any group of similar resources that differ only in a few settings. Replace the queue settings in the config object with that resource’s arguments, and the module call, the map in locals, and the outputs stay almost identical.

Each piece of information lives in exactly one place, and adding a new resource becomes a one-line change that is easy to review.

Author

maki96milosavljevic@gmail.com

Practical notes about AWS, Terraform, DevOps, automation, and building systems that are easier to operate.