Terraform on Software Engineering Learning Paths

Infrastructure as Code Concepts

Mon, 01 Jan 0001 00:00:00 +0000

The Problem IaC Solves#

Before IaC, infrastructure was managed through manual processes — clicking through cloud consoles, running ad-hoc scripts, and documenting steps in wikis that quickly became outdated. This created several problems:

flowchart TD
 Manual["Manual Infrastructure"] --> Drift["Configuration Drift<br>Servers differ over time"]
 Manual --> Undoc["Undocumented Changes<br>Who changed what? When?"]
 Manual --> Slow["Slow Provisioning<br>Hours/days to set up environments"]
 Manual --> Error["Human Error<br>Typos, missed steps, wrong order"]
 Manual --> NoRepro["Not Reproducible<br>Can't recreate environments reliably"]

IaC eliminates these problems by treating infrastructure the same way we treat application code — written in files, stored in version control, reviewed by peers, and deployed through automation.

HCL Syntax

Mon, 01 Jan 0001 00:00:00 +0000

What is HCL?#

HCL (HashiCorp Configuration Language) is a domain-specific language designed for infrastructure configuration. It’s not a general-purpose programming language — it’s purpose-built to be human-readable, machine-parseable, and expressive enough to describe complex infrastructure.

Block Structure#

Everything in HCL is organized into blocks:

block_type "label_1" "label_2" {
 argument = value
 
 nested_block {
 nested_argument = value
 }
}

Common Block Types#

Block	Labels	Purpose
`resource`	type, name	Create infrastructure
`data`	type, name	Read existing infrastructure
`variable`	name	Declare input
`output`	name	Expose values
`locals`	(none)	Define computed values
`module`	name	Use a reusable module
`provider`	name	Configure a provider
`terraform`	(none)	Terraform settings

# Resource block: 2 labels (type + local name)
resource "aws_s3_bucket" "data_lake" {
 bucket = "my-data-lake"
}

# Variable block: 1 label (name)
variable "region" {
 type = string
 default = "eu-central-1"
}

# Locals block: no labels
locals {
 prefix = "myapp-${var.environment}"
}

Types and Values#

Primitive Types#

# String
name = "web-server"

# Number (integer or float)
port = 8080
timeout = 3.5

# Boolean
enabled = true

Collection Types#

# List (ordered, duplicates allowed)
availability_zones = ["eu-central-1a", "eu-central-1b", "eu-central-1c"]

# Map (key-value pairs)
tags = {
 Environment = "prod"
 Team = "platform"
}

# Set (unordered, no duplicates)
allowed_ips = toset(["10.0.0.1", "10.0.0.2"])

Structural Types#

# Object (fixed structure with named attributes)
variable "database" {
 type = object({
 engine = string
 instance_class = string
 storage_gb = number
 multi_az = bool
 })
}

# Tuple (fixed-length, mixed types)
variable "rule" {
 type = tuple([string, number, bool])
 # Example: ["allow", 443, true]
}

Type Constraints#

variable "ports" {
 type = list(number) # list of numbers only
}

variable "tags" {
 type = map(string) # map with string values only
}

variable "services" {
 type = list(object({
 name = string
 port = number
 path = optional(string, "/") # optional with default
 }))
}

Expressions#

String Interpolation and Templates#

# Simple interpolation
name = "app-${var.environment}-${var.region}"

# Heredoc (multi-line strings)
policy = <<-EOT
 {
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": "s3:GetObject",
 "Resource": "arn:aws:s3:::${var.bucket_name}/*"
 }]
 }
EOT

# Directive (template logic)
user_data = <<-EOT
 #!/bin/bash
 %{ for port in var.ports ~}
 ufw allow ${port}
 %{ endfor ~}
EOT

Operators#

# Arithmetic
total = var.base_count * var.multiplier

# Comparison
is_large = var.instance_count > 10

# Logical
needs_scaling = var.is_production && var.high_traffic

Conditional Expressions#

# condition ? true_value : false_value
instance_type = var.environment == "prod" ? "t3.large" : "t3.micro"

# Nested (avoid — use locals for readability)
size = var.tier == "large" ? 100 : var.tier == "medium" ? 50 : 20

For Expressions#

# Transform a list
upper_names = [for name in var.names : upper(name)]

# Filter a list
prod_instances = [for i in var.instances : i if i.environment == "prod"]

# List to map
instance_map = { for i in var.instances : i.name => i.id }

# Map transformation
uppercased = { for k, v in var.tags : k => upper(v) }

Splat Expressions#

# Shorthand for [for o in list : o.attribute]
instance_ids = aws_instance.web[*].id
subnet_cidrs = aws_subnet.private[*].cidr_block

# Equivalent to:
instance_ids = [for i in aws_instance.web : i.id]

References#

Resource References#

resource "aws_vpc" "main" {
 cidr_block = "10.0.0.0/16"
}

resource "aws_subnet" "public" {
 vpc_id = aws_vpc.main.id # reference another resource
 cidr_block = cidrsubnet(aws_vpc.main.cidr_block, 8, 0)
}

Reference Types#

Reference	Syntax	Example
Resource attribute	`type.name.attribute`	`aws_vpc.main.id`
Variable	`var.name`	`var.environment`
Local	`local.name`	`local.common_tags`
Module output	`module.name.output`	`module.vpc.vpc_id`
Data source	`data.type.name.attribute`	`data.aws_ami.latest.id`
Count index	`count.index`	`count.index`
Each key/value	`each.key`, `each.value`	`each.value.port`
Self (provisioners)	`self.attribute`	`self.public_ip`

Dynamic Blocks#

Generate repeated nested blocks from a collection:

Providers, Resources, and Data Sources

Mon, 01 Jan 0001 00:00:00 +0000

Providers#

Providers are Terraform’s plugin system. Each provider is a bridge between Terraform and an external API — AWS, Azure, GCP, Kubernetes, GitHub, Datadog, or any service with an API.

flowchart TD
 TF["Terraform Core"] --> AWS["AWS Provider"]
 TF --> Azure["Azure Provider"]
 TF --> K8s["Kubernetes Provider"]
 TF --> GH["GitHub Provider"]
 
 AWS --> AWSAPI["AWS API"]
 Azure --> AzureAPI["Azure API"]
 K8s --> K8sAPI["Kubernetes API"]
 GH --> GHAPI["GitHub API"]

Provider Configuration#

terraform {
 required_providers {
 aws = {
 source = "hashicorp/aws"
 version = "~> 5.0"
 }
 random = {
 source = "hashicorp/random"
 version = "~> 3.6"
 }
 }
}

provider "aws" {
 region = var.aws_region
 
 default_tags {
 tags = {
 Environment = var.environment
 ManagedBy = "terraform"
 Team = var.team
 }
 }
}

Version Constraints#

Constraint	Meaning	Example
`= 5.0.0`	Exact version	Only 5.0.0
`>= 5.0`	Minimum version	5.0 or higher
`~> 5.0`	Pessimistic (minor)	≥ 5.0, < 6.0
`~> 5.0.0`	Pessimistic (patch)	≥ 5.0.0, < 5.1.0
`>= 5.0, < 6.0`	Range	Between 5.0 and 6.0

Best practice: Use ~> MAJOR.MINOR for providers (allows patch updates, blocks breaking changes).

State

Mon, 01 Jan 0001 00:00:00 +0000

What is State?#

State is Terraform’s memory. It’s a JSON file that maps your configuration to real-world infrastructure. Without state, Terraform wouldn’t know which cloud resources it created, what their current attributes are, or what needs to change.

flowchart TD
 Config["Configuration (.tf files)<br>What you WANT"] 
 State["State File<br>What Terraform KNOWS"]
 Reality["Real Infrastructure<br>What ACTUALLY EXISTS"]
 
 Config --> |"terraform plan"| Compare["Compare all three"]
 State --> Compare
 Reality --> Compare
 Compare --> Plan["Execution Plan<br>(create, update, destroy)"]

Why State Exists#

Terraform needs state because:

Variables, Outputs, and Locals

Mon, 01 Jan 0001 00:00:00 +0000

Input Variables#

Variables parameterize your Terraform configuration — the same code works across environments by changing variable values.

Declaration#

variable "environment" {
 description = "Deployment environment (dev, staging, prod)"
 type = string
 default = "dev"
}

variable "instance_count" {
 description = "Number of EC2 instances to create"
 type = number
 # No default — Terraform will prompt or require a value
}

variable "enable_monitoring" {
 description = "Whether to enable detailed monitoring"
 type = bool
 default = false
}

Type Constraints#

# Simple types
variable "name" { type = string }
variable "port" { type = number }
variable "enabled" { type = bool }

# Collections
variable "availability_zones" {
 type = list(string)
}

variable "tags" {
 type = map(string)
}

variable "allowed_ports" {
 type = set(number)
}

# Complex objects
variable "database_config" {
 type = object({
 engine = string
 instance_class = string
 storage_gb = number
 multi_az = bool
 backup_window = optional(string, "03:00-04:00")
 })
}

# List of objects
variable "services" {
 type = list(object({
 name = string
 port = number
 cpu = number
 memory = number
 replicas = optional(number, 1)
 }))
}

Validation#

variable "environment" {
 type = string
 
 validation {
 condition = contains(["dev", "staging", "prod"], var.environment)
 error_message = "Environment must be dev, staging, or prod."
 }
}

variable "cidr_block" {
 type = string
 
 validation {
 condition = can(cidrhost(var.cidr_block, 0))
 error_message = "Must be a valid CIDR block (e.g., 10.0.0.0/16)."
 }
}

variable "instance_type" {
 type = string
 
 validation {
 condition = startswith(var.instance_type, "t3.") || startswith(var.instance_type, "t4g.")
 error_message = "Only t3 and t4g instance families are allowed."
 }
}

Sensitive Variables#

variable "db_password" {
 description = "Database master password"
 type = string
 sensitive = true # hidden in plan/apply output
}

Setting Variable Values#

flowchart TD
 A["How does Terraform get variable values?"]
 A --> B["1: -var flag (highest priority)"]
 A --> C["2: -var-file flag"]
 A --> D["3: *.auto.tfvars (auto-loaded)"]
 A --> E["4: terraform.tfvars (auto-loaded)"]
 A --> F["5: TF_VAR_* env variables"]
 A --> G["6: default value in variable block"]
 A --> H["7: Interactive prompt (lowest)"]

tfvars Files#

# config/variables/prod.tfvars
environment = "prod"
instance_count = 3
instance_type = "t3.large"
enable_monitoring = true

database_config = {
 engine = "postgres"
 instance_class = "db.r6g.large"
 storage_gb = 100
 multi_az = true
}

terraform plan -var-file="config/variables/prod.tfvars"

Environment Variables#

export TF_VAR_environment="prod"
export TF_VAR_db_password="secret123"
terraform plan

Auto-loaded Files#

Files matching these patterns are loaded automatically:

Modules

Mon, 01 Jan 0001 00:00:00 +0000

What is a Module?#

A module is a container for multiple resources that are used together. Every Terraform configuration is a module — the root module. When you call another module, it becomes a child module.

flowchart TD
 Root["Root Module<br>(environments/prod/main.tf)"]
 Root --> VPC["module.vpc<br>VPC + Subnets + IGW + NAT"]
 Root --> ECS["module.ecs_cluster<br>Cluster + Capacity Providers"]
 Root --> DB["module.database<br>RDS + Subnet Group + SG"]
 
 ECS --> SVC["module.api_service<br>Service + Task Def + ALB Rule"]
 
 VPC -.->|"vpc_id, subnet_ids"| ECS
 VPC -.->|"vpc_id, subnet_ids"| DB
 DB -.->|"connection_string"| SVC

Modules provide:

Lifecycle Rules and Dependencies

Mon, 01 Jan 0001 00:00:00 +0000

Dependency Graph#

Terraform builds a directed acyclic graph (DAG) of all resources and their dependencies. This graph determines the order of operations.

flowchart TD
 VPC["aws_vpc.main"] --> Subnet["aws_subnet.public"]
 VPC --> SG["aws_security_group.web"]
 Subnet --> Instance["aws_instance.web"]
 SG --> Instance
 Instance --> EIP["aws_eip.web"]

Terraform creates resources top-down (VPC first, EIP last) and destroys bottom-up (EIP first, VPC last).

Implicit Dependencies#

Most dependencies are automatic — Terraform detects them from attribute references:

resource "aws_vpc" "main" {
 cidr_block = "10.0.0.0/16"
}

# Implicit dependency: references aws_vpc.main.id
resource "aws_subnet" "public" {
 vpc_id = aws_vpc.main.id
 cidr_block = "10.0.1.0/24"
}

# Implicit dependency: references aws_subnet.public.id
resource "aws_instance" "web" {
 subnet_id = aws_subnet.public.id
 ami = data.aws_ami.latest.id
}

Terraform automatically knows: VPC → Subnet → Instance.

Workspaces and Environment Management

Mon, 01 Jan 0001 00:00:00 +0000

The Problem#

You need the same infrastructure in multiple environments (dev, staging, prod) with different configurations. How do you avoid duplicating your entire Terraform codebase?

flowchart TD
 Code["Single Terraform Codebase"]
 Code --> Dev["Dev<br>t3.micro, 1 instance, no HA"]
 Code --> Staging["Staging<br>t3.small, 2 instances, single AZ"]
 Code --> Prod["Prod<br>t3.large, 3 instances, multi-AZ"]

Approach 1: Workspaces#

Terraform workspaces maintain separate state files for the same configuration:

# Create workspaces
terraform workspace new dev
terraform workspace new staging
terraform workspace new prod

# Switch workspace
terraform workspace select prod

# List workspaces
terraform workspace list
# default
# dev
# staging
# * prod

# Show current
terraform workspace show
# prod

Using Workspace in Configuration#

locals {
 env_config = {
 dev = {
 instance_type = "t3.micro"
 count = 1
 multi_az = false
 }
 staging = {
 instance_type = "t3.small"
 count = 2
 multi_az = false
 }
 prod = {
 instance_type = "t3.large"
 count = 3
 multi_az = true
 }
 }
 
 config = local.env_config[terraform.workspace]
}

resource "aws_instance" "web" {
 count = local.config.count
 instance_type = local.config.instance_type
}

Workspace State Storage#

With S3 backend, workspaces create separate state paths:

Provisioners

Mon, 01 Jan 0001 00:00:00 +0000

What Are Provisioners?#

Provisioners execute scripts or commands on resources after creation. They’re Terraform’s escape hatch for tasks that can’t be expressed declaratively.

flowchart LR
 Create["Resource Created"] --> Prov["Provisioner Runs"]
 Prov --> Done["Resource Ready"]
 
 Prov -.->|"If fails"| Taint["Resource Tainted<br>(marked for recreation)"]

Important: Provisioners are a last resort. HashiCorp explicitly recommends avoiding them when possible because they break Terraform’s declarative model.

Types of Provisioners#

remote-exec#

Runs commands on the remote resource via SSH or WinRM:

Import, State Manipulation, and Refactoring

Mon, 01 Jan 0001 00:00:00 +0000

Why Manipulate State?#

As infrastructure evolves, you need to:

Import existing resources into Terraform management
Rename resources without destroying them
Move resources into or out of modules
Remove resources from Terraform without destroying them
Split large state files into smaller ones

All of these require state manipulation — changing what Terraform knows without changing the actual infrastructure.

Importing Existing Resources#

The Problem#

You have infrastructure created manually (or by another tool) that you now want Terraform to manage.

CI/CD Integration

Mon, 01 Jan 0001 00:00:00 +0000

Why Automate Terraform?#

Running Terraform manually introduces risk:

Wrong environment targeted
Unapproved changes applied
No audit trail
State corruption from concurrent runs
Inconsistent workflows between team members

CI/CD pipelines enforce a consistent, safe, auditable workflow.

flowchart LR
 Dev["Developer"] -->|"Push code"| MR["Merge Request"]
 MR -->|"Auto"| Plan["terraform plan"]
 Plan -->|"Review"| Approve["Peer Approval"]
 Approve -->|"Merge"| Apply["terraform apply"]
 Apply -->|"Auto"| Notify["Notification"]

The Standard Workflow#

flowchart TD
 subgraph MR["On Merge Request"]
 Fmt["terraform fmt -check"]
 Fmt --> Validate["terraform validate"]
 Validate --> Plan["terraform plan"]
 Plan --> Comment["Post plan as MR comment"]
 end
 
 subgraph Merge["On Merge to Main"]
 Init["terraform init"]
 Init --> PlanProd["terraform plan -out=tfplan"]
 PlanProd --> ApplyProd["terraform apply tfplan"]
 ApplyProd --> Notify["Notify team"]
 end
 
 MR -->|"Approved + Merged"| Merge

Key Principles#

Plan on MR — everyone sees what will change before it happens
Apply on merge — only approved changes reach infrastructure
Save plan artifact — what was reviewed is exactly what gets applied
Never auto-approve production — require manual gate for prod

GitLab CI Example#

# .gitlab-ci.yml
include:
 - project: dx/cloud/ci-cd/terraform/terraform-project-pipeline
 ref: v2.6.2
 file: terraform-pipeline.gitlab-ci.yml

variables:
 TERRAFORM_IMAGE_VERSION: "1.11.4"
 ENABLE_TEST_ENV: "true"
 ENABLE_PRE_ENV: "true"
 ENABLE_PROD_ENV: "true"
 AWS_DEPLOY_ACCOUNT_ALIAS: "TUIDS"

stages:
 - lint
 - plan
 - apply

Custom Pipeline (Without Shared Template)#

stages:
 - validate
 - plan
 - apply

variables:
 TF_ROOT: "states/main"

.terraform-base:
 image: hashicorp/terraform:1.11.4
 before_script:
 - cd $TF_ROOT
 - terraform init -backend-config=../../config/backend/${ENVIRONMENT}.tfbackend

validate:
 extends: .terraform-base
 stage: validate
 variables:
 ENVIRONMENT: test
 script:
 - terraform fmt -check -recursive
 - terraform validate
 rules:
 - if: $CI_MERGE_REQUEST_IID

plan:test:
 extends: .terraform-base
 stage: plan
 variables:
 ENVIRONMENT: test
 script:
 - terraform plan -var-file=../../config/variables/test.tfvars -out=tfplan
 artifacts:
 paths: [states/main/tfplan]
 expire_in: 1 day
 rules:
 - if: $CI_MERGE_REQUEST_IID

plan:prod:
 extends: .terraform-base
 stage: plan
 variables:
 ENVIRONMENT: prod
 script:
 - terraform plan -var-file=../../config/variables/prod.tfvars -out=tfplan
 artifacts:
 paths: [states/main/tfplan]
 expire_in: 1 day
 rules:
 - if: $CI_COMMIT_BRANCH == "main"

apply:test:
 extends: .terraform-base
 stage: apply
 variables:
 ENVIRONMENT: test
 script:
 - terraform apply tfplan
 dependencies: [plan:test]
 rules:
 - if: $CI_COMMIT_BRANCH == "main"

apply:prod:
 extends: .terraform-base
 stage: apply
 variables:
 ENVIRONMENT: prod
 script:
 - terraform apply tfplan
 dependencies: [plan:prod]
 when: manual  # require human click
 rules:
 - if: $CI_COMMIT_BRANCH == "main"

GitHub Actions Example#

name: Terraform
on:
 pull_request:
 branches: [main]
 push:
 branches: [main]

jobs:
 plan:
 runs-on: ubuntu-latest
 if: github.event_name == 'pull_request'
 steps:
 - uses: actions/checkout@v4
 - uses: hashicorp/setup-terraform@v3
 with:
 terraform_version: 1.11.4
 
 - run: terraform init
 - run: terraform fmt -check
 - run: terraform validate
 - run: terraform plan -no-color -out=tfplan
 
 - uses: actions/upload-artifact@v4
 with:
 name: tfplan
 path: tfplan

 apply:
 runs-on: ubuntu-latest
 if: github.ref == 'refs/heads/main' && github.event_name == 'push'
 environment: production  # requires approval
 steps:
 - uses: actions/checkout@v4
 - uses: hashicorp/setup-terraform@v3
 - run: terraform init
 - run: terraform apply -auto-approve

Plan Artifacts#

The plan file captures the exact changes that were reviewed:

Best Practices

Mon, 01 Jan 0001 00:00:00 +0000

Code Organization#

File Structure#

project/
├── main.tf # Primary resources or module calls
├── variables.tf # All input variables
├── outputs.tf # All outputs
├── locals.tf # Local values
├── data.tf # Data sources
├── providers.tf # Provider configuration
├── versions.tf # Required providers and Terraform version
├── backend.tf # Backend configuration
└── README.md # Documentation

For larger projects, group by service:

states/main/
├── backend.tf
├── provider.tf
├── variables.tf
├── locals.tf
├── data.tf
├── outputs.tf
└── services/
 ├── api/
 │ ├── ecs.tf
 │ ├── alb.tf
 │ └── outputs.tf
 └── worker/
 ├── ecs.tf
 ├── sqs.tf
 └── outputs.tf

Naming Conventions#

# Resources: snake_case, descriptive
resource "aws_security_group" "api_server" { }
resource "aws_iam_role" "ecs_task_execution" { }
resource "aws_s3_bucket" "application_logs" { }

# Variables: snake_case, include unit if ambiguous
variable "cache_ttl_seconds" { }
variable "max_instance_count" { }
variable "volume_size_gb" { }

# Outputs: include resource context
output "vpc_id" { }
output "api_load_balancer_dns" { }
output "database_endpoint" { }

# Locals: snake_case, descriptive
locals {
 name_prefix = "${var.project}-${var.environment}"
 is_production = var.environment == "prod"
}

Single Resource Naming#

When a module has only one of a resource type, use this or main:

Testing & Validation

Mon, 01 Jan 0001 00:00:00 +0000

Testing & Validation#

Terraform code manages production infrastructure. Bugs don’t just break builds — they delete databases, expose services to the internet, or incur unexpected costs. Testing Terraform is not optional.

The Terraform Testing Pyramid#

 ╱╲
 ╱ ╲ E2E (apply to real cloud, verify)
 ╱────╲
 ╱ ╲ Integration (plan against real providers)
 ╱────────╲
 ╱ ╲ Static analysis (lint, validate, scan)
╱────────────╲

Level	Speed	Cost	Catches
Static analysis	Seconds	Free	Syntax errors, policy violations, security misconfigurations
Integration (plan)	Minutes	Minimal (API calls for plan, no creates)	Configuration logic errors, dependency issues
E2E (apply + verify)	Minutes-hours	Cloud resource costs	Real infrastructure behaviour

Level 1: Static Analysis#

`terraform validate`#

Checks syntax and internal consistency (no cloud API calls):

Terraform on Software Engineering Learning Paths

Infrastructure as Code Concepts

The Problem IaC Solves#

HCL Syntax

What is HCL?#

Block Structure#

Common Block Types#

Types and Values#

Primitive Types#

Collection Types#

Structural Types#

Type Constraints#

Expressions#

String Interpolation and Templates#

Operators#

Conditional Expressions#

For Expressions#

Splat Expressions#

References#

Resource References#

Reference Types#

Dynamic Blocks#

Providers, Resources, and Data Sources

Providers#

Provider Configuration#

Version Constraints#

State

What is State?#

Why State Exists#

Variables, Outputs, and Locals

Input Variables#

Declaration#

Type Constraints#

Validation#

Sensitive Variables#

Setting Variable Values#

tfvars Files#

Environment Variables#

Auto-loaded Files#

Modules

What is a Module?#

Lifecycle Rules and Dependencies

Dependency Graph#

Implicit Dependencies#

Workspaces and Environment Management

The Problem#

Approach 1: Workspaces#

Using Workspace in Configuration#

Workspace State Storage#

Provisioners

What Are Provisioners?#

Types of Provisioners#

remote-exec#

Import, State Manipulation, and Refactoring

Why Manipulate State?#

Importing Existing Resources#

The Problem#

CI/CD Integration

Why Automate Terraform?#

The Standard Workflow#

Key Principles#

GitLab CI Example#

Custom Pipeline (Without Shared Template)#

GitHub Actions Example#

Plan Artifacts#

Best Practices

Code Organization#

File Structure#

Naming Conventions#

Single Resource Naming#

Testing & Validation

Testing & Validation#

The Terraform Testing Pyramid#

Level 1: Static Analysis#

terraform validate#

`terraform validate`#