How to Use Terraform With Local AWS (Emulated With LocalStack)?

Recently I was working on a project that involved provisioning an environment on AWS using Terraform, in order to test the setup during development I leaned on LocalStack to emulate AWS services to have a fast feedback loop for testing and avoid incurring costs.

I faced an issue as I couldn’t figure out how could Terraform connect to my local AWS (LocalStack) instead of the actual one seamlessly, some online resources suggeted installing additional tools like tflocal or awslocal OR changing the aws provider configuration to use LocalStack endpoints.

As I wasn’t very keen on neiter installing additional tools nor changing my terraform code just for testing, I keept digging into it till I came up with another solution.

This post will show you how to use Terraform with AWS LocalStack to test your infrastructure code locally.

TL;DR

  1. Define a LocalStack profile in ~/.aws/config
  2. Use the profile with aws --profile xyz commands or set AWS_PROFILE environment variable (Optionally use direnv to automatically set the profile for a directory)
  3. Having done that, you can use test your terraform code locally using LocalStack as an emulated cloud provider.
  4. Sourcecode for this post can be found at my terraform-aws-localstack-demo repository.

Prerequisites

LocalStack

LocalStack is a pretty cool cloud service emulator that allows you to mimic cloud services locally and for free, it supports different cloud providers like AWS and Azure.

AWS CLI

Typically one would use AWS CLI tool to interact with AWS services from the console, issuing commands like aws s3 ls for listing S3 buckets for example.

But how can one use the same commands to connect to LocalStack under the hood instead of the AWS cloud? Named AWS CLI Profiles come to the rescue!

If there’s no profile specified, AWS CLI uses the default profile, to use a different profile you can use the --profile flag like so aws s3 ls --profile xyz after defining xyz profile in the AWS CLI config file - typically located at ~/.aws/config.

In order to leverage the named profiles for LocalStack, we could define a profile named localstack in the ~/.aws/config file with the following content:

[profile localstack]
region = eu-central-1
output = json
endpoint_url = http://localhost:4566
  • endpoint_url is the URL of the LocalStack instance, change it if you’re running LocalStack on a different port or host than the default.
Info

s3 SDK

Note there’s a special case for s3 on LocalStack, In order for LocalStack to be able to parse the bucket name from your request, your endpoint needs to be prefixed with s3. otherwise some commands might fail (e.g. bucket creation). For more details check LocalStack S3 documentation

In order to fix this we can specifiy a specific endpoint for s3 service in our AWS CLI config file, the final configuration would be:

[profile localstack]
region = eu-central-1
output = json
endpoint_url = http://localhost:4566
services = localstack-s3

[services localstack-s3]
s3 =
  endpoint_url = http://s3.localhost:4566

In order to use this profile, you can use the --profile flag like so aws s3 ls --profile localstack, or set the AWS_PROFILE environment variable and aws command will pick up this profile.

export AWS_PROFILE=localstack
aws s3 ls
Info

I personally prefer to use direnv to manage environment variables on the directory level, in order to use it after installation run the following commands:

cd /path/to/your/project
echo 'export AWS_PROFILE=localstack' > .envrc
direnv allow .
aws s3 ls

Terraform

Terraform is an infrastructure as code tool that allows you to define and provision infrastructure resources in a declarative way. It supports different cloud providers like AWS, Azure, and GCP.

An example Terraform project that provisions an ec2 instance and s3 bucket:

# main.tf
locals {
  aws_region = "eu-central-1"
  aws_ec2_ami = "ami-0387413ed05eb20af"
  aws_ec2_instance_type = "t3.micro"
  aws_ec2_instance_name = "my-server"
  aws_s3_bucket_name = "my-bucket"
}

provider "aws" {
  region = local.aws_region
}

resource "aws_instance" "my_server" {
  ami           = local.aws_ec2_ami
  instance_type = local.aws_ec2_instance_type

  tags = {
    Name = local.aws_ec2_instance_name
  }
}

resource "aws_s3_bucket" "my_bucket" {
  bucket = local.aws_s3_bucket_name
}

After provision the infrastructure, you can verify the resources are created:

Verify s3 bucket is created ✅

aws s3api get-bucket-location --bucket my-bucket
#{
#    "LocationConstraint": "eu-central-1"
#}

Verify ec2 instance is created ✅

aws ec2 describe-instances \
  --filters "Name=tag:Name,Values=my-server" \
  --query "Reservations[].Instances[].{ID: InstanceId, State: State.Name}" \
  --output table
# ---------------------------------------
# |          DescribeInstances          |
# +----------------------+--------------+
# |          ID          |    State     |
# +----------------------+--------------+
# |  i-313c9fa1249eade53 |  running     |
# +----------------------+--------------+

I hope that this post was helpful for you, if you experienced the same issue and had a different solution let me know in the comments below!

A demo project for this post can be found at https://gitlab.com/bigsolom/terraform-aws-localstack-demo

comments

comments powered by Disqus