An intro to Docker, Terraform, and Amazon ECS

A quick intro to
Docker,
Terraform, and
Amazon ECS
TERRAFORM
Amazon ECS

In this talk, we’ll show how to
deploy two apps:

A Rails Frontend and a
Sinatra Backend

Slides and code from this talk:
ybrikman.com/speaking

class ApplicationController < ActionController::Base
def index
url = URI.parse(backend_addr)
req = Net::HTTP::Get.new(url.to_s)
res = Net::HTTP.start(url.host, url.port) {|http|
http.request(req)
}
@text = res.body
end
end
The rails frontend calls the sinatra
backend…

<h1>Rails Frontend</h1>
<p>
Response from the backend: <strong><%= @text %></strong>
</p>
And renders the response as
HTML.

We’ll package the two apps as
Docker containers…

Amazon ECS
Deploy those Docker containers
using Amazon ECS…

TERRAFORM
And define our infrastructure-as-
code using Terraform.

I’m
Yevgeniy
Brikman
ybrikman.com

Co-founder of
Gruntwork
gruntwork.io

gruntwork.io
We offer DevOps
as a Service

gruntwork.io
And DevOps
as a Library

Author of
Hello,
Startup
hello-startup.net

And
Terraform:
Up & Running
terraformupandrunning.com

1. Docker
2. Terraform
3. ECS
4. Recap
Outline

Docker allows you to build and
run code in containers

Containers are like lightweight
Virtual Machines (VMs)

Like an isolated process that
happens to be an entire OS

> docker run –it ubuntu bash
root@12345:/# echo

> time docker run ubuntu echo "Hello, World"
Hello, World
real 0m0.183s
user 0m0.009s
sys 0m0.014s
Containers boot quickly, with
minimal CPU/memory overhead

You can define a Docker image
as code in a Dockerfile

FROM gliderlabs/alpine:3.3
RUN apk --no-cache add ruby ruby-dev
RUN gem install sinatra --no-ri --no-rdoc
RUN mkdir -p /usr/src/app
COPY . /usr/src/app
WORKDIR /usr/src/app
EXPOSE 4567
CMD ["ruby", "app.rb"]
Here is the Dockerfile for the
Sinatra backend

FROM gliderlabs/alpine:3.3
RUN apk --no-cache add ruby ruby-dev
RUN gem install sinatra --no-ri --no-rdoc
COPY . /usr/src/app
EXPOSE 4567
CMD ["ruby", "app.rb"]
It specifies dependencies, code,
config, and how to run the app

> docker build -t gruntwork/sinatra-backend .
Step 0 : FROM gliderlabs/alpine:3.3
---> 0a7e169bce21
(...)
Step 8 : CMD ruby app.rb
---> 2e243eba30ed
Successfully built 2e243eba30ed
Build the Docker image

> docker run -it -p 4567:4567 gruntwork/sinatra-backend
INFO WEBrick 1.3.1
INFO ruby 2.2.4 (2015-12-16) [x86_64-linux-musl]
== Sinatra (v1.4.7) has taken the stage on 4567 for
development with backup from WEBrick
INFO WEBrick::HTTPServer#start: pid=1 port=4567
Run the Docker image

> docker push gruntwork/sinatra-backend
The push refers to a repository [docker.io/gruntwork/sinatra-
backend] (len: 1)
2e243eba30ed: Image successfully pushed
7e2e0c53e246: Image successfully pushed
919d9a73b500: Image successfully pushed
(...)
v1: digest: sha256:09f48ed773966ec7fe4558 size: 14319
You can share your images by
pushing them to Docker Hub

Now you can reuse the same
image in dev, stg, prod, etc

> docker pull rails:4.2.6
And you can reuse images created
by others.

FROM rails:4.2.6
COPY . /usr/src/app
RUN bundle install
EXPOSE 3000
CMD ["rails", "start"]
The rails-frontend is built on top of
the official rails Docker image

rails_frontend:
image: gruntwork/rails-frontend
ports:
- "3000:3000"
links:
- sinatra_backend
sinatra_backend:
image: gruntwork/sinatra-backend
ports:
- "4567:4567"
Define your entire dev stack as
code with docker-compose

rails_frontend:
image: gruntwork/rails-frontend
ports:
- "3000:3000"
links:
- sinatra_backend
sinatra_backend:
image: gruntwork/sinatra-backend
ports:
- "4567:4567"
Docker links provide a simple
service discovery mechanism

> docker-compose up
Starting infrastructureascodetalk_sinatra_backend_1
Recreating infrastructureascodetalk_rails_frontend_1
sinatra_backend_1 | INFO WEBrick 1.3.1
sinatra_backend_1 | INFO ruby 2.2.4 (2015-12-16)
sinatra_backend_1 | Sinatra has taken the stage on 4567
rails_frontend_1 | INFO WEBrick 1.3.1
rails_frontend_1 | INFO ruby 2.3.0 (2015-12-25)
rails_frontend_1 | INFO WEBrick::HTTPServer#start: port=3000
Run your entire dev stack with one
command

Terraform is a tool for
provisioning infrastructure

Terraform supports many
providers (cloud agnostic)

And many resources for each
provider

You define infrastructure as code
in Terraform templates

provider "aws" {
region = "us-east-1"
}
resource "aws_instance" "example" {
ami = "ami-408c7f28"
instance_type = "t2.micro"
}
This template creates a single EC2
instance in AWS

> terraform plan
+ aws_instance.example
ami: "" => "ami-408c7f28"
instance_type: "" => "t2.micro"
key_name: "" => "<computed>"
private_ip: "" => "<computed>"
public_ip: "" => "<computed>"
Plan: 1 to add, 0 to change, 0 to destroy.
Use the plan command to see
what you’re about to deploy

> terraform apply
aws_instance.example: Creating...
ami: "" => "ami-408c7f28"
instance_type: "" => "t2.micro"
key_name: "" => "<computed>"
private_ip: "" => "<computed>"
public_ip: "" => "<computed>”
aws_instance.example: Creation complete
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Use the apply command to apply
the changes

Now our EC2 instance is running!

resource "aws_instance" "example" {
ami = "ami-408c7f28"
tags {
Name = "terraform-example"
}
}
Let’s give the EC2 instance a tag
with a readable name

> terraform plan
~ aws_instance.example
tags.#: "0" => "1"
tags.Name: "" => "terraform-example"
Plan: 0 to add, 1 to change, 0 to destroy.
Use the plan command again to
verify your changes

> terraform apply
aws_instance.example: Refreshing state...
aws_instance.example: Modifying...
tags.#: "0" => "1"
tags.Name: "" => "terraform-example"
aws_instance.example: Modifications complete
Use the apply command again to
deploy those changes

Now our EC2 instance has a tag!

resource "aws_elb" "example" {
name = "example"
availability_zones = ["us-east-1a", "us-east-1b"]
instances = ["${aws_instance.example.id}"]
listener {
lb_port = 80
lb_protocol = "http"
instance_port = "${var.instance_port}"
instance_protocol = "http”
}
}
Let’s add an Elastic Load Balancer
(ELB).

name = "example"
listener {
lb_port = 80
instance_protocol = "http”
}
}
Terraform supports variables,
such as var.instance_port

name = "example"
listener {
lb_port = 80
instance_protocol = "http"
}
}
As well as dependencies like
aws_instance.example.id

name = "example"
listener {
lb_port = 80
}
}
It builds a dependency graph and
applies it in parallel.

After running apply, we have an ELB!

> terraform destroy
aws_instance.example: Refreshing state... (ID: i-f3d58c70)
aws_elb.example: Refreshing state... (ID: example)
aws_elb.example: Destroying...
aws_elb.example: Destruction complete
aws_instance.example: Destroying...
aws_instance.example: Destruction complete
Use the destroy command to
delete all your resources

For more info, check out The Comprehensive Guide
to Terraform

EC2 Container Service (ECS) is a
way to run Docker on AWS

ECS Overview
EC2 Instance
ECS Cluster
ECS Scheduler
ECS Agent
ECS Tasks
ECS Task Definition
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Service Definition
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}

ECS Cluster: several servers
managed by ECS
EC2 Instance
ECS Cluster

Typically, the servers are in an
Auto Scaling Group
Auto Scaling Group
EC2 Instance

Each server must run the ECS
Agent
ECS Agent
EC2 Instance
ECS Cluster

ECS Task: Docker container(s)
to run, resources they need
{
"name": "example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
ECS Agent
EC2 Instance
ECS Task Definition
ECS Cluster

ECS Service: long-running ECS
Task & ELB settings
{
"name": "example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"desiredCount": 2
}
ECS Agent
EC2 Instance
ECS Task Definition ECS Service Definition
ECS Cluster

ECS Scheduler: Deploys Tasks
across the ECS Cluster
{
"name": "example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"desiredCount": 2
}
ECS Agent
ECS Tasks
EC2 Instance
ECS Task Definition ECS Service Definition ECS Scheduler
ECS Cluster

You can associate an ALB or
ELB with each ECS service
{
"name": "example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"desiredCount": 2
}
ECS Agent
ECS Tasks
EC2 Instance
ECS Cluster

This allows you to distribute
load across your ECS Tasks
{
"name": "example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"desiredCount": 2
}
ECS Agent
ECS Tasks
EC2 Instance
ECS Cluster

You can also use it as a simple
form of service discovery
{
"name": "example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"desiredCount": 2
}
ECS Agent
ECS Tasks
EC2 Instance
ECS Cluster

Let’s deploy our apps on ECS
using Terraform

Define the ECS Cluster as an
Auto Scaling Group (ASG)
EC2 Instance
ECS Cluster

resource "aws_ecs_cluster" "example_cluster" {
name = "example-cluster"
}
resource "aws_autoscaling_group" "ecs_cluster_instances" {
name = "ecs-cluster-instances"
min_size = 5
max_size = 5
launch_configuration =
"${aws_launch_configuration.ecs_instance.name}"
}

Ensure each server in the ASG
runs the ECS Agent
ECS Agent
EC2 Instance
ECS Cluster

# The launch config defines what runs on each EC2 instance
resource "aws_launch_configuration" "ecs_instance" {
name_prefix = "ecs-instance-"
# This is an Amazon ECS AMI, which has an ECS Agent
# installed that lets it talk to the ECS cluster
image_id = "ami-a98cb2c3”
}
The launch config runs AWS ECS
Linux on each server in the ASG

Define an ECS Task for each
microservice
{
"name": "example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
ECS Agent
EC2 Instance
ECS Task Definition
ECS Cluster

resource "aws_ecs_task_definition" "rails_frontend" {
family = "rails-frontend"
container_definitions = <<EOF
[{
"name": "rails-frontend",
"image": "gruntwork/rails-frontend:v1",
"cpu": 1024,
"memory": 768,
"essential": true,
"portMappings": [{"containerPort": 3000, "hostPort": 3000}]
}]
EOF
}
Rails frontend ECS Task

resource "aws_ecs_task_definition" "sinatra_backend" {
family = "sinatra-backend"
[{
"name": "sinatra-backend",
"image": "gruntwork/sinatra-backend:v1",
"cpu": 1024,
"memory": 768,
"essential": true,
"portMappings": [{"containerPort": 4567, "hostPort": 4567}]
}]
EOF
}
Sinatra Backend ECS Task

Define an ECS Service for each
ECS Task
{
"name": "example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"desiredCount": 2
}
ECS Agent
EC2 Instance
ECS Cluster

resource "aws_ecs_service" "rails_frontend" {
cluster = "${aws_ecs_cluster.example_cluster.id}"
task_definition =
"${aws_ecs_task_definition.rails-fronted.arn}"
desired_count = 2
}
Rails Frontend ECS Service

resource "aws_ecs_service" "sinatra_backend" {
family = "sinatra-backend"
cluster = "${aws_ecs_cluster.example_cluster.id}"
task_definition =
"${aws_ecs_task_definition.sinatra_backend.arn}"
desired_count = 2
}
Sinatra Backend ECS Service

Associate an ELB with each
ECS Service
{
"name": "example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"desiredCount": 2
}
ECS Agent
ECS Tasks
EC2 Instance
ECS Cluster

resource "aws_elb" "rails_frontend" {
name = "rails-frontend"
listener {
lb_port = 80
instance_port = 3000
}
}
Rails Frontend ELB

resource "aws_ecs_service" "rails_frontend" {
(...)
load_balancer {
elb_name = "${aws_elb.rails_frontend.id}"
container_name = "rails-frontend"
container_port = 3000
}
}
Associate the ELB with the Rails
Frontend ECS Service

resource "aws_elb" "sinatra_backend" {
name = "sinatra-backend"
listener {
lb_port = 4567
instance_port = 4567
}
}
Sinatra Backend ELB

resource "aws_ecs_service" "sinatra_backend" {
(...)
load_balancer {
elb_name = "${aws_elb.sinatra_backend.id}"
container_name = "sinatra-backend"
container_port = 4567
}
}
Associate the ELB with the Sinatra
Backend ECS Service

Set up service discovery
between the ECS Services
{
"name": "example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"desiredCount": 2
}
ECS Agent
ECS Tasks
EC2 Instance
ECS Cluster

resource "aws_ecs_task_definition" "rails_frontend" {
[{
...
"environment": [{
"name": "SINATRA_BACKEND_PORT",
"value": "tcp://${aws_elb.sinatra_backend.dns_name}:4567"
}]
}]
EOF
}
Pass the Sinatra Bckend ELB URL
as env var to Rails Frontend

It’s time to deploy!
{
"name": "example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"desiredCount": 2
}
ECS Agent
ECS Tasks
EC2 Instance
ECS Task Definition ECS Service Definition ECS Scheduler
ECS Cluster

> terraform apply
aws_ecs_cluster.example_cluster: Creating...
name: "" => "example-cluster"
aws_ecs_task_definition.sinatra_backend: Creating...
...
Use the apply command to deploy
the ECS Cluster & Tasks

See the cluster in the ECS console

For more
info, see
Hello,
Startup
hello-startup.net

gruntwork.io
For DevOps help, see
Gruntwork

An intro to Docker, Terraform, and Amazon ECS

Recommended

More Related Content

What's hot (20)

Similar to An intro to Docker, Terraform, and Amazon ECS (20)

More from Yevgeniy Brikman (20)

Recently uploaded (20)

An intro to Docker, Terraform, and Amazon ECS

Rails Frontend