Running Cockroach DB in Production with Nomad

For those that know me, they know I have been a huge fan of HashiCorp Nomad for the past year or so. I have been very open with my thoughts that I don’t believe that sweetspot of the tool is for scheduling containers. I believe the ship has sailed in the, very fashionable, direction of Kubernetes for container scheduling. I have been using Nomad in a similar vein to how others are supervising their services with tooling like systemd.

Building an Autodiscovering Apache Zookeeper Cluster in AWS using Packer, Ansible and Terraform

Following my pattern of building AMIs for applications, I create my Apache Zookeeper cluster with Packer for my AMI and Terraform for the infrastructure. This Zookeeper cluster is auto-discovering of the other nodes that are determined to be in the cluster ###Building Zookeeper AMIs with Packer The packer template looks as follows: { "variables": { "ami_id": "", "private_subnet_id": "", "security_group_id": "", "packer_build_number": "", }, "description": "Zookeeper Image", "builders": [ { "ami_name": "zookeeper-{{user `packer_build_number`}}", "availability_zone": "eu-west-1a", "iam_instance_profile": "app-server", "instance_type": "t2.

Replacing a node in a Riak Cluster

The instances that run in my infrastructure get a lifespan of 14 days. This allows me to continually test that I can replace my environment at any point. People always ask me if I follow the same principal for data nodes. I posted previously about replacing nodes is an ElasticSearch cluster, this post will detail how I replace nodes in a Riak cluster NOTE: This post assumes that you have the Riak Control console enabled for Riak.

Building a Riak Cluster in AWS with Packer and Terraform

Following my pattern of building AMIs for applications, I create my riak cluster with Packer for my AMI and Terraform for the infrastructure ###Building Riak AMIs with Packer {% raw %} { "variables": { "ami_id": "", "private_subnet_id": "", "security_group_id": "", "packer_build_number": "", }, "description": "Riak Image", "builders": [ { "ami_name": "riak-{{user `packer_build_number`}}", "availability_zone": "eu-west-1a", "iam_instance_profile": "app-server", "instance_type": "t2.small", "region": "eu-west-1", "run_tags": { "role": "packer" }, "security_group_ids": [ "{{user `security_group_id`}}" ], "source_ami": "{{user `ami_id`}}", "ssh_timeout": "10m", "ssh_username": "ubuntu", "subnet_id": "{{user `private_subnet_id`}}", "tags": { "Name": "riak-packer-image" }, "type": "amazon-ebs" } ], "provisioners": [ { "type": "shell", "inline": [ "sleep 10" ] }, { "type": "shell", "script": "install_dependencies.

Replacing the Nodes in an AWS ElasticSearch Cluster

In a previous post, I talked about how I have tended towards the philosophy of ‘Immutable Infrastructure’. As part of that philospohy, when a box is created in my environment, it has a lifespan of 14 days. On the 14th day, I get a notification to tell me that the box is due for renewal. When it comes to ElasticSearch nodes, there is a process I follow to renew a box.

Deploying Kibana Using Nginx as an SSL Proxy

In my last post, I described how I use Packer and Terraform to deploy an ElasticSearch cluster. In order to make the logs stored in ElasticSearch searchable, I use Kibana. I follow the previous pattern and deploy Kibana using Packer to build an AMI and then create the infrastructure using Terraform. The Packer template has already taken into account that I want to use nginx as a proxy. ###Building Kibana AMIs with Packer and Ansible

Building an ElasticSearch cluster in AWS with Packer and Terraform

As discussed in a previous post, I like to build separate AMIs for each of my systems. This allows me to scale up and recycle nodes easily. I have been doing this with ElasticSearch for a while now. I usually build an AMI with Packer and Ansible and I use Terraform to roll out the infrastructure ###Building ElasticSearch AMIs with Packer The packer template looks as follows: {% raw %}

Autoscaling Group Notifications with Terraform and AWS Lambda

I use Autoscaling Groups in AWS for all of my systems. The main benefit for me here was to make sure that when a node died in AWS, the Autoscaling Group policy made sure that the node was replaced. I wanted to get some visibility of when the Autoscaling Group was launching and terminating nodes and decided that posting notifications to Slack would be a good way of getting this. With Terraform and AWS Lambda, I was able to make this happen.

Changing my focus

For those that know me, you know that I am very passionate about infrastructure and DevOps. Infrastructure is something that we, as developers, don’t really give a lot of thought to. DevOps is something that will make our businesses more successful. So, I am changing the job role that I have. From now, I am no longer focusing on writing application code. Instead I am going to spend my time focusing on the following areas: