vim, git, aws and other three-letter words

Software Development Notes

AWS OpsWorks: set up MongoDB EBS volume backups

I described how to set up mongodb on EC2 using OpsWorks and here is how to set up mongo data backups.

In my case all mongo data is stored on the same EBS volume so I just need to make a volume snapshot. The relevant part from the mongodb docs:

Backup with –journal

The journal file allows for roll forward recovery. The journal files are located in the dbpath directory so will be snapshotted at the same time as the database files. If the dbpath is mapped to a single EBS volume then proceed to Backup the Database Files. If the dbpath is mapped to multiple EBS volumes, then in order to guarantee the stability of the file-system you will need to Flush and Lock the Database. NOTE Snapshotting with the journal is only possible if the journal resides on the same volume as the data files, so that one snapshot operation captures the journal state and data file state atomically.

This is my case - I have all data on the same EBS volume and journal option is enabled by default for MongoDB 2.0 and higher on 64-bit systems. In other cases it is necessary to flush and lock the database. This method is supported by the ec2-consistent-snapshot tool.

My solution is based on the modified version of aws-snapshot-tool. I wanted a completely automated setup where I don't need to take manual steps like assigning tags to the volumes I need to backup.

The process I have now does this:

The recipe (ec2-backup/recipes/default.rb) looks like this: 

# see https://github.com/stuart-warren/chef-aws-tag

# Assign tag to the instance
include_recipe "aws"
tags = {
  "MakeSnapshot" => "True"
}
aws_resource_tag node['ec2']['instance_id'] do
    tags(tags)
    action :update
end

# Create directory where snapshot tool will be stored
directory "/srv/backup" do
  owner 'root'
  group 'root'
  mode '0644'
  action :create
end

# Copy the snapshot tool to /srv/backup
cookbook_file "makesnapshots.py" do
  path "/srv/backup/makesnapshots.py"
  action :create
end

# Copy the snapshot tool config to /srv/backup
cookbook_file "config.py" do
  path "/srv/backup/config.py"
  action :create
end

# Setup a cron job for daily backups
cron "backup-daily" do
  path "/usr/local/bin:$PATH"
  hour "1"
  minute "30"
  weekday '1-6'
  command "cd /srv/backup && /usr/bin/python makesnapshots.py day 2>&1 |/usr/bin/logger -t \"CRON: makenapshot\""
end

# Setup a cron job for weekly backups
cron "backup-weekly" do
  path "/usr/local/bin:$PATH"
  hour "2"
  minute "30"
  weekday '7'
  command "cd /srv/backup && /usr/bin/python makesnapshots.py week 2>&1 |/usr/bin/logger -t \"CRON: makenapshot\""
end

# Setup a cron job for monthly backups
cron "backup-monthly" do
  path "/usr/local/bin:$PATH"
  hour "3"
  minute "30"
  day '1'
  command "cd /srv/backup && /usr/bin/python makesnapshots.py month 2>&1 |/usr/bin/logger -t \"CRON: makenapshot\""
end

Under the recipe files folder (ec2-backup/files/default/) I have a modified copy of the aws-snapshot-tool. The recipe folder layout is this: 

├── Berksfile
├── ec2-backup
│   ├── files
│   │   └── default
│   │       ├── config.py
│   │       ├── makesnapshots.py
│   ├── metadata.rb
│   └── recipes
│       └── default.rb

The Berksfile contains dependency info (only aws is relevant to ec2-backup recipe): 

source 'https://supermarket.getchef.com'

cookbook 'mongodb'
cookbook 'aws', '>= 0.2.4'

My version of the makesnapshots.py is here. My modification was to allow tagging instances instead of volumes. There is already a similar pull request in the original repository so I didn't submit my change back. There is also one new option in the config.py (see the full config example here): 

...
# Set to True to use instance tags, False - volume tags
'use_instance_tag': True,
...

That's all, now I have mongo data backups with email notifications.

Tags: aws

Tags