The following describes my personal case on how I maintained snapshots of my AWS EBS. The following requirements guided me with the scripts and tools I decided to use:

1. Open source scripts and solutions within budget.
2. Incremental snapshots maintain controlled and cost efficient growth
3. Manage all volume’s snapshots for several AWS accounts from one central server.
4. Advanced snapshot rotation mechanism and configuration options.

I decided to use a central server that will run the automatic Cron service (just a simple micro instance on Amazon EC2), which responsible to run snapshots scripts in the desired time.

Learn how-to Automated an EBS Volume Snapshot

Alestic Consistent Snapshot Script

Searching the web, I found many simple scripts in most scripting languages (Perl, php, bash, python) which can make snapshots of your AWS volumes including basic rotation tasks. In order to make snapshots, I chose the Alestic.com script - https://github.com/alestic/ec2-consistent-snapshot, which is simple and easy to use. Consistent script ensures consistent live snapshots, eliminating the need to stop instances to make a snapshot of the root disk or the MySQL DB disk. If you are running a MySQL server on XFS formatted EBS volume, this script also freezes the file system and flushes the MySQL tables’ data. This functionality can minimize data loss of live snapshotting processes. In order to install this process, you need to make sure that you have perl libsincluded on your Amazon instance yum repo.

[Newvem analyzes your baseline disaster recovery (DR) status, reflecting how well AWS DR best practices have been implemented, and recommends AWS features and best practices to reach optimal availability, increase outage protection, and quick recovery. Learn more about Newvem]

That’s Geeky: Snapshot Rotation

This Alestic script does not perform any rotation or deletion of snapshots, so additional script was needed for this operation. For backup rotation, I used - http://www.thatsgeeky.com/2011/06/rotating-ebs-snapshots-ec2-prune-snapshots/. The script worked on the same basic command line arguments as an EC2-consistent-snapshot, and it had more advanced rotation time setup than other scripts. You can setup the number of hourly, daily, weekly, and monthly snapshots to be kept. It is similar to the Grandfather-Father-Son style rotation.

Grandfather-father-son backup refers to a common rotation scheme for backup media. Originally designed for tape backup, it works well for any hierarchical backup strategy. The basic method is to define three sets of backups, such as daily, weekly and monthly. The daily, or son, backups are rotated on a daily basis with one graduating to father status each week. The weekly or father backups are rotated on a weekly basis with one graduating to grandfather status each month. In addition, quarterly, biannual, and/or annual backups can also be separately retained. Often one or more of the graduated backups is removed from the site for safekeeping and disaster recovery purposes. Learn More about the Grandfather-Father-Son style rotation

Pic credit - http://resources.filevaultusa.com/grandfather-father-son/

By simply putting these scripts on crontab file, you can manage all volume snapshots from one AWS host instance important for security purposes, don’t forget to use AWS IAM Access Management service. Simply create an Amazon IAM group or user only for snapshot operations (defining a custom snapshots policy) and use this group’s credentials on a host that is running the Cron scripts. It will save your AWS account services from getting hacked. If you lose your snapshot group/user credentials file, only your Amazon EBS snapshots can be compromised but not the entire AWS service.

About the Author

Jonas Mardosas, Cloud Solution Architect/CloudOps Administrator/Software Engineer. Cloud Solutions Architect, System Administrator with strong background in Cloud Computing (IaaS, PaaS) services and Linux environment. Experienced in HPC (High Performance Computing) and Virtualizations (VMware, XEN) platforms.

[Newvem analyzes your EBS volume and snapshot usage patterns to help you increase control and enhance your backup policies. Create your free Newvem account]

Keywords: Amazon web services, Amazon AWS console, Amazon Cloud Services, Cloud Scalability, Cloud Performance, AWS Console, EC2 instance, amazon EBS, cloud Volume,  cloud Backup, Cloud Disaster Recovery, AWS DR, EC2 Backups, Cloud Outage, cloud Compliance, AWS Regulations, RTO, RPO, Cloud Continuity, High Cloud Availability, AWS outage