Automate EBS Snapshots using Lambda function

Note: This post was contributed by our Cloud Support Engineer, Bakrudeen

Note:This scripts used in this post are actually written by Ryan S. Brown of serverlesscode, here. We are a great fan of his work, go check his site.

We recently wrote about delivering AWS inventory details in CSV using lambda. Today we will use Lambda function to automate EBS snapshots, snapshot retention and schedule a job.

You can follow the previous post step by step for creating lambda function. What changes is just the definition of the function and an IAM role.

Create an IAM role that has enough previleges to create/delete snapshots:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:*"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": "ec2:Describe*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:CreateSnapshot",
                "ec2:DeleteSnapshot",
                "ec2:CreateTags",
                "ec2:ModifySnapshotAttribute",
                "ec2:ResetSnapshotAttribute"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Make sure to set your role type as AWS Lambda

Next - make sure that all the Instances whose EBS volumes you would like to snapshot are tagged. The script filters on a tag to enumerate all instances and snapshot the volumes attached to them. We used the tag key/value as Backup/True. The script also has retention period hardcoded, you might want to change it to your liking. Its currently 7.

Create Snapshots

Below is the script for lambda function

import boto3  
import collections  
import datetime

#Please mention your region name
#below line code is call cross region
ec = boto3.client('ec2', region_name='ap-southeast-1')

#begins lambda function
def lambda_handler(event, context):  
    # mention your tag values below example "Backup-snap"
    reservations = ec.describe_instances(
        Filters=[
            {'Name': 'tag-key', 'Values': ['Backup', 'True']},
        ]
    ).get(
        'Reservations', []
    )

    instances = sum(
        [
            [i for i in r['Instances']]
            for r in reservations
        ], [])

    print "Number of the Instances : %d" % len(instances)

    to_tag = collections.defaultdict(list)

    for instance in instances:
        try:
            retention_days = [
                int(t.get('Value')) for t in instance['Tags']
                if t['Key'] == 'Retention'][0]
        except IndexError:
            # Please give your retention period day
            retention_days = 7

        for dev in instance['BlockDeviceMappings']:
            if dev.get('Ebs', None) is None:
                continue
            vol_id = dev['Ebs']['VolumeId']
            for name in instance['Tags']:
                # To store the instance tag value
                Instancename= name['Value']
                # To store the instance key value
                key= name['Key']
                # Below the code is create Snapshot name as instance Name 
                if key == 'Name' :
                    ins_name = Instancename
                    print "Found EBS volume %s on instance %s" % (
                    vol_id, instance['InstanceId'])

            #To get all the instance tags deatils
            for name in instance['Tags']:
                # To store the instance tag value
                Instancename= name['Value']
                # To store the instance key value
                key= name['Key']
                # Below the code is create Snapshot name as instance Name 
                if key == 'Name' :
                    snap = ec.create_snapshot(
                    VolumeId=vol_id,
                    Description=Instancename,
                    )
                    print "snap %s" %snap

            to_tag[retention_days].append(snap['SnapshotId'])

            print "Retaining snapshot %s of volume %s from instance %s for %d days" % (
                snap['SnapshotId'],
                vol_id,
                instance['InstanceId'],
                retention_days,

            )
            for retention_days in to_tag.keys():
                delete_date = datetime.date.today() + datetime.timedelta(days=retention_days)
                snap = snap['Description'] + str('_')
                # Here to get current date 
                snapshot = snap + str(datetime.date.today())   
                # to mention the current date formet
                delete_fmt = delete_date.strftime('%Y-%m-%d')
                print "Will delete %d snapshots on %s" % (len(to_tag[retention_days]), delete_fmt)
                # below code is create the name and current date as instance name
                ec.create_tags(
                Resources=to_tag[retention_days],
                Tags=[
                {'Key': 'DeleteOn', 'Value': delete_fmt},
                {'Key': 'Name', 'Value': snapshot },
                ]
                ) 
        to_tag.clear()

Delete Snapshots

Below is the lambda function. We have already specified retention period in the above function, this will just delete based on that.

import boto3  
import re  
import datetime

#Please mention your region name
#below line code is call cross region
ec = boto3.client('ec2', region_name='us-west-2')  
iam = boto3.client('iam')

#begins lambda function
def lambda_handler(event, context):  
    account_ids = list()
    try:         
        iam.get_user()
    except Exception as e:
        # use the exception message to get the account ID the function executes under
        account_ids.append(re.search(r'(arn:aws:sts::)([0-9]+)', str(e)).groups()[1])

    delete_on = datetime.date.today().strftime('%Y-%m-%d')
    filters = [
        {'Name': 'tag-key', 'Values': ['DeleteOn']},
        {'Name': 'tag-value', 'Values': [delete_on]},
    ]
    snapshot_response = ec.describe_snapshots(OwnerIds=account_ids, Filters=filters)

    for snap in snapshot_response['Snapshots']:
        print "Deleting snapshot %s" % snap['SnapshotId']
        ec.delete_snapshot(SnapshotId=snap['SnapshotId'])

You can now schedule these two functions using Cloudwatch events.

That's about it. Happy snapshotting! :)

Note: These scripts are available on github. Please feel free to fork.

comments powered by Disqus