AWS Tagging Script

Challenge Objectives

• Create a script to tag the EC2 instances, the volumes and volume snapshot automatically

Project Background

• Volume tagging script that copies tags from running instances to volumes, and snapshots from those volumes, and identifies the previous AMI that was used to generate the AMI / volumes.

• EC2 instances all have tags that are used for various purposes.  The goal is get those tags onto the other resources so that we can track them.

Technology Stack

• AWS SDK / API

• Python 2.7

• CSV & SMTP

• Jenkins

Individual Requirements

1. Create a python script, which will be executed from Jenkins.

The script should implement the following functionalities:

• Ensure ALL resources have right tags (if NO tag we need to TAG Them)

  • The volumes have the same tags as their associated EC2 instance

  • The volume snapshots have the same tags as their corresponding volume

  • Identifies the previous AMI that was used to generate the AMI/volumes. The idea there is a tag on AMIs that contains the previous AMI id.  For example, if Instance 1 is created from AMI A, and then AMI B is generated from Instance 1, AMI B will have a tag, "Original AMI" that contains AMI A's id.

• No Override of existing TAG

• Needs to work for existing / new resources

• At the end of the execution, it should generate a report (CSV file), upload it to s3 bucket, and email the file link to multiple users.

  • A record should be generated for the resource only if its tags are changed, and the record should contain the following data: Application, Owner, Environment, Role, Stack, Resource Id/Type, Old Tags, New Tags

 

The image above describes the relationships between the instances, volumes, snapshots and AMI.

 

You can implement the tagging script in this flow:

• Call describe-instances AWS API to load all the instances

• For each instance, get its volumes, and copy tags from the instance to the volumes. Add an “AMI” tag for the volumes as well with instance’s AMI id as the tag value.

• For each volume, call describe-snapshots AWS API to load the volume’s snaphosts, and then copy the tags from the volume to the snapshots

• Call describe-images AWS API to load all the images, for each image, if the snapshot the image is created from has an “AMI” tag, copy the tag to the image with “Original AMI” tag name.

• Call describe-volumes AWS API to load all volumes, for each volume, if the snapshot the volume is created from has an “AMI” tag, copy the tag to the volume with “Original AMI” tag name.

 

You are required to optimize it to make it as efficient as possible.

The script should be able to be executed from command line. Logging should be added, and proper logs should be generated to indicate the job is successful or failed. Please configure it to log to console only for now. The script should follow OOP pattern, and it should be well documented as well.  

The following properties should be configurable (please use an external json or YAML file for configuration):

• Either AWS client id/key pair, or an IAM instance profile for authentication. Both should be supported
• Report Emails
• SMTP info
• Email Template
• s3 bucket to store the reports

2. Jenkins Integration

Provides steps and screeshots to run the script from the Jenkins
 

 

 

Final Submission Guidelines

- Source Code
- Deployment and Verification Guide
- The steps of creating ec2 instances, volumes, snapshots and AMI images for testing must be well described as well.