Optimizing AWS Costs by Identifying Unused Resources

❓ Challenge :

Sometimes, developers create EC2 instances with attached volumes by default and take snapshots for backup. When they no longer need the instance and delete it, they might forget to remove the snapshots. As a result, they keep paying for these unused snapshots, even though they're not needed anymore. Lets solve this problem!

💡Approach :

To solve the above problem, we will use a Lambda function, which will check our snapshots and EC2 instances. If the Lambda function finds a snapshot that is not linked to any active EC2 instances, it will delete the snapshot. By doing so, it will save us money and help reduce unnecessary costs. Let me show you, step by step!

👁 Observation :

There are many similar issues to concern, for e.g: Unused EBS Volumes(storage cost-$), Unused Load Balancers(costs for services-$), Unused Elastic IP attached with EC2 Instances($) and many more.

🪜 Steps :

Step 1:

1. Log into your AWS Console, navigate to the EC2 Console, go to the Instances section, select 'Instances,' and click on 'Launch Instance'. Create a EC2 Instance (I am using Free Tier).
   

   
   
   

2. Navigate to the 'Elastic Block Store' section, select 'Volumes,' and observe the default volume that has been automatically created.
   

   
   
   

3. Go to the 'Snapshots' section, click 'Create Snapshot,' set 'Resource type' to 'Volume,' and choose the automatically created volume from the 'Volume ID' dropdown. Finally, click the 'Create Snapshot' button.
   

   
   
   

Step 2 :

1. Navigate to the 'Lambda Console', click on 'Functions,' select 'Author from Scratch,' enter a Function name, choose the latest Python version, scroll down, and click 'Create Function'.
   

   
   
   

2. After creating the function, scroll down to the Code section, clear the existing code, replace it with 'Python_script_CleanEBS.py', then click Deploy to save your changes. Finally, click Test, which will prompt a page similar to the image below.
   

   
   
   

3. When 'Test' is clicked, select 'Create new test event', Give an 'event name' & click 'Save'.

   
   
   

4. After creating the event, go to the 'IAM Console' and navigate to the 'Policies section' to 'create a new policy'. Select 'EC2' as the service and, in the Actions section, grant permissions for

   • DescribeInstances
     

   • DescribeVolumes
     

   • DescribeSnapshots
     

   • DeleteSnapshots
     

     
     
     

5. Give a 'Policy Name', and Click 'Create Policy'.
   

   
   
   

6. Go to the 'Lambda function' page, select the newly created function, navigate to Configurations → Permissions, and click on the 'Role name'.

   
   
   

7. Click on 'Add Permissions' and then select 'Attach Policy', select the correct policy, you just created.
   

   
   
   

8. After that, you can go to the 'Lambda function' page and run the code; it will display some outputs as shown below.
   

   
   
   

Step 3 :

1. To test the 'Lambda function', navigate to the 'EC2 Console' and 'terminate' the EC2 instance.
   
   
   

2. Go to the 'Lambda Console', navigate to the 'Lambda Function' page, and under the 'Code' section, click 'Test code' to run the function. It will display an output similar to the one shown below.
   

   
   
   

3. If the snapshot was linked to a missing volume, the Lambda function successfully deleted it. VOILA! 🎉

   
   
   

🔍 Architectural Insights:

Additionally, We can use 'Amazon CloudWatch' to automatically trigger the Lambda function at predefined intervals like every hour, day, minute, or second. However, this may result in higher costs because our Lambda execution time increases when triggered automatically. Nevertheless, manually triggering this function is a better choice because it allows us to trigger it when needed.

CloudWatch/ EventBridge Implementation :

Steps :

1. Navigate to CloudWatch Console, Go to 'Events'-> 'Rules'-> 'Create Rule'.
   

2. Under 'Rule detail', give a 'name', 'description', 'Rule Type' = 'Schedule, click 'Continue in EventBridge Scheduler'.
   
   
   

3. Under 'Schedule Pattern', select 'Recurring schedule', 'Schedule Type'='Rate-based schedule' & 'Rate expression' = '1 hour' (We set for hourly)
   
   
   

4. Select Target as 'Templated targets' -> 'AWS Lambda', select the lamdba functrion from dropdown, & 'Next'.
   
   
   

5. On the next page, choose 'None' for the 'Action after Schedule' option and 'Next'-> 'Review the details -> 'Create Schedule'.
   
   
   

6. You have successfully created the CloudWatch scheduler, which will trigger the Lambda function every hour.However, please note that this setup will incur some costs since the function is triggered continuously every hour. Alternatively, we can configure it to run on specific days and times as needed.
   

🟢Happy Learning ! 🥁🙌🫶