Schedule check for forgotten running EC2 with AWS Lambdas

Have you ever got surprised by an AWS invoice charging you for resources you forgot running in strange regions around the world, while doing some experiments with your account?

Well, I did, more than once.

After spend lot of time looking for a tutorial on how to write a serverless function to monitor forgotten resources running consuming my credits in AWS with no success, I decided to my research and create my own solution.

I came up with a free tier AWS set of resources solution consisting of a Lambda Function running every day on a particular AWS region, checking for running EC2 instances around all AWS regions for more than a pre configured time, and notifying me via email.

In this post, I will go through step by step on how to create your own global EC2 resource monitoring, spamming across a variety of AWS technologies:

  1. Lambda functions
  2. Simple Notification Service SNS
  3. CloudWatch Events and Logs
  4. IAM Roles and Policies
  5. NodeJS and AWS SDK

To complete this exercise, you require an AWS account with access to all these resources. If you don’t have one, I recommend creating your own account, as AWS offers a free tier which allows you to practice all different services they provide, without spending any money.

This exercise can be executed and left running forever under the AWS Free Tier.

Ready? Let’s start!

The monitoring code

AWS Lambda supports various language. I decided to use NodeJs as I feel particularly comfortable with the async/await and Promises way to write concurrent code.

Before starting configuring AWS resources, let’s look at the monitoring code and understand what it does

On the first block (lines 1–5) we are initialising AWS SDK, alongside with SNS and EC2 clients. As I’m based in Sydney, all the resources are going to be created on ap-southeast-2 region, but you can use your own region to run the EC2 instance check. We are also setting the Topic SNS ARN which you will have to update accordingly after creating your own Topic, and configuring how many minutes an EC2 has to be running to trigger the Email.

On line 9, we are querying a list of existing AWS regions and just after, lines 11–13 we are creating a list of Promises, ie, code that will be executed asynchronously, to get a list of EC2 instances in each region.

On line 15, we wait for all Promises to complete, and them, using some functional programming, we build a list of all EC2 instances that are running (State.Code=16) for more than maxMinutes.

Lastly, the block from line 29 to 38 we are publishing an SNS event for every EC2 instance running, that will result in notification Emails.

Simple Notification Service SNS

First we will create the service responsible for sending emails. We will do that using SNS, which works on a Topic/Subscriber fashion. Every message published to a topic will be distributed to every subscriber. Subscriber’s can be either HTTP or Email endpoints. In our case, we will use Email.

Select Simple Notification Service
Click on Create Topic from SNS Dashboard
Type in the topic name and click on Create topic
On the ec2-check-instances topic screen, click on Create subscription
Select Email for Protocol, type in the desired Email on Endpoint and click Create subscription
SNS will send a confirmation email. Click in the confirmation link to confirm the subscription
Subscription added

Done! At this point, we setup a SNS topic that will forward every message to the subscribed email. Take note of your generated Topic ARN, as it is the universal resource identifier that will be used in the Lambda code later.

Lambda function

Now let’s create the core of the system, the serverless Lambda Function that will run on a scheduled basis, looking for running EC2 instances in every AWS region across the globe.

From the services menu, select Lambda
In the Lambda Functions list, click on Create Functions

In the new Lambda function dialog, type in:

Name: check-running-ec2-instances
Runtime: Node.js 8.10
Role: Create new role from template
Role name: check-running-ec2-instances
Policy Templates: SNS publish policy

Select Author from scratch, enter the above information and click on Create functions
After created, on the Lambda Function dashboard, select the Lambda icon to load the code editor
Paste the NodeJS code into the editor, and update the snsTopicArn to your generated ARN from the previous step
As this Lambda Function will talk to every AWS Region endpoint, it’s necessary to increase the Timeout. 20 secs should be enough.

IAM Roles and Policies

Every Lambda function requires an IAM Role to run under. When we created the Lambda function in the previous example, a new IAM Role was created to grant pre-defined permissions to the function.

In order to inspect EC2 instances, and publish events to an SNS topic, this role requires extra policies to do so.

From the Services list, select IAM
On the IAM service, select Roles, and look for your check-running-ec2-instance Role created from Lambda

We can see in the pre created Role, there is already two attached Policies.

- AWSLambdaBasicExecutionRole: is added by default, and basically grants write access to ClodWatch Logs, so the function can record it’s execution logs.

- AWSLambdaSNSPublishPolicy: we selected on the Lambda creation dialog, and grants permissions to publish on the SNS Topic.

In order to monitor our EC2 instances, we need to add a new EC2ReadOnly Policy to this role

On the check-running-ec2-instances role summary page, click on Attach policies
Search for ec2readonly, select the AmazonEC2ReadOnly policy and click on Attach policy

CloudWatch Events

Lambda Functions accepts a variety of triggers to run. One of them is CloudWatch Events, which can be configured to trigger a event on a scheduled basis.

We want our function to run at least once per day, so we are going to create a new CloudWatch Event scheduled rule, and plug it as a trigger in our Lambda Function

From the Services menu, select CloudWatch
On the CloudWatch dashboard select Rules on the left, and them click on Create rule

Our new CloudWatch Event rule should have a schedule as Event Source, with a Fixed rate of 1 day. On the Targets, we select Lambda Function, and our check-running-ec2-instance function. To correctly trigger the function execution, CloudWatch needs a input body. As our function doesn’t expect anything, let’s just put a random Constant JSON text.

Configure the new rule and click Configure details
Enter ec2-check-instance for the rule name, and click Create rule

CloudWatch Logs

CloudWatch is also the service where AWS resources direct their logs. By default, every new Lambda function created will also have a Log Group created in CloudWatch, whose messages never expires by default.

As our function will execute every day, let’s change the log retention period so we don’t store unnecessary data.

On the CloudWatch page, select Logs on the left, find the check-running-ec2-instances log group, and click on Never Expires to change
Change the Retention to 1 day and click Ok

Testing the whole thing

Now with the whole infrastructure in place, let’s create an EC2 instance and test our Lambda Function.

On AWS Services menu, click on EC2
On EC2 dashboard, click on Launch Instance
On AMI List, select Amazon Linux 2 AMI
On Instance Type list, select t2.micro Free Tier and click on Review and Launch Instance (it will skip and assume defaults for the next steps)
On the Review page, click on Launch. You will be asked to setup SSH Key pairs or to use one existent. Follow the instructions and launch the instance

Now that we have a running EC2 instance, we can go back to our Lambda Function and manually test it.

On the check-running-ec2-instances Lambda Function dashboard, click on Test
You will be asked to setup test data the first time you’re testing. As our function doesn’t expect any input, just create a new dummy test event with the defaults provided and click on Create
After the text is executed, you can inspect the outputs. As our Lambda Function is configured to locate any instance running for more than 1 minute, the instance we created on the previous step should appear
The Lambda Function is instructed to post this event to the SNS topic, which will result in an Email with the EC2 running instances list

Summary

Congratulations! You finished the tutorial, and now you have a free serverless infrastructure checking for forgotten EC2 instances around the globe in your AWS account.

With this exercise, you learned how to

  1. create Topic SNS and configure emails subscriptions
  2. create new Lambda functions and how to test them
  3. Attach IAM Policies in IAM Roles
  4. Create scheduled CloudWatch Events and configure CloudWatch logs expiration

On the AWS Javascript SDK documentation, you can learn how to tune your script to check for other resources, and perhaps add different behaviours.

No more surprises on AWS invoices at the end of the month.

--

--

--

Passionated by cloud architecture, process automation and agile environments. Expertise in leading teams aiming high productivity balancing throughput and risk

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Reverse Nodes in k-Group

Transfer Your Wordle Save State/Stats Between Different Browsers and Devices

Time to Effect — Operations of large number of Kubernetes Clusters

Java / Kotlin Testing

Top Performance issues every developer/architect must know — part 1

Micropython on STM32 Nucleo Board in easy steps

Different ways & strategies to build and deploy Node.js for Production

Screenshot 2021-09-18 at 9.58.40 AM.png

The Alternative Easy-Peasy Way of Parsing JSON Data With Dart/Flutter.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Marcio Ghiraldelli

Marcio Ghiraldelli

Passionated by cloud architecture, process automation and agile environments. Expertise in leading teams aiming high productivity balancing throughput and risk

More from Medium

Asynchronously Invoking Lambda Function from Another Lambda

AWS DynamoDB ORM (Dynamoose) with Node.js #1

aws dynamodb

Cross-stack AWS SQS Permissions

Get your Serverless API in the Cloud