There are chances where the employees/users either intentionally or unintentionally make changes or delete the AWS resources. These scenarios cannot be traced or brought to our notice unless we have a proper monitoring and alerting mechanism to take action immediately to avoid any business interruptions. Proactive monitoring is one of the key items in maintaining a Secure infrastructure. To catch such activities, we can make use of the AWS services such as CloudTrail, CloudWatch, and SNS topic with subscribers to actively monitor the activities happening in the AWS account, log them and notify the subscribers when such an anomaly occurs.
In this blog, we will demonstrate how to track AWS account activities using AWS CloudTrail with step-by-step instructions.
Hands-on
As part of this implementation, we will do followings:
- Create CloudTrail using AWS Console
- Creating CloudWatch Log group
- Set Up an IAM Role
- Configure CloudTrail Logging to CloudWatch Log group
- Disable CloudTrail event logging to CloudWatch Log group
- Create SNS Topic
- Add Subscribers to SNS Topic
- Publishing Messages to Subscribers
- Finding CloudTrail event Patterns
- Creating CloudWatch Metric Filters
- Creating CloudWatch alarm
Lists of AWS Services used in this implementation
- CloudTrail
- CloudWatch
- SNS
- S3
- IAM
Create CloudTrail using AWS Console
Let’s go ahead and create a trail for the AWS account.
Login to CloudTrail console, you can see the events recorded by CloudTrail by default.
Here are some of the events such as ConsoleLogin, CreateRole, Change Password etc recorded by CloudTrail.
Check the event and then choose view event record for more details about the event.
In the left navigation pane, choose trails and you can see that the account doesn’t have any trails created yet.
To create a trail, click create trail, provide a name for the trail.
For storage location, we are going to send all the CloudTrail events to S3 bucket.
We can either create a new S3 bucket for this log delivery to S3 bucket or we can use the existing bucket.
Choose to create a new S3 bucket, CloudTrail will automatically set up a unique name for the S3 bucket.
Encrypt log files with SSE-KMS – you can encrypt files using SSE-KMS instead of SSE-S3.
For which we need to either create a new KMS key or use the existing Customer Managed AWS KMS key.
Optionally, send out a notification for every log file delivery to S3 bucket.
Add a tag for the CloudTrail, which is again optional.
And then click Next.
Under events, choose the event type such as
Management events – records management operations performed on aws resources.
Data events – record operations that are performed within a resource.
Insights events – identifies unusual activities, errors and user activities in AWS accounts.
For Management events,
Choose the type of activities you want the CloudTrail to record and log it.
Read – read API operations such as Describe.
Write – create, update, delete API operations.
Click Next, review the settings and then click create trail.
CloudTrail is successfully created.
To enable or disable the CloudTrail logging, click the trail name.
Click Stop logging.
We can also delete the CloudTrail itself using the delete option.
To check the logs stored in the S3 bucket, go to the S3 bucket console.
It has a directory structure that follows AWSLogs, Account, CloudTrail, Region, Year, Month and Day.
There you can find a file of type gz.
Create a CloudWatch Log Group
For the CloudTrail to store all the events in the CloudWatch log group.
First we need to create a CloudWatch log group.
To create a CloudWatch log group, log in to the CloudWatch console.
In the Navigation pane, under Logs, choose Log Groups.
Click Create Log group, enter a name for the Log group, and click Create.
The Log group will be used to store all the events captured by CloudTrail.
Set up an IAM Role
For CloudTrail to send events to the CloudWatch log group, the trail needs permission to put log events to the CloudWatch log group.
So we need to create an IAM Role, and the role will be used for CloudTrail to be able to send the events to the CloudWatch log group.
Basically, the Role will have permission to create LogStream and PutLogEvents.
We are not going to manually configure the IAM policy and IAM Role.
Instead, we will be using the CloudTrail console to configure it while enabling the CloudWatch logging.
Configure CloudTrail event Logging to CloudWatch Log Group
Go to the CloudTrail console, and choose trails in the navigation pane.
Select the trail for which you want to set up CloudWatch logging.
Under CloudWatch Logs, click Edit.
Make sure CloudWatch Logs is Enabled.
For the log group, choose existing and then provide the name of the CloudWatch log group.
For IAM Role, choose new and provide a name for the Role and click Save changes.
CloudTrail event logging to cloudwatch log group is implemented.
To check the CloudTrail logs, go to CloudWatch console, click the Log group and you can find the lists of log streams holding the events captured by the CloudTrail.
Select it.
Disable Sending of CloudTrail events to CloudWatch Log group
To disable event logging to CloudWatch log group.
Go to the CloudTrail console.
Click the trail for which CloudWatch logging should be disabled.
Under CloudWatch Logs, click edit.
And then Uncheck, Enabled, click Save changes.
Create a SNS Topic
Go to SNS Console
In the navigation pane, choose topics and then click click create topic.
Choose the topic type to be Standard and enter a name for the topic.
Other optional elements such as,
ENCRYPTION:
By default, delivery of messages by SNS is performed in an encrypted way.
To enable server side encryption, i.e Encryption at rest.
ACCESS POLICY
Restricts access to the Topic.
By default, the owner / Topic creator is allowed.
Custom policy can be applied to allow Publish and Subscribe for other IAM users.
DELIVERY RETRY POLICY
Retry sending messages to the configured endpoints.
By default, SNS tries 3 times to send the message to the subscribers.
DELIVERY STATUS LOGGING
Logging the delivery status of the SNS topic will help us to analyse and resolve if the SNS topic is not able to deliver the message successfully.
TAGS
You can add a tag to the SNS topic and then,
Click create topic. SNS topic is created.
Lets go ahead and add subscribers for this topic.
Adding Subscribers to SNS Topic
Subscribers are the one who receives a notification.
To add subscribers, click Create subscription.
Choose the protocols such as lambda, SMS, Email, Email JSON, SQS, HTTP, HTTPS endpoints.
Let me choose Email.
Enter the email address and click create subscription.
When a subscriber is added to the topic, it should be validated / confirmed.
Subscribers failed to confirm the subscription will fail to receive notifications from the SNS.
Go to Inbox, Amazon SNS must send a message as shown below.
Click Confirm subscription.
Now go back to the SNS console and choose the topic and Under Subscriptions, you should see the subscription is confirmed.
Now let’s test the pub/sub messaging system, by publishing a message to the SNS topic and the subscriber (email) should receive the message.
Publishing Message to Subscribers
To do this, select the topic and click Publish message,
Enter a Subject for the message,
For the Message body, enter the message to publish to the subscribers.
Then click the Publish message.
As I have checked my Mailbox, I have received a message from an SNS topic.
We should identify the type of activities to be monitored and notified.
The following are the filter patterns which can be used to monitor IAM authentication and authorization activities.
1: Monitoring Changes to IAM
Monitor changes to an IAM account.
{ ( ($.eventSource = "iam.amazonaws.com") && (($.eventName = "Add") || ($.eventName = "Attach") || ($.eventName = "Change") || ($.eventName = "Create") || ($.eventName = "Deactivate") || ($.eventName = "Delete") || ($.eventName = "Detach") || ($.eventName = "Enable") || ($.eventName = "Put") || ($.eventName = "Remove") || ($.eventName = "Set") || ($.eventName = "Update") || ($.eventName = "Upload*")) ) }
It will track events that begin with Add, Create, Deactivate, Change, Delete, Enable, Remove, Put, Upload and Update.
2: Monitoring All Calls to IAM
Monitor all the IAM related activity
{ ($.eventSource = "iam.amazonaws.com") }
If you’re using IAM for a lot of services, you will get a lot of alerts.
3: Monitoring Changes to Authentication & Authorization Configurations
Using the below filter pattern, you can monitor changes to security credentials and policy configuration changes.
{ ( ($.eventSource = "iam.amazonaws.com") && (($.eventName = "PutPolicy") || ($.eventName = "Attach") || ($.eventName = "Detach") || ($.eventName = "Create") || ($.eventName = "Update") || ($.eventName = "Upload") || ($.eventName = "Delete") || ($.eventName = "Remove") || ($.eventName = "Set*")) ) }
Creating CloudWatch Metric Filter Pattern
Create the CloudWatch metric for the IAM changes, then a CloudWatch alarm will be created for the metric pattern.
Go to the CloudWatch console, under Logs, choose Log groups.
Click the log group name which you have created before, and then under Actions, choose create metric filter.
On the Define pattern page,
For Create filter pattern, enter the filter pattern.
For Test pattern, choose the log stream and then click Next.
We need to assign a metric, and provide a Filter name.
Enter a Metric namespace.
For the Metric name,
Enter IAMAuthnAuthzActivity and enter the metric Value as 1.
Click Next, review the settings and then click Create metric filter.
Next we need to create a CloudWatch alarm.
Creating an CloudWatch Alarm
Under the CloudWatch Log group, you can find the metric filter created.
Choose it and click create alarm.
For Statistics, choose Sum and the Period 5 mins.
Under Conditions, choose static threshold,
The alarm condition is Greater/Equal and value to be 1.
The alarm will be triggered when the metric value is >=1 for 1 consecutive period.
and click Next,
For Notification, SNS topic will be used.
Choose In alarm
For Select an SNS topic, choose Select an existing SNS topic.
Choose the SNS topic which was created before and then click Next.
Enter a name for the CloudWatch alarm and then click Next .
Review the settings and click Create alarm.
Whenever the metric pattern matches,
CloudWatch Alarm will be triggered and we will be notified by SNS to all the Subscribers as shown below.
The below image shows that there is a change in the IAM service, hence the subscribers are notified by the SNS.
Conclusion
In this blog, with this implementation using AWS services such as CloudTrail, CloudWatch, and SNS we will be able to monitor IAM changes such as Authentication and Authorization, and also any other activities that occur in the AWS account can be tracked, Monitored, and Notified. Stay tuned to keep getting all updates about our upcoming new blogs on AWS and relevant technologies.
Meanwhile …
Keep Exploring -> Keep Learning -> Keep Mastering
This blog is part of our effort towards building a knowledgeable and kick-ass tech community. At Workfall, we strive to provide the best tech and pay opportunities to AWS-certified talents. If you’re looking to work with global clients, build kick-ass products while making big bucks doing so, give it a shot at workfall.com/partner today.
