← All articles

Advanced Cloud Cost Optimization Strategy: Automated Rightsizing

Why an automated rightsizing strategy matters

Most engineering and platform teams treat rightsizing as a quarterly manual chore: they open the Compute Optimizer console, copy a handful of recommendations, and apply changes one by one. The process is time‑consuming, error‑prone, and quickly becomes stale as workloads evolve. An automated rightsizing pipeline removes the human bottleneck, enforces consistent policies, and provides a repeatable audit trail. By letting AWS generate recommendations, storing them centrally, and letting a Lambda function act on them, teams can continuously shrink under‑utilized resources while preserving performance.

Prerequisites: permissions, services, and data collection

Before building the pipeline, verify that you have the following in place:

• IAM role with ComputeOptimizerReadOnlyAccess, AmazonS3FullAccess, AWSLambdaBasicExecutionRole, and AmazonEC2FullAccess (or more granular scoped policies).
• AWS CLI version 2 installed and configured with a profile that can assume the above role.
• Compute Optimizer enabled for the target AWS account (or organization root).
• S3 bucket for recommendation export, with versioning and server‑side encryption enabled.
• EventBridge (formerly CloudWatch Events) permission to invoke the Lambda on a schedule.

If any of these are missing, the steps below include the exact CLI commands to create them.

Step 1 – Enable AWS Compute Optimizer and export recommendations

Compute Optimizer analyses historical CloudWatch metrics and produces instance, volume, and load‑balancer recommendations. Enabling it is a one‑time action:

aws compute-optimizer enable-recommendation-export \
  --account-ids $(aws sts get-caller-identity --query Account --output text) \
  --s3-bucket my‑optimizer‑exports \
  --export-format CSV

• Replace my‑optimizer‑exports with the name of the bucket you created.
• The export runs daily and writes a CSV file named recommendations-YYYY-MM-DD.csv.
• Verify the export by checking the bucket after 24 hours.

Step 2 – Store recommendations in S3 with versioning

Versioning protects against accidental deletion and lets you compare day‑over‑day changes. Enable it with:

aws s3api put-bucket-versioning \
  --bucket my‑optimizer‑exports \
  --versioning-configuration Status=Enabled

Create a folder structure that separates instance, volume, and load‑balancer data:

aws s3api put-object --bucket my‑optimizer‑exports --key recommendations/instances/
aws s3api put-object --bucket my‑optimizer‑exports --key recommendations/volumes/
aws s3api put-object --bucket my‑optimizer‑exports --key recommendations/lb/

The Lambda will read the latest file from the instances/ prefix.

Step 3 – Build a Lambda function that evaluates recommendations

The core of the automation is a Lambda written in Python (or Node.js) that:

1. Retrieves the newest CSV file from S3.
2. Parses each row and extracts InstanceArn, CurrentInstanceType, RecommendedInstanceType, and UtilizationMetric.
3. Applies a policy threshold – for example, only act on instances whose average CPU utilization is below 20 % for the last 7 days.
4. Calls the EC2 API to either stop the instance (if it is a dev/test box) or modify its instance type.

Below is a minimal Python snippet you can paste into the Lambda console:

import boto3, csv, io, os

def lambda_handler(event, context):
    s3 = boto3.client('s3')
    ec2 = boto3.client('ec2')
    bucket = os.getenv('BUCKET')
    prefix = 'recommendations/instances/'
    # Find latest object
    resp = s3.list_objects_v2(Bucket=bucket, Prefix=prefix)
    latest = max(resp['Contents'], key=lambda x: x['LastModified'])
    obj = s3.get_object(Bucket=bucket, Key=latest['Key'])
    csv_body = io.StringIO(obj['Body'].read().decode('utf-8'))
    reader = csv.DictReader(csv_body)
    for row in reader:
        cpu = float(row['CpuUtilizationPercentage'])
        if cpu < 20.0:
            instance_id = row['InstanceArn'].split('/')[-1]
            new_type = row['RecommendedInstanceType']
            # Stop dev instances, modify prod
            tags = ec2.describe_tags(Filters=[{'Name':'resource-id','Values':[instance_id]}])
            env = next((t['Value'] for t in tags['Tags'] if t['Key']=='Environment'), 'unknown')
            if env == 'dev':
                ec2.stop_instances(InstanceIds=[instance_id])
            else:
                ec2.modify_instance_attribute(InstanceId=instance_id, Attribute='instanceType', Value=new_type)
    return {'status':'complete'}

Important settings - Set the environment variable BUCKET to your S3 bucket name. - Increase the Lambda timeout to at least 5 minutes for large fleets. - Attach the IAM role created earlier.

Step 4 – Trigger the Lambda with EventBridge on a schedule

A daily run aligns with the Compute Optimizer export cadence. Create a rule that fires at 02:00 UTC:

aws events put-rule \
  --name DailyRightsizeRule \
  --schedule-expression "cron(0 2 * * ? *)" \
  --state ENABLED

Add the Lambda as a target:

aws events put-targets \
  --rule DailyRightsizeRule \
  --targets "Id"="1","Arn"="arn:aws:lambda:us-east-1:123456789012:function:RightsizeLambda"

Finally, grant EventBridge permission to invoke the function:

aws lambda add-permission \
  --function-name RightsizeLambda \
  --statement-id EventBridgeInvoke \
  --action "lambda:InvokeFunction" \
  --principal events.amazonaws.com \
  --source-arn arn:aws:events:us-east-1:123456789012:rule/DailyRightsizeRule

Now the pipeline runs automatically every day, evaluates the latest recommendations, and applies changes without human interaction.

Step 5 – Apply changes safely with EC2 ModifyInstanceAttribute or Stop/Start

Automated actions must be reversible. Follow these safety practices:

• Create snapshots of EBS volumes before downsizing. Add a pre‑action in the Lambda: python ec2.create_snapshot(VolumeId=vol_id, Description='Pre‑rightsizing snapshot')
• Tag instances after modification with Rightsized=YYYYMMDD for audit.
• Use a dry‑run flag during the first week: python ec2.modify_instance_attribute(..., DryRun=True) Review the response before removing DryRun.
• Notify Slack or email using an SNS topic at the end of each run. Example: python sns = boto3.client('sns') sns.publish(TopicArn=os.getenv('SNS_TOPIC'), Message='Rightsizing run completed')

These steps keep the automation transparent and give you a rollback path if performance degrades.

Comparison: Manual rightsizing vs Automated pipeline

The table makes it clear why most teams overlook this tactic: the perceived upfront effort masks long‑term operational gains.

Frequently asked questions

How often should I run the rightsizing Lambda?

A daily schedule matches Compute Optimizer’s export cadence and catches workload changes quickly. If your environment is very stable, a weekly run may be sufficient, but daily runs provide the fastest feedback loop.

Will stopping a dev instance affect my CI/CD pipelines?

Only if the pipeline runs on that instance. Tag dev resources with Environment=dev and configure the Lambda to stop rather than terminate them. You can also add a whitelist tag like RightsizeSkip=true for critical dev boxes.

What if a recommended instance type is not available in my AZ?

The Lambda should include a fallback check:

try:
    ec2.modify_instance_attribute(...)
except ec2.exceptions.InvalidInstanceAttributeValue as e:
    # Log and skip, or try a different AZ

Logging the failure to CloudWatch lets you manually intervene.

Does this approach work for Spot Instances?

Yes, but Spot pricing adds volatility. The Lambda can be extended to convert On‑Demand instances to Spot by creating a new Spot request with the same configuration, then terminating the original instance after the Spot instance is healthy.

Key takeaways

• Automating rightsizing removes the manual bottleneck and delivers savings within 24 hours of a recommendation.
• Enable Compute Optimizer, export CSV to an S3 bucket with versioning, and let a Lambda evaluate utilization thresholds.
• Use EventBridge to schedule the Lambda, and incorporate safety steps like snapshots, tagging, and dry‑run validation.
• The automated pipeline scales to thousands of resources, provides an audit trail, and reduces human error.
• Start with a small pilot, monitor performance, then expand to the entire fleet.

CloudBudgetMaster can automate the entire workflow described above. Today the platform scans AWS accounts in read‑only mode, surfaces idle and wasted resources, and reports the dollar impact of each recommendation. Support for GCP, Azure, and Snowflake is coming soon. Use our free AWS waste finder to see immediate opportunities, then create a free account to let CloudBudgetMaster keep the optimization loop running for you.