← All articles

Advanced Cloud Cost Optimization Strategy Teams Overlook

Why a policy‑as‑code approach matters for cloud cost optimization

Most engineering and platform teams treat cost as a reporting problem: they look at the bill, tag resources, and hope someone notices waste. The deeper issue is that cost decisions are made outside of the code that creates the resources. When infrastructure is defined in Terraform, CloudFormation, or CDK, the same source of truth that launches a server can also enforce a cost‑effective configuration. By moving cost rules into version‑controlled policy, you eliminate manual gate‑keeping, reduce drift, and guarantee that every new resource meets a predefined efficiency baseline.

Key benefits of a policy‑as‑code strategy:

• Consistency – every pull request is evaluated against the same cost rules.
• Speed – violations are caught before resources are provisioned, avoiding remediation cycles.
• Auditability – policy changes are tracked in Git, providing a clear compliance trail.
• Scalability – the same policy set can be applied across multiple accounts and regions.

Gather accurate utilization data with AWS Compute Optimizer

AWS Compute Optimizer analyzes historical CloudWatch metrics for EC2, Auto Scaling groups, EBS volumes, and Lambda functions. It produces recommendation files that include projected monthly savings for each suggested change. To use these recommendations as the basis for a cost policy, you first need to export them in a machine‑readable format.

Step‑by‑step: Enable and export recommendations

1. Open the AWS Management Console and navigate to Compute Optimizer → Settings.
2. Turn on Data collection for the services you want to analyze. The console will show a status like Enabled – collecting data.
3. In the left navigation, choose Export recommendations.
4. Select the target S3 bucket (create one if needed) and click Export. The bucket will receive a JSON file such as compute-optimizer-recommendations-2024-10-06.json.
5. Verify the export with the CLI: bash aws s3 cp s3://my‑optimizer‑bucket/compute-optimizer-recommendations-2024-10-06.json - | jq '.recommendations | length' The command returns the number of recommendation objects.

The exported JSON contains fields like instanceArn, instanceType, utilizationMetrics, and estimatedMonthlySavings. These are the raw inputs for the policy engine.

Export recommendations and integrate with IaC

Most teams store their infrastructure as code in Git. To make Compute Optimizer data actionable, you can import the JSON into the same repository and generate a cost‑baseline file that the policy engine reads.

Example: Generating a Terraform cost‑baseline

Assume you have a Terraform module that creates EC2 instances. Create a script generate_baseline.py:

import json, pathlib

src = pathlib.Path('compute-optimizer-recommendations-2024-10-06.json')
recommendations = json.loads(src.read_text())['recommendations']

baseline = {}
for rec in recommendations:
    arn = rec['instanceArn']
    suggested = rec['recommendationOptions'][0]['instanceType']
    baseline[arn] = suggested

pathlib.Path('cost_baseline.tfvars.json').write_text(json.dumps(baseline, indent=2))

Run the script after each export and commit the resulting cost_baseline.tfvars.json to Git. Terraform can then read the file with:

variable "cost_baseline" {
  type = map(string)
}

resource "aws_instance" "example" {
  for_each = var.instances
  ami      = each.value.ami
  instance_type = var.cost_baseline[each.key] != "" ? var.cost_baseline[each.key] : each.value.instance_type
}

In this example the instance type defaults to the Compute Optimizer recommendation when it exists, ensuring that new deployments start with the most efficient size.

Enforce cost‑optimized configurations with policy engines

Having the baseline in version control is only half the solution. You still need a gate that prevents a developer from overriding the recommendation without justification. Two popular open‑source options integrate well with AWS:

Enforcing with OPA in a CI pipeline

1. Add a policy directory to your repo and create cost_policy.rego: ```rego package cost

# Map of instance ARN to approved type from baseline file approved[arn] = type { data.baseline[arn] = type }

deny[msg] { input.resource_type == "aws_instance" not approved[input.resource_arn] = input.instance_type msg = sprintf("Instance %s uses %s, which differs from approved %s", [input.resource_arn, input.instance_type, approved[input.resource_arn]]) } 2. In your CI workflow (GitHub Actions example):yaml name: Cost Policy Check on: [pull_request] jobs: opa: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Install OPA run: curl -L -o opa https://openpolicyagent.org/downloads/v0.64.0/opa_linux_amd64 && chmod +x opa - name: Load baseline run: cp cost_baseline.tfvars.json data.json - name: Run policy check run: | ./opa eval -i data.json -d policy/cost_policy.rego "data.cost.deny" `` If the evaluation returns anydeny` messages, the PR fails and the developer must either accept the recommendation or provide a documented exception.

Automate remediation: Lambda functions that downsize or stop resources

Even with strict policies, legacy resources may already exist in the account. A lightweight Lambda can read Compute Optimizer recommendations daily, compare them to live resources, and take corrective actions such as: - Downsize an EC2 instance type. - Stop an idle RDS instance. - Delete unattached EBS volumes that exceed a size threshold.

Sample Lambda (Python) for EC2 down‑sizing

import boto3, json, os

s3 = boto3.client('s3')
ec2 = boto3.client('ec2')
BUCKET = os.getenv('RECOMMENDATION_BUCKET')
KEY = os.getenv('RECOMMENDATION_KEY')

def lambda_handler(event, context):
    obj = s3.get_object(Bucket=BUCKET, Key=KEY)
    recs = json.loads(obj['Body'].read())['recommendations']
    for r in recs:
        arn = r['instanceArn']
        opt_type = r['recommendationOptions'][0]['instanceType']
        instance_id = arn.split('/')[-1]
        # Get current type
        cur = ec2.describe_instances(InstanceIds=[instance_id])['Reservations'][0]['Instances'][0]
        cur_type = cur['InstanceType']
        if cur_type != opt_type:
            print(f"Resizing {instance_id} from {cur_type} to {opt_type}")
            ec2.modify_instance_attribute(InstanceId=instance_id, InstanceType={'Value': opt_type})
    return {'status': 'completed'}

Deploy the function via the console or SAM, set the environment variables RECOMMENDATION_BUCKET and RECOMMENDATION_KEY, and schedule it with an EventBridge rule (rate(1 day)). The Lambda runs without needing write permissions to other services, keeping the IAM footprint minimal.

Monitoring, alerting, and continuous improvement

A policy‑as‑code system is only as good as the feedback loop that tells you when it fails or succeeds. Combine three AWS services to close the loop:

1. CloudWatch Logs – Capture Lambda output and OPA CI results.
2. CloudWatch Alarms – Trigger an SNS notification if the number of policy violations exceeds a threshold in a 24‑hour window.
3. Cost Explorer – Create a custom report that groups spend by the CostCategory you defined for Optimized vs Non‑Optimized resources. Review the report weekly to quantify savings.

Quick alert setup

aws cloudwatch put-metric-alarm \
  --alarm-name "PolicyViolationCount" \
  --metric-name "PolicyViolations" \
  --namespace "Custom/FinOps" \
  --statistic Sum \
  --period 86400 \
  --threshold 5 \
  --comparison-operator GreaterThanOrEqualToThreshold \
  --evaluation-periods 1 \
  --alarm-actions arn:aws:sns:us-east-1:123456789012:FinOpsAlerts

The metric PolicyViolations can be emitted by your CI pipeline using the PutMetricData API.

Frequently asked questions

How often should I refresh Compute Optimizer data?

Compute Optimizer updates its metrics every 24 hours. For most workloads a daily export is sufficient. If you run a high‑frequency CI pipeline, you can trigger an on‑demand export using the aws compute-optimizer get-recommendations CLI, but be aware of API rate limits.

Does policy‑as‑code replace human cost reviews?

No. The policy engine enforces baseline efficiency, but strategic decisions—such as purchasing Savings Plans or reserving capacity—still require human analysis. The automation frees the team to focus on those higher‑level choices.

Can I apply the same approach to other clouds?

The concept works across clouds, but the tooling differs. AWS provides Compute Optimizer and Config Rules; GCP and Azure have similar recommendation APIs that will be supported by CloudBudgetMaster soon. Until then, you can manually export usage data and feed it into a generic policy engine.

What if a recommendation would break a critical workload?

Policy violations can be overridden with a documented exception. Store the exception in a separate exceptions.yaml file, and modify the OPA rule to allow the specific ARN when input.exception == true. This keeps the audit trail while preserving stability.

Key takeaways

• Export Compute Optimizer recommendations to a version‑controlled baseline file.
• Use policy‑as‑code (OPA or Config Rules) to block non‑optimal resources in CI.
• Deploy a daily Lambda to remediate existing drift automatically.
• Wire CloudWatch metrics and alarms to maintain visibility on policy violations.
• Leverage Cost Explorer reports to quantify the financial impact of the strategy.
• Start with the free AWS waste finder to identify low‑hanging idle resources, then scale the policy framework.
• When ready, create a free account to let CloudBudgetMaster continuously scan your AWS environment, report dollar impact of idle and wasted resources, and soon support GCP, Azure, and Snowflake.

CloudBudgetMaster automates this workflow by scanning AWS in read‑only mode today, delivering a clear dollar impact of idle and wasted resources. Support for GCP, Azure, and Snowflake is coming soon.