← All articles

Advanced Cloud Cost Optimization Strategy Teams Overlook

The hidden cost of always‑on non‑production environments

Most engineering and platform teams treat development, test, and staging environments as permanent fixtures. The result is a steady stream of EC2 instances, RDS databases, and Elasticache clusters that run 24/7 even when nobody is using them. The dollar impact shows up as a line item labeled "running instances" on the AWS bill, but it rarely triggers a deeper investigation because the resources appear "required" for the workflow.

Answer: Implement a dynamic instance‑scheduling strategy that automatically stops and starts non‑production resources based on a calendar or usage pattern. This tactic is supported by native AWS tooling, requires only a few CLI commands, and can reduce compute spend by 20‑40 % without affecting production workloads.

1. Identify candidate resources for scheduling

Before you can automate anything, you need a reliable inventory of the workloads that are safe to stop during off‑hours.

1.1 Tagging convention

Create a tagging policy that distinguishes production from non‑production. A simple two‑tag approach works well:

• Environment=prod – never stop
• Environment=dev|test|stage – eligible for scheduling

Apply the tags using the AWS CLI or console:

# Example: tag an EC2 instance
aws ec2 create-tags \
  --resources i-0abcd1234efgh5678 \
  --tags Key=Environment,Value=dev

1.2 Generate a list of schedulable resources

Use the aws resourcegroupstaggingapi to pull all resources with the non‑production tag:

aws resourcegroupstaggingapi get-resources \
  --tag-filters Key=Environment,Values=dev,test,stage \
  --resource-type-filters ec2:instance rds:db elasticache:cluster \
  --output table

Export the list to a CSV for review and future automation:

aws resourcegroupstaggingapi get-resources \
  --tag-filters Key=Environment,Values=dev,test,stage \
  --resource-type-filters ec2:instance rds:db elasticcache:cluster \
  --query 'ResourceTagMappingList[*].ResourceARN' \
  --output text > schedulable_resources.txt

2. Choose a scheduling implementation

Three common approaches exist. The table below compares them on cost, flexibility, and operational overhead.

For most teams, the AWS Instance Scheduler offers the best balance of zero additional licensing cost and robust features.

3. Deploy the AWS Instance Scheduler solution

AWS provides a ready‑to‑use CloudFormation template that creates all required resources: DynamoDB tables, Lambda functions, IAM roles, and CloudWatch Events.

3.1 Download the template

curl -O https://aws-solutions.s3.amazonaws.com/instance-scheduler/v2.0.0/instance-scheduler.template

3.2 Launch the stack

Replace MySchedulerStack with a stack name that fits your naming convention.

aws cloudformation create-stack \
  --stack-name MySchedulerStack \
  --template-body file://instance-scheduler.template \
  --capabilities CAPABILITY_NAMED_IAM \
  --parameters ParameterKey=TagName,ParameterValue=Environment ParameterKey=TagValue,ParameterValue=dev,test,stage ParameterKey=ScheduleName,ParameterValue=OffHours

The stack creates:

• DynamoDB tables that store schedule definitions and resource mappings.
• Lambda functions (scheduler-start, scheduler-stop) that start/stop resources.
• CloudWatch Event rules that trigger the Lambdas at the defined times.

3.3 Define an off‑hours schedule

Edit the SchedulerConfig DynamoDB table (via the console or CLI) to add a schedule named OffHours:

aws dynamodb put-item \
  --table-name SchedulerConfig \
  --item '{
    "Name": {"S": "OffHours"},
    "Period": {"S": "Daily"},
    "StartTime": {"S": "20:00"},
    "StopTime": {"S": "08:00"},
    "Timezone": {"S": "UTC"},
    "Enforced": {"BOOL": true}
  }'

This schedule stops resources at 20:00 UTC and restarts them at 08:00 UTC each day.

4. Map resources to the schedule automatically

The Instance Scheduler can automatically associate resources based on tags. Ensure your non‑production resources have the Schedule=OffHours tag, or let the scheduler use the Environment tag you already created.

# Tag an RDS instance for the OffHours schedule
aws rds add-tags-to-resource \
  --resource-name arn:aws:rds:us-east-1:123456789012:db:my-test-db \
  --tags Key=Schedule,Value=OffHours

If you prefer a bulk operation, use the list you exported earlier:

while read -r ARN; do
  aws resourcegroupstaggingapi tag-resources \
    --resource-arn-list "$ARN" \
    --tags Schedule=OffHours
done < schedulable_resources.txt

The scheduler Lambda reads the DynamoDB ScheduleTag configuration (default Schedule) and applies the defined schedule to any resource carrying that tag.

5. Verify that start/stop actions work as expected

5.1 Dry‑run with CloudWatch Logs

The Lambda functions write detailed logs to CloudWatch. Open the log group /aws/lambda/InstanceScheduler-Stop and look for entries like:

2024-10-19T20:00:01.123Z INFO Stopping instance i-0abcd1234efgh5678 (Environment=dev)

If you see the expected messages, the stop action succeeded.

5.2 Manual trigger for testing

You can invoke the stop Lambda manually to confirm it can stop a specific instance:

aws lambda invoke \
  --function-name InstanceScheduler-Stop \
  --payload '{"resourceId":"i-0abcd1234efgh5678"}' \
  response.json

Check response.json for a success status and then verify the instance state in the console.

6. Measure the financial impact

After a week of operation, compare the compute spend before and after scheduling.

1. Open the Cost Explorer console.
2. Filter by TagKey=Environment and TagValue=dev,test,stage.
3. Set the time range to the last 7 days.
4. Export the CSV and calculate the difference.

For a quick CLI view, use the aws ce get-cost-and-usage API:

aws ce get-cost-and-usage \
  --time-period Start=$(date -d '-7 days' +%Y-%m-%d),End=$(date +%Y-%m-%d) \
  --granularity DAILY \
  --filter '{"Tags":{"Key":"Environment","Values":["dev","test","stage"]}}' \
  --metrics UnblendedCost \
  --output table

Document the dollar savings and feed the numbers back into your FinOps dashboard.

7. Automate continuous improvement

Scheduling is not a set‑and‑forget activity. As teams create new environments, they must inherit the tagging policy and schedule.

7.1 Enforce tags with Service Control Policies (SCP)

If you use AWS Organizations, attach an SCP that denies creation of EC2, RDS, or Elasticache resources without the required Environment tag:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Deny",
    "Action": ["ec2:RunInstances", "rds:CreateDBInstance", "elasticache:CreateCacheCluster"],
    "Condition": {"Null": {"aws:TagKeys": true}},
    "Resource": "*"
  }]
}

7.2 Use CloudFormation macros for tag propagation

Add a macro that automatically adds the Schedule=OffHours tag to any resource defined in a stack that includes the Environment=dev|test|stage parameter.

Parameters:
  Environment:
    Type: String
    AllowedValues: [dev, test, stage]
Resources:
  MyInstance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: t3.micro
      Tags:
        - Key: Environment
          Value: !Ref Environment
        - Key: Schedule
          Value: OffHours

Frequently asked questions

How does stopping an RDS instance affect data?

When you stop an Amazon RDS instance, the underlying storage remains intact and you are not charged for instance hours. However, you cannot stop Aurora clusters; instead, you can pause Aurora Serverless v2. Always ensure backups are up‑to‑date before stopping.

Will my CI/CD pipelines break if a dev environment is stopped?

Schedule the stop time after the last expected build of the day. Most pipelines can be configured to skip runs when the target environment is unavailable. If a pipeline must run on demand, add a pre‑step that starts the environment via the aws ec2 start-instances CLI.

Is there any risk of data loss when using the Instance Scheduler?

The scheduler only issues start/stop API calls. Stopping a compute instance does not delete attached EBS volumes (unless you set DeleteOnTermination). For databases, the stop operation preserves the data. Review each service's stop behavior in the AWS documentation.

Can I apply this strategy to spot instances?

Yes. Spot instances can be stopped (if they support hibernation) or terminated. The scheduler treats them like any other instance; just be aware that stopping a spot instance may cause it to lose the spot price advantage when restarted.

Key takeaways

• Tag non‑production resources with a clear Environment value.
• Deploy the native AWS Instance Scheduler via CloudFormation for zero‑cost automation.
• Define off‑hours schedules in DynamoDB and map resources using the Schedule tag.
• Verify actions through CloudWatch logs and manual Lambda invocations.
• Use Cost Explorer or the aws ce CLI to quantify savings.
• Enforce tagging at creation time with SCPs or CloudFormation macros to keep the system self‑sustaining.

Next steps with CloudBudgetMaster

CloudBudgetMaster can automatically scan your AWS account in read‑only mode, identify idle and under‑utilized resources, and report the dollar impact of each wasteful asset. Our platform currently supports AWS scanning today, with GCP, Azure, and Snowflake support coming soon. Use our free AWS waste finder to get an instant view of hidden spend, then create a free account to start automating remediation.