← All articles

Unified Multi-Cloud Cost Visibility: Consolidate AWS, GCP & Azure Spend

Why a Single View of Spend Matters

Most engineering teams treat each cloud provider as a silo. That works for provisioning but makes cost control impossible: you can’t compare an idle t3.medium in AWS with an under‑utilized e2-standard-2 in GCP, nor see whether a shared Azure SQL Database is the cheapest option for a workload. A unified cost view lets you:

• Spot duplicate workloads across clouds.
• Align budgeting with corporate finance that expects a single line‑item.
• Apply consistent tagging and chargeback rules.
• Reduce manual reconciliation time from hours to minutes.

The challenge isn’t the data—it’s pulling it, normalizing it, and visualizing it in a way that’s comparable.

Pull Raw Cost Data from Each Provider

AWS

1. Enable Cost Explorer in the console (Settings → Enable).
2. Use the CLI to export the last 30 days of usage:

aws ce get-cost-and-usage \
  --time-period Start=$(date -d '-30 days' +%Y-%m-%d),End=$(date +%Y-%m-%d) \
  --granularity DAILY \
  --metrics "UnblendedCost" "UsageQuantity" \
  --group-by Type=DIMENSION,Key=TAG

Replace TAG with your cost allocation tag key (e.g., Project). The output is JSON; pipe it to jq to flatten:

aws ce get-cost-and-usage ... | jq -r '.ResultsByTime[].Groups[] | [.Keys[0], .Metrics.UnblendedCost.Amount] | @tsv'

GCP

1. Create a billing export to a BigQuery dataset (Billing → Settings → Export to BigQuery).
2. Query the export with bq:

bq query --use_legacy_sql=false \
'SELECT service.description, sku.description, usage_start_time, cost, labels.project_id \
 FROM `my-billing.project_id.gcp_billing_export_v1_001` \
 WHERE usage_start_time >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)'

The labels field holds any user‑defined tags.

Azure

1. Enable the Consumption API by granting the Reader role on the subscription.
2. Pull daily usage with Azure CLI:

az consumption usage list \
  --subscription <SUB_ID> \
  --start-date $(date -d '-30 days' +%Y-%m-%d) \
  --end-date $(date +%Y-%m-%d) \
  --query '[].{date: usageStart, cost: pretaxCost, meter: meterCategory, tags: tags}' \
  -o json > azure_usage.json

For price information, also pull the price sheet:

az consumption price-sheet list \
  --subscription <SUB_ID> \
  -o json > azure_prices.json

Normalize Tags and Cost Allocation

Each cloud has its own tag model: - AWS: Cost allocation tags (must be activated). - GCP: Labels on resources, exported as labels. - Azure: Tags on resources, appear under tags in the Consumption API.

Create a mapping table that translates each provider’s tag key to a common taxonomy, e.g., Project, Environment, Owner. A simple Python script can read the three JSON/CSV files, rename keys, and output a unified CSV:

import pandas as pd, json, sys
aws = pd.read_csv('aws_cost.tsv', sep='\t', names=['Tag','Cost'])
gcp = pd.read_json('gcp_query.json')
az = pd.read_json('azure_usage.json')
# Standardize column names
aws.rename(columns={'Tag':'Project'}, inplace=True)
gcp.rename(columns={'labels.project_id':'Project'}, inplace=True)
az.rename(columns={'tags.Project':'Project'}, inplace=True)
# Concatenate
combined = pd.concat([aws[['Project','Cost']],
                     gcp[['project_id','cost']].rename(columns={'project_id':'Project','cost':'Cost'}),
                     az[['tags.Project','cost']].rename(columns={'tags.Project':'Project','cost':'Cost'})])
combined.to_csv('unified_cost.csv', index=False)

Now every row has a Project column you can aggregate across clouds.

Build a Single Dashboard

Option 1: Open‑Source Stack (Grafana + Prometheus)

1. Ingest CSV into Prometheus using the textfile collector. Place unified_cost.csv in /var/lib/prometheus/textfile_collector/ and add a custom exporter script that converts each line to a metric:

while IFS=, read -r project cost; do
  echo "cloud_cost{project=\"$project\"} $cost"
 done < unified_cost.csv > /var/lib/prometheus/textfile_collector/cloud_cost.prom

2. Reload Prometheus and create a Grafana panel: - Query: sum by (project) (cloud_cost) - Visualization: Bar gauge or table.

Option 2: SaaS BI (Google Data Studio / Power BI)

1. Upload unified_cost.csv to Google Sheets.
2. Connect Data Studio to the sheet, create a blended data source that adds a static column cloud (AWS, GCP, Azure) for each row.
3. Build a time‑series chart of Cost by Project and a pie chart of Cost by cloud.

Both approaches let you slice by Environment or Owner once you add those tags to the mapping.

Ongoing Governance and Alerts

Enforce Tagging at Provision Time

• AWS: Use an IAM policy that denies RunInstances unless the request includes required tags.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Deny",
    "Action": "ec2:RunInstances",
    "Resource": "*",
    "Condition": {"StringNotEquals": {"aws:TagKeys": ["Project","Owner"]}}
  }]
}

• GCP: Deploy an Organization Policy constraints/compute.requireOsLogin combined with a custom gcloud wrapper that adds --labels project=....
• Azure: Apply an Azure Policy that requires tags:

{
  "if": {"field": "tags.Project", "equals": ""},
  "then": {"effect": "deny"}
}

Automated Cost Alerts

• AWS Budgets: aws budgets create-budget --account-id <id> --budget file://budget.json
• GCP Budgets: gcloud billing budgets create --billing-account <ACCOUNT> --budget-file budget.yaml
• Azure Budgets: az consumption budget create --category cost --amount 5000 --time-grain Monthly --name "Team‑X"

Set the alert threshold to 80 % of the monthly allocation and route notifications to Slack or email.

Turning Insight into Action

Once you see a line item like Project=demo‑app, Cloud=AWS, Cost=$120 you can: 1. Verify utilization via CloudWatch metrics (CPUUtilization, NetworkIn). 2. Compare with the same project on GCP (e2-medium usage) and decide whether to consolidate. 3. Right‑size or switch to spot/preemptible instances based on the unified view.

Regularly schedule the extraction script (e.g., a daily cron job) so the dashboard always reflects the latest spend.

How CloudBudgetMaster helps: Our platform automatically pulls cost data from AWS Cost Explorer, GCP Billing Export, and Azure Consumption APIs, normalizes tags across clouds, and surfaces the dollar impact of idle resources in a single dashboard—so you can act without building the pipeline yourself.