You open AWS Cost Explorer on the first Monday of the month. The number is bigger than last month. Again. You can see it came from EC2 and SageMaker. But which model? Which team? Which experiment that someone spun up on Thursday and forgot to shut down?
Cost Explorer doesn't know. And neither do you.
This is the reality for most ML engineering teams running production workloads on AWS. The billing infrastructure was built for traditional cloud workloads — web servers, databases, storage. It was never designed to answer the questions that ML teams actually need answered: what does it cost to train this specific model? How much are we spending on inference for this endpoint? Which team's experiments are eating 40% of our GPU budget?
The Gap Between AWS Billing and ML Reality
AWS gives you granular billing data. That's not the problem. The Cost and Usage Report (CUR) can deliver line-item records for every resource-hour consumed across your account. You can slice by service, region, instance type, and tag.
The problem is that ML workloads don't map cleanly to the categories AWS bills by.
A single model training run might touch SageMaker training instances, S3 for data and checkpoints, ECR for container images, CloudWatch for logging, and possibly EFS or FSx for shared storage. The inference pipeline adds SageMaker endpoints (or self-managed EC2 instances), load balancers, and API Gateway. A data preprocessing step might run on EMR or Glue.
All of these show up as separate line items in your AWS bill, scattered across different services with no thread connecting them back to the model they served. It's like getting a utility bill that shows you used electricity in the kitchen, bedroom, and living room — but can't tell you how much the refrigerator costs to run.
Why Tagging Alone Doesn't Solve It
The standard answer from AWS and most FinOps practitioners is "just tag your resources." And yes, tagging is necessary. But for ML workloads, it's far from sufficient.
Here's why. ML teams work iteratively. A data scientist spins up a notebook instance to test a hypothesis, runs three training jobs with different hyperparameters, evaluates results, adjusts, and runs three more. In a single afternoon, they might create a dozen transient resources across four AWS services. Expecting every one of those to be properly tagged, in real time, with the correct project, team, and model identifiers is optimistic at best.
Even when tags are applied, they're often inconsistent. One team tags their SageMaker jobs with project:fraud-detection. Another uses Project:fraud_model_v2. A third doesn't tag at all because they're using spot instances through a custom Kubernetes scheduler on EKS that doesn't propagate tags to the underlying EC2 instances.
The Real Cost of Cost Blindness
The operational pain is obvious — you can't explain your bill, you can't forecast accurately, and you can't charge back costs to the teams generating them. But the financial impact is larger than most leaders realize.
When an ML engineer can see that their model's inference endpoint costs $14,000/month and a comparable endpoint using a smaller distilled model would cost $3,200/month, they make different architectural choices. When a team lead can see that 60% of their GPU budget is consumed by a single legacy model that serves 5% of production traffic, they prioritize the migration.
Without model-level AWS ML cost visibility, these decisions don't get made. The spend grows. The bill goes up. And the engineering team shrugs because they have no data to act on.
What Model-Level Cost Attribution Actually Looks Like
True ML cost attribution means connecting every dollar of AWS spend to the model, pipeline, team, and business function that generated it. Not just at the service level, but at the workload level.
This requires stitching together data from multiple sources. The AWS CUR provides the raw cost data. The FOCUS 1.0 standard (recently adopted by AWS in their billing exports) helps normalize that data. But you still need ML-specific context: which SageMaker training job belongs to which model, which EC2 instances are running inference for which endpoint, which S3 costs are training data vs. model artifacts vs. checkpoint storage.
When you have this picture, you can answer questions that were previously impossible:
"What does our fraud detection model cost per month, fully loaded?" Not just the SageMaker endpoint, but the training runs, the data pipeline, the storage, the monitoring. The full cost of owning that model in production.
"How much did Team Alpha spend on experiments last quarter?" Not a rough estimate based on their AWS account, but an accurate number based on every resource their experiments actually consumed.
"What's our cost per prediction for the risk scoring model?" Divide the total attributed cost by the number of predictions served. Now you have a unit economics number you can track, optimize, and report to finance.
"If we migrate this model from real-time SageMaker endpoints to batch inference on Spot instances, what would we save?" You can model the answer because you know the current fully-loaded cost.
The AWS CUR Is Your Foundation — But Not Your Solution
If you're starting from zero on ML cost visibility, the AWS Cost and Usage Report is where to begin. Enable the CUR with FOCUS 1.0 export in your billing account, and you'll have detailed line-item data landing in S3 that you can query with Athena.
But as we discussed, CUR data alone doesn't give you model-level attribution. You'll need to enrich it with metadata from your ML platform — training job IDs, endpoint names, pipeline run identifiers — and build the mapping logic that connects AWS resources to ML workloads.
Some teams build this in-house. They write Athena queries, build dbt models on top of CUR data, and create Grafana dashboards. This works until it doesn't. The maintenance burden grows with every new service, every new team, every new model. The engineer who built it leaves, and suddenly nobody understands the pipeline. Six months later, the dashboards are stale and the team is back to spreadsheets.
This is the exact problem ML Cost Intel was built to solve. We ingest your CUR data, automatically map AWS resources to ML workloads using metadata from SageMaker, EKS, and your CI/CD pipeline, and give you a real-time dashboard that shows cost per model, per team, per environment. Setup takes minutes, not quarters. And it stays accurate as your ML platform evolves because the mapping is automated, not manual.
Where to Start This Week
If you're an engineering leader or FinOps practitioner at a company running ML on AWS, here are three things you can do this week to start closing the cost visibility gap:
Enable the CUR with FOCUS export. If you haven't already, set up the Cost and Usage Report in your AWS Billing console with the FOCUS 1.0 data export option. Choose Parquet format and land it in an S3 bucket you control. This is the raw data foundation everything else builds on.
Audit your tagging compliance for ML resources. Run a report in AWS Config or Tag Editor to see what percentage of your SageMaker, EC2 GPU, and EKS resources carry your required cost allocation tags. If it's below 80%, you have a tagging problem that needs to be addressed before any cost tool (including ours) can be fully effective.
Calculate your cost-per-model for one production model. Pick your most important model. Manually trace every AWS resource it touches — training, inference, storage, networking, monitoring. Add up the monthly cost. This exercise alone will reveal how much hidden cost exists that your current tooling doesn't surface.
Or, if you'd rather skip the manual work and see your full ML cost picture in minutes, try ML Cost Intel free. We connect to your AWS account, ingest your CUR data, and show you what every model actually costs — before next month's bill arrives.