AI Infrastructure Costs: 5 Moves to Protect Gross Margin
In this article
Key takeaways
- Compute and storage were named among the top AI forecast overruns more often than token costs.
- Healthy reported margins can coexist with pressure from AI infrastructure costs.
- Finance teams need to forecast the full delivery cost, attribute it to customers and products, and test whether pricing recovers it.
Tokens ranked third. If your AI forecast starts and ends with the model bill, it misses the categories finance leaders most often named among their top overruns.
In Cloud Capital’s The Cost of Compute: AI Edition, compute topped that list at 64%, followed by data and storage at 60%. Foundation-model API and token costs came next at 47%, with networking and egress at 44%.[1]
The uncomfortable question for a CFO is whether AI infrastructure costs are growing inside the cloud bill without being visible in the product forecast.
Research note: Sapio Research surveyed 100 senior finance leaders at US and UK SaaS and technology companies with 50–1,000 employees in June 2026. Respondents could name up to three overrun categories; the percentages measure how often categories were selected, not the size of overspend.
Healthy margins can hide rising AI infrastructure costs
The report’s most revealing tension is that AI infrastructure is putting pressure on margins while reported margins across the sample are mostly rising.[2]
That is a reason to look beneath the headline number. A healthy company margin does not tell you whether an individual AI feature earns its keep. Revenue growth, pricing or changes elsewhere in the business can offset an expensive delivery model. The board sees the result; finance needs to understand what is sustaining it.
The risk is approving the next phase of growth on economics that were never fully measured.
Consider a hypothetical AI document-analysis feature generating $100,000 in monthly revenue. Its launch budget includes $10,000 for model API and tokens. At the planned usage level, the actual delivery stack also needs $7,000 of application compute, $5,000 of retrieval and storage, and $3,000 of networking and monitoring.
The token budget was accurate. The cost scope was not. Identified delivery cost is 2.5 times the budgeted amount, leaving $75,000 after these costs instead of the expected $90,000. Those are illustrative contributions before any other COGS, not a complete gross margin calculation.
The question to take into the next product review is simple: “What does one successful customer outcome cost us, including everything needed to deliver it?”
Forecast the workload, not just the model calls
Give supporting AI infrastructure its own forecast lines, separate from token spend, and reconcile them to the cloud bill. Then ask engineering what changes when adoption grows: interactions per customer, processing per interaction, retrieval demand and data retained.
AI compute costs can change even when the number of customers does not. A feature that performs more steps per request may cost more to serve. Cloud cost forecasting needs to reflect those design choices as well as the sales plan.
Simply increasing last month’s bill by the revenue growth rate leaves changes in usage intensity and architecture out of the forecast.
Build a driver-based forecast with a base case and a higher-usage case, making the cost assumptions explicit enough for finance and engineering to challenge together.
Find the customers changing your AI unit economics
An average cost per customer can look acceptable while a handful of heavy users consume the contribution from everyone else. Break costs down by product, customer segment and team, starting with the most material workload.
Connect model spend and supporting infrastructure to revenue and usage. Compare the economics of typical users with heavy users before extending unlimited access or bundling more AI into a subscription.
Cloud Capital’s AI research found that 72% of AI-native companies had excellent customer-level cost attribution, compared with 29% of AI-enabled companies.[3] That gap is a useful challenge for businesses adding AI to an established product: can finance see the new economics at customer level?
Cloud Capital’s financial control layer is designed to help separate AI infrastructure from the rest of cloud spend and connect costs to their drivers.
Cloud Capital: visibility across cloud and AI infrastructure
Bring that analysis into a profitability view showing delivery cost and contribution by product or customer segment. Use it to identify where pricing, usage limits or delivery efficiency need attention.
Check whether better margins are just a classification change
Many teams carry development and experimentation spend in COGS, which understates product margin. Correcting it is legitimate, as long as the basis is documented and the board can tell a classification change from a delivery improvement.
Separate customer-serving production, development and experimentation, and shared services. Document the basis for allocating shared costs, and test whether it still reflects usage after AI features launch.
A useful board check is to bridge changes in reported AI gross margin into revenue, delivery efficiency, unit rates, product mix and classification. That makes operational improvement distinguishable from a reporting change.
Keep a classification map alongside the margin bridge so the board can see which changes reflect delivery economics and which reflect reporting treatment.
Put the approval before the spending decision
Budget alerts help teams spot a problem. Approval rules help them address the economics before a launch or architecture change creates one.
Give each product or team an owner, spending thresholds and an escalation route. Require material changes to include an estimate of incremental delivery cost, expected customer value and any pricing response. Agree what happens when an experiment becomes a production workload.
An alert without an owner or an agreed response is a notification, not a spending control.
Document the thresholds, owners and required actions in a short escalation policy that product teams can use before committing the spend.
Test the commitment, not just the discount
Separate the workload you expect to keep from the demand that is still uncertain. Negotiate rates against the durable baseline, then test what happens if usage falls or engineering changes the delivery stack. A cheaper rate can still produce a more expensive bill if the business pays for capacity it cannot use.
Cloud Capital gives customers committed-use rates while taking on the utilization risk itself, so the business gets the discount without carrying the downside if usage falls short. Customers also start with 5% cashback on qualifying AWS spend from month one, before making any new commitment.
Cloud Capital: infrastructure savings
Before approving a commitment, compare total cost under expected usage and a downside case, including any payment for unused capacity. Make the exposure explicit: who carries it, how long it lasts and which product or architecture changes could make it material. Review those assumptions monthly with engineering. In Cloud Capital’s separate The Cost of Compute 2026 study, 39% of teams with joint finance and engineering ownership reported highly predictable forecasts, compared with 16% under engineering-only ownership.[4]
Record the decision in a monthly commitment plan: the demand assumptions, capacity covered, downside exposure and owner responsible for reviewing it.
Bring one question to your next product review
“Are we pricing the full cost of delivering AI, or just the part we can see?”
Finance needs to know what drives AI infrastructure costs, which customers create them, how they are classified and whether pricing recovers them. A healthy headline margin doesn’t answer that question. It just makes it easier not to ask.
Sources and research notes
[1] The Cost of Compute: AI Edition, page 9
[2] The Cost of Compute: AI Edition, page 12
The AI study did not measure the wider drivers of margin change. Potential offsets discussed in this article are explanations to investigate, not measured causes. The report suggests that the visibility of token costs may contribute to the forecast blind spot; it does not establish causation.
[3] The Cost of Compute: AI Edition, page 13
[4] The Cost of Compute 2026, Finance Ownership
Ownership and forecast predictability are associated in the survey; the comparison does not prove that changing ownership alone produces the outcome. The example above is hypothetical and is not a Cloud Capital customer case or research finding.
