FinOps for AI: Foundations

By the end of this lesson, you can…

• Explain why 98% of FinOps teams now manage AI spend, up from 31% two years ago, and how that breaks cost-allocation habits built for predictable cloud bills.
• Read an LLM API bill's token pricing mechanics: asymmetric input/output rates, context-caching multipliers, and batch discounts.
• Compare GPU/accelerator on-demand, reserved, and spot pricing, and explain why AI utilization math differs from general compute.
• Place a given AI workload inside the 2026 Framework's FinOps for AI Technology Category and name which of its three reused capabilities applies.
• Match a cost driver on a real AI bill to the persona who actually owns that decision, including the two personas new to this table.

Why the Old FinOps Playbook Broke

Two years ago, AI spend was a rounding error on most cloud bills — a research team's GPU box, maybe an experimental API key. That's no longer true. FinOps Foundation survey data shows 98% of FinOps teams now manage AI spend, up from just 31% two years earlier. That's not gradual growth; that's a discipline that had to absorb an entirely new spend category in two budget cycles.

The problem isn't that AI spend is bigger — plenty of line items are big. It's that the habits built for the old bill don't transfer cleanly:

Bursty, not predictable

A viral feature or a runaway agent loop can 10x token spend in a day. General compute forecasting assumes gradual, seasonal change -- AI spend doesn't.

Metered by a unit finance has never budgeted in

Tokens and GPU-hours, not instance-months. A budget built around "how many servers" doesn't map onto "how many tokens did the model read."

Waste that looks identical to growth

A cost spike from a genuinely more-used feature and one from an inefficient prompt or an unbounded retry loop produce the exact same chart shape until someone digs in.

Owned by people outside the usual room

Data Science and Product now make decisions that move the bill as much as Engineering does -- more on this below.

Everything else in this course is really just working through the consequences of those four differences, one FinOps capability at a time.

Token Economics: How LLM API Pricing Actually Works

Every major LLM API provider prices around the same core mechanic, and it's the first thing that trips up anyone used to reading a compute bill: input and output tokens are priced differently, and it's not a small gap.

A few things worth sitting with in that table:

• Output costs 5x input, on both models. Generating a token is more compute-intensive than reading one, and every major provider's rate card reflects that. A cost model that treats "tokens" as one undifferentiated number is already wrong before it does anything else.
• Context caching isn't free to write, only cheap to reuse. Anthropic's own pricing (per its prompt-caching documentation) charges a 5-minute cache write at 1.25x the standard input rate and a 1-hour cache write at 2x — you're paying a premium to store the context. A cache read (cache hit) is 0.1x input, a 90% discount. The trade only pays off once you re-read that cached context at least once; a system prompt or long document sent once and never reused is cheaper left uncached.
• Batch and cache discounts stack. A real, documented example: Claude Haiku 4.5's standard input is $1.00/MTok. A cached input hit brings that to $0.10/MTok. Route that same cached read through the Batch API's flat 50% discount for asynchronous work, and it lands at $0.05/MTok — 20x cheaper than the standard input rate, for tokens that don't need a real-time response.

That compounding is the single highest-leverage lever most teams haven't pulled yet: nothing about the model changes, only when and how the same tokens are sent.

GPU & Accelerator Cost Structures

GPU capacity is priced the same three ways as general compute — on-demand, committed, and spot — but the gaps between them, and what "committed" even means, are different for accelerators. Take AWS's p5.48xlarge (8x NVIDIA H100, the workhorse instance for large-scale training and high-throughput inference) as a real example:

The part that differs from a general-purpose EC2 fleet: "reserved" doesn't mean a 1- or 3-year Reserved Instance here. AWS's GPU commitment vehicle is EC2 Capacity Blocks for ML — you reserve a specific cluster size for a specific future date range, and you pay for that entire window whether you fill every hour with training runs or not. On the other cloud side, GCP's standard flexible Committed Use Discounts explicitly exclude the A3 (H100) and A2 (A100) accelerator-optimized machine families — GPU flex-CUD eligibility is limited to the G2/G4 machine series, and a discounted commitment on A2 Ultra is a sales conversation, not a self-serve toggle. That's why AI utilization math differs from general compute: the commitment product itself is less standardized across providers, so the "is this commitment still worth it" question in Week 5 needs a different answer per cloud, not one universal formula.

The 2026 Framework: FinOps for AI as a Technology Category

It's tempting to assume AI needed a whole new FinOps discipline. It didn't, and this is worth being precise about: the FinOps Foundation's 2026 Framework treats "FinOps for AI" as a Technology Category, not a set of new capabilities. It reuses three capabilities you'd apply to any cloud spend — Usage Optimization, Governance, and Architecting & Workload Placement — and applies them across a wider AI Scope that explicitly tolerates more waste than a traditional compute estate would.

New Personas at the Table

The classic FinOps room was Engineering, Finance, and Procurement. AI spend adds two more, and both change what "the bill" even measures:

Exercise: Read a Real LLM Bill and a Real GPU Bill, Side by Side

Same skill from FinOps 101 — flag the top drivers, tag each to a persona — applied to the two bill shapes that are new to AI spend. Work through both, then reveal each analysis.

Quick Check: Which Technology Category Is This?

Exercise: This Week's Deliverable

Reflection

Putting it together

AI spend didn't break FinOps — it broke the assumption that every cloud bill behaves like general compute. Tokens price input and output asymmetrically and reward caching and batching; GPUs price commitment differently across providers and tolerate a different utilization curve; and the room now includes Data Science and Product alongside Engineering, Finance, and Procurement. None of that requires new capabilities, just Usage Optimization, Governance, and Architecting & Workload Placement, aimed at a wider scope. Week 2 starts applying that scope for real: ingesting the billing data itself, and catching where the usage API and the invoice quietly disagree.