The org chart and the bill are the same diagram.
Through Stages 1–7 you learned to create things: accounts, identities, compute, networks, storage, resilient failover. Every one of those things has two properties you have not yet been taught to design — who is allowed to touch it, and who pays for it. At small scale you can hold both in your head. At scale you cannot, so you encode them in structure.
This stage is about that structure. It is the shortest stage to describe and the hardest to get right, because the failure mode is silent: nothing breaks, nothing pages you, and then an invoice arrives.
Multi-account (AWS) / multi-subscription (Azure) / multi-project (GCP) structure is not paperwork. It is simultaneously the blast-radius boundary and the budget boundary — one line on the diagram doing two jobs.
One hierarchy, two readings
Read the tree below left-to-right and it is a governance diagram: policy attaches at a level and flows down, capping what anyone beneath can do. Read it right-to-left and it is a cost diagram: charges are born at the leaves and roll up, aggregating into whatever the level above represents. Same tree. The reason architects care about hierarchy design is that these two readings have to agree — if your security boundaries and your cost-ownership boundaries disagree, you will be unable to answer either "who can do this?" or "who pays for this?" without a spreadsheet and a meeting.
The surprise bill, and where it comes from
Ungoverned sprawl
The surprise bill almost never comes from one dramatic mistake. It comes from an accumulation of resources that are untagged (so no report shows them), unowned (so no human feels responsible), and unmonitored (so nothing fires when they grow). Each is individually forgivable. Together they compound into a line item nobody can explain.
The three properties are the same three properties, negated, that this stage teaches you to enforce structurally rather than by asking people nicely.
Why it is an architecture problem
You cannot solve sprawl with discipline, because discipline does not survive headcount growth or a 2 a.m. incident. You solve it the way you solve any other reliability problem: with boundaries that hold by default, controls that fail closed, and telemetry that fires before the damage is large.
That reframing is the whole stage. Governance is reliability engineering pointed at the invoice.
What you will be able to do at the end
- Design a multi-account / subscription / project structure that serves as both a governance and a cost boundary, and defend the number of units you chose.
- Apply preventive, detective and proactive guardrails, and say which one a given risk deserves.
- Write a tagging / labelling strategy that survives contact with reality, including the resources that cannot be tagged.
- Do the commitment-discount arithmetic — break-even utilisation, blended spot cost with a real interruption rate, right-sizing savings — especially for GPUs.
- Translate AWS Organizations / Control Tower / Savings Plans into their Azure and GCP equivalents in both directions.
- Map your own LLM cost ladder onto the FinOps framework and name which lever is which.
How the four sessions build
S1 is deliberately vendor-free. Every concept in S2, S3 and S4 is an implementation of something S1 already named, so if a vendor service ever confuses you, the move is to ask "which S1 concept is this?" rather than to memorise another product page.
What this stage assumes you already have
The four sources, and where I corrected the plan
Three corrections and one significant staleness warning:
section verification and corrections
AWS §14 teaches SCPs as the only authorization policy. AWS has since shipped two more organization policy types: resource control policies (RCPs) and declarative policies, both GA since late 2024. §14 also teaches guardrails as preventive-or-detective; Control Tower now has a third kind, proactive. Details in S2.
Azure §10 omits Azure savings plan for compute entirely, teaching only Reservations. The savings plan is the flexible sibling and changes the decision. Details in S3.
The session plan's "Blueprints" is retiring. Azure Blueprints (Preview) was deprecated on 11 July 2026 with phased retirement beginning 31 July 2026 and full retirement 31 January 2027. I teach the replacement pair instead. Details in S3.
Both AWS and GCP headline discount numbers in the materials are rounded marketing figures and the underlying mechanics have moved — GCP in particular migrated its spend-based commitment billing model in January 2026. Every rate in S2–S4 is re-verified with a source link.
In a GenAI infrastructure bill, accelerator hours are not a line item, they are the line item — commonly 70–90% of the total, with everything else rounding error by comparison. That has two consequences that shape all four sessions.
First, the commitment decision matters more than every other optimisation combined: a wrong on-demand-versus-committed call on a serving fleet costs more than every untagged S3 bucket in the org. Second, GPU capacity is the one resource where the discount mechanisms behave differently from general compute — sustained-use discounts silently do not apply to accelerator-optimised machine families on GCP, spot interruption rates for H100s differ from A100s by an order of magnitude, and capacity, not price, is often the binding constraint. Each session flags where your workload class diverges from the generic case.
The industry caught up recently: the FinOps Foundation's State of FinOps 2026 found 98% of practitioners now actively manage AI spend, up from 63% in 2025 and 31% in 2024 — the fastest adoption curve in the survey's six-year history. data.finops.org
S1 builds the vocabulary and the arithmetic with no vendor names attached, so that S2–S4 are three translations rather than three curricula.
Governance & FinOps concepts
Six ideas, no product names. Hierarchy as a control boundary. Preventive versus detective guardrails. Tagging strategy. Showback versus chargeback. The commitment-discount curve. The FinOps loop.
Your patterns catalog — the patterns catalog's the cloud application patterns catalog — has no governance or cost section, so nothing in this session comes from it. It is
Nobody decides to overspend
Here is the shape of essentially every cloud cost incident. A resource gets created for a good reason. The reason expires. The resource does not. Because nothing labelled it, no report shows it; because no report shows it, no human owns it; because no human owns it, nobody deletes it. Repeat a few hundred times across a few dozen engineers over a few years.
The important thing about that story is that no step in it is a mistake. There is no bad actor and no incompetent engineer. It is an emergent property of a system with no structural memory. That is why "be more careful" is not a fix and why this is an architecture topic rather than a management topic.
Six ideas, three passes each
A · Hierarchy as a control boundary
Think about an apartment building. Every unit has its own front-door lock, so a burglary in 4B does not reach 7C. Every unit also has its own electricity meter, so 4B's habit of running space heaters does not show up on 7C's bill. Those two facts are independent design decisions in a real building — you could imagine a building with individual locks and one shared meter — but in the cloud the same wall does both jobs. When you draw a boundary around a set of resources, you have simultaneously drawn a security perimeter and a billing perimeter, whether you meant to or not.
That is why "how many accounts should we have?" is a real architecture question and not an administrative one. You are deciding where the walls go.
Every major cloud gives you the same four-layer tree, with different words. A root at the top representing the whole company. A grouping layer beneath it for organising by environment, business unit, or compliance regime. An isolation unit — the thing that is genuinely a hard boundary. And resources at the leaves.
Two rules make the tree a control surface rather than just a filing cabinet:
- Policy inherits downward and intersects. A restriction placed at a level applies to everything beneath it, and cannot be relaxed by an administrator lower down. Permissions are the intersection of every level's allowance, never the union.
- Cost aggregates upward. Charges occur at the leaves and sum into every ancestor, so each level is automatically a reportable, budgetable unit without you configuring anything.
The second rule is why splitting into more isolation units improves cost visibility for free — a separate project is a separate cost report line. The first rule is why splitting also improves security for free. That coupling is the single most useful thing in this session.
If splitting is free value, why not give every workload its own isolation unit? Because splitting has costs that are invisible on day one and painful on day four hundred:
What splitting costs you
Structural limits to design against
These are real ceilings, verified July 2026, and they shape how deep a tree can usefully go:
Sources: AWS Organizations quotas Azure CAF, management groups
The nuance worth internalising: depth is cheap, width is expensive. Adding a grouping level costs almost nothing and buys you a policy attach point. Adding isolation units costs plumbing forever. So the mature pattern is a shallow-but-deliberate grouping tree with a modest number of isolation units, not a deep tree or a wide one.
The counter-failure is real too. Microsoft's landing-zone guidance explicitly warns against duplicating your org chart into the hierarchy, because org charts reorganise every eighteen months and hierarchies do not. Group by what a policy would say about it — production versus sandbox, regulated versus not — because those categories are stable.
B · Preventive, detective, and proactive guardrails
Three ways to stop someone building an unsafe balcony. You can lock the door to the balcony so it can never be used — nothing bad can happen, and also nothing good can happen out there. You can put a camera on it and review the footage — everything stays possible, and you find out afterwards. Or you can require plan review before construction — the design gets rejected at drawing stage, before any concrete is poured.
Those are preventive, detective, and proactive controls. The word "guardrail" gets used loosely for all three, which is why it is worth pinning the definitions down now.
Preventive guardrails work by capping, not granting. This is the part that trips people up coming from C2's IAM. A guardrail never gives anyone a permission — it only removes permissions from the set that identity policy might otherwise allow. The effective permission is the intersection.
Here is what a preventive rule actually looks like as an artifact. This one is the canonical cost guardrail: it blocks launching anything outside two approved regions, which stops both accidental-region sprawl and the classic crypto-mining-in-an-unwatched-region incident.
// preventive guardrail: deny outside approved regions { "Effect": "Deny", "Action": "*", "Resource": "*", "Condition": { "StringNotEquals": { "RequestedRegion": ["us-east-1", "us-west-2"] } } }
And here is how a request is evaluated as it descends the hierarchy. Step through it:
An engineer runs "launch 8 × H100 in eu-west-1"
She has an identity role that grants full compute permissions. In a single-account world, that is the end of the analysis and the instance launches. In a governed hierarchy, her identity permission is only the last of several gates, and the earlier ones are ones she cannot see or change.
Root: is there an org-wide deny?
The organisation has a root-level rule denying every region except us-east-1 and us-west-2. eu-west-1 is not in that list. The intersection of "everything" and "not eu-west-1" already excludes this request.
At this point the outcome is decided. Nothing beneath the root can widen it back out — that is the meaning of inheritance. The remaining steps still run, but only to narrow further.
Grouping layer: what does the Production node allow?
Her account sits under a Production grouping node carrying a rule that denies deleting audit-log buckets and denies disabling the configuration recorder. Neither applies to a compute launch, so this level neither helps nor hurts. Most levels are like this — a policy set is a sparse filter, not a full permission list.
Isolation unit: any account-level cap?
The account has a rule limiting instance types to a published list, which for this team excludes 8-GPU nodes and permits single-GPU nodes only. Independently of the region problem, this request was going to fail here too.
Two independent denies is common and is a feature: guardrails are written by different people for different reasons and they compose without coordination.
Identity: does her role allow it?
Yes — her role grants full compute. This is the only gate that grants anything. Everything above only subtracts. If you remember one sentence from this concept: identity policy proposes, hierarchy policy disposes.
Denied — and the failure mode is the interesting part
She gets an access-denied error that, in most clouds, does not clearly say which level denied her or why. She has full compute permissions in her own view of the world, so the error looks like a bug.
This is the real operational cost of preventive guardrails, and the reason mature organisations pair every deny with a documented, discoverable reason and an exception path. A guardrail nobody can understand becomes a guardrail somebody eventually disables.
Preventive is wrong when…
…the rule has legitimate exceptions you cannot enumerate in advance. A deny with no exception path routes around itself: the team files a ticket, someone grants a carve-out under time pressure, and the carve-out is permanent. You now have a guardrail with a hole and false confidence.
Also wrong when the blast radius of a false positive exceeds the risk. A too-broad region deny applied during an incident can prevent failover.
Detective is wrong when…
…the damage is done by the time you detect it. Detection latency for cost is measured in hours to days, because billing data is itself delayed. A misconfigured autoscaling group on GPU nodes can spend a quarter's budget inside that window.
And detection without a wired-up action is theatre. An alert that lands in an unread inbox is not a control.
Proactive is wrong when…
…people can bypass the IaC path. Proactive controls hook the deployment pipeline, so a console click or a direct API call sails straight past them. They are excellent as a fast-feedback layer — the engineer learns at pull-request time instead of at deploy time — but they are not a boundary.
Treat them as developer experience, not as security.
Layer them by cost of being wrong. Prevent the small set of things that are never acceptable and have no legitimate exception — deleting audit logs, disabling the config recorder, leaving the approved regions, using the root credential. Proactively check the large set of things that are usually wrong but occasionally justified, so engineers get told at review time. Detect everything else, and wire each detection to an automated response, not just a notification.
The ratio in a healthy organisation is roughly a dozen preventive rules, dozens of proactive checks, and hundreds of detective ones. If you have hundreds of preventive rules, you have built a system nobody can deploy into.
C · Tagging and labelling strategy
Airline baggage tags. The bag itself does not know where it is going or who owns it; the tag carries that information, and every downstream system — sorting, routing, claims — reads the tag rather than inspecting the bag. Attach the tag at check-in and everything works. Attach it later and the bag is already in the wrong pile.
Cloud tags are exactly this, with one cruel difference: tags are not retroactive on cost data. Tagging a resource today does not attribute yesterday's spend. The bag that travelled untagged stays untagged in the record forever.
A working taxonomy needs surprisingly few keys. Four is usually right, and every one of them answers a question someone will actually ask:
| Key | Answers | Values | Why it earns its place |
|---|---|---|---|
| owner | Who do I ask before deleting this? | team-slug | A team, never a person. People leave; teams persist. This is the single highest-value tag. |
| env | What are the rules here? | prod | stage | dev | sandbox | Drives both guardrail strictness and the expected cost profile. Also the cleanup filter. |
| cost-center | Whose budget does this hit? | finance code | The one key finance cares about; keep values identical to the finance system's own codes. |
| workload | What is this for? | service or model name | Lets you compute unit economics — cost per model, per feature, per request. |
The enforcement chain is what turns a taxonomy into data. A definition alone gets you maybe 40% coverage. The chain looks like this:
Cardinality: why "owner: alice@" ruins your dashboard
Free-text tag values produce unbounded distinct values. Cost tools group by exact string match, so Platform, platform and platform-team become three rows that never reconcile. Worse, high-cardinality tags make reports slow and charts unreadable. Closed value lists, enforced at admission, are not bureaucracy — they are the difference between a dimension and noise. Roughly: if a key can take more than about fifty values, it is a description, not a dimension.
Untaggable spend, and what to do about it
Some charges have no resource to tag. Inter-zone data transfer is billed against the flow, not an instance. A shared NAT gateway serves twenty workloads. Support plans are a percentage of the whole bill. These are shared costs, and you have three honest options: leave them unallocated and report them as a visible "shared" line; split them proportionally to tagged spend; or split them evenly. Proportional splitting is the usual choice — but say which you did, because the three give very different per-team numbers and an unexplained method destroys trust in the whole report.
This connects directly to Stages 4–5: your NAT and egress line items are exactly the ones that resist tagging. Design the allocation rule for them at the same time you design the network.
Tag drift, and why audit-mode-first is the right rollout
Turning on deny-on-missing-tag in an existing estate breaks deployments on day one and gets your policy revoked by lunchtime. The rollout that works is: publish the taxonomy → add tags to IaC modules → run the policy in audit mode and publish the non-compliance list per team → give a deadline → flip to deny for new resources only. Existing untagged resources get remediated by their owners or bulk-tagged from a best-guess mapping, and you accept that their historical cost data is permanently unattributable.
D · Showback versus chargeback
Showback is an itemised receipt taped to the fridge in a shared house: here is what each of you used. Chargeback is actually taking the money out of each housemate's account. The information is identical; the consequences are not, and so the political temperature is not either.
Mechanically they are the same pipeline — tags and hierarchy produce attributed spend — and differ only in what happens at the end. Which is precisely why the technical work is identical and the organisational work is not.
The reason the distinction matters to an architect: chargeback demands accuracy that showback does not. Nobody disputes a receipt taped to a fridge. Everybody disputes a debit. The moment money moves, your shared-cost allocation method, your untaggable residual, and your tag coverage all become contested, and any weakness in the pipeline surfaces as an argument. So the sequencing rule is: get to high tag coverage and a defensible shared-cost method under showback first, and only then consider chargeback.
Chargeback creates real incentive — a team that pays for its own idle GPUs turns them off — but it also creates perverse ones. Teams start optimising for their line item rather than the whole: refusing to adopt a shared platform because the shared cost lands on them, or under-provisioning something the company needs. Showback creates awareness without those distortions, but awareness without consequence decays; the second monthly report gets read less than the first.
The pattern most organisations land on is showback everywhere, chargeback on the one or two line items large enough to change behaviour — which in a GenAI shop means accelerator hours and inference spend, and essentially nothing else.
E · The commitment-discount curve
Three ways to get to the gym. Pay at the door each visit — most expensive per visit, zero obligation, always available. Buy an annual membership — much cheaper per visit if you actually go, dead money if you stop. Or take whatever slot is left over at 5 a.m. when the gym is empty — nearly free, but they can turn you away at the door and you have to be able to leave immediately.
On-demand, committed, and spot. The whole pricing model is that trade: you are paid for reducing the provider's uncertainty. A commitment removes demand uncertainty; accepting interruption removes capacity-planning uncertainty. Both are worth money to them, so both are discounted for you.
Plotted, the curve looks like this. The vertical axis is effective price; the horizontal axis is how much certainty you have given up.
The curve is intuition. The decision is arithmetic, and it rests on one number: break-even utilisation.
The formula is embarrassingly simple. If a commitment costs you a fraction c of the on-demand price and you pay it whether or not you use it, then:
Three refinements that the simple formula hides, and that separate a good commitment decision from a bad one:
- Utilisation is measured against the commitment, not the machine. A spend-rate commitment of $10/hour is wasted if you only run $7/hour of eligible usage that hour — the other $3 evaporates. Unused commitment does not bank forward.
- Stranded commitment is the real failure. You commit to a GPU family, a better one launches nine months later, and you are now paying for hardware you do not want. This is why flexible commitments are worth their few points of discount for fast-moving workloads and rigid ones are for boring, stable ones.
- Ladder rather than lump. Buying your whole commitment on one day means it all expires on one day, at whatever prices exist then. Buying in monthly tranches gives you a rolling portfolio, continuous repricing, and the ability to shrink coverage gradually if demand falls.
Commitment coverage is the share of eligible spend that a commitment discount actually applies to. The industry benchmark most practitioners aim for is 80–85%, with the remainder left on-demand deliberately as a shock absorber for demand that turns out to be temporary. Median observed coverage sits well below that — around 55% on AWS — which is where most of the "we could save 20% tomorrow" claims come from.
Committing to 100% guarantees you will strand some of it, because no forecast is that good.
F · The FinOps loop
A thermostat. It measures the temperature, compares it to a target, acts, and then measures again. What makes it a thermostat rather than a switch is the again. Cost management done once is a cleanup project; done as a loop it is a practice.
Its engine is a three-phase loop that a given workload cycles through continuously and independently of other workloads.
Inform — make spend visible, attributed, and forecastable
Tagging, hierarchy, dashboards, allocation, showback, budgets, forecasts, anomaly detection. Nothing is optimised in this phase. The output is a shared, trusted picture of where money goes.
The failure mode is skipping it. Teams that jump straight to optimisation cut whatever is most visible rather than whatever is most wasteful, and cannot prove afterwards that they saved anything.
Optimize — reduce cost without reducing the value delivered
Four families of lever, in rough order of effort-to-payoff: eliminate (delete idle, unattached, orphaned things), right-size (match provisioned capacity to observed use), re-rate (buy commitments, move to spot, change storage class), and re-architect (scale to zero, batch, quantise, cache).
Note that only the first three are pure cost work; the fourth is engineering work with a cost benefit, and it is where the biggest wins live.
Operate — make it continuous and make it stick
Guardrails that prevent the waste class from recurring, automated cleanup, policy enforcement, KPIs, and unit-economics reporting. This is the phase that converts a one-off saving into a permanent floor.
It is also where governance and FinOps stop being two subjects. A preventive guardrail against untagged resources is the operate phase of the tagging problem.
Back to inform — with a changed baseline
The loop's value comes from repetition against a moving target. Your workload changed, prices changed, a new instance family launched, your commitment aged. Practitioners consistently report that the first pass finds large obvious waste and later passes find smaller, harder items — which is a sign of maturity, not of diminishing value.
The 2026 State of FinOps captured this directly: teams describe having already captured the "big rocks" and now facing a high volume of smaller opportunities requiring more effort per unit of saving.
The inform → optimize → operate loop is unchanged and remains the engine. Two structural additions are worth knowing because they change what a FinOps practice is for:
Scopes (added 2025, deepened 2026). A Scope is a defined segment of technology spend aligned to a business construct — a product line, an environment, an AI initiative. The framework now explicitly expects different Scopes to run at different cadences and tolerate different levels of waste. An innovation-focused AI Scope is supposed to tolerate more waste than a mature production Scope; applying uniform efficiency standards across both is now considered an anti-pattern.
Executive Strategy Alignment (new capability, 2026). Reflects the discipline's shift from "how do we spend less" to "how do we spend better" — connecting technology value to business strategy rather than reporting savings. Several capabilities were also renamed to drop the word "cloud", because the framework now covers SaaS, licensing, data centre and AI spend, not just public cloud.
Source: FinOps Foundation, 2026 FinOps Framework framework home
The framework describes maturity as crawl / walk / run, and the useful insight is that maturity is per-capability, not global. It is entirely normal — and correct — to be at "run" on commitment management because it is where your money is, and at "crawl" on sustainability reporting because nobody has asked. Trying to be uniformly mature is how FinOps programmes turn into overhead.
The second trade-off is where in the lifecycle you intervene. Acting after deployment (detect and clean up) is easy to implement and permanently reactive. Acting before deployment — "shift left", pricing an architecture at design review — is far more valuable and far harder, because it requires cost estimates for things that do not exist yet. The 2026 survey found pre-deployment architecture costing was the single most requested missing capability, with most teams building their own calculators. If you build one thing beyond the basics, build that.
The three-cloud Rosetta Stone
Every concept above, in three dialects. This is the table to come back to — S2, S3 and S4 each expand one column.
| Concept (S1 name) | AWS | Azure | GCP |
|---|---|---|---|
| Root | Organization (management account) | Tenant root management group | Organization node |
| Grouping layer | Organizational Unit (OU), ≤5 deep | Management group, ≤6 deep | Folder, ≤10 deep |
| Isolation unit | Account | Subscription | Project |
| Sub-grouping inside the unit | — (tags only) | Resource group (a real container) | — (labels only) |
| Preventive guardrail | Service control policy (SCP); resource control policy (RCP) | Azure Policy with Deny effect | Organization policy constraint |
| Detective guardrail | AWS Config rules; Security Hub | Azure Policy with Audit effect; Defender for Cloud | Security Command Center; Config Validator |
| Proactive guardrail | Control Tower proactive controls (CloudFormation hooks) | Policy evaluated at ARM deployment; deployment stacks | Policy Simulator / dry-run enforcement |
| Governed environment factory | Control Tower + Account Factory | Azure Landing Zone accelerator | Cloud Foundation Toolkit / project factory |
| Metadata for billing | Tags (must be activated as cost allocation tags) | Tags | Labels — GCP also has separate Tags for policy; different things |
| Cost reporting | Cost Explorer; Data Exports (FOCUS 1.2) | Cost Management cost analysis; exports | Billing reports; billing export to BigQuery |
| Budget + alert | AWS Budgets (can trigger actions) | Azure Budgets + action groups | Cloud Billing budgets — alert-only by default |
| Anomaly detection | Cost Anomaly Detection | Cost Management anomaly detection | Cost anomaly detection in billing reports |
| Right-sizing advice | Compute Optimizer; Trusted Advisor; Cost Optimization Hub | Azure Advisor | Recommender (Active Assist) |
| Rigid commitment | Standard / Convertible Reserved Instance | Reservation (Reserved VM Instance) | Resource-based committed use discount |
| Flexible spend commitment | Compute / EC2 Instance / Database Savings Plan | Azure savings plan for compute | Compute flexible (spend-based) CUD |
| Interruptible capacity | Spot Instances (2-min warning) | Spot Virtual Machines (configurable eviction) | Spot VMs (30-sec default notice) |
| No-commitment auto discount | — none | — none | Sustained use discounts — unique to GCP |
Every row re-verified against current vendor documentation in July 2026. The two rows worth memorising are the last one — GCP is alone in giving a discount for nothing — and the "metadata" row, where GCP's labels versus Tags distinction catches people who learned AWS first.
Do the arithmetic
Worked example 1 — what a tagging gap actually costs you
The claim "30% of our spend is unattributable" sounds like a reporting nuisance. Work it through and it is a decision-making failure. Take a mid-sized GenAI platform at $180,000/month. Tag coverage is 70% by spend.
Consequence one — you cannot compute unit economics. Suppose you want cost-per-1M-tokens for your flagship model. Its tagged spend is $28,000/month for 400M tokens.
Consequence two — the optimisation you cannot even attempt. Suppose 20% of the untagged spend is genuinely idle. You cannot find it, because finding it requires knowing what it is for, and that is precisely the information the tag would have carried.
Worked example 2 — break-even on a steady serving instance
Generic numbers, so the method transfers to any cloud. A model-serving node costs $4.00/hour on-demand. A 1-year commitment prices it at $2.40/hour (40% off); a 3-year at $1.40/hour (65% off). You believe you will run roughly one node continuously, but you are not certain the product survives.
Break-even utilisation for each, from the formula:
Now price the failure case honestly. Suppose the product is cancelled at month 14 — past both break-evens, but with 22 months of a 3-year commitment left.
Everything above assumes the hardware stays desirable for the term. For general compute that is a safe assumption — an m-series VM is much the same machine in year three. For accelerators it is not. A 3-year commitment bought on an A100 fleet in 2023 spent its final year competing against H100s at better performance-per-dollar, and a 3-year H100 commitment made today runs into successive generations before it expires.
So for GPUs, add a fourth term to the arithmetic: the risk that you keep the commitment and still want to migrate. That is the specific reason flexible spend-based commitments — which follow you across families — are worth their few points of shallower discount in accelerator-heavy estates, and why 1-year terms dominate 3-year ones here despite the worse headline rate.
Three grounded examples
AWS billed customers trillions — and its own alarms did not stop it
On 17 July 2026 a unit-pricing error in AWS's estimated-billing computation subsystem produced customer bill estimates in the billions and trillions of dollars for over 24 hours. AWS's own alarms detected the anomaly, but did not halt bill generation or page engineers; customer escalations reached the company roughly 4.5 hours later. During mitigation, budget and cost anomaly alerts were disabled platform-wide.
The lesson is exactly the S1 one: a detective control whose finding is not wired to an action is not a control. And note the second-order failure — the customers' own budget alerts went dark during the incident, which is why guardrails should never depend on a single vendor-side signal.
State of FinOps 2026: optimisation is no longer the whole job
The FinOps Foundation's sixth annual survey found waste reduction is still the single top current priority — but that governance, forecasting and scope expansion collectively now outweigh optimisation as forward-looking priorities. Practitioners describe having captured the "big rocks" and facing a long tail of smaller items.
The AI figure is the headline: 98% now manage AI spend, versus 63% a year earlier and 31% two years earlier. AI cost management is the most sought-after skill teams plan to hire for.
The Framework grew past "cloud"
The 2026 Framework revision renamed capabilities to drop "cloud" (Policy & Governance became Governance, Policy & Risk; Workload Optimization became Usage Optimization), added Executive Strategy Alignment as a new capability, and published Technology Category pages spanning public cloud, SaaS, data centre, data platforms and AI.
Why it matters to you: the framework now explicitly says an AI innovation Scope should tolerate higher waste and run at a faster cadence than a mature production Scope. If you are experimenting with models, that is official permission not to optimise the sandbox.
Your context, and one hands-on lab
Your cost ladder is already a FinOps artifact — here is its real name
You have documented a ladder from roughly $324/month baseline down to ~$45/month with optimisations, plus per-request and per-model cost tracking in your LLM stack. That is not "some cost work you did". Mapped onto the framework, it is a complete inform → optimize cycle with a partially-built operate phase. Here is the mapping, because naming the levers correctly is what lets you reason about which one to pull next:
| Your lever | FinOps phase | Optimisation family | What generalises |
|---|---|---|---|
| Per-request and per-model cost tracking | Inform | Allocation / unit economics | This is the thing most organisations never build. You have cost-per-unit-of-value, which the 2026 survey lists as the capability teams are climbing toward. |
| Quantisation | Optimize | Re-architect | Reduces the resource required per unit of work. The highest-leverage family and the one no FinOps tool can recommend for you — it needs domain knowledge. |
| Batching | Optimize | Re-architect (utilisation) | Raises work-per-GPU-hour. Economically identical to right-sizing: you are closing the gap between provisioned and used capacity. |
| Spot instances (your Vertex config) | Optimize | Re-rate | Same resource, cheaper price, in exchange for interruption tolerance. Pure movement along the commitment curve. |
| Scale-to-zero (your Modal setup) | Optimize + Operate | Eliminate, automated | The interesting one — see below. |
| Autoscaling 1–3 (your Vertex config) | Operate | Right-size, continuously | Right-sizing that re-runs itself. This is what "operate" means in practice. |
Your Modal deployment scales to zero when idle. Most people file that under "saves money", which undersells it. Look at it through this session's lens instead: scale-to-zero makes idle spend structurally impossible for that workload. It is not a detective control that finds idle capacity, nor a preventive control that forbids creating it — it is an architecture in which the failure mode cannot occur.
That is the highest tier of the operate phase, and it is why the FinOps framework's Architecting & Workload Placement capability sits above the optimisation ones. Every other lever in this stage is a control layered on top of a system that can waste money. Scale-to-zero removes the capability to waste it.
The trade-off, which you will already have felt: cold-start latency. That is the real currency you paid. When you get to S2–S4, notice that the same trade shows up as committed capacity versus on-demand — reserved GPUs are, precisely, the decision to pay for warmth.
The one gap in your ladder
Your levers are strong on optimize and reasonable on operate, but your inform phase is workload-scoped rather than estate-scoped. You know what each request and model costs. You do not yet have the thing that catches a resource you forgot you created — because per-request tracking only sees traffic that flows through your application, and the surprise bill by definition does not.
That is precisely what a budget alert plus an anomaly detector plus a tag policy provide, and it is why the lab below is the one worth doing before any of the vendor sessions.
Time: ~40 minutes. Cloud: whichever you already have credentials for; GCP is the natural choice given your Vertex work. Nothing here creates a billable resource — budgets, alerts, and tag policies are free to configure on all three clouds.
- Write the taxonomy first, on paper. Four keys, closed value lists, following the table in Pass 2 above. Include an explicit answer for your untaggable spend: which allocation rule will you use for shared network egress? Write it down now, because you will not want to invent it under pressure later.
- Set a budget at the isolation-unit level — the project / subscription / account holding your GPU work — with thresholds at 50%, 80% and 100%. Set the amount deliberately below what you can afford, to absorb billing-data delay.
- Add a forecast-based threshold as well as an actual-based one. Actual-based alerts tell you what happened; forecast-based alerts fire while you can still act. For a GPU workload, the forecast alert is the one that matters, because a misconfigured autoscaler can exhaust a month's budget in a day.
- Turn on anomaly detection for the same scope. It is a different signal from a budget: budgets catch too much in total, anomaly detection catches unusual shape, and a GPU cost incident usually shows up in shape first.
- Route one alert somewhere you will actually see it — a Slack channel or your phone, not an email alias. Then re-read the AWS billing-incident example above and ask what your alert would do, not just say.
- Write the tag policy in audit mode only. Do not enable deny. Look at the non-compliance list it produces against your existing resources — that list is your actual current tag coverage, and it is usually a surprise.
What good looks like: you can answer "what did my GPU work cost last month, broken down by workload?" from a dashboard rather than from reasoning, and you would find out within a day if it doubled.
Every concept above now gets a product name. AWS's version is the most elaborate of the three — it is the only cloud with three distinct organization policy types and four distinct commitment vehicles — which is why it comes first and why your AWS source is the richest source in the stage.
Watch for two things in particular: how Control Tower turns the S1 hierarchy into an automated factory, and how Savings Plans versus Reserved Instances is the flexible-versus-rigid commitment trade-off from the curve above, with real GPU numbers attached.
AWS governance & cost
Organizations and OUs. Three kinds of organization policy — the material teaches one. Control Tower landing zones. Cost Explorer, Budgets, Anomaly Detection, Cost Optimization Hub. Savings Plans versus Reserved Instances versus Spot, with GPU arithmetic.
This is the deepest governance material in your four sources, and the most out of date. §14 was written when SCPs were the only organization-level authorization policy; AWS has shipped two more policy types since, plus a third guardrail category. Every stale claim is flagged with the current fact and a source.
AWS made the account the atom, and everything follows
AWS's original design has exactly one hard boundary: the account. There is no sub-container inside it with real teeth — no equivalent of an Azure resource group as a policy target. That single design decision cascades into everything in this session. Because the account is the only real wall, AWS customers end up with many accounts, which means AWS needed a way to manage many accounts, which is Organizations; then a way to bootstrap them consistently, which is Control Tower; then a way to share resources back across the walls they just built, which is Resource Access Manager; then a way to see one bill across all of it, which is consolidated billing.
Read that chain backwards and the entire AWS governance product line is a consequence of one architectural choice. That is a more useful mental model than five separate service descriptions.
The AWS dialect
A · Organizations, OUs, and consolidated billing
Organizations is a free service that puts many AWS accounts under one umbrella so you can apply rules to groups of them and get one bill. That is genuinely the whole idea. The complexity is entirely in what rules and which groups.
The OU structure AWS itself recommends — and that Control Tower creates for you — is deliberately not an org chart:
Hard numbers, verified July 2026
Traps worth knowing before you build
B · Three policy types — the material teaches one
§14.2 introduces SCPs as the mechanism for centrally limiting permissions, and §14.6 contrasts SCPs with IAM policies as if that were the complete picture. It was, when written. AWS has since added two more organization policy types, both generally available since December 2024: resource control policies (RCPs) and declarative policies. They close two gaps that SCPs structurally cannot.
Sources: Declarative policies — AWS Organizations Organizations FAQ, policy types
Three different questions, three different policy types.
An SCP answers "what may our people do?" — it constrains the principals inside your accounts. But your S3 bucket can also be accessed by someone else's principal from outside your org entirely, and an SCP has nothing to say about that, because that person is not your people. So an RCP answers the mirror question: "what may be done to our resources, by anyone?"
Both are about permission. Neither can express "all our VPCs should have public access blocked", because that is not a permission, it is a configuration. A deny on the API that enables public access is brittle: AWS adds a new API next year and your policy does not cover it. So a declarative policy answers a third question: "what configuration should this service always be in, regardless of what APIs exist?"
| Policy type | Constrains | Enforced at | Survives new AWS APIs? | Canonical use |
|---|---|---|---|---|
| SCP service control policy | Principals in your accounts | Authorization (IAM evaluation) | Only if written with wildcards | Deny regions, deny root user, deny disabling logging |
| RCP resource control policy · 2024 | Anyone acting on your resources, including external principals | Authorization (resource-side) | Only if written with wildcards | Block cross-org data access; require TLS on every bucket in the org |
| Declarative policy 2024 | Service configuration, not permissions | The service's own control plane | Yes — that is the point | Block VPC public internet access org-wide; restrict AMI providers |
| Tag policy | Tag keys and value casing | Reporting + optional enforcement | n/a | Standardise your four-key taxonomy from S1 |
| Backup policy | Backup plans | AWS Backup | n/a | Org-wide retention — the C7 tie-in |
The declarative-policy distinction is the subtle and important one. Authorization policies regulate access to APIs. Declarative policies are applied at the service's control plane to enforce durable intent — so when AWS ships a new way to make a VPC public next year, the declarative policy already covers it, whereas your SCP denying ec2:CreateInternetGateway does not.
Here is the cost-relevant SCP everyone should have, and the reason it is a cost control and not just a security one:
// SCP: only these instance families may be launched. // A GPU typo — p5.48xlarge instead of g6.xlarge — is a $55/hour mistake. { "Effect": "Deny", "Action": "ec2:RunInstances", "Resource": "arn:aws:ec2:*:*:instance/*", "Condition": { "ForAnyValue:StringNotLike": { "ec2:InstanceType": ["g6.*", "g5.*", "t3.*", "m7i.*"] } } }
Deny lists versus allow lists — and why almost everyone should use deny lists
AWS attaches a policy called FullAWSAccess to every node by default, which allows everything. Since permissions are the intersection of all levels, you can govern in two styles. A deny list keeps FullAWSAccess and adds explicit denies for the handful of things you forbid. An allow list removes FullAWSAccess and enumerates every service anyone may use.
Allow lists sound more secure and are usually a mistake, because AWS has hundreds of services and adds more constantly, so you are signing up to maintain an enumeration forever — and every gap presents as a confusing access-denied to an engineer who has the IAM permission. Use deny lists for the org, and reserve allow lists for genuinely locked-down OUs like a regulated-data enclave.
SCPs don't affect service-linked roles — and this is load-bearing
Service-linked roles are exempt from SCPs. That is deliberate: without the exemption, a broad deny would break Auto Scaling, Elastic Load Balancing and other services that act on your behalf, in ways that are extremely hard to diagnose. It also means an SCP is not a complete control against everything happening in the account, and you should not reason about it as one.
The 10,240-character ceiling, and how people hit it
SCPs are small. Even after May 2026's doubling, a comprehensive deny list runs out of room fast — condition blocks with long region or instance-type lists are the usual culprit. Three techniques: collapse statements that share an Effect and Resource into one with a list of actions; drop every Sid (they count); and split across the hierarchy, putting the truly universal denies at the root and specific ones on OUs, since inherited SCPs don't count against the child's own limit of ten.
Testing SCPs safely
There is no dry-run mode for SCPs. The safe procedure is: create a throwaway OU, move one non-critical account into it, attach the policy, verify the intended things break and nothing else does, then roll outward. IAM Access Analyzer's policy validation catches syntax and some logic errors but cannot tell you what will break operationally. Have the "move the account back out of the OU" command ready before you attach.
C · Control Tower and the landing zone
If Organizations is the filing system, Control Tower is the person who sets it up correctly and keeps checking it. You could build everything it builds by hand — an OU structure, a log-archive account, an audit account, centralised CloudTrail, a set of SCPs and Config rules, a way to vend new accounts. Control Tower does it in an afternoon and then continuously verifies it has not drifted.
§14.10.1 lists guardrail types as "Preventive vs Detective". There are now three. Control Tower implements preventive, detective, and proactive controls, exactly matching the three-way distinction from S1.
About controls in AWS Control Tower AWS re:Post, three control types
Controls also carry a guidance level, which is the part that matters when you are deciding what to turn on: mandatory (always on, cannot be disabled), strongly recommended (AWS best practice, opt-in), and elective (useful for some organisations only). A sensible starting posture is all mandatory plus all strongly recommended on Prod, and mandatory only on Sandbox.
Control Tower is opinionated, and its opinions cost you flexibility. It expects to own the OU structure it creates; it enables Config in every governed account and Region, which is not free and is a real line item on a large estate; and its landing-zone updates occasionally require a coordinated re-enrolment across accounts.
Reach for raw Organizations plus your own IaC instead when you have a small number of accounts and strong Terraform practice, when you need an OU structure that fights Control Tower's model, or when Config's per-item recording cost is material relative to your total spend. Reach for Control Tower when you are vending accounts regularly, when you need auditable evidence that controls are applied, or when nobody on the team wants to own landing-zone drift.
D · Sharing back across the walls — Resource Access Manager
Your §16 covers this at length. The governance-and-cost framing that the section under-emphasises: RAM is how you avoid paying for the same thing N times after you split into N accounts. The canonical case is a shared VPC — one networking account owns the subnets, NAT gateways and Transit Gateway attachments, and shares subnets to workload accounts. Twenty accounts then use one NAT gateway instead of twenty, which given NAT's hourly-plus-per-GB pricing from C4 is often the single largest structural saving in a multi-account migration.
The cost catch, and it is a good exam-style trap: with a shared VPC, the subnet owner pays for the NAT gateway and data processing, while the participant account pays for its own instances. So your networking account develops a large, growing bill that belongs to nobody in particular — the untaggable shared-cost problem from S1, in its most common concrete form. Decide the allocation rule for it when you design the sharing, not when finance asks.
E · The cost toolchain
AWS has a lot of cost tools and their names do not tell you how they relate. They sort cleanly into the S1 FinOps loop: some tell you what happened, some tell you what is about to happen, some tell you what to do about it.
INFORM — what happened
DETECT — what is about to happen
OPTIMIZE — what to do
The Budgets / Cost Explorer / Trusted Advisor triad in your §17 is still accurate but is now the older half of the toolchain. The 2025-era additions above — Cost Optimization Hub, the Efficiency Score, managed anomaly monitors, FOCUS 1.2 — postdate the material. AWS CFM, re:Invent 2025 launches
Budget versus forecast versus anomaly
Three different questions, and teams commonly configure one and think they are covered.
Blended versus unblended versus amortised
Three columns in your cost data that answer three different questions. Choosing wrong makes savings appear or vanish.
F · Purchasing options — the commitment curve, AWS edition
AWS now has four commitment vehicles plus spot. Your §18 covers three of them.
| Vehicle | You commit to | Max discount | Flexibility | Best for |
|---|---|---|---|---|
| Standard RI | Instance family, size, region, OS, tenancy | ~72% | Lowest — can be sold on the RI Marketplace but not exchanged | Genuinely static infrastructure |
| Convertible RI | Same, but exchangeable for a different config | ~66% | Manual exchange process | Largely superseded by Savings Plans |
| EC2 Instance Savings Plan | $/hour within one instance family in one region | ~72% | Change size, OS and tenancy freely; family and region locked | Stable family, changing sizes |
| Compute Savings Plan | $/hour, any family, any region | ~66% | Highest — also covers Fargate and Lambda | The default for most teams |
| Database Savings Plan new · re:Invent 2025 | $/hour across database services | ~35% | Move between RDS, Aurora, ElastiCache, DynamoDB, Neptune, DocumentDB without restarting the commitment | Databases — finally |
| Spot | Nothing | up to ~90% | Total — and no guarantee | Interruptible: training, batch, HPO |
Discount ceilings are 3-year, all-upfront figures and vary by instance and region. The Compute-vs-EC2-Instance gap is about 6 percentage points — the price of total flexibility. 2026 comparison, sourced to AWS pricing AWS Savings Plans
Database Savings Plans launched December 2025, at up to 35% for a 1-year no-upfront commitment, covering RDS, Aurora, ElastiCache, Neptune, DocumentDB, DynamoDB and Timestream. Before this, database commitments were rigid reservations per engine; now the commitment follows you if you move from ElastiCache to DynamoDB.
Also new: RI and Savings Plan sharing preferences, letting you prioritise or restrict which accounts receive shared discount benefit — which fixes a long-standing annoyance where a sandbox account would consume commitment intended for production.
Layering. These stack without conflict, and AWS applies them in a fixed order — RIs first, then Savings Plans, with the highest-savings usage covered first within each tier. So the practical strategy is a base layer of Compute Savings Plans covering your steady floor, RIs only where the extra points justify the rigidity, and spot on everything interruptible. You cannot double-discount an instance-hour, and you cannot accidentally conflict them.
Savings Plans returns are narrowly bounded. Only plans with an hourly commitment of up to $100/hour, purchased within the last 7 days and in the same calendar month, can be returned — with a maximum of 10 returns per year. Above $100/hour there is no return at all.
At GPU scale you cross $100/hour quickly: a single 8×H100 node at on-demand list is $55/hour, so a Savings Plan sized for two such nodes is already effectively permanent. Treat any commitment above that threshold as irreversible and size it to your floor, never your average.
Spot mechanics you need for the math below. AWS gives a 2-minute interruption warning via instance metadata. Spot Instance Advisor publishes trailing 30-day interruption frequency by instance type and region — check it before choosing a family, because the variation is enormous. Current published figures: H100 spot on p5.48xlarge runs under 5% interruption, while A100 spot on p4d.24xlarge runs 15–20%. That is not a small difference; it changes which one is cheaper in practice.
Cloud GPU spot interruption rates, 2026 · check live figures in the AWS Spot Instance Advisor before committing a fleet design.
Three-cloud comparison — the AWS things you just learned, translated
Organizations + OUs
AWSFree account grouping with policy inheritance and consolidated billing. Account = hard boundary; OU = policy attach point.
QuirkNo real container inside the account — tags do all sub-account grouping.
Management groups
AzureSame shape, one extra real layer: resource groups inside a subscription are genuine containers you can target with policy and RBAC.
QuirkThat extra layer means Azure needs fewer subscriptions than AWS needs accounts.
Folders + projects
GCPSame shape. Folders nest deeper (10).
QuirkBilling is a separate axis — a billing account is linked to projects independently of the folder tree, so the governance and cost hierarchies can genuinely diverge.
SCP / RCP / declarative policy
AWSThree distinct policy types: constrain your principals, constrain access to your resources, and pin service configuration.
QuirkThe only cloud that separates these three. Most granular, most to learn.
Azure Policy
AzureOne engine, many effects. Deny ≈ SCP, Audit ≈ Config rule, DeployIfNotExists and Modify auto-remediate.
QuirkCan fix things, not just block them — no AWS equivalent.
Organization policy constraints
GCPBoolean or list-based constraints on resource configuration, inherited down the hierarchy. Custom constraints available.
QuirkCloser to AWS declarative policies than to SCPs — constrains configuration, not principals.
Control Tower + Account Factory
AWSManaged landing zone with mandatory / strongly-recommended / elective controls in three enforcement types.
QuirkOpinionated about your OU structure and turns on Config everywhere, which costs money.
Azure Landing Zone accelerator
AzureBicep/Terraform reference implementation of the CAF architecture — management-group tree, ~100 policy assignments, hub-spoke network.
QuirkIaC you own and modify, not a managed service. More flexible, more yours to maintain.
Cloud Foundation Toolkit
GCPTerraform blueprints for org structure, projects, networking and policy.
QuirkLeast managed of the three. Google assumes you have Terraform practice.
Savings Plans + RIs + Spot
AWSFour commitment vehicles plus spot, layered automatically in a fixed order, highest-saving usage first.
QuirkMost vehicles of any cloud, and no automatic no-commitment discount.
Reservations + savings plan
AzureReservations up to ~72%, savings plan for compute up to ~65%. Plus Azure Hybrid Benefit for Windows/SQL licences.
QuirkHybrid Benefit stacks on top, pushing total discount past 80% on eligible Windows workloads.
CUDs + SUDs + Spot
GCPResource-based CUDs (~55–70%), flexible CUDs (28% / 46%), Spot (60–91%) — and sustained-use discounts requiring no commitment.
QuirkThe only automatic no-commitment discount in the industry. Sharp eligibility edges — see S4.
GPU arithmetic on AWS
p5.48xlarge (8 × H100 80GB): on-demand $55.04/hour, which is $6.88 per GPU-hour. AWS cut P5 pricing by 44% in June 2025; older sources quoting $98.32/hour are pre-cut. Spot for the same instance has recently traded around $20.78–$30.64/hour. Always re-check the live rate — this is the fastest-moving number in the stage. EC2 on-demand pricing
Worked example 1 — commitment break-even on a steady serving fleet
You serve a quantised model continuously and need two p5.48xlarge nodes up 24/7. Discounts applied: Compute Savings Plan at 1 year ≈ 40% and at 3 years ≈ 66%.
Now the break-even, and then the risk:
A Savings Plan covering two p5 nodes commits roughly $33–66/hour. That is under the $100/hour return threshold, so you would have a 7-day, same-calendar-month window to unwind — but a four-node commitment crosses $100/hour and becomes permanently unreturnable. Sizing your commitment just below that line, and adding a second plan later, buys you a real option for free.
Worked example 2 — the blended cost of a spot GPU training fleet
This is the calculation people get wrong, because they compare the spot price to the on-demand price and stop. The real question is the cost per unit of completed work, and interruptions destroy work.
Setup. A fine-tuning run needing 400 GPU-hours of useful compute on H100s. You checkpoint every 30 minutes. On p5.48xlarge (8 GPUs) that is 50 instance-hours of useful work. Spot at $24.00/hour for the instance versus on-demand at $55.04.
Take the published H100 p5 interruption rate of under 5% — use 4% per instance-hour as a working figure, and re-check the Spot Instance Advisor for your region before relying on it.
Now change one variable and watch the conclusion invert. Same job on A100s (p4d.24xlarge), where the published interruption rate is 15–20% — use 18% — and suppose the team checkpoints every 2 hours instead of every 30 minutes:
Checkpoint interval is a pricing decision. Halving it halves your expected lost work, which is a direct, computable dollar saving. The optimal interval is where the marginal cost of checkpointing (write time, storage, throughput hit) equals the marginal expected loss — and for expensive GPUs on volatile spot pools, that interval is much shorter than most teams set.
Interruption rate is an instance-selection criterion, not a footnote. The p5-versus-p4d difference here is roughly 4% against 18%, and it flips which family is cheaper per unit of completed work regardless of the sticker price. Check the Spot Instance Advisor before choosing a family, the same way you check memory and interconnect.
The right architecture is a blend. Steady serving on a Compute Savings Plan, training and evaluation on diversified spot, and a small on-demand buffer for the deadline-critical run. Those three sit at three different points on the S1 curve, deliberately.
Three grounded examples
The loudest cheer at re:Invent was for a discount
After two hours of LLM, chip and agent announcements, the biggest audience reaction at Matt Garman's keynote was for Database Savings Plans — a pricing change, delivered in the closing lightning round. The FinOps community had asked for it for roughly six years.
Why it belongs in a governance session: it is evidence that commitment flexibility, not commitment depth, is the constraint practitioners actually feel. The discount is only ~35%, well below RI rates — and people cheered anyway, because being able to migrate engines without restarting the commitment is worth more than the extra points.
Half of eligible compute still runs at full price
Industry data puts commitment adoption at 64% of organisations using some form of commitment pricing, but with a median coverage rate of only 55% against a recommended target of 80–85%. Nearly half of eligible compute spend runs at on-demand rates.
The useful reading: the gap is not caused by ignorance of Savings Plans. It is caused by forecast uncertainty — teams cannot convince themselves the workload will persist, so they default to on-demand. That makes it a forecasting problem wearing a pricing costume, which is exactly why the FinOps loop puts inform before optimize.
Why AWS added two policy types
RCPs exist because SCPs have a structural blind spot: they constrain your principals, so an externally-owned principal accessing your S3 bucket is entirely outside their reach. RCPs close it by evaluating on the resource side, org-wide, without editing thousands of individual resource policies. AWS raised the RCP quota to 2,000 per organization in July 2026 — a signal of real adoption at scale.
Declarative policies solve the other gap: durability. A configuration pinned at the service control plane survives AWS shipping new APIs, which an action-based deny never does.
Your context, and one hands-on lab
Translating your Vertex spot configuration to AWS
Your Vertex AI deployment script uses T4 GPUs, autoscaling 1–3, with spot instances. That configuration encodes three decisions that map exactly onto this session's material, and the AWS equivalents differ in ways worth knowing:
| Your Vertex decision | AWS equivalent | What differs |
|---|---|---|
| Spot instances for GPU nodes | EC2 Spot in an Auto Scaling group with a mixed-instances policy | AWS gives 2 minutes of warning; GCP gives 30 seconds by default. That single difference means an AWS spot handler can do meaningfully more work before dying — flush a checkpoint, drain connections — where GCP's forces you to pre-stage everything. |
| Autoscaling 1–3 | ASG min=1, max=3, target tracking on GPU utilisation | A floor of 1 means you never scale to zero, so that one node is a permanent, predictable cost — and therefore the ideal Savings Plan candidate. Commit to the floor, run the peak on-demand or spot. |
| T4 (cost-optimised inference GPU) | g4dn (T4) or g6 (L4) — g6 is the current generation | Google is retiring T4 instances; AWS's equivalent modern step is g6/L4. If you rebuild, go to the newer family — better performance per dollar, and a commitment on a current generation strands less. |
Your autoscaling floor is a commitment opportunity and your ceiling is not. An ASG with min=1 guarantees one instance runs 8,760 hours a year. That is not a forecast — it is a configuration you control. Committing to exactly your floor has near-zero stranding risk, because the only way to strand it is to delete the service.
Meanwhile everything between the floor and the ceiling is by definition variable and belongs on-demand or on spot. This gives you a clean rule that needs no forecasting skill at all: commit the floor, never the peak. Applied across a fleet of autoscaled services, it typically lands you close to the 80% coverage benchmark automatically.
Where Modal's scale-to-zero has no AWS equivalent — and what to use instead
Your Modal setup scales GPU capacity to zero when idle. On raw EC2 that is not achievable for GPU serving: an ASG with min=0 gives you cold starts measured in minutes, because the instance must boot, pull a multi-gigabyte container, and load weights onto the GPU. The AWS-native options that get closest are SageMaker Serverless Inference (real scale-to-zero, but with cold-start latency and model-size limits) and SageMaker asynchronous inference (queues requests, scales to zero between batches — an excellent fit for non-interactive work). For a Kubernetes path that matches your KServe experience, EKS with Karpenter plus KEDA scale-to-zero on a spot GPU node pool is the closest architectural match, and you keep the KServe abstractions you already know.
Time: ~35 minutes. Everything here is free to configure: Organizations, Budgets, Cost Anomaly Detection and tag policies carry no charge. The lab deliberately provisions nothing — the point is to build the control plane before you have anything expensive to control.
Why this lab and not "set up Control Tower": Control Tower enables AWS Config across every governed account and Region, which is genuinely billable and can surprise you on a personal account. Save it for an employer's estate.
- Create a cost budget scoped to your GPU work — filtered by a tag if you have one, or by the EC2 service otherwise. Set the amount to something that would genuinely alarm you.
- Add two alert thresholds: 80% forecasted and 100% actual. The forecast one is the useful one; the actual one is the receipt. Notice in the console that they are separate alert types — this is the distinction from Pass 3 made concrete.
- Attach a budget action to the 100% threshold. This is the step that makes it a control instead of a notification. Choose "apply an IAM policy" and select a deny policy targeting
ec2:RunInstances. Set it to require manual approval the first time so you can see exactly what it would do without risking a self-inflicted outage. - Enable Cost Anomaly Detection with an AWS-managed monitor. Set the alert threshold low — for a personal account, $20 of unexpected spend is a real signal.
- Open Cost Explorer and switch the cost type between unblended and amortised. If you have any commitment, watch the shape of the graph change. If you have none, note that the two are identical — and that this is exactly why the distinction only starts mattering after your first Savings Plan.
- If you have an Organization: create a tag policy for the four keys from S1 and attach it to a non-production OU. Do not enable enforcement. Read the compliance report.
What good looks like: you can point at a mechanism that would stop runaway GPU spend, not merely tell you about it — and you have seen the manual-approval prompt, so you know what it looks like at 3 a.m.
Azure took the opposite architectural bet: instead of three separate policy types, one policy engine with many effects — including effects that repair non-compliant resources rather than just blocking or reporting them. That capability has no AWS analogue and changes what governance can mean.
Also watch the hierarchy: Azure's resource group is a real container inside the subscription, which means Azure needs fewer subscriptions than AWS needs accounts to achieve the same separation.
Azure governance & cost
Management groups. Azure Policy as one engine with many effects — including effects that fix things. What replaced Blueprints. Cost Management, Advisor, Reservations versus savings plan for compute, and Hybrid Benefit.
Azure's governance model is the most conceptually elegant of the three and the least like AWS. Where AWS separates permission-denial, resource-protection and configuration into three policy types, Azure has one policy engine that does all of it — plus something neither other cloud offers: automatic remediation.
Azure gave itself one more real container — and it changes everything downstream
AWS's account is the only hard boundary, so AWS estates grow many accounts. Azure has a resource group inside every subscription: a genuine container that you can target with RBAC assignments, with policy, with locks, with budgets, and that you can delete as a unit. That is not a filing convenience — it is a policy scope.
The consequence is that an Azure organisation achieves the same separation with materially fewer isolation units than an AWS one. Where an AWS shop might vend an account per team per environment, an Azure shop often runs one subscription per environment with resource groups per team inside it. Fewer subscriptions means less plumbing, fewer cross-boundary trust relationships, and a shallower hierarchy — at the cost of a weaker wall, because a resource group is a management boundary, not a security or quota boundary the way a subscription is.
Hold that trade in mind through the whole session. Almost every "why does Azure do it this way?" question resolves to it.
The Azure dialect
A · Management groups and the scope ladder
Azure calls the place a rule applies a scope, and there are four of them stacked: management group, subscription, resource group, resource. Assign anything — a policy, a role, a budget, a lock — at any scope, and it applies to that scope and everything beneath it. One word, one mechanism, four levels. That consistency is the nicest thing about Azure governance.
The reference hierarchy that the Azure Landing Zone accelerator builds is worth memorising, because you will meet it in every enterprise Azure estate:
By default, any principal in the tenant can create a new management group under the root, and all new subscriptions land under the tenant root group — outside whatever policy you carefully assigned to your intermediate root. Two fixes, both first-day work: enable RBAC authorization on management-group operations to restrict who can create them, and configure a default management group for new subscriptions so nothing ever lands ungoverned.
Hard numbers, verified July 2026
The resource-group trap
Because resource groups are cheap and real, the temptation is to use them where a subscription is needed. Three things a resource group is not:
Use resource groups for lifecycle and delegation; reach for a subscription when you need quota, hard isolation, or a distinct bill.
B · Azure Policy — one engine, many effects
In S2 you learned that AWS needs three policy types because permission-denial, resource-protection and configuration-pinning are structurally different jobs. Azure's answer is different: one policy language, and each policy declares an effect — what to do when a resource matches the rule. Change the effect and the same rule becomes preventive, detective, or self-healing.
The analogy: AWS gives you three tools. Azure gives you one tool with a dial on it. The dial includes settings the AWS tools do not have — most notably "fix it".
| Effect | What it does | S1 guardrail type | Cost-governance use |
|---|---|---|---|
Deny | Rejects the create/update request | Preventive | Block expensive SKUs in dev; block non-approved regions |
Audit | Allows it, records non-compliance | Detective | The safe first step for any new rule — see what would break |
AuditIfNotExists | Flags a resource missing a related resource | Detective | Find VMs with no diagnostic settings, no backup |
DeployIfNotExists | Creates the missing thing automatically | Remediating | Auto-enable diagnostics, auto-attach backup policy |
Modify | Adds, updates or removes properties/tags on the resource | Remediating | Auto-inherit cost-centre tags from the resource group — the single most useful cost policy in Azure |
Append | Adds fields during creation | Preventive-ish | Force a default tag onto every new resource |
Disabled | Turns the assignment off without deleting it | — | The rollback button. Use it. |
DeployIfNotExists and Modify do not block and do not merely report — they remediate. And they run not only on new resources but on existing ones, through a remediation task that sweeps everything already in scope.
For cost governance this is transformative. In AWS, the untagged-resource problem is solved by denying creation without tags (which breaks deployments) or by reporting and nagging (which does not work). In Azure you write a Modify policy that inherits cost-center from the parent resource group onto every child resource, run a remediation task once across the estate, and the tagging-gap problem from S1 largely evaporates — without denying anything or asking anyone to change their pipelines.
Here is that policy's core, which is the artifact worth recognising on sight:
// Modify effect: inherit cost-center tag from the resource group "then": { "effect": "modify", "details": { "roleDefinitionIds": ["/providers/.../Contributor"], "operations": [{ "operation": "addOrReplace", "field": "tags['cost-center']", "value": "[resourceGroup.tags['cost-center']]" }] } }
Effect precedence — Deny always wins, and it wins before Modify runs
Azure evaluates effects in a defined order: Disabled is checked first, then Append and Modify, then Deny, then Audit. The practical consequence is that a Modify policy adding a required tag runs before a Deny policy requiring that tag — so the pair composes correctly and the deployment succeeds with the tag applied. If you reason about them in the other order, you will expect breakage that does not happen.
Also note: policy assignments do not conflict the way SCPs "intersect". Multiple assignments all evaluate, and any Deny match rejects the request.
Remediation needs a managed identity — and that identity needs permissions
DeployIfNotExists and Modify act on your resources, so the assignment needs an identity to act as. Azure creates a managed identity for the assignment, and you must grant it a role with sufficient permission at the target scope. This trips up nearly everyone the first time: the policy shows as assigned, non-compliance is detected, and remediation silently fails because the identity lacks Contributor. Check the remediation task's failure detail, not the compliance blade.
Compliance evaluation is not instant
Azure Policy evaluates on resource create/update immediately, but the periodic full compliance scan runs roughly every 24 hours. So after assigning a new Audit policy, an empty compliance report means "not yet evaluated", not "fully compliant". You can trigger an on-demand scan, and you should, before drawing conclusions.
The audit-first rollout, in Azure terms
Azure makes S1's audit-first discipline unusually easy because effect is a parameter on many built-in policies. Assign with effect = Audit, read the compliance report, fix or exempt the outliers using policy exemptions (a first-class object with an expiry date — use the expiry), then re-assign with effect = Deny. Same definition, same scope, one parameter changed. No cloud makes this cycle smoother, so there is no excuse for skipping it.
C · Landing zones — and the Blueprints correction
The session plan lists "Azure Policy and Blueprints/landing zones". Azure Blueprints never reached general availability and is being retired. The timeline, from Microsoft's own retirement documentation:
Microsoft's recommended replacement is a pair, because Blueprints did two jobs: Azure Deployment Stacks take over assignment, lifecycle management and deny-assignment locking; template specs (or a Git repository) take over storing and versioning the definitions.
Your Azure source is ahead of the session plan here. §16 teaches Deployment Stacks and does not teach Blueprints — so follow the material, not the plan. Azure Blueprints retirement Migration to deployment stacks
Microsoft.Resources namespace.DenyDelete and DenyWriteAndDelete. This is how a landing zone stops teams from editing the platform underneath them.The distinction from AWS is worth stating plainly. Control Tower is a managed service — AWS operates it and it has opinions you cannot easily override. The Azure Landing Zone accelerator is infrastructure-as-code you own — you fork it, modify it, and maintain it. More flexibility, more responsibility. Practitioners report the accelerator handles roughly 80% of an enterprise landing zone with the remaining 20% being customisation.
D · Cost Management, budgets, and Advisor
The Azure-specific move worth learning: budget → action group → Automation runbook. Because action groups can invoke a runbook or Function, an Azure budget can do more than notify — it can deallocate VMs in a scope, scale a set down, or apply a policy. §10 mentions budgets integrate with action groups but does not draw out the consequence, which is that Azure's budget-to-action path is more flexible than AWS's fixed set of budget actions, at the cost of you having to write the runbook.
A budget does not cap spending on any cloud. Azure budgets alert; they stop nothing by themselves. The stopping happens only if you wire an action group to something that acts. This is the same trap as GCP's (see S4) and the same as the AWS billing incident in S1 — worth stating once per cloud because people keep assuming the word "budget" implies a limit.
E · Reservations, savings plan, Hybrid Benefit, and Spot
§10's "Quick comparison of Azure pricing models" lists exactly three: pay-as-you-go, Reserved Instances (up to 72%), and Spot. It omits Azure savings plan for compute entirely. That is the flexible spend-based commitment — Azure's answer to AWS Compute Savings Plans — and its absence removes the most important modern option from the decision. The 72% Reservation figure the material quotes is still current and correct. Azure savings plans
| Vehicle | You commit to | Max discount | Covers | Best for |
|---|---|---|---|---|
| Reservation (Reserved VM Instance) | A VM series, size flexibility group, and region for 1 or 3 years | ~72% | VMs, plus separate reservations for SQL, Cosmos DB, App Service, Redis and others | Steady-state workloads in a known SKU |
| Azure savings plan for compute | A fixed $/hour of eligible compute spend for 1 or 3 years | ~65% | VMs, Dedicated Hosts, Container Instances, App Service Premium v3 & Isolated v2, Functions Premium, AKS node VMs, Azure Virtual Desktop | Anything whose SKU mix will change |
| Azure Hybrid Benefit | Nothing new — you bring existing Windows Server or SQL Server licences with Software Assurance | stacks | Removes the OS/SQL licence surcharge | Applied on top of a Reservation or savings plan; combined discounts can exceed 80% on eligible Windows workloads |
| Spot Virtual Machines | Nothing | up to ~90% | Evictable VM capacity | Interruptible batch, training, rendering |
The 7-percentage-point gap between Reservations (72%) and savings plan (65%) is the price of flexibility, and closely mirrors AWS's ~6-point gap between EC2 Instance and Compute Savings Plans. Microsoft's 65% figure is based on a specific configuration (an M64dsv2 in East US over 36 months) — treat all published ceilings as best-case. 2026 comparison
Azure-specific mechanics worth knowing
Spot VMs — how Azure differs
Three-cloud comparison — Azure-anchored
Azure Policy (one engine, many effects)
AzureDeny, Audit, AuditIfNotExists, DeployIfNotExists, Modify, Append, Disabled. Bundled into initiatives, parameterised at assignment.
QuirkThe only engine that can remediate. Effect is a parameter, so audit→deny is a one-value change.
SCP + Config + CFN hooks
AWSThree separate systems for the same three jobs. SCPs deny, Config detects, hooks pre-check IaC.
QuirkNo remediation effect. Auto-fix requires you to write EventBridge + Lambda yourself.
Org policy constraints
GCPBoolean and list constraints on resource configuration, plus custom constraints. Dry-run mode built in.
QuirkConstrains configuration only — cannot express "deny this principal this action" the way an SCP does.
Management group + subscription + resource group
AzureFour scope levels, one consistent assignment model for policy, RBAC, budgets and locks.
QuirkResource groups are lifecycle units — deleting one deletes its contents, making ephemeral environments trivial.
OU + account
AWSThree levels; tags do all sub-account grouping.
QuirkMore accounts needed for equivalent separation, hence Control Tower and Account Factory.
Folder + project
GCPThree levels; folders nest to 10.
QuirkProjects are cheap and disposable — closer in spirit to an Azure resource group than to an AWS account, despite being the hard boundary.
Deployment Stacks + template specs
AzureStacks manage lifecycle and apply deny assignments; template specs store and version the templates. Replaces Blueprints.
QuirkDeny settings on a stack lock platform resources against modification by the teams using them.
Control Tower + Service Catalog
AWSManaged landing zone; Account Factory vends accounts from approved blueprints.
QuirkManaged, so less flexible — and enables Config everywhere, which bills.
Cloud Foundation Toolkit
GCPTerraform blueprints for the org, projects, network and policy.
QuirkFully DIY. Google's assumption is that you already run Terraform.
Budget + action group
AzureBudget at any scope; action group can invoke Logic Apps, Functions or Automation runbooks.
QuirkMost flexible response path of the three — but you write the runbook.
AWS Budgets + budget actions
AWSFixed menu of actions: apply an IAM policy or SCP, stop EC2/RDS instances. Optional manual approval.
QuirkLeast flexible, most turnkey — no code required to get a real stop.
Budget + Pub/Sub
GCPBudget publishes to Pub/Sub; you write a Cloud Run function to respond.
QuirkAlert-only by default and the docs say so explicitly. The kill switch is entirely yours to build.
Right-sizing arithmetic, and the commitment that outlived its SKU
Worked example 1 — what right-sizing an over-provisioned fleet is actually worth
Right-sizing is the least glamorous lever and frequently the largest one, because over-provisioning compounds silently. Azure Advisor flags a fleet of 12 × Standard_D16s_v5 (16 vCPU) running an inference API. Observed peak CPU over 30 days is 22%; observed peak memory is 31%. Pay-as-you-go for D16s_v5 in East US is approximately $0.768/hour; D8s_v5 is approximately $0.384/hour; D4s_v5 approximately $0.192/hour.
Now the part most analyses skip — stacking right-sizing with a commitment, and the order in which you do them:
Eliminate, then right-size, then commit. Every commitment is a bet on your current consumption shape, so any optimisation that changes that shape must happen first. Teams that buy commitments early because "we know we'll always need something" consistently end up over-committed on a shape they then cannot improve without stranding the commitment. The commitment is the last step, not the first.
Worked example 2 — the Reservation that outlived its SKU
This quantifies the flexibility premium concretely. You need one GPU node for inference, steady, 24/7, for three years. Assume a GPU VM at $3.60/hour pay-as-you-go, a 3-year Reservation at 62% off, and a 3-year savings plan at 55% off on the same VM. In month 18 a newer GPU series launches at 40% better performance per dollar, and you want to migrate.
Three grounded examples
Blueprints reaches phased retirement in three days
Deprecated 11 July 2026; phased retirement begins 31 July 2026; full retirement 31 January 2027. At retirement, unexported definitions and assignments are permanently deleted and all remaining Blueprint locks are removed automatically — meaning resources currently protected by a Blueprint deny assignment will silently become modifiable.
The governance lesson beyond the migration: a preview service was used for production locks, and its sunset removes those locks. When choosing a governance mechanism, weight GA status heavily — the cost of a governance service being withdrawn is not just migration work, it is a silent reduction in enforcement.
The accelerator does 80%; the last 20% takes as long
Practitioner reports put a basic Azure Landing Zone deployment — management group hierarchy, policies, hub-spoke networking — at 2–6 weeks using the Bicep or Terraform accelerator, and a full enterprise deployment with regulated-industry custom policies, ExpressRoute and CI/CD at 8–16 weeks. Migrating an existing unstructured estate adds another 4–8 weeks for subscription reorganisation.
Why this belongs in a cost session: those weeks are the real price of the governance you get. It is why the S1 decision tree defaults to not creating a new isolation unit — every one you add is a share of that ongoing cost.
Azure lets you trade a Reservation for a savings plan
Microsoft supports trading in recently purchased Reservations for a savings plan, and both vehicles can be held simultaneously — the savings plan absorbs compute the Reservations do not match. Reservation exchange was also extended to let you move a Reservation to a different VM family or region.
The strategic reading: Azure's escape hatches are better than AWS's, where Savings Plans above $100/hour are permanently unreturnable. If you are running the same workload on both clouds, that asymmetry is a real, if unglamorous, input to placement decisions.
Your context, and one hands-on lab
Working Azure literacy for an AWS/GCP-primary engineer
Your stated goal is equal AWS and GCP fluency with working Azure literacy. For governance and cost, "working literacy" means four specific translations you can do in an interview or a design review without looking anything up:
| If someone says… | You should immediately think… | And know the one non-obvious difference |
|---|---|---|
| "We enforce it with Azure Policy" | SCP or Config rule or auto-remediation — ask which effect | "Azure Policy" alone does not tell you whether it blocks, reports, or fixes. Always ask for the effect. |
| "It's in a different resource group" | Roughly a tag boundary plus a lifecycle unit, not an account | It shares the subscription's quota — including GPU quota. |
| "We bought Reservations" | Rigid commitment, ~72%, family-locked | Instance size flexibility means it is less rigid than an AWS Standard RI, and it can be traded for a savings plan. |
| "The landing zone handles that" | Terraform/Bicep the team owns, not a managed service | Ask who maintains their fork. The answer is often "nobody since the consultants left". |
Two things that differ from your GCP experience and would catch you out. First, GPU quota is per-subscription per-region and starts at zero — the same as GCP, but Azure's quota-request flow is slower and NC/ND-series capacity is more regionally concentrated. Plan the quota request before the architecture review, not after.
Second, AKS node VMs are covered by the savings plan for compute, but the AKS service charge is not. If you were to port your KServe stack to AKS, the node pool commits cleanly while the control-plane charge sits outside — a small but easy-to-miss gap when modelling costs.
The Azure equivalent of your scale-to-zero Modal setup is KEDA on AKS with a spot GPU node pool scaling to zero, or Azure Container Apps with serverless GPU for lighter models. Neither matches Modal's cold-start, for the same reason nothing does: someone still has to pull the image and load weights.
Time: ~40 minutes. Azure Policy, management groups, Cost Management and budgets are all free. The lab uses Azure's remediation capability, which has no AWS or GCP equivalent, so it is the one worth doing here specifically.
Prerequisite: an Azure subscription — the free tier is fine. You need permission to create a management group, or you can do the whole lab at subscription scope instead.
- Create a management group and move your subscription into it. Then set it as the default management group for new subscriptions, so nothing ever lands at tenant root ungoverned. That one setting is the highest-value five seconds in the lab.
- Tag a resource group with
cost-center = experiments. This is the value the policy will propagate. - Assign the built-in policy "Inherit a tag from the resource group if missing" at management-group scope, parameterised with
cost-center. This is aModifypolicy. Let Azure create the managed identity and confirm it gets a role assignment — this is where the lab usually fails silently. - Create a resource that would be untagged — a storage account on the free/lowest tier is fine and costs pennies. Confirm the tag appears automatically. You have now watched a governance system repair rather than obstruct.
- Run a remediation task across existing resources. Watch it fix things you created before the policy existed. This is the step with no AWS equivalent, and it is the answer to S1's "tags are not retroactive" problem — at least for the resource itself, though not for historical cost data.
- Assign "Allowed virtual machine size SKUs" with effect = Audit, scoped to your management group, permitting only small SKUs. Read the compliance report. Then change the effect parameter to
Denyand re-read. Same definition, one parameter — this is the audit-first cycle made concrete. - Create a budget at management-group scope with 50/80/95 thresholds, and attach an action group. Even if the action group only emails you, create it as a separate object so you can see the indirection that lets a budget invoke a runbook later.
What good looks like: you can explain to an AWS person why Azure needs fewer accounts and how a policy can fix a resource instead of rejecting it — and you have seen both happen.
Deny on VM SKUs will confuse you badly in three months. Delete the budget. Leave the management group if you want, but remember you set it as the default for new subscriptions — either keep that deliberately or clear it. Finally, check Cost Management → cost analysis the next day rather than trusting the resource list, because Azure storage and disks can persist after their parent looks deleted.
GCP's hierarchy looks like the other two but has one structural difference that changes everything about cost: billing is a separate axis. A billing account attaches to projects independently of the folder tree, so your governance hierarchy and your cost hierarchy can legitimately disagree.
And GCP is the only cloud that hands you a discount for nothing — sustained-use discounts. It has sharp edges that your GCP source does not mention, and one of them lands squarely on accelerator workloads like yours.
GCP governance & cost
The Resource Manager hierarchy, and the billing axis that runs beside it. Organization policy constraints. Labels versus Tags — two different things. Budgets that do not cap. Committed-use and sustained-use discounts, including the eligibility trap that hits GPUs. Billing export to BigQuery.
A note on the section: §12's real title is broader than the session plan suggested, and its front half is IAM you covered in C2. The governance and billing material is the back half — Structuring Google Cloud Resources, Managing Billing Accounts, and Introduction to the Organization Policy Service. It is also the thinnest of the three vendor section on cost, so more of this session comes from grounding than from the material.
In GCP, the cost tree is not the governance tree
Tab 0 opened with a claim: the org chart and the bill are the same diagram. GCP is the cloud where that is least automatically true, and understanding why is the key to the whole session.
In AWS, an account belongs to exactly one organization and pays through it. In Azure, a subscription sits in one management group and bills through it. In GCP, a project sits in the folder tree and separately is linked to a billing account — and the billing account is not part of the resource hierarchy at all. Two projects in the same folder can bill to different billing accounts. One billing account can pay for projects scattered across the whole tree. The governance boundary and the budget boundary are genuinely independent axes, and keeping them aligned is your design work, not a property you get for free.
That flexibility is occasionally useful — a shared services project billing to a platform cost centre while living in a workload folder — and is much more often a source of orphaned projects, mystery spend, and the specific GCP failure mode where a project's billing gets disabled and everything in it stops.
The GCP dialect
A · Resource Manager and the billing account
Same four-level tree as the other two clouds: organization → folders → projects → resources, with policy inheriting downward. §12 states this clearly and correctly. The GCP-specific instinct to build is that projects are cheap. Unlike an AWS account — which needs an email address, a payment method and a plumbing pass — a GCP project is a lightweight thing you create with one command. That changes the default: where an AWS architect asks "can we avoid another account?", a GCP architect more often asks "why is this sharing a project?"
§12's organisation best practices are sound and match the other two clouds: separate projects per environment, folders per department or function, and a dedicated project for shared infrastructure so networking and logging are not duplicated. The section's suggestion of multiple billing accounts for larger enterprises is where GCP genuinely diverges — and it deserves a caution the material does not give, which appears in section D.
Things that surprise people
Quota versus limit — the distinction people blur
Why this belongs in a cost session: quota is the only mechanism in GCP that acts as a preventive spend control on a specific resource type. A budget cannot stop you launching 100 GPUs; a quota of 8 can. Deliberately setting quota below what you could afford is one of the most effective and least-used cost guardrails on any cloud.
B · The Organization Policy Service
Where an AWS SCP says "this principal may not call this API", a GCP organization policy says "resources in this scope may not be configured this way". It is closer to AWS's declarative policies than to SCPs — it constrains the shape of your infrastructure rather than who may act on it.
That difference has a practical consequence worth holding: GCP's answer to "stop people using expensive regions" is a constraint on where resources may exist, not a deny on the principals who create them. The effect is similar; the mental model is not.
Audit effect, and a capability AWS SCPs simply do not have.The two constraints that do the most cost work, and are worth setting on day one:
# constraints/gcp.resourceLocations — where resources may exist # stops region sprawl and the unwatched-region mining incident constraint: constraints/gcp.resourceLocations listPolicy: allowedValues: [ in:us-central1-locations, in:us-east4-locations ] # constraints/compute.vmExternalIpAccess — no public IPs # a security control that is also an egress-cost control constraint: constraints/compute.vmExternalIpAccess listPolicy: deniedValues: [ ALL ]
Both are list constraints. allowedValues and deniedValues are mutually exclusive on a single policy — you pick a posture, exactly as you pick between an AWS deny list and allow list.
§12 introduces the service but is thin on the resolution rules, which is where the real behaviour lives. Three that matter:
Inheritance is the default, and it is overridable
A child node inherits the parent's policy unless it sets inheritFromParent: false, in which case the child's policy replaces the parent's outright. This is a real difference from AWS SCPs, where a child can never re-grant what a parent denied. GCP org policy is closer to a default than to an absolute ceiling — unless you lock it.
enforce and the deny-override
For boolean constraints, a child can set enforce: false and switch a parent's restriction off for its subtree. Whoever holds orgpolicy.policyAdmin at that node can do it. Governance here is only as strong as the IAM on the policy service itself — a point worth making in any design review.
Dry run first, always
Set the policy in dry-run, read the audit logs for what would have been denied, fix the legitimate cases, then enforce. This is S1's audit-first cycle, and GCP gives it to you as a first-class mode rather than as a separate policy you rewrite. Use it especially for resourceLocations, which breaks more things than anyone expects — including some global and multi-region services.
Why resourceLocations breaks things, and how to roll it out without an incident
The constraint evaluates against the location of the resource being created, and a surprising number of things are global or multi-regional: Cloud Storage multi-region buckets, some Cloud Build and Artifact Registry operations, Firestore's location model, and a variety of managed services that provision a hidden regional resource on your behalf. Allowing only us-central1 will therefore block things nobody associates with a region.
The safe rollout is: dry-run at organization scope for two weeks, read the audit log, then add value groups — GCP provides pre-defined location groups like in:us-locations that cover a whole geography including the multi-regions — rather than enumerating individual zones. Enumerating zones by hand is the usual cause of the incident.
There is no built-in organization policy constraint that says "every resource must carry a cost-center label." AWS has tag policies (weak, but present) and Azure has Modify with remediation (strong). In GCP, enforcing label hygiene means writing a custom constraint against the resource's label field, or accepting detection-after-the-fact from the BigQuery billing export. That is a meaningful hole in a cost-governance story, and it is why the GCP allocation section below leans harder on detection than the Azure one did.
C · Labels, Tags, and network tags — three things, confusingly named
In AWS, a tag is one thing that does two jobs: it groups costs and it can gate access. GCP splits those jobs across two different systems with almost the same name, and then adds a third unrelated thing also called a tag.
The intuition to hold: labels are for the accountant, Tags are for the auditor, network tags are for the firewall. If you take nothing else from this section, take that sentence — it resolves most GCP naming confusion in design reviews.
| Mechanism | What it actually is | Where it shows up | What it cannot do |
|---|---|---|---|
| Label | A free-form key/value pair attached to a resource. Lowercase keys, up to 64 per resource. Anyone with write access to the resource can set one. | Billing reports, the BigQuery billing export, cost breakdowns, and resource listings. This is your cost-allocation dimension. | Cannot be used in an IAM condition or an org policy condition. Cannot be enforced by a built-in constraint. Not validated against any list. |
| Tag (resource manager tag) | An organization-level resource: you define a tag key and its permitted values centrally, control who may attach them with IAM, then bind them to resources. Inherits down the hierarchy. | Conditional IAM bindings and conditional org policies — "this policy applies only where env=prod". A governance primitive. |
Historically not a billing dimension the way labels are. Do not reach for Tags to slice a cost report. |
| Network tag | A bare string attached to a VM. No key, no value, no governance. | VPC firewall rules and routes target them. | Everything else. It is not metadata, it is a firewall selector. C4 territory, mentioned here only so you never confuse it with the other two. |
The reason this split exists is sound, even if the naming is not: a cost dimension needs to be cheap, unvalidated and editable by whoever owns the resource, while a policy dimension needs to be centrally defined and IAM-controlled. AWS uses one mechanism for both jobs and consequently has a weaker policy story around tags. GCP separated them. Tags overview
Where labels quietly fail to appear
The GCP-specific answer: the project is the label
Because projects are cheap and every cost row carries a project ID unconditionally, the most reliable allocation strategy in GCP is structural rather than metadata-based: make the project boundary match the thing you want to bill, and treat labels as a secondary dimension inside it.
This is the opposite of the AWS instinct, where accounts are expensive so you push allocation into tags. It is worth stating explicitly in a design review, because an AWS-trained architect will reflexively build one big project and label heavily — producing exactly the gaps listed to the left. In GCP, a per-team or per-experiment project costs nothing and gives you allocation that cannot be forgotten, plus a quota boundary and a clean delete.
D · Budgets that do not budget
S1 made the general point that a budget is a smoke detector, not a sprinkler. GCP is the cloud where that is most literally true, and where the documentation says so in its own words: a budget's purpose is to trigger notifications. It does not cap spend, it does not stop resources, and by default the only thing that happens when you blow through it is that some email arrives.
Where AWS gives you budget actions from a fixed menu and Azure lets a budget invoke a runbook, GCP hands you an event on a message queue and says: build what you want. That is more work and more power. It is also the single most common gap between what a team believes it has configured and what it actually has. Cloud Billing budgets
A GCP cost-response system is four components you assemble yourself. The stepper walks the path a single dollar of overspend takes:
A budget is scoped, and the scope is more flexible than you'd expect
You create it on a billing account, then narrow the scope: to specific projects, specific services, specific labels, or a combination. That label filter is the payoff for the labelling discipline in section C — it lets you budget workload=inference across every project at once, which no folder-based scheme could express.
Set the amount as either a fixed figure or "last month's spend", and set thresholds as percentages. Use both actual and forecast rules: actual at 50/90/100, forecast at 100. The forecast rule is the one that gives you time to act, and it is the one people forget to add.
The threshold fires — on data that is hours old
Budget evaluation runs against billing data that lags actual consumption. For most services that lag is a matter of hours. This is the design constraint that matters for GPUs: eight A3 instances left running by a broken autoscaler can accrue several thousand dollars before the budget data catches up, and the alert then tells you about money you have already spent.
The conclusion is not "budgets are useless" — it is that budgets are the backstop, and the real preventive control for accelerators is quota, which is enforced synchronously at launch. Set both. Treat the budget as the thing that catches what quota did not anticipate.
Pub/Sub is the seam — and it must exist before the alert does
Attaching a Pub/Sub topic to the budget is optional and off by default. Without it, the budget emits email to billing-account role holders and nothing else. With it, every threshold crossing publishes a JSON message containing the budget name, the cost amount, the threshold exceeded, and the currency.
Two practical notes: the message is published on every update once you are over a threshold, not once, so your consumer must be idempotent. And the messages arrive whether or not anything is listening, so create the topic and subscription first — a budget pointing at a topic with no subscription silently discards everything. Programmatic budget notifications
Your function decides — and graduated responses beat one big switch
A Cloud Run function subscribes to the topic and acts. The useful pattern is a ladder keyed to the threshold in the message, not a single action:
The 100% action is the one worth building carefully, because scaling to a floor is reversible and safe, while the alternatives are neither.
Disabling billing: the real kill switch, and why it is a last resort
The function can call the Cloud Billing API to unlink the billing account from the project. This genuinely stops spend, because resources in a project with no billing account stop working. Google publishes this as the example capping pattern, and it does what it says.
It is also close to detonation. Compute stops, possibly not gracefully. Data in some services becomes inaccessible and, past a retention window, can be lost. Re-enabling billing does not automatically restore everything to its prior state. Never point this at a production project. It belongs on sandboxes, on individual training projects, and on the demo environment somebody's proof-of-concept left running — which is precisely where runaway spend actually happens.
Notice what this whole ladder is: you have rebuilt, by hand, a slightly better version of what AWS gives you as a checkbox. That is the trade GCP makes throughout this session.
Given all of the above, here is the honest ordering of GCP's spend controls by how reliably they stop money leaving, strongest first. It is not the order most teams configure them in.
| Control | Enforced | What it actually prevents | Why it is under-used |
|---|---|---|---|
| Quota (per project, per region) | Synchronous | The launch itself. A GPU quota of 4 means the fifth accelerator cannot exist, regardless of anyone's IAM permissions or intentions. | Feels like an obstacle rather than a control, and raising it is a ticket. That friction is the feature. |
Org policy (resourceLocations, machine-type constraints) | Synchronous | The wrong shape of resource, in the wrong place. Stops the expensive-region and oversized-SKU classes of waste. | Requires knowing the constraint catalogue, and the rollout risk means people postpone it. |
| IAM on billing and compute | Synchronous | Who can link a project to a billing account at all — the root of the ungoverned-project problem. | Considered a security concern, so it never lands on the cost team's checklist. |
| Budget → Pub/Sub → function | Hours late | Nothing, until you write the function. Then: whatever you coded, on data that lags. | Everyone configures the budget and stops there, believing the job is done. |
| Budget email alerts | Hours late | Nothing at all. It informs a human who may be asleep. | Not under-used — over-trusted. This is the one everybody has. |
Run the ranking against a realistic accelerator failure: a training job's retry loop that requests a new A3 node on every failure. Quota stops it at n nodes, immediately, every time. Org policy does nothing, because each node is a legitimate shape in a legitimate region. The budget notices some hours later, by which point the meter has been running at accelerator rates the whole time. For GPU spend specifically, quota is not a secondary control — it is the primary one, and it should be set deliberately per project rather than accepted as whatever the last increase request happened to grant.
E · The discount stack: sustained use, committed use, Spot
GCP is the only major cloud that gives you a volume discount for doing nothing. Run a VM for most of a month and the price drops automatically — no purchase, no term, no commitment. That is the sustained-use discount, and it is genuinely unusual: on AWS and Azure, the discount for steady usage requires you to sign something first.
The analogy is a coffee shop that quietly starts charging you less once you have come in most days this month, versus one that sells you a punch card up front. The first is friendlier. It is also the reason GCP list prices and AWS list prices are not directly comparable, and the reason a naïve three-cloud price comparison spreadsheet is usually wrong in GCP's disfavour.
And there is a catch that lands squarely on your workloads. Hold that thought until Pass 3.
| Vehicle | Discount | Commitment | How it works | Locked to |
|---|---|---|---|---|
| Sustained-use discount (SUD) | up to ~30% (N1) up to ~20% (N2/N2D) |
None | Applied automatically as a credit, based on the fraction of the month an eligible resource ran. Compute Engine also combines partial usage across instances of the same type in a region to maximise the discount. | Machine family eligibility. See Pass 3. |
| Resource-based CUD | ~55–70% | 1 or 3 years | You commit to a quantity of vCPU, memory, GPU or local SSD in one region. Billed hourly whether you use it or not. The deepest discount available without accepting interruption. | Region + machine family. Cannot be moved. This is the rigid one. |
| Spend-based / flexible CUD | 28% (1 yr) 46% (3 yr) |
1 or 3 years | You commit to a dollar-per-hour of spend; the discount applies to eligible compute spend wherever it lands. GCP's answer to AWS Compute Savings Plans. The flexible variant spans machine families and regions. | A spend floor, not a resource. Far more portable. |
| Spot VMs | 60–91% | None | Reclaimable capacity. Preemption gives a 30-second shutdown notice — a quarter of AWS's two minutes. Unlike the old preemptible VMs, Spot VMs have no 24-hour maximum runtime. | Nothing — but you must survive interruption. |
How they stack. Spot is priced separately and does not combine with commitments. SUD and CUD do interact: committed-use discounts apply first to the resources they cover, and sustained-use applies to eligible on-demand usage beyond that. In practice, the shape of a well-run GCP compute bill is a committed floor, sustained-use credits on the on-demand layer above it, and Spot handling anything interruptible. CUD overview Sustained use discounts
Google migrated the billing model for spend-based commitments in January 2026, changing how commitment charges and the discounts against them appear as line items in reports and in the BigQuery export. Nothing about the economics changed, but any cost query, dashboard or FinOps script written against the previous line-item shape needs re-checking — and any tutorial written before 2026, including §12, describes the old presentation.
§12 and most GCP cost material present sustained-use discounts as an automatic benefit of running things steadily. Two eligibility conditions go almost universally unmentioned, and both matter for accelerator work:
1 · Accelerator-optimised machine families do not receive sustained-use discounts. The A-series and G-series — A2, A3, A4, and the G2 family your L4 work runs on — are excluded. GPUs attached to N1 instances do earn SUD, which is why older T4-on-N1 setups quietly benefited and newer L4-on-G2 setups do not. Migrating from N1+T4 to G2+L4 therefore loses a discount you never explicitly bought and may never have noticed you had.
2 · Eligibility varies by billing-account type. Sustained-use discounts apply to self-service (online) billing accounts; invoiced customers should confirm their own terms rather than assume the published percentages. §12 introduces the self-service/invoiced distinction in a purely administrative context and never connects it to discounts — but the connection exists, and on an enterprise agreement it can invalidate a forecast.
The consequence: for modern GPU families there is no automatic discount at all. On-demand is the real starting price, and every reduction from there must be deliberately purchased — a committed-use discount, or Spot. A forecast that assumes 20–30% will appear on its own is wrong by exactly that amount. Eligible resources — Compute Engine docs
Resource-based or spend-based? The question is about hardware lifetime
A resource-based CUD buys a much deeper discount in exchange for naming a region and a machine family for up to three years. For CPU fleets that is often fine — N2 will exist in three years. For accelerators it is a bet on a hardware generation, and accelerator generations turn over faster than commitment terms. The reality-check example below puts numbers on exactly that bet.
A spend-based flexible CUD gives up roughly a third of the discount and buys the right to change your mind about hardware. For GPU spend specifically, that optionality is usually worth more than the extra percentage points.
Spot on GCP versus Spot on AWS — the 30-second problem
S2's spot arithmetic carries over, with one design difference that changes implementation: GCP's default preemption notice is 30 seconds, against AWS's two minutes. Thirty seconds is not enough to serialise a large model's optimiser state to Cloud Storage.
So the checkpoint strategy has to change rather than tighten: checkpoint on a timer to a local SSD and sync asynchronously, or use a framework with in-memory redundancy, and treat the shutdown hook as time to deregister from the load balancer rather than time to save state. For inference, 30 seconds is comfortable for connection draining — which is why Spot is a much easier sell for serving than for training on GCP. Spot VMs
F · All three clouds, from the GCP side
The third and final pass over the same comparison. Read these as GCP's answers with the other two as contrast — by now you should be able to predict most of the AWS and Azure cells before reading them.
Organization → folder → project
GCPFolders nest to 10; projects are the hard boundary for IAM, quota and service enablement.
QuirkProjects are cheap enough to use as an allocation dimension. Neither other cloud's equivalent is.
Organization → OU → account
AWSOUs nest 5 deep; accounts are heavyweight and vended by Control Tower.
QuirkAccount creation friction pushes allocation into tags, and tag hygiene then becomes the whole problem.
MG → subscription → resource group
AzureFour levels; the resource group adds a lifecycle unit the others lack.
QuirkOnly cloud where the deepest container is also a single-command delete of everything inside.
Billing account (separate axis)
GCPLinked to projects independently of the folder tree. One account can span the org; one org can hold many.
QuirkThe only cloud where the governance tree and the payment tree are genuinely independent — power and hazard in equal measure.
Payer account + consolidated billing
AWSThe management account pays; every member account rolls up to it. Structure follows the org.
QuirkConsolidation is also what lets RI and Savings Plan benefit float across accounts.
Billing account → billing profile → invoice section
AzureA parallel commercial hierarchy that subscriptions attach to, distinct from management groups.
QuirkAlso two hierarchies, but the billing one is itself a tree — arguably the most complex of the three.
Labels (cost) + Tags (policy)
GCPTwo separate systems: labels flow to billing, Tags are IAM-governed and usable in policy conditions.
QuirkClean separation, but no built-in constraint requiring a label. Enforcement is custom or detective only.
Tags + tag policies
AWSOne mechanism for cost and access control; tag policies standardise casing and values.
QuirkCost allocation tags must be activated in the billing console before they appear in reports — and are not retroactive.
TagsModify policy
AzureOne mechanism, plus a policy engine that can add missing tags and remediate existing resources.
QuirkThe only cloud that repairs tag gaps automatically. Tags do not inherit from resource group without that policy.
Budget → Pub/Sub → your code
GCPAlert-only by default. A real response is a function you write; the documented cap is unlinking billing.
QuirkMost powerful and least turnkey. Also the most honest — the docs state plainly that budgets do not cap.
Budgets + budget actions
AWSFixed menu: apply an IAM policy or SCP, stop EC2/RDS. Optional approval step.
QuirkThe only no-code path to an automated stop.
Budget + action group
AzureAny scope; action groups invoke Logic Apps, Functions or runbooks.
QuirkSits between the other two on both flexibility and effort.
SUD + CUD (resource / spend-based) + Spot
GCPAutomatic discount for steady use, two commitment shapes, and Spot at 60–91% with 30-second notice.
QuirkOnly cloud with a no-commitment discount — and it excludes exactly the accelerator families GenAI runs on.
RI + Savings Plans + Spot
AWSStandard/Convertible RIs, EC2 Instance and Compute Savings Plans, Database Savings Plans, Spot to ~90%.
QuirkRichest menu, hardest to reason about. Two-minute interruption notice is the best of the three.
Reservations + savings plan + Hybrid Benefit + Spot
Azure~72% reserved, ~65% savings plan, licence benefit stacking past 80% combined.
QuirkBest escape hatches — exchanges, and trading a Reservation for a savings plan.
The discount that never arrived, and the commitment that outlived its silicon
Both examples below are about accelerators, because that is where your money is and because GCP's discount rules treat accelerators as a special case in ways the documentation does not advertise. The first is an error of forecasting; the second is an error of term selection. The second is roughly three times more expensive.
Hourly rates below are stated as model inputs, not as quoted list prices — GPU pricing moves, and it moved sharply in 2025–26. Take the arithmetic and the ratios as the lesson, and substitute the current figure from the price calculator when you run this for real. The discount percentages, by contrast, are the published ones and were verified for this session.
Example 1 · Forecasting a sustained-use discount onto a G2 fleet
An inference tier of three L4-backed instances, running continuously. The team builds next year's budget in a spreadsheet, applies "GCP sustained use, about 30%" the way every GCP cost article told them to, and takes that number to finance.
Now the part that makes this worth teaching rather than just embarrassing. The forecast was not unachievable. It was unpurchased:
Suppose this fleet was previously N1 + T4. Those instances were earning sustained-use discounts automatically. Moving to L4 on G2 is a performance and efficiency upgrade on every technical axis — and it silently deletes a discount the team never bought and never saw as a line item. The per-hour comparison in the migration proposal will therefore understate the true increase. Whenever you migrate accelerator families on GCP, compare effective post-discount cost, never list price.
Example 2 · A three-year commitment on a two-year GPU
This is the decision your own Vertex deployment will face, so it is worth doing slowly. A steady inference floor of four T4-backed N1 instances. Finance asks the reasonable question: one year or three, and which commitment type?
The right question is not "which discount is deeper" but "will this hardware still be here at the end of the term?" T4 is an ageing accelerator and Google has been steering workloads toward L4 and the newer accelerator-optimised families. Assume the migration lands at month 18 — well within a three-year term. GPU platforms — Compute Engine
The general form. Solve for the migration month m at which the two vehicles break even, and you get a number worth carrying around:
1 · The industry has decided this is a real job. The State of FinOps 2026 data shows 98% of surveyed organisations now managing AI and GenAI spend as a distinct category — up from 63% a year earlier and 31% the year before that. Two years ago this session would have been an appendix. State of FinOps
2 · And the most-wanted capability is the one this session just did. The same survey's most-requested missing practice is pre-deployment architecture costing — pricing a design before it is built, rather than reporting on it afterward. Both examples above are exactly that exercise, and both were decided by arithmetic done before any resource existed. 2026 FinOps Framework
3 · Even the vendor's own billing pipeline can fail loudly. On 17 July 2026 an AWS unit-pricing error generated cost estimates in the billions and trillions of dollars for over a day. Internal alarms detected the anomaly but did not halt bill generation or page anyone; customers escalated roughly four and a half hours later, and budget and anomaly alerts were disabled platform-wide during mitigation. The transferable lesson for a GCP estate is the one from section D: a detection system with no enforced response is a system that watches things happen. InfoQ
Your Vertex deployment, audited
This is the most directly applicable session in the stage, because you have already built the thing it governs. You wrote a Vertex AI deployment script — T4 accelerators, autoscaling from one to three replicas, spot enabled — and you have run vLLM on L4. Everything above lands on that script. Here is the audit, in the order the findings matter:
1 · Your min_replica_count = 1 is your commitment floor
Not the max, not the average. One replica is the only capacity you can be certain will be running in twelve months' time, so it is the only capacity worth committing. If you ever purchase a CUD against this deployment, size it at one replica — or, following the 80% coverage benchmark from S1, slightly under. Replicas two and three are precisely what Spot exists for.
2 · Your T4 setup is earning a discount you did not buy
T4 attaches to N1, and N1 is SUD-eligible. Your always-on replica has been accruing sustained-use credits automatically. This is worth knowing for one reason above all: it is the thing you lose the day you move to L4, and it will not appear as a line item on the migration proposal.
3 · Your L4 work has no automatic discount at all
G2 is accelerator-optimised, therefore SUD-excluded. If the vLLM-on-L4 path becomes the production path, on-demand is the true baseline and every reduction from it must be purchased deliberately. Model the migration as example 1 does: effective post-discount cost on both sides.
4 · Spot plus a 30-second notice changes your readiness design, not your checkpointing
For inference this is comfortable — 30 seconds is ample for connection draining. Your KServe experience transfers directly: the preemption hook should fail the readiness probe, let the load balancer stop routing, and drain in-flight requests. What it must not do is try to persist anything large. That instinct, correct for training, is wrong here.
5 · Vertex AI has its own commitment vehicle
Vertex AI prediction can be committed separately from raw Compute Engine. If your serving runs on Vertex endpoints rather than GKE nodes you manage, a Compute Engine CUD is not the instrument — check the Vertex-specific commitment terms before assuming your compute commitment covers it. This is the same gap Azure has between AKS nodes and the AKS service charge.
6 · Your project probably needs to be several projects
Following section C's default: separate serving from experimentation at the project boundary rather than by label. You get independent GPU quota — which, per section D, is the only synchronous spend control that works on accelerators — plus unforgeable cost allocation and a clean delete. In GCP this costs nothing.
Translating what you already built
Your documented cost ladder — roughly $324/month down to about $45/month — is the artefact worth reframing here. In FinOps terms it was a complete inform → optimize → operate cycle run by one engineer. In GCP terms:
| What you built | The GCP mechanism it corresponds to | What the managed version buys you, and what it costs |
|---|---|---|
| Per-request cost tracking in your LLM stack | BigQuery billing export + labels | The export gives you every SKU at project and label granularity, but never per-request — it cannot see tokens. Your application-level instrumentation is more granular than anything the platform offers, and the two should be joined, not replaced. This is the single most valuable thing you have already built. |
| Per-model cost distribution | Labels on endpoints, queried from the export | Requires the label to be set at endpoint creation. Retrofitting it does not fix historical rows — S1's non-retroactivity rule, in the place it will actually bite you. |
| Scale-to-zero on Modal | Cloud Run with GPU, or GKE with KEDA scaling a Spot node pool to zero | Modal's bill has no hierarchy at all — one line item, no folders, no projects, no org policy. That simplicity is exactly why you had to build cost tracking yourself. Moving to GCP trades that DIY instrumentation for a governance hierarchy you must design. |
| The $324 → $45 ladder itself | An unrecorded FinOps cycle | The gap is not technical. Nobody but you knows the assumptions behind either number. Writing down the ladder — workload, assumption, action, realised saving — converts a personal optimisation into an organisational capability, and is what "operate" means in the framework. |
For whichever GPU tier you run next: take the autoscaler's minimum replica count, multiply by 8,760 hours and the current on-demand rate, and compare that annual figure against the same figure at 28%. That difference is the entire value of a one-year flexible commitment on your floor, computed in about thirty seconds. If it is small, you have correctly established that commitments are not your lever and you should stop thinking about them. If it is large, you now know the ceiling on what any commitment conversation can be worth — before anyone builds a spreadsheet. Both outcomes are useful, which is why the calculation is worth doing every time.
Time: ~50 minutes, plus a 24-hour wait for billing export data to appear. Cloud Billing budgets, the BigQuery export configuration, and the Organization Policy Service are all free. Pub/Sub and Cloud Run stay inside their free tiers at this volume, and BigQuery's free tier covers 1 TiB of query processing per month against a dataset that will hold a few megabytes.
Prerequisite: a project with a billing account you can administer. If you have an organization node, do steps 4–5; if you are on a personal account without one, skip them and do everything else at project scope.
- Create a dedicated project —
gov-lab-<yourname>— rather than using an existing one. Remember the ID is permanent and globally unique. Doing this deliberately is the section C default in action, and it makes teardown a single operation. - Enable the BigQuery billing export for standard usage cost, into a dataset in that project. Do it first, because data takes up to 24 hours to appear and nothing else in the lab depends on it. Note that the export is forward-only — it will never contain yesterday.
- Create a Pub/Sub topic and a subscription on it, in that order, before touching the budget. A budget publishing to a topic with no subscription discards every message silently, which is the most common way this pipeline appears to work and does not.
- Create a budget scoped to the project, amount $10, with actual thresholds at 50/90/100 and a forecast threshold at 100. Connect it to the Pub/Sub topic under "Manage notifications". Confirm the topic is attached — this checkbox is off by default and is the entire difference between an alert and a pipeline.
- Deploy a Cloud Run function triggered by the subscription that does nothing but log the decoded message. Do not build the kill switch. The objective is to see the JSON — the budget name, the threshold, the cost amount — and understand what you would be branching on. Building the enforcement without first reading the payload is how people write a kill switch that fires on every 50% notification.
- Set an organization policy in dry-run mode:
gcp.resourceLocationsallowing onlyin:us-central1-locations. Then create a small resource somewhere else — ane2-microineurope-west1, or a bucket in another region. Watch it succeed, then find the dry-run violation in the audit logs. This is the audit-first cycle with training wheels, and no other cloud offers it as cleanly. - Switch the policy to enforced and try the same creation. Read the error message carefully — it names the constraint, which is the detail that makes these failures debuggable in production.
- Inspect your GPU quota under IAM & Admin → Quotas for the project. Do not request an increase. Just observe that it is almost certainly zero, that it is per-region, and that it is the only control in this entire lab that would stop an accelerator from launching at the moment of the API call.
- The next day, query the export. Group cost by service and by label. With a handful of dollars of activity the numbers are trivial, but the schema is the real lesson: look at how credits appear as their own rows, and how a discount is a negative line rather than a reduced price. Understanding that shape is what makes the difference between reading a GCP bill and guessing at one.
What good looks like: you can state, from having watched it, why a GCP budget alone stops nothing; you have seen a policy violation logged without being blocked and then blocked; and you can explain to an AWS-trained colleague why the project boundary is the right allocation unit here.
resourceLocations behind at org level), and the billing export configuration on the billing account, which points at a dataset that will no longer exist. Remember the 30-day soft delete: the project stays recoverable, but confirm in Billing → Reports two days later that the daily cost has actually gone to zero rather than trusting the resource list.
Three claims opened this stage. The hierarchy is a blast-radius boundary and a budget boundary — and GCP, by splitting billing onto its own axis, is the cloud that proves it by making the alignment optional. Guardrails must be enforced rather than hoped for — and a GCP budget, which enforces nothing by default, is the cleanest demonstration of the difference. FinOps is engineering, not finance — and you have now done the arithmetic that decides a $22,705 question about a discount that looked worse on paper.
C9 turns from governing what you have built to migrating what already exists — assessment, the seven Rs, landing zones as a destination rather than a diagram, and the cost modelling that decides whether a migration is worth starting. The commitment arithmetic from this stage is a direct input: the business case for a migration is largely a claim about post-migration run rate, and you can now check that claim.