Every primitive you are about to learn already existed in a physical building: a fenced campus, VLANs, a router, a firewall, a load balancer appliance, a leased line, a DNS server. The cloud did not invent them. It made them API calls. Learn the topology once, then translate the names three times.
Click any row of the rack to see what it maps to.
One big subnet where everything can reach everything. It is not a configuration mistake so much as a default — it is what you get when you accept every wizard's suggestion and never think about topology. It works perfectly. It stays working perfectly right up until one component is compromised, at which point the attacker inherits your entire estate, because nothing between the machines ever said no.
In C2 you built the identity layer — who a caller is, and what they're allowed to ask for. Networking is the second layer, and it answers a different question: can the packet even arrive? They compose, and the composition is the point. A private endpoint with no scoped role is reachable by any identity inside the network. A tightly scoped role on a publicly reachable endpoint still invites the whole internet to try. You want both: a private path and a least-privilege identity on it. Every "apply it" section in this stage names the identity half explicitly.
Almost every connectivity question in this stage — and in the certification exams — is really the same question: what is the lowest rung that meets the requirement? Climbing higher than you need buys you a bigger bill and more moving parts, not more safety.
You are learning three clouds, so here is the one question the three genuinely answer differently. Every other difference in this stage is vocabulary.
Design a multi-tier network with CIDR blocks that still work when someone asks you to peer with another team's network eighteen months from now.
L4 or L7, regional or global, public or private — and say why in one sentence a reviewer would accept.
Write security groups / NSGs / firewall rules that reference each other rather than IP ranges, so the policy survives a redeploy.
Estimate NAT, egress and cross-zone charges for a GPU workload on paper, and know which architectural moves cut them.
Three corrections to the plan, and a running ledger of staleness that each tab repeats where it matters.
AWS — AWS guidance, AWS architecture guidance. Your section numbers are right; the exact titles are §6 AWS Networking Concepts and Services VPC Design and Network Segmentation Hybrid Networking and DNS Integration Network Traffic Monitoring and Troubleshooting Network Security and Encryption. This really is the deepest of the four — roughly 275 pages on networking alone — and S2 leans on it.
Azure — vendor guidance, vendor guidance, vendor guidance, Mastering Azure. §3 Networking and §4 Advanced Networking are correct. One gap worth naming: Front Door is mentioned only in passing covers Load Balancer, Application Gateway, Traffic Manager and Firewall in depth but not Front Door. S3 teaches Front Door from current Microsoft documentation and says so at the point of use.
GCP — GCP guidance, a leading GCP architecture reference. §4 GCP Networking is correct, but its scope is narrower than the session plan assumed. It covers VPC and subnets, firewall rules, VPC peering, Cloud VPN, Cloud Interconnect and Cloud CDN. It lists Shared VPC as a topic in its own section structure and then does not teach it; Cloud DNS does not appear in the section at all; and Cloud Load Balancing lives in §2, Autoscaling and Load Balancing, not §4. S4 covers all three anyway and flags each as taught from documentation rather than the material.
Patterns — A cloud patterns catalog, the cloud application patterns catalog. Confirmed: ten section, none about networking. Its useful contribution to S1 is the set of patterns that define where the network boundary goes — Service API and Backend Service , Dispatcher , Public API . Everything else in S1 is taught from first principles and labelled as such.
| costPublic IPv4 is metered on AWS | $0.005 per address per hour, charged whether the address is in use or idle. About $3.65/month per address; a three-AZ NAT setup quietly adds three of them. VPC pricing |
| newRegional NAT Gateway (AWS) | Introduced late 2025: one NAT resource that expands and contracts across AZs with your workload, billed per AZ-hour. Not in any 2025-vintage guide. VPC pricing |
| retiredAzure Basic SKUs | Basic public IP addresses and Basic Load Balancer retired 30 Sep 2025. Any source screenshot offering you a "Basic" tier is showing a dead option. Microsoft Learn |
| changingAzure default outbound access | The free implicit internet path for VMs is going away. For API versions released after 31 Mar 2026, new VNets default to private subnets — you must attach an explicit egress method. Microsoft Learn |
| renamedGCP load balancer names | "HTTP(S) Load Balancing", "TCP Proxy" and "TCP/UDP Network LB" are gone as product names. It is now Application Load Balancer, proxy Network Load Balancer, and passthrough Network Load Balancer, each in external/internal and global/regional flavours. Cloud docs |
| supersededGCP firewall policies | Classic per-network VPC firewall rules still work, but new capability ships on Cloud NGFW policies — hierarchical (org/folder), global, and regional — with IAM-governed tags. Sources that teach only tag-based VPC rules are a generation behind. Cloud docs |
| policyEgress-to-leave is now free | All three providers waive egress charges for customers migrating their data off the platform, and the EU Data Act bans switching-related egress charges outright from 12 Jan 2027. Ordinary day-to-day egress is unaffected and still costs what it always did. |
Verified mid-2026, US regions, list price. Internet egress: AWS $0.09/GB after 100 GB/month free · Azure $0.087/GB after 100 GB/month free · GCP Premium Tier $0.12/GB (Standard Tier $0.085/GB). NAT: AWS $0.045/hour + $0.045/GB processed · Azure $0.045/hour + $0.045/GB · GCP Cloud NAT ~$0.044/hour per gateway-VM plus ~$0.045/GB. Cross-zone traffic inside one region: $0.01/GB each way on all three. Ingress is free everywhere. Regional variation is real — always re-check before you commit a number to a design document.
Sources: aws.amazon.com/vpc/pricing azure.microsoft.com bandwidth pricing cloud.google.com/vpc/network-pricing
You can resize a VM, swap a database, rewrite a service. But the address ranges you pick on day one, and the tiers you do or don't create, are load-bearing for the life of the system. This session teaches the primitives with no vendor names attached, because the primitives are the same three times and only the labels change.
A model server runs on your laptop, bound to 127.0.0.1:8000. Nothing can reach it but you, and that feels like a limitation, so when you move it to a cloud GPU you bind it to 0.0.0.0 and open the port. It works immediately. That is the entire problem: the insecure version works first and works fine. There is no error message. The bill and the breach both arrive later.
Internet-wide scanning over 293 days found roughly 175,000 publicly reachable Ollama hosts across 130 countries — real production inference servers on real GPUs, most with no authentication, because the framework ships without any and binding to 0.0.0.0 is what the quickstart tells you to do. Researchers traced an end-to-end criminal operation: scan for exposed Ollama, vLLM and OpenAI-compatible endpoints, validate response quality, then resell the stolen inference capacity through a gateway. By May 2026 a critical flaw in Ollama's quantisation path (CVE-2026-7482, CVSS 9.1) turned that exposure into memory disclosure — system prompts, environment variables, API keys — on around 300,000 servers.
Everything in this session exists to make the secure version the one that is easy to build.
What it does contribute is the question of where the boundary belongs: its Service API and Backend Service patterns , Dispatcher and Public API all describe a component whose contract is deliberately exposed while its implementation is deliberately not. That is the application-architecture statement of the same idea this session states in packets. Everything else in S1 is taught from first principles — CIDR, routing, gateways, load-balancer tiers and DNS are protocol-level facts that predate all three clouds.
The provider owns a metropolis of hardware. You rent a fenced campus inside it and get to decide the street numbering, where the internal roads go, and where the gates are. Your neighbours are metres away physically and unreachable logically. The fence is not a firewall you configured — it is the default state, and the interesting work is deciding where to cut doors into it.
This is the mental model to keep for the rest of the stage. For traffic to flow from A to B, all three of these must be true, and they are configured in three different places:
Nearly every "why can't these talk?" ticket is one of these three being absent. Nearly every accidental exposure is all three being present when you only meant to configure one. Diagnose in that order and you will be right faster than people who start with tcpdump.
The isolation is enforced in the provider's software-defined network — packets are encapsulated and tagged with your network's identity, and the hypervisor drops anything mislabelled. It is strong, and no publicly documented breach of a major provider has come from tenant-to-tenant network leakage. But two consequences follow from it being software:
base-address/prefix-length, where the prefix length says how many leading bits are fixed. 10.0.0.0/16 means "the first 16 bits are locked; the remaining 16 bits are yours" — 65,536 addresses.Layer: addressing. Appears in subnet definitions, route tables, and every firewall rule you will ever write.A postcode narrows down where you are. Give more digits and you narrow to a street; give fewer and you cover a county. The prefix length is exactly that: the number of digits you have committed to. A small number after the slash means a big area. A big number means a small one. /8 is a county, /24 is a street, /32 is a single letterbox, and 0.0.0.0/0 — zero digits committed — means "anywhere in the world", which is why it is the notation for "the entire internet" and why seeing it in a firewall rule should make you look twice.
An IPv4 address is 32 bits. The prefix locks the leading n; the remaining 32 − n are free to vary, so the block holds 2(32−n) addresses. That is the whole calculation.
Three ranges are reserved by IANA for private use and are never routed on the public internet. You will build inside one of them:
# RFC 1918 private ranges — pick one and stay inside it
10.0.0.0/8 16,777,216 addresses — the default choice for cloud, room to plan
172.16.0.0/12 1,048,576 — where several providers put their default networks
192.168.0.0/16 65,536 — home routers live here; avoid, it will collide with a laptop VPNToo small is annoying: you run out of addresses in a subnet and have to add another one. All three providers now let you add secondary ranges to a network after creation, so this costs you an afternoon.
Overlapping is close to unfixable. Two networks that both use 10.0.0.0/16 can never be peered, because a router cannot decide which 10.0.1.5 you meant. The fix is renumbering a live system: new subnets, redeployed workloads, every hard-coded address chased down. Teams pay six-figure sums for this. It is caused by one person, in one afternoon, accepting a default.
Every provider offers a starter network, and they all use very similar ranges. Two teams who both clicked "create with defaults" have a 50/50 chance of colliding. Treat your organisation's private address space as a single allocated resource with an owner — even if the "registry" is a spreadsheet with fifteen rows. Assign each environment and region a non-overlapping block before anyone creates anything.
Some real limits to design inside: providers reserve addresses in every subnet (five on AWS and Azure, four on GCP — the arithmetic is in the Reality Check below), the largest practical network block is a /16 on AWS and Azure, and the smallest usable subnet is a /28 on AWS and a /29 on Azure.
There is no "public" checkbox. A subnet is public if and only if its route table contains a route for 0.0.0.0/0 pointing at an internet gateway. Nothing else about it differs. Two identical subnets, same size, same tags, same firewall rules — one is public and one is private, and the only difference is one line in a routing table. Once you internalise this, "is this thing exposed?" becomes a question you can answer by reading, in about fifteen seconds, instead of guessing.
The conventional layout is three tiers, repeated in each failure domain. Click any element below.
Each floor of the building has a signpost by the lift. "Anything for the east wing, that way. Anything for the loading bay, that way. Anything else — the street." The signpost does not check credentials; it just points. And if there is no sign for where you want to go, you do not go.
Here is a private subnet's table. The config is the mechanism, so it is worth reading closely:
DESTINATION TARGET WHY IT'S THERE
10.20.0.0/16 local implicit — everything inside the network, always present, cannot be removed
10.90.0.0/16 peering-conn-1 the partner network, reached directly over the backbone
192.168.4.0/24 vpn-gateway the office, over an encrypted tunnel
0.0.0.0/0 nat-gateway the catch-all: everything else exits one-way via NATA packet for 10.20.5.9 matches both 10.20.0.0/16 (16 bits) and 0.0.0.0/0 (0 bits); the /16 is longer, so it stays local. A packet for 52.94.1.1 matches only the catch-all and goes to NAT. Change the last line's target to the internet gateway and this subnet becomes public — that is the entire operation.
Two consequences worth having reflexes about:
local route is why everything inside a network can reach everything else by default. Segmentation between your own subnets is therefore a firewall job, not a routing job. Routing gets you to other networks; rules keep your own tiers apart.Non-transitivity. If A peers with B and B peers with C, A cannot reach C. Ever. Not by adding routes — the provider drops it. This is deliberate: transitive peering would let a partner's partner into your network. The consequence is that a full mesh of n networks needs n(n−1)/2 peerings — 45 for ten networks — which is why all three clouds sell a hub product (Transit Gateway, Virtual WAN, Network Connectivity Center) whose entire job is to be the one thing everything peers with.
Route tables are capped, typically around 50–100 entries with a raisable limit. Summarise: if the office is 192.168.0.0/24 through 192.168.7.0/24, write 192.168.0.0/21 — one entry instead of eight. This is the practical reason to allocate contiguous blocks per site rather than scattering them.
The most specific route wins even when it is wrong. A hand-added /32 for a debugging session outranks every carefully designed policy above it, silently, until someone finds it eight months later. Route tables deserve to be in version control with the rest of your infrastructure.
The internet gateway is the front door: people come in, people go out, and anyone who knows the address can knock. The NAT gateway is the staff exit with a one-way turnstile. Staff go out for coffee and come back with what they bought. No one on the street can come in through it — there is no handle on the outside. And crucially, everyone who leaves through it appears to the outside world to be the building, not themselves: that is the address translation.
When a private machine at 10.20.16.7 opens a connection to a model registry at 203.0.113.9:443, the NAT gateway rewrites the source to its own public address and a free ephemeral port, and records the mapping in a translation table. The reply comes back to the gateway's address and port; the gateway looks up the row and forwards the packet inward.
An unsolicited packet arriving from the internet has no matching row. There is nothing to forward it to — not "denied by policy", but genuinely undeliverable. The one-way property is a consequence of how the state table works, not a rule someone configured, which is why it is reliable.
Follow one request all the way out and back:
It bills on three axes at once, and they compound:
And the recommended high-availability pattern is one per failure domain, so a reference text three-zone design costs about $97/month before any traffic at all. For a workload that pulls multi-gigabyte model weights on every node start, the per-GB half of that dwarfs the hourly half. The fix is almost never a cheaper NAT — it is not using NAT for that traffic, via private endpoints to the object store and registry. S2 does the arithmetic with real numbers.
There is also a private NAT variant on AWS and Azure, used when two networks with overlapping ranges must talk: it translates into a non-overlapping range so the routers can tell them apart. Rare, but it is the escape hatch when someone has already made the overlap mistake.
DNS returns the load balancer's public address. The client opens TLS to port 443. The packet arrives at the internet gateway, which finds a route to the public subnet, and the load balancer's firewall rule permits 443/tcp from 0.0.0.0/0 — the one place in this whole design where that source range is correct, because a public API is genuinely public.
What made this work: a gateway, a route, and a rule. All three. Remove any one and the connection times out or is refused.
At layer 7 the load balancer terminates the client's TLS, reads the HTTP request, picks a healthy backend and opens its own connection to 10.20.16.7:8000. The GPU node's rule does not say "allow 8000 from anywhere" — it says "allow 8000 from the load balancer's security group". Identity-by-group rather than identity-by-address, so it keeps working when the load balancer's addresses change.
Why this hop is the whole design: the node has no public address and no inbound route. The only way to reach port 8000 is to already be inside, on a permitted path.
Traffic stays inside the network — the local route covers it, no gateway involved, no charge unless it crosses a zone boundary ($0.01/GB each way, which for a chatty retrieval workload is a real number). The data tier's rule permits its one port from the app tier's group and nothing else. The database itself has no route to 0.0.0.0/0.
Because the firewalls in front of your instances are stateful, no return rule is needed anywhere: the reply to an allowed request is allowed automatically. This is why security-group rules are usually inbound-only and still work. (Stateless filters at the subnet level, where they exist, do need explicit return rules — that asymmetry is a favourite exam trap and gets full treatment in S2.)
Meter check: tokens leaving via the load balancer to the internet are billed as egress. A verbose 4,000-token answer is a few kilobytes — trivial. It is bulk data, not chat, that generates real egress bills.
The node has no public address, so the pull goes out through NAT: 30 GB × $0.045 = $1.35 in NAT processing alone, per node start. Scale to 3 replicas cycling twice daily and that is $240/month for traffic that never left the provider's own network.
The architectural fix, not the frugal one: put the weights in the same region's object store and reach it through a private endpoint. On AWS a gateway endpoint for object storage has no hourly and no per-GB charge at all, so this line goes to zero. Same pull, same security posture, different route table entry.
Someone port-scans the NAT gateway's public address and sends a SYN to port 8000. The gateway checks its translation table for a matching outbound flow, finds none, and drops the packet. There is no rule to misconfigure here — the reachability simply does not exist.
Contrast with the failure mode from the top of this session: a GPU VM with its own public address and 0.0.0.0/0:8000 open answers that same knock with a list of the models it is serving.
An L4 load balancer is a switchboard operator who connects your call to whichever line is free. She never hears a word you say — she is fast, she handles enormous volume, and she cannot help you if you asked for the wrong department.
An L7 load balancer is a receptionist who reads your form. "Anything starting /v1/embeddings goes to the small-model pool; /v1/chat goes to the GPU pool; if you have no valid token you don't go anywhere." Slower per request, vastly more useful.
There is a third tier the materials tend to under-teach: the global / anycast layer. One address announced from hundreds of edge locations worldwide; a client in Sydney and a client in Dublin dial the same address and land on different edges. Traffic then rides the provider's private backbone to your region rather than the public internet. This is Amazon CloudFront and Global Accelerator, Azure Front Door, and Google's global external Application Load Balancer.
Timeouts are the number-one production surprise. L7 load balancers default to an idle timeout of about 60 seconds. A large generation that takes 90 seconds to produce its first token gets its connection cut mid-thought, and the client sees a 504 with no useful error. Either raise the idle timeout deliberately, or stream — with server-sent events the first token arrives in a second and the connection is never idle. Streaming is the better answer for a second reason: it also gives you a real time-to-first-token metric.
Buffering breaks streaming. Some L7 proxies buffer the whole response before forwarding, which converts your beautiful token-by-token stream into one lump at the end. If tokens arrive in a batch, look for a response-buffering setting before you look at your model server.
Least-connections beats round-robin for GPUs. Round-robin assumes requests are interchangeable. Inference requests are not: one may be 50 tokens, the next 4,000. Distributing by active connection count tracks actual load far better. Where available, least-outstanding-requests is the setting you want.
Health checks need a real endpoint. A TCP check passes the moment the port is listening — which on a GPU node is several minutes before the model has finished loading into VRAM. That window sends live traffic to a server that will fail every request. Health-check an HTTP path that returns 200 only after the model is loaded and can complete a token.
You ask for a name and get back an address. The important, always-underestimated detail is the notepad: the answer is cached at every hop for as long as the TTL says. Change a record with a 24-hour TTL and some clients keep dialling the old address all day. This is why every DNS cutover plan starts by lowering the TTL, waiting for the old one to expire, and only then making the change.
Two cloud-specific capabilities matter more than the mechanics:
weights.internal resolves to 10.20.48.9 for anything inside your network and returns nothing at all to the outside world. This is how you name internal services without publishing your topology.Health-check-driven DNS failover costs a few dollars a month and takes as long as your TTL plus the health-check interval to converge — realistically 60–180 seconds, and longer for clients that ignore TTLs (some JVMs cache DNS answers for the life of the process unless told otherwise). An anycast global front door fails over in seconds because the address never changes; only the routing behind it does. You are choosing between a few dollars a month with minutes of downtime, and a few tens of dollars a month with seconds. State the recovery-time requirement first and the choice makes itself.
Two operational notes worth keeping: split-horizon confusion (the same name resolving differently inside and outside) is powerful and is also the cause of a large fraction of "works on my machine" incidents — document it where the developers are, not just where the network engineers are. And a dangling CNAME pointing at a decommissioned provider resource is a real subdomain-takeover path: delete the record when you delete the resource.
The full three-cloud treatment is in S2–S4; the two axes that organise it are worth fixing now, because every vendor's product is a point on this grid.
Remembers connections. Allow an inbound request and its reply is automatically allowed — no return rule. Usually allow-only: you cannot write a deny, because anything not allowed is already denied.
The superpower: rules can reference other groups rather than address ranges — "allow 8000 from the load-balancer group". The rule keeps working when addresses change, which under autoscaling is constantly.
AWS security group · Azure NSG · GCP firewall rule
Evaluates every packet independently with no memory. Allow inbound and you must also allow the outbound reply, on ephemeral ports. Supports explicit deny, which the stateful kind cannot.
What it is genuinely for: a coarse, subnet-wide backstop — blocking a specific hostile range, or an entire protocol, in one place that no individual team can override.
AWS network ACL · (Azure and GCP have no direct equivalent)
Do your segmentation with the stateful, workload-attached one. It composes with autoscaling, it is what code review can read, and it cannot be silently defeated by a missing return rule. Reach for the stateless subnet filter only when you need an explicit deny or a blanket rule that outranks everything — and expect to spend a debugging session on ephemeral ports the first time you do.
Print this. Every later tab is an expansion of one column.
| Neutral concept | AWS | Azure | GCP |
|---|---|---|---|
| Private network | VPCVirtual Private Cloudregional — a VPC cannot span regions | VNetVirtual Networkregional; a resource group is billing/admin, not a boundary | VPC networkGLOBAL — one network spans every region. The genuine differentiator; S4 dwells on it. |
| Address slice | Subnetlives in exactly one AZ5 addresses reserved per subnet; min /28 | Subnetspans the AZs of its region5 reserved; min /29; some subnets must have exact names | Subnetregional, spans all zones4 reserved; can be expanded in place without downtime |
| Two-way internet door | Internet Gatewayexplicit object you attach; free | Public IP + rulesno gateway object; the address is the door | Default internet routeno gateway object; an external IP plus the route |
| One-way egress | NAT Gatewayper-AZ resource (or the new Regional mode); $0.045/hr + $0.045/GB | NAT Gatewaynow mandatory — implicit outbound access is being retired | Cloud NATnot an appliance — distributed SDN, no bandwidth chokepoint |
| Stateful workload firewall | Security Groupallow-only, attached to the interfacerules can reference other groups | NSGallow AND deny, priority-orderedattaches to NIC or subnet; ASGs give group-referencing | Firewall rule / policynetwork-wide, priority-orderedtargets by tag or service account — no per-instance object |
| Stateless subnet filter | Network ACLthe only one of the three with a true stateless subnet filter | — (use NSG deny)NSG on the subnet is stateful but supports deny rules | — (use policies)hierarchical policies give the org-wide backstop instead |
| Forwarding | Route tableexplicit, per subnetpublic vs private is decided here and only here | System routes + UDRsensible defaults exist; you override with user-defined routes | Routesnetwork-wide, not per subnetfollows from the global VPC model |
| L7 load balancer | ALBregional+ CloudFront / Global Accelerator for the global tier | Application Gatewayregional+ Front Door for global; four LB products total, S3 disambiguates | Application Load Balancerglobal or regionalrenamed from "HTTP(S) Load Balancing"; one anycast IP worldwide |
| L4 load balancer | NLBstatic IP per AZ; supports PrivateLink as a provider | Load Balancer (Standard)Basic SKU retired Sep 2025 | Network Load Balancertwo kinds: passthrough (DSR) and proxy |
| Network ↔ network | VPC Peering / Transit Gatewaypeering is non-transitive; TGW is the hub | VNet Peering / Virtual WANnon-transitive; hub-and-spoke is the house style | VPC Peering / NCCoften unnecessary — one global VPC or a Shared VPC instead |
| Encrypted tunnel | Site-to-Site VPNtwo tunnels per connection by default | VPN Gatewayneeds a subnet literally named GatewaySubnet | Cloud VPN (HA VPN)HA VPN carries a 99.99% SLA; classic is legacy |
| Dedicated circuit | Direct Connectvirtual interfaces: private, public, transit | ExpressRouteunlimited-data plan makes egress free above ~7 TB/mo | Cloud InterconnectDedicated, Partner, and Cross-Cloud to another provider |
| Private path to a service | PrivateLink / VPC endpointsgateway endpoints (object store) are FREE — use them | Private Link / private endpointservice endpoints are the older, weaker cousin | Private Service Connecthow you take a Vertex AI endpoint off the public internet |
| Managed DNS | Route 537 routing policies + health checks; $0.50/zone/month | Azure DNS / Private DNSTraffic Manager does the DNS-level steering | Cloud DNS100% availability SLA; public and private zones |
| Flow visibility | VPC Flow Logshas an ACCEPT/REJECT action field — the fastest firewall debugger there is | VNet flow logsNSG flow logs retired; Traffic Analytics on top | VPC Flow Logssampled by default; sampling rate is tunable |
The brief. A model-serving platform. Three tiers (public, app, data), three availability zones, one region now and a second region "eventually". Today the app tier needs 40 nodes; the team says it might be 400. There is another team on the same cloud whose network you will probably be asked to peer with next year. Design the addressing.
Address space costs nothing. Allocate per region from a registry that exists before anyone opens a console:
ORG ALLOCATION — one row per environment × region, written down BEFORE anything is created
prod / region-1 10.20.0.0/16 65,536 addrs — this design
prod / region-2 10.21.0.0/16 reserved now, empty for a year
stage / region-1 10.30.0.0/16
dev / region-1 10.40.0.0/16
partner-team 10.60.0.0/16 reserved so a future peering cannot collideThe last line is the whole trick. Peering fails on overlap, and overlap is decided on day one by people who have not met yet.
Give each tier a contiguous block so route tables and firewall rules can summarise it in one line. Then split each block three ways for the zones.
| Public tier 10.20.0.0/22 3 × /24 (one per zone), one /24 spare | 1,024 |
| → 10.20.0.0/24 · 10.20.1.0/24 · 10.20.2.0/24 | 251 usable each |
| App tier 10.20.16.0/20 3 × /22 (one per zone), one /22 spare | 4,096 |
| → 10.20.16.0/22 · 10.20.20.0/22 · 10.20.24.0/22 | 1,019 usable each |
| Data tier 10.20.48.0/22 3 × /24, one /24 spare | 1,024 |
| Reserved for growth 10.20.64.0/18 | 16,384 |
| Allocated / total — 66% of the block is deliberately unused | 22,528 / 65,536 |
10.20.28.0/22 — still inside the summarisable 10.20.16.0/20, so no route table or firewall rule has to change.Nobody gets all the addresses in a subnet. A /22 is 1,024 addresses; here is what you can actually schedule onto:
| Per subnet | AWS | Azure | GCP |
|---|---|---|---|
| Reserved by the platform | 5network, router, DNS, future, broadcast | 5first four + last | 4network, gateway, second-to-last, broadcast |
| Usable in a /22 (1,024) | 1,019 | 1,019 | 1,020 |
| Usable in a /24 (256) | 251 | 251 | 252 |
| Smallest subnet allowed | /2811 usable | /293 usable | /29practical floor |
| Can you grow it later? | Noadd another subnet or a secondary block | Yesresize if the space next to it is free | Yesexpand in place, no downtime |
Does the plan hold? 400 app nodes across three zones is ~134 per zone against 1,019 usable — comfortable. But if this becomes a Kubernetes cluster where every pod gets a real network address, 134 nodes × 30 pods is 4,020 addresses per zone, and the /22 is four times too small. That is the single most common address-exhaustion failure in cloud networking, and it is why the plan reserves a /18. S4 does the pod-range arithmetic properly.
Two structural facts fall out of the diagram before any traffic flows:
| NAT gateway × 3 (one per zone, the HA recommendation) | $98.55 |
| Public IPv4 × 3 (one per NAT) — AWS meters these at $0.005/hr | $10.95 |
| L7 load balancer, base hourly | ~$16.43 |
| Before a single byte moves | ~$125.93 / month |
| + 1 TB/month of model weights pulled through NAT (1,024 × $0.045) | +$46.08 |
| + the same 1 TB again if it came from outside the cloud (egress $0.09) | +$92.16 |
Researchers documented a complete criminal pipeline in January 2026: internet-wide scanning for unauthenticated Ollama, vLLM and OpenAI-compatible endpoints, automated validation of which ones respond well, and resale of the stolen capacity through a gateway service. This is the commercial incentive that turns "I left a port open" into "someone is billing my GPUs to third parties".
175,000 exposed Ollama servers across 130 countries — The Hacker News, Jan 2026
Google's Cloud Threat Horizons report for H1 2026 puts misconfiguration at 21% of initial access in the second half of 2025 (down from 29.4% in H1, as automated guardrails bite). The number that should change your behaviour is the speed: throughout 2025 Google observed attackers finding misconfigured compute and Kubernetes instances and deploying payloads within an hour of the resource being created. "I'll lock it down tomorrow" is not a plan.
All three providers now waive egress fees for customers taking their data off the platform entirely, and the EU Data Act bans switching-related egress charges outright from 12 January 2027. Read the scope carefully: this covers leaving. Your day-to-day serving traffic, cross-region replication and model-weight pulls are unaffected and still cost $0.087–$0.12 per GB. Design for the prices, not the headlines.
Azure bandwidth pricing — free egress on exit AWS free data transfer out when moving off AWS
Gemma on vLLM via Modal. Modal is doing all of this for you and not telling you. Your function has an HTTPS endpoint on Modal's domain: their global edge terminates TLS, their scheduler is the load balancer, and your container runs on their private network with no address you control. That is rungs 1 and 2 of the ladder, pre-built. The reason it felt effortless is that someone else made every decision in this session on your behalf. The cost of that is that you cannot put it inside your network, and the security boundary is a bearer token rather than a network path.
Vertex AI, T4s, autoscaling 1–3, spot. By default a Vertex endpoint is a public HTTPS endpoint — reachable from the internet, protected by IAM rather than by topology. That is a defensible design (it is layer-one defence in depth doing its job) but it is rung 1, and if the requirement is "prediction traffic must never traverse the public internet", the answer is a private endpoint. S4 covers exactly how.
vLLM on a GCP L4. This is the one to look at hardest. If that VM has an external IP and a firewall rule allowing 8000/tcp from 0.0.0.0/0, it is one of the 175,000 — a scanner will find it within hours of creation. The five-minute fix: remove the external IP, add Cloud NAT for outbound, and reach it over an SSH tunnel or a bastion. The production fix is the tiered layout in Fig 1.2.
S1 is concepts, so its lab is paper. Twenty minutes, and it is the exercise that most reliably separates people who can design a network from people who can click through a wizard.
/16 into three tiers × three zones, leaving at least a quarter unallocated. Do the usable-address arithmetic by hand — subtract five per subnet.ipcalc 10.20.16.0/22 or any CIDR calculator. Do it after, not before.Nothing in this lab costs money, but from S2 onward every lab creates billable resources. Set this up now, before you need it:
You now have the shapes: private network, tiered subnets, route tables, two kinds of gateway, two tiers of load balancer, and a name system that also steers traffic. Session 2 puts AWS labels on every one of them and adds the parts that only exist once you get specific — security groups versus network ACLs, VPC endpoints that make the NAT bill disappear, Transit Gateway, Route 53's routing policies, and flow logs as the fastest firewall debugger you will ever use.
Azure and GCP give you working defaults and let you override them. AWS hands you an empty region and a box of parts. Nothing routes until you write the route; nothing is reachable until you attach the gateway. It takes longer and it is the best possible way to learn, because every concept from Session 1 appears as an object you can point at.
Singh — AWS Networking Concepts and Services, VPC Design and Network Segmentation, Hybrid Networking and DNS Integration, Network Traffic Monitoring and Troubleshooting, Network Security and Encryption. Roughly 275 pages, and the deepest networking source of your four sources. Where I depart from it — the metered public IPv4 address, Regional NAT Gateway, current per-GB prices — it is flagged inline.
Two engineers deploy the same containerised model server on AWS. One takes ninety minutes and produces something a security review passes; the other takes fifteen and produces a GPU instance with a public address and port 8000 open to the world. The console will help you build either one with equal enthusiasm. The difference is knowing which six objects have to exist and how they refer to each other.
Click through the diagram — each object names the one it depends on.
The security group is a bouncer at each door who remembers faces. He let you in, so he lets you out — no second check. He only has a guest list; there is no "banned" list, because not being on the list is being banned.
The network ACL is a gate guard at the perimeter with a numbered clipboard and no memory whatsoever. Every crossing is a fresh decision, in both directions. He does have a banned list, and he stops reading the moment he finds a matching line.
The NACL evaluates its numbered rules lowest number first and stops at the first match. Rule 100 ALLOW 443 from 0.0.0.0/0 matches, so it passes — and rule 200 DENY for the same range is never read, because a lower number wins even if a later rule is more specific. The implicit final rule, shown as *, denies anything unmatched.
AWS convention: number rules in hundreds so you can insert between them later.
The security group checks its inbound rules. Order is irrelevant — all rules are evaluated and any match permits. The rule here is 443/tcp from sg-alb, referencing a group rather than a range. The connection is recorded in the group's state table.
No outbound rule is consulted for this packet. The group recognises it as the return of a permitted connection. This is why almost every real security group has a wide-open outbound rule and it does not matter — the inbound side is where the segmentation lives.
Outbound evaluation, fresh. The reply's destination port is the client's ephemeral port — some number between 1024 and 65535 that the client's OS chose. So the outbound NACL needs ALLOW 1024–65535 to 0.0.0.0/0, or the reply is dropped after the request was successfully served.
Symptom: the server logs show the request arriving and a 200 being written, and the client sees a timeout. Every application-level trace says success. It looks like an application bug and is not.
Diagnosis: open VPC Flow Logs and look at the ACTION column. Inbound ACCEPT followed by outbound REJECT on ephemeral ports is a missing NACL return rule, every time. AWS guidance makes exactly this point and it is the single most practically useful debugging heuristic in AWS networking:
The rules themselves, since the configuration is the mechanism:
sg-alb (on the load balancer)
IN 443/tcp from 0.0.0.0/0 correct — a public API is public
OUT 8000/tcp to sg-app only to the app group
sg-app (on the GPU nodes)
IN 8000/tcp from sg-alb GROUP as source — survives every scale event
OUT all wide open outbound is normal and fine
sg-data (on the vector store)
IN 5432/tcp from sg-app one port, one source group, nothing elseGroup-referencing does not cross a peering connection to another account's group in every case, and it never crosses regions. In multi-account designs you fall back to prefix lists — a named, shareable set of CIDR blocks that you update in one place. Worth knowing before you design a hub topology around group references.
Quotas that bite: 60 inbound and 60 outbound rules per security group, 5 groups per interface by default. A group with 60 hand-maintained IP rules is a design smell — collapse them into a prefix list.
NACLs are numbered 1–32766 and evaluated ascending. A low-numbered ALLOW beats a high-numbered DENY. Renumbering a live NACL is genuinely risky, which is the practical argument for keeping them coarse and rare.
The rule that ages badly: ALLOW 22/tcp from 0.0.0.0/0, added during a debugging session, never removed. Don't add it — use Session Manager, which reaches an instance through the SSM API with no inbound rule, no bastion, and no open port at all, and logs every session. It is strictly better than a bastion for almost every case, and it is free.
Public IPv4 addresses are now metered. $0.005 per address per hour, charged for in-use and idle addresses alike. That is $3.65/month each. A three-AZ NAT design carries three of them; an old account with forgotten Elastic IPs is paying for every one. Official pricing
Regional NAT Gateway exists. Introduced in late 2025: a single NAT resource that automatically extends to the AZs where your workload actually is, and stops billing for an AZ when the workload leaves it. Billing is still per AZ-hour, so it is not automatically cheaper — the win is operational, one object instead of three, with the AZ set tracking reality. Worth knowing about in an exam and in a design review. Official pricing
Your private instances need things from AWS's own services all day: object storage, container images, secrets, log destinations. Sending that traffic out through the NAT gateway is like walking out of your building, round the block, and back in through the neighbour's public entrance — when the two buildings share a wall. A VPC endpoint cuts a staff door through that wall. Shorter, private, and in the object-storage case, free.
They work in completely different ways, which is why one is free and one is not.
A gateway endpoint is a route. AWS adds a prefix-list destination to your route table pointing at the endpoint; packets for S3 or DynamoDB match it instead of the default route and never reach the NAT gateway. There is no interface, no address, no capacity — it is an entry in a table:
DESTINATION TARGET
10.20.0.0/16 local
pl-63a5400a (S3) vpce-0a1b2c3d ← the gateway endpoint. This line is the whole feature.
0.0.0.0/0 nat-0f3d9eAn interface endpoint is a real elastic network interface with a private address in each subnet you choose, backed by AWS PrivateLink. Private DNS makes the service's normal hostname resolve to that address, so your code needs no change. Because it is an interface, it has a security group and it bills per hour per AZ — and because it is per AZ, a three-AZ deployment costs three times one.
Longest-prefix match is what makes the gateway endpoint work silently. The prefix list for S3 is far more specific than 0.0.0.0/0, so it wins without you disabling anything. Add the endpoint and traffic reroutes itself.
Gateway endpoints are regional and route-table-scoped. They only work for S3 and DynamoDB, only within the same region, and only for subnets whose route table you actually associated. The most common failure is adding the endpoint and associating it with two of three private route tables — traffic from the third AZ keeps paying NAT charges, silently, and the only symptom is a cost line that will not go down.
Gateway endpoints do not work from on-premises over Direct Connect or VPN, because they are a routing trick inside the VPC. If your datacenter needs private access to S3, you need an interface endpoint. That is a genuine architectural difference, not a limitation to work around.
Interface endpoints multiply. Each AWS service needs its own: ECR needs two (the API and the Docker registry), plus S3 for the image layers themselves. At three AZs and $0.01/hour each, a handful of endpoints reaches $100/month before any traffic. Add them where the per-GB saving justifies the hourly cost — high-volume services first — and use the free gateway endpoint everywhere it applies.
Endpoint policies are the security upside. Both types accept a resource policy restricting what can be reached through them: "this endpoint may only access these two buckets". Combined with a scoped IAM role on the instance, that is two independent controls that must both fail before data can leave. This is the concrete form of the identity-plus-network composition from C2.
The three ways a private instance can reach S3, which is the clearest illustration of why this section is the money section:
| Path | Hourly | Per GB | Cost of 1 TB/month | Notes |
|---|---|---|---|---|
| Via NAT gateway the default if you do nothing | $0.045/hr $32.85/mo | $0.045 processing + egress if leaving AWS | $78.93 | Same-region S3 has no transfer charge, but NAT processing applies to every byte regardless. |
| Via interface endpoint PrivateLink | ~$0.01/hr per AZ $21.90/mo for 3 AZ | ~$0.01 | $32.14 | Right answer for ECR, Secrets Manager, CloudWatch — services with no gateway endpoint. |
| Via gateway endpoint S3 and DynamoDB only | $0 | $0 | $0.00 | Free. Traffic never touches the internet. There is no argument against adding one. |
A gateway endpoint for S3 in every VPC that has a NAT gateway costs nothing, takes two minutes, and removes model-weight pulls, log shipping and backup traffic from the NAT meter entirely. Practitioner write-ups consistently report 30–80% reductions in NAT spend from this plus interface endpoints for ECR — and the AWS pricing page itself points at gateway endpoints as the way to avoid the NAT data-processing charge in its own worked example. If you take one operational habit from this entire session, take this one.
AWS VPC pricing — NAT gateway example VPC endpoints documentation
One-to-one, non-transitive, uses the AWS backbone. No hourly charge at all — you pay only $0.01/GB for traffic that crosses an AZ boundary. Cross-region and cross-account both work. Requires non-overlapping CIDRs, permanently.
Reach for it when: two or three VPCs, stable relationships, cost matters.
A regional hub. Every VPC, VPN and Direct Connect gateway attaches once and the hub routes between them — transitively. Multiple route tables let you build "spokes can reach shared services but not each other", which peering cannot express.
Cost: ~$0.05 per attachment-hour (~$36/mo each) plus $0.02/GB through the hub. Six VPCs ≈ $220/month standing.
IPsec over the internet to your building. Two tunnels per connection for redundancy, ~$0.05/hr plus egress. Live the same day.
Reach for it when: starting hybrid, or as the always-on backup for Direct Connect.
A physical circuit into an AWS location. Virtual interfaces come in three flavours: private (to one VPC), public (to AWS public services over the circuit), transit (to a Transit Gateway). Port fee plus much lower per-GB egress.
Resiliency note the exam loves: one circuit is not highly available. Two circuits at two locations, or one circuit plus a VPN backup, is the minimum defensible design.
All of these are soft limits you can raise by request, which is exactly why they surprise people — nothing warns you as you approach one.
The habit worth building: when a design review reaches "and then we add another VPC", check the peering count and the route-table count before agreeing.
The routing policies are the part worth knowing cold — they turn DNS into a control plane:
| Policy | What it does | Reach for it when |
|---|---|---|
| Simple | One record, one answer (or several, returned in random order). | A single endpoint. The default. |
| Weighted | Splits answers by assigned weight — 95/5, 50/50. | Canary releases; shifting traffic to a new model version gradually. |
| Latency | Returns the region with the lowest measured latency to that resolver. | Multi-region serving where response time matters. The usual choice for global inference. |
| Failover | Primary while its health check passes, secondary when it fails. | Active-passive DR. Pair with a health check on a real readiness path. |
| Geolocation | Answers by the user's country or continent. | Data-residency and compliance: EU users must hit the EU endpoint, full stop. |
| Geoproximity | By geographic distance, with a bias dial to expand or shrink a region's catchment. | Shifting load between regions by geography rather than by measured latency. |
| Multivalue answer | Up to eight healthy records, returned at random, each health-checked. | Poor-man's load balancing where you have no load balancer. Not a substitute for one. |
Latency and geolocation are different and the difference matters. Latency routing answers "which region is fastest for this resolver?" Geolocation answers "where is this user, legally?" A user in Ireland routed by latency might land in London; routed by geolocation they land in Frankfurt because your compliance policy says EU data stays in the EU. Choosing the wrong one is a compliance finding, not a performance one.
Hybrid DNS is the other half of §8. Route 53 Resolver endpoints let queries cross the boundary between AWS and your datacenter: an inbound endpoint lets on-prem resolvers look up your private zones; an outbound endpoint plus forwarding rules lets your VPC resolve names that live on your corporate DNS. Without them, hybrid connectivity works at the packet level and fails at the name level, which produces exceptionally confusing tickets.
Three patterns, and the honest guidance on which to use:
Beyond TLS, AWS added VPC Encryption Controls (2025) — a per-VPC setting with monitor and enforce modes that gives visibility into, and can require, encryption of traffic within and between VPCs at the hardware layer. It is billed per non-empty VPC per hour (about $0.15 in us-east-1), which is substantial, so it is a compliance tool rather than a default. Worth knowing it exists; not in any 2025 source.
| AWS object | What it is | Azure | GCP |
|---|---|---|---|
| Security Group | Stateful allow-only firewall on the interface; can reference other groups | NSG + ASGNSG also does deny and has priorities; ASG gives you the group-reference trick | Firewall rule w/ service accountno per-instance object; target by service account for the same effect |
| Network ACL | Stateless numbered subnet filter | — no equivalentuse NSG deny rules on the subnet instead | — no equivalenthierarchical firewall policies play the org-wide backstop role |
| NAT Gateway | Per-AZ managed NAT (or the new Regional mode) | NAT Gatewaygoing from optional to required as implicit egress retires | Cloud NATdistributed SDN, not an appliance — no throughput chokepoint |
| Gateway VPC endpoint | Free private path to S3/DynamoDB via a route | Service endpointfree, but keeps the public address — weaker than a private endpoint | Private Google Accessa subnet flag; free; same idea |
| Interface endpoint (PrivateLink) | Private IP for a service, ~$0.01/hr + $0.01/GB | Private Link / private endpoint | Private Service Connect |
| Transit Gateway | Regional transitive hub with route domains | Virtual WANplus hub-and-spoke peering as the manual version | Network Connectivity Centeroften unnecessary — one global VPC already spans regions |
| Direct Connect | Physical circuit; private/public/transit VIFs | ExpressRouteunlimited-data plan bundles egress — changes the crossover maths | Cloud InterconnectCross-Cloud Interconnect links you to another provider directly |
| Route 53 | 7 routing policies, health checks, alias records | Azure DNS + Traffic Managersteering is a separate product from the zone | Cloud DNSless DNS-level steering because the global anycast LB does it instead |
| VPC Flow Logs | Per-flow records with ACCEPT/REJECT | VNet flow logsNSG flow logs are retired; use VNet flow logs + Traffic Analytics | VPC Flow Logssampled by default — raise the rate before you debug with them |
| ACM | Free auto-renewing certs for AWS endpoints | App Gateway / Front Door managed certs | Google-managed SSL certs |
The workload. Three g5.xlarge nodes across three AZs serving a 14B model. Each node pulls 28 GB of weights from S3 on start and recycles roughly twice a day. Container images from ECR: 4 GB, same cadence. Application logs to CloudWatch: 40 GB/month. Serving traffic out to the internet: 300 GB/month of JSON responses. Everything in us-east-1, list price, 730-hour month.
| NAT gateway hourly × 3 AZs (3 × 730 × $0.045) | $98.55 |
| Elastic IP × 3 for the NAT gateways (3 × 730 × $0.005) | $10.95 |
| Model weights: 28 GB × 3 nodes × 60 starts/mo = 5,040 GB @ $0.045 | $226.80 |
| Container images: 4 GB × 3 × 60 = 720 GB @ $0.045 | $32.40 |
| Logs to CloudWatch: 40 GB @ $0.045 | $1.80 |
| NAT data processing subtotal (5,800 GB) | $261.00 |
| ALB hourly + capacity units (approx.) | $22.00 |
| Internet egress: 300 GB − 100 GB free = 200 GB @ $0.09 | $18.00 |
| Monthly network cost | $410.50 |
| NAT gateway hourly × 3 + 3 Elastic IPs (kept for occasional egress) | $109.50 |
| S3 gateway endpoint — 5,040 GB of weights | $0.00 |
| ECR interface endpoint: 2 endpoints × 3 AZ × 730 × $0.01 | $43.80 |
| ECR data: 720 GB @ $0.01 | $7.20 |
| CloudWatch Logs interface endpoint: 1 × 3 AZ × 730 × $0.01 + 40 GB @ $0.01 | $22.30 |
| Residual NAT traffic ≈ 60 GB @ $0.045 | $2.70 |
| ALB hourly + capacity units | $22.00 |
| Internet egress: 200 billable GB @ $0.09 | $18.00 |
| Monthly network cost | $225.50 |
| NAT gateway × 1 + 1 Elastic IP | $36.50 |
| Cross-AZ charge for the two AZs routing to it: ~60 GB × $0.01 × 2 directions | $1.20 |
| Endpoints (as Design B) | $73.30 |
| ALB + egress | $40.00 |
| Monthly network cost | $151.00 |
| Dimension | NLB (L4) | ALB (L7) | Which wins for inference |
|---|---|---|---|
| Added latency | ~100 µs | ~1–5 ms | Irrelevant. Your model takes 200–2,000 ms. A 4 ms proxy hop is noise. |
| Path routing | No | Yes | ALB. /v1/chat to the big pool, /v1/embeddings to the small one, one endpoint. |
| Client IP at backend | Preserved | X-Forwarded-For | Tie. Read the header. |
| Load algorithm | Flow hash | Least outstanding requests | ALB. Variable-length generations make request-count balancing much better than hashing. |
| WAF / auth at the edge | No | Yes | ALB. Rate limiting in front of a GPU is a cost control, not just a security control. |
| Static IP for allow-listing | Yes, per AZ | No, DNS name | NLB — the one real reason to choose it. Enterprise clients with IP allow-lists. |
| gRPC / non-HTTP | Any TCP | HTTP/1.1, 2, gRPC | Tie for gRPC; NLB for anything else. |
Verdict: ALB, unless a client needs a fixed IP to allow-list. If you need both, the documented pattern is an NLB in front of an ALB — the NLB gives the static addresses, the ALB gives the L7 features.
AWS's own guidance converges on exactly the layout in Fig 2.1: workloads in private subnets, a load balancer as the only public entry point, per-AZ NAT for egress, and endpoints for AWS-service traffic. The Security Pillar of the Well-Architected Framework states the network-layer position — control traffic at all layers, use security groups as the primary segmentation, and prefer private paths to AWS services — and the VPC user guide's own pricing page recommends gateway endpoints specifically to avoid NAT data-processing charges.
Well-Architected · Infrastructure Protection NAT gateway pricing guidance
Sysdig's threat research team coined "LLMjacking" for the pattern of compromising credentials or open endpoints purely to consume inference capacity at someone else's expense. By June 2026 they documented an attacker using a publicly exposed, unauthenticated model server as the reasoning engine inside a multi-stage offensive pipeline — not reselling it, using it. Treat model-serving infrastructure as a high-value target on par with a production database, because that is now its market price.
You already know the compute half — containers, KServe, autoscaling groups behind a load balancer are C3 material. The AWS-specific additions are exactly five objects: the VPC and its subnets (Fig 2.1), two route tables (one public, one private per AZ), three security groups that reference each other rather than addresses, an S3 gateway endpoint for the weights, and an ALB with a readiness health check. Wire those and the compute layer you already understand slots straight in.
On EKS specifically: the AWS VPC CNI gives every pod a real VPC address from the node's subnet. That is elegant — pods get security groups and flow logs like any other resource — and it is the number-one cause of address exhaustion on AWS. Size node subnets for pods, not nodes: 50 nodes × 30 pods is 1,500 addresses, so a /22 per AZ, not a /24. Prefix delegation raises the per-node pod ceiling and consumes addresses in /28 chunks, which makes the arithmetic worse, not better.
The identity half, from C2: the node role should be scoped to s3:GetObject on exactly the weights bucket prefix, and the S3 gateway endpoint should carry an endpoint policy restricting it to that bucket. Then even a fully compromised node can reach one bucket, read-only, over a path that does not touch the internet. Network says where from; identity says as whom; the endpoint policy says to what. Three independent controls, all cheap.
Roughly 45 minutes. Use the free tier and the smallest instance sizes: t4g.micro or t3.micro, no GPU anywhere. The point is the topology, and the topology is identical at any instance size.
10.20.0.0/16 using the "VPC and more" wizard: 2 AZs, 2 public and 2 private subnets, NAT gateway "In 1 AZ" (not one per AZ — that is $65/month you do not need for a lab), and tick the S3 gateway endpoint box the wizard offers. Read the resource map it draws before you click create; it is Fig 2.1.t3.micro in a private subnet, no public IP, IAM role with AmazonSSMManagedInstanceCore. Connect with Session Manager — no bastion, no key pair, no inbound rule. Then curl https://checkip.amazonaws.com and watch it return the NAT gateway's address.aws s3 ls from that instance succeeds even though it has no public address. Remove the S3 endpoint's route and it still works — via NAT, now metered. Put it back.t3.micro in the private ones, with sg-app allowing 80 from sg-alb. Delete it the moment you have seen it work — this is the second most expensive thing in the lab.Every shape you just built has an Azure name, and Azure's defaults sit in different places — routing is implicit until you override it, the firewall does both allow and deny, and there are four load-balancer products where AWS has two. Session 3 maps them, and covers the two current changes that invalidate every Azure networking screenshot printed before late 2025: the Basic SKU retirement and the end of free implicit outbound access.
Azure gives you working defaults and asks you to override them, which is faster on day one and means the exposure is sometimes something you inherited rather than something you built. Two structural changes landed in 2025 that make older Azure material actively misleading, and this session leads with them.
vendor guidance, vendor guidance and vendor guidance (Networking) and 4 (Advanced Networking). Between them they cover VNets and subnets, VPN Gateway, ExpressRoute, VNet peering, NSGs and ASGs, DNS, Virtual Network Manager, Virtual WAN, Load Balancer, Application Gateway, Traffic Manager, Azure Firewall, service endpoints and Private Link, and routing. Front Door is mentioned only in passing — the section do not teach it — so the Front Door material below comes from current Microsoft documentation and is marked where it appears.
1 · Basic SKUs are gone. Basic public IP addresses and Basic Load Balancer retired on 30 September 2025. New Basic public IPs could not be created after 31 March 2025. Any walkthrough that says "choose Basic to save money" is describing an option that no longer exists. Standard is the only tier, it requires an NSG to permit inbound (secure by default rather than open by default), and it supports zone redundancy. Microsoft Learn
2 · Free implicit outbound access is ending. Historically an Azure VM with no public IP and no NAT still reached the internet through a shared address Microsoft supplied invisibly. That is default outbound access, and it is being retired. For API versions released after 31 March 2026, new virtual networks default to private subnets — no implicit egress at all. You must attach an explicit method: a NAT gateway, load balancer outbound rules, or a public IP on the resource. Existing VNets keep working; new ones do not. Microsoft Learn
3 · Front Door (classic) retires 31 March 2027, and has not accepted new profiles since March 2025. Standard and Premium are the tiers; Premium is the one with Private Link to origin, which is the feature that lets you keep your backend off the public internet entirely. Microsoft Learn
The second of these is the one that will change your designs. Azure is converging on the AWS and GCP model where egress is a thing you build, not a thing you get.
The subnet model is where Azure differs from AWS in a way that changes your diagrams. An AWS subnet is pinned to one AZ, so a three-AZ tier needs three subnets. An Azure subnet spans the whole region, so a three-zone tier is one subnet, and zone redundancy is a property of the resources inside it. Fewer subnets, simpler address plan, and the zone decision moves from the network layer to the resource layer.
The genuinely surprising part is that several Azure services require a dedicated subnet with a specific, exact name. Get the name wrong and the deployment fails with an error that does not always say so:
| Required name | For | Minimum / recommended size | Note |
|---|---|---|---|
| GatewaySubnet | VPN Gateway and ExpressRoute gateway | /27 recommended | Exactly one per VNet. Never put anything else in it — and do not attach an NSG. |
| AzureFirewallSubnet | Azure Firewall | /26 required | The size is enforced, not advisory. |
| AzureBastionSubnet | Azure Bastion | /26 recommended | Browser-based RDP/SSH with no public IP on the VM. |
| (any name) | Application Gateway | /24 recommended | Dedicated to the gateway; v2 autoscaling consumes addresses as it scales. |
| (any name) | Private endpoints | /28–/26 | Each private endpoint takes one address. They multiply faster than teams expect. |
The address arithmetic: Azure reserves five addresses in every subnet — the first four (network, gateway, and two for DNS mapping) and the last. The smallest supported IPv4 subnet is a /29, which leaves three usable addresses, and the largest is a /2 though a VNet in practice tops out around a /16. A /24 gives you 251. Unlike AWS, Azure subnets can be resized after creation if the adjacent space is free — a genuine relief when you underestimate.
Internet, Storage, AzureLoadBalancer, VirtualNetwork) that updates itself.Layer: policy. Roughly "AWS security group and network ACL merged into one object".Where AWS splits the job between a stateful bouncer who only has a guest list and a stateless gate guard who has a banned list, Azure gives you one bouncer with a numbered rulebook that contains both. He reads from the lowest number up and acts on the first line that matches. And he remembers faces, so replies are automatic.
Rules are numbered 100–4096, evaluated ascending, and evaluation stops at the first match. Below your rules sit three immovable defaults at 65000+:
PRIO NAME SOURCE ACTION
100 allow-https-from-internet Internet Allow 443
200 allow-app-from-gateway asg-appgw Allow 8000
300 deny-ssh-from-anywhere * Deny 22
-- your rules end; Azure's immovable defaults begin --
65000 AllowVnetInBound VirtualNetwork Allow any
65001 AllowAzureLoadBalancerInBound AzureLoadBalancer Allow any
65500 DenyAllInBound * Deny anyRule 65000 is the one to internalise. By default everything inside the VNet can reach everything else inside the VNet, on every port. Azure's default posture is flat-inside, closed-outside. Tier segmentation therefore requires you to explicitly deny intra-VNet traffic — on AWS you get that for free because security groups are allow-only and start empty.
Two mechanisms make rules durable:
Allow 443 to Storage.WestEurope keeps working when Microsoft adds address ranges — which they do, without telling you.asg-inference, the gateway's in asg-appgw, and write allow 8000 from asg-appgw to asg-inference. No addresses anywhere in the rule.When an NSG is on both the subnet and the interface, inbound is evaluated subnet-first then NIC; outbound is NIC-first then subnet. Both must allow. Two NSGs on one path is the most common source of "the rule looks right and traffic is still blocked" — check both.
NSGs are layer 3–4 only: address, port, protocol. They cannot express "allow this VM to reach pypi.org but nothing else on the internet", because that is a name, not an address, and the addresses behind it change hourly. That requirement is what Azure Firewall exists for — a managed stateful firewall with FQDN filtering, threat intelligence feeds, and its own public address for egress. It bills at roughly $1.25/hour for the Standard tier (~$912/month) plus per-GB processing, which makes it an organisation-level shared service in a hub VNet, never a per-project component.
Azure Virtual Network Manager is the answer at scale: define network groups and push connectivity and security-admin rules across many VNets from one place, with security-admin rules evaluated before NSGs so project teams cannot override them. This is the Azure equivalent of GCP's hierarchical firewall policies, and it is the right tool the moment you have more than a handful of VNets.
Rather than every team building their own reception, mail room and security desk, one floor holds all of them and every tenant floor connects to it. Tenants get their own locked space, their own budget and their own keys — but the way out of the building, and the rules about who may leave, belong to the landlord. A tenant cannot walk to another tenant's floor directly; they go via the shared floor, where security sees them.
A bare peering is just a private path between two VNets. Two settings promote it into the pattern:
0.0.0.0/0 to the hub's firewall private address. Because routing is set by the platform team on the spoke's subnets, the spoke owner cannot bypass inspection even if they want to. This is the difference between a convention and a control.Together they mean a spoke is a workload container with no exit of its own — every packet leaving it takes a path someone else defined.
Peering is not transitive on Azure either. Two spokes peered to the same hub cannot reach each other. Your options are a direct spoke-to-spoke peering (fine for two, unmanageable for twenty) or routing them through the hub firewall with UDRs. Most real hubs run a firewall largely for this reason, not for internet inspection.
Peering charges apply in both directions — roughly $0.01/GB leaving the sender and $0.01/GB arriving at the receiver. A spoke-to-hub-to-spoke conversation is therefore metered on four legs, plus the firewall's own per-GB processing. Chatty cross-spoke traffic is the line item that surprises people; co-locate things that talk constantly.
Global peering connects VNets in different regions and works the same way, at higher per-GB rates. Note one restriction worth remembering: some load-balancer scenarios behave differently across a global peering than a regional one, so validate rather than assume when you span regions.
When to skip the pattern entirely: below roughly four VNets with no central governance requirement, direct peering plus NSGs plus a NAT gateway does the same job for under $100/month against the hub's ~$1,200. Hub-and-spoke buys you enforcement across teams who do not report to you. If that is not your problem, you are paying for governance you will not use.
Routing on Azure is implicit until you override it. Azure creates system routes automatically — within the VNet, to peered VNets, to the gateway, and a default 0.0.0.0/0 to the internet. You do not build a route table to get a working network, which is why Azure feels faster to start. You add a UDR when you need to change that, and the canonical UDR is 0.0.0.0/0 → virtual appliance at 10.0.1.4, applied to every spoke subnet, which forces all egress through the hub firewall. That single route is what makes the hub design enforceable rather than merely conventional.
ExpressRoute has two billing models and choosing between them is a straight arithmetic problem, not a preference.
Metered — a lower monthly port fee plus per-GB outbound (around $0.025/GB in Zone 1, well under the ~$0.087/GB internet rate). Unlimited — a single fixed monthly port fee, roughly $5,700 for a 1 Gbps circuit in Zone 1, with all inbound and outbound data included.
The crossover for a 1 Gbps circuit lands somewhere around 65% sustained utilisation — below that, metered is cheaper; above it, unlimited wins. Two things people miss: the circuit's own egress rate is already far below internet egress, so ExpressRoute starts paying for itself against internet transit at roughly 7 TB/month before you even consider the unlimited plan; and ExpressRoute Global Reach, which links two circuits so your own sites talk to each other over Microsoft's backbone, is billed separately and is excluded from the unlimited plan.
The design consequence for a GenAI platform: if you are moving training datasets between an on-premises cluster and Azure at multi-terabyte scale, the circuit is a cost reduction, not a cost. If you are running a hybrid control plane with a trickle of traffic, a VPN Gateway at ~$139/month does the same job for 2% of the price.
This is the Azure question that trips people up, because AWS has two products and Azure has four. They divide cleanly on two axes: global or regional, and L4 or L7.
Azure offers two ways to reach a managed service without traversing the public internet, and they are frequently confused. The difference is worth knowing precisely because it is a common exam question and a common design mistake.
Extends your VNet's identity to the service over the Azure backbone. Traffic stays on Microsoft's network, and the service can restrict access to your subnet. But the service keeps its public address and public DNS name, and the connection is initiated outbound from you.
Free. Reasonable for simple cases. Does not help on-premises callers, and does not remove the public endpoint from the internet.
Puts a network interface with a private IP from your subnet in front of the service. The service becomes an address inside your VNet. You can then disable its public access entirely. On-premises callers reach it over VPN or ExpressRoute like any other internal address.
Costs an hourly fee per endpoint plus per-GB. Requires a private DNS zone so the service's own hostname resolves to the private address — this is the step people miss, and the symptom is that everything appears configured and traffic still goes out the public path.
A private endpoint without its private DNS zone is a very common half-finished configuration. Your application connects to myaccount.blob.core.windows.net; without a private zone linked to the VNet, that name still resolves to the public address and your traffic still leaves through the public path — while the portal shows a healthy private endpoint. Link the zone, then verify with nslookup from inside the VNet. It should return a 10.x address.
Nothing in the application changes when you adopt Private Link. The SDK still connects to myaccount.blob.core.windows.net, because that is the name in the connection string and in every Microsoft code sample. The whole mechanism works by changing what that name resolves to, which is why it is transparent to the app and opaque during debugging.
Azure's VNet resolver checks the private zone privatelink.blob.core.windows.net, which is linked to this VNet, and returns 10.1.36.4 — the private endpoint's address in snet-privatelink. A caller outside the VNet asking the same question gets the public address, because the private zone is not linked to their network. Split-horizon DNS, doing exactly its job.
10.1.36.4 is inside the VNet's own range, so Azure's system route sends it there with no UDR needed — this traffic never approaches the internet path at all. The NSG on snet-app must permit outbound to that address on 443; if you have replaced the permissive defaults with explicit rules, remember the private endpoint subnet is a destination you now have to allow.
On the storage account, set public network access to disabled. Your VNet keeps working through the private endpoint; your laptop, a partner, and every internet scanner get nothing. This is the step that converts "traffic prefers the private path" into "the public path does not exist", and it is the one that a compliance reviewer actually asks about.
The portal shows a healthy private endpoint with a green tick. Connectivity works. Everyone signs it off. But the name still resolves publicly, so every byte is leaving through the public path — and the moment someone disables public access, the application breaks with a connection error that points at the storage account rather than at DNS.
The three-second check: from a VM inside the VNet, nslookup the service hostname. A 10.x answer means the private path is real. A public address means you have a private endpoint that is decorative.
| Azure object | What it is | AWS | GCP |
|---|---|---|---|
| VNet | Regional private network; resource groups are not boundaries | VPCalso regional | VPC networkglobal — the structural difference |
| Subnet | Spans the region's zones; 5 reserved; resizable; some need exact names | Subnetone AZ each; 5 reserved; not resizable | Subnetregional; 4 reserved; expandable in place |
| NSG | Stateful, allow AND deny, priority 100–4096, NIC or subnet | Security group + NACLthe two halves split across two objects | Firewall rules / policiesnetwork-scoped, priority 0–65535 |
| ASG | Named group of NICs usable as a rule source | Security group referencethe group is both the policy and the label | Network tag / service accountservice accounts are the IAM-governed way |
| Service tag | Self-updating name for a service's ranges | Managed prefix list | Named ranges / netblocks |
| UDR | Override for Azure's automatic system routes | Route table entryAWS has no automatic routes to override — you write them all | Custom route |
| Azure Firewall | Managed L3–L7 firewall with FQDN filtering, ~$1.25/hr | Network Firewall | Cloud NGFW |
| Front Door | Global anycast L7 + CDN + WAF; Premium adds Private Link origin | CloudFront (+ Global Accelerator) | Global external Application LBGCP's default LB is already global — no separate product |
| Application Gateway v2 | Regional L7 + WAF, dedicated subnet | ALB | Regional external Application LB |
| Traffic Manager | DNS-level global steering, never in the data path | Route 53 routing policiessame job, built into the DNS service | Cloud DNS routing policies |
| Private endpoint | Private IP for a managed service; needs a private DNS zone | Interface endpoint | Private Service Connect |
| ExpressRoute | Private circuit; unlimited-data plan bundles egress | Direct Connect | Cloud Interconnect |
| Virtual WAN / VNet Manager | Managed hub-of-hubs; centrally pushed rules that outrank NSGs | Transit Gateway | NCC / hierarchical policies |
The estate. One hub, three spokes. Each spoke sends 500 GB/month to shared services in the hub and 200 GB/month to the internet. On-premises connectivity via VPN Gateway. All in a Zone 1 region, list price.
| Azure Firewall (Standard) — 730 hrs × ~$1.25 | $912.50 |
| Firewall data processing: 2,100 GB × ~$0.016 | $33.60 |
| VPN Gateway (VpnGw1) — 730 hrs × ~$0.19 | $138.70 |
| NAT Gateway in the hub — 730 hrs × $0.045 | $32.85 |
| NAT data processing: 600 GB internet-bound × $0.045 | $27.00 |
| Peering — charged in BOTH directions: 1,500 GB out @ $0.01 + 1,500 GB in @ $0.01 | $30.00 |
| Internet egress: 600 GB − 100 GB free = 500 GB @ $0.087 | $43.50 |
| Standard public IPs × 3 — 730 hrs × ~$0.0036 each | $7.88 |
| Monthly network cost | ~$1,226 |
AWS charges $0.01/GB for cross-AZ peering traffic. Azure charges roughly $0.01/GB on egress from the sending VNet and $0.01/GB on ingress to the receiving VNet — about $0.02/GB round trip. Now consider a spoke-to-spoke conversation routed through a hub firewall: spoke A → hub is metered twice, hub → spoke B is metered twice, and the firewall charges its own per-GB processing in between. A single gigabyte between two spokes can be billed four times on peering plus once on the firewall. For a chatty retrieval-augmented workload pulling embeddings across spokes, put the vector store in the same VNet as the thing that queries it, or accept a line item that looks nothing like the diagram.
Same three-tier design as S1, adapted for Azure's regional subnets. Because subnets span zones, the plan is simpler — one subnet per tier, not one per tier per zone — but the named-subnet requirements consume space you must remember to allocate:
| snet-appgw 10.1.0.0/24 — dedicated, v2 autoscaling needs room | 251 usable |
| snet-inference 10.1.16.0/20 — GPU nodes / AKS | 4,091 usable |
| snet-data 10.1.32.0/22 | 1,019 usable |
| snet-privatelink 10.1.36.0/26 — one address per private endpoint | 59 usable |
| AzureBastionSubnet 10.1.36.64/26 — exact name required | 59 usable |
| (hub only) GatewaySubnet /27 AzureFirewallSubnet /26 — exact names required | — |
| Reserved for growth · 10.1.64.0/18 | 16,384 |
The Cloud Adoption Framework's landing-zone guidance assumes exactly Fig 3.1: a platform-owned hub carrying shared connectivity and security services, workload spokes peered to it, and centrally-managed policy the workload teams cannot override. The Well-Architected Framework's security pillar adds the segmentation position — treat the network as one layer of defence among several, and prefer private endpoints over public ones for platform services.
The default-outbound-access retirement is not just a portal change. Once the API default flips for new VNets after 31 March 2026, infrastructure code that creates a VNet and VMs without an explicit egress resource will produce machines that cannot reach the internet — and the failure appears at runtime as package installs and image pulls timing out, not at deploy time as an error. If you maintain Azure modules, add an explicit NAT gateway or outbound rule to them now.
1 · AKS networking has two modes and the choice is an address-planning decision. Azure CNI (flat) gives every pod a real VNet address — pods are first-class network citizens with NSGs and flow logs, and you burn addresses at nodes × max-pods, exactly like the EKS problem in S2. Azure CNI Overlay puts pods on a separate overlay range that does not consume VNet space, so nodes take VNet addresses and pods do not. Overlay is the right default for GPU clusters, where node counts are small but pod density and future growth are unpredictable. Choose flat only when something outside the cluster must address a pod directly.
2 · Azure Machine Learning managed online endpoints are the Vertex AI equivalent, and they have the same public-by-default posture. You deploy a model, you get an HTTPS endpoint on a Microsoft domain protected by a key or by Entra ID. To make it private you use a managed VNet for the workspace plus a private endpoint for the scoring endpoint — the same rung-2 move as everywhere else. If your evaluation criterion is "can prediction traffic be kept off the public internet", the answer on Azure is yes, via Private Link, and it is worth being able to say so.
3 · Front Door Premium with Private Link to origin is Azure's best answer to "expose a model endpoint globally without a public backend". Front Door terminates TLS at the edge, applies the WAF and rate limits, then reaches your origin over a private link. The origin has no public address. For a global inference API this is a genuinely strong pattern and it has no exact one-product equivalent on AWS, where you would compose CloudFront with a VPC origin or an internal ALB.
About 40 minutes. Do not deploy Azure Firewall — at ~$1.25/hour it is the single most expensive thing you can accidentally leave running in this entire stage. Use NSGs instead.
10.1.0.0/16 with subnets snet-app (10.1.16.0/24) and snet-privatelink (10.1.36.0/26).snet-app. Look at the three default rules first — find AllowVnetInBound at 65000 and register that intra-VNet traffic is open by default. Add a deny rule at priority 4000 for everything from Internet, then an allow at 100 for what you actually want.B1s VM (free-tier eligible) in snet-app, no public IP. Reach it with Azure Bastion if you have the /26, or simply use the serial console — free, and it makes the point that you do not need an inbound path to administer a machine.snet-privatelink, accepting the prompt to create the private DNS zone. From the VM, nslookup yourstorage.blob.core.windows.net — it must return a 10.1.36.x address. If it returns a public one, the DNS zone is not linked, which is exactly the failure mode described above.AWS and Azure both scope the private network to a region, so multi-region means multiple networks and something to join them. Google made the opposite choice, and one VPC spans every region on Earth. Session 4 works through what that actually changes — in routing, in firewall design, in how many networks you need — and finishes by putting your Vertex and GKE inference endpoints behind a private path.
On AWS and Azure the private network stops at the region boundary. On Google Cloud it does not: one VPC spans every region on Earth, and a machine in Tokyo can reach a machine in São Paulo by private address with no peering, no gateway, and no tunnel. That single choice changes routing, firewall design, how many networks you create, and how you think about multi-region. It is worth dwelling on, because it is the one place where "same idea, three dialects" genuinely breaks down.
GCP guidance (GCP Networking): VPC and subnets, auto vs custom mode, firewall rules, VPC peering, Cloud VPN, Cloud Interconnect, Cloud CDN. Three things the session plan expected from that section are not in it: Shared VPC is listed in the section's own structure and then never taught; Cloud DNS does not appear at all; and Cloud Load Balancing lives in §2 (Autoscaling and Load Balancing), not §4. All three are covered below from current Google documentation and marked at the point of use. The load-balancer product names in §2 also predate Google's renaming — see the note in section 6.
You already deployed to Vertex AI with T4s, spot instances and autoscaling 1–3, and it worked without you touching a network. That is the GCP experience: the defaults are good enough that networking stays invisible until the day someone asks "can prediction traffic be kept off the public internet?" or "why is our egress bill $1,400?" This session is about being able to answer both.
AWS and Azure give you a separate company per city, and if the London office needs to call Tokyo you set up an inter-company agreement. Google gives you one company with offices everywhere and a single internal directory: extension 4471 works from any desk in any building. You still pay for the long-distance call, but you never had to negotiate the ability to make it.
The flip side is exactly what you would expect from one company: a policy written at head office applies in every building simultaneously. That is a feature when the policy is right and a global incident when it is not.
A new project comes with a default network in auto mode: it creates a /20 subnet in every region automatically, from a predictable range starting at 10.128.0.0/9. Convenient for a first VM, and wrong for anything real — the ranges are predictable, which means two auto-mode networks in different projects have overlapping subnets and can never be peered.
# The one structural decision, made at creation and hard to undo
auto mode → a /20 in every region, from 10.128.0.0/9, chosen for you
predictable = collides with every other auto-mode network
custom mode → no subnets at all until you create them, with your own ranges
the only correct choice for anything that might ever peerAlways create custom-mode networks. You can convert auto to custom later; you cannot convert back, and you cannot undo an overlap.
Two GCP-specific address facts worth having:
/24 gives you 252 usable rather than AWS's 251.There is also Private Google Access: a per-subnet flag that lets instances with no external address reach Google APIs and services — Cloud Storage, BigQuery, Artifact Registry — over internal paths. It is free, it is one checkbox, and it is the GCP counterpart to an AWS S3 gateway endpoint. Turn it on for every private subnet; there is no reason not to.
Isolation now needs deliberate effort. On AWS, "prod and dev are separate VPCs in separate accounts" is the path of least resistance. On GCP the path of least resistance is one network with subnets for everything, and separation has to be a decision — separate networks per environment, or Shared VPC with per-project subnet grants, or firewall policies scoped by service account. The global model is more convenient and slightly more dangerous, and the mitigation is organisational rather than technical.
Quotas are global too. Routes, firewall rules and peerings are counted per network, and a single network serving every region hits those ceilings faster than five regional ones would. Worth checking before you consolidate.
"Global" is not "free" and it is not "one failure domain". Inter-region traffic is metered per GB, and a regional outage still takes out the subnet in that region. What you avoid is the connectivity plumbing, not the physics.
AWS and Azure attach the guest list to each door. GCP posts one set of house rules on the wall and says which rooms each rule applies to. There is no per-instance firewall object at all — you write "allow 8000 to anything labelled inference", and every machine wearing that label is covered, in every region, immediately.
Every network has two implied rules at the lowest possible priority (65535) that you cannot delete: allow all egress and deny all ingress. Everything you write sits above them.
PRIORITY DIRECTION TARGET (by service account) SOURCE ACTION
1000 ingress sa-inference@… sa-frontend@… allow tcp:8000
1000 ingress sa-data@… sa-inference@… allow tcp:5432
2000 egress sa-data@… 0.0.0.0/0 deny all (no internet for the DB)
-- implied, undeletable --
65535 egress all 0.0.0.0/0 allow all
65535 ingress all 0.0.0.0/0 deny allLower number wins, and evaluation stops at the first match — same as an Azure NSG. The interesting choice is the target mechanism:
allow-ssh to their VM and grant themselves the rule. They are convenient and they are not a security boundary.iam.serviceAccountUser on it, so the firewall rule inherits IAM's access control. This is the setting where C2 and C4 physically join: the network rule is scoped by the identity the workload runs as.Prefer service accounts for anything that matters. It is more setup and it is the difference between a policy and a suggestion.
Google now organises firewalls as policies rather than loose per-network rules, and all new Cloud NGFW capability lands on the policy constructs. Three scopes:
goto_next to delegate downward. Lower levels cannot override a higher-level decision, which is how you enforce "no SSH from the internet, anywhere, ever" across every project including ones created next year.Default evaluation order is hierarchical → regional system → classic VPC rules → global network policy → regional network policy → implied. That order is itself configurable per network, which is a detail you will be glad you know the first time a rule seems to be ignored.
The practical guidance: use a hierarchical policy for organisation-wide prohibitions, network policies with service-account targets for workload segmentation, and treat classic tag-based rules as legacy you are migrating away from.
This is the GCP answer to a problem AWS solves with Transit Gateway plus RAM sharing and Azure solves with hub-and-spoke. Instead of many networks joined together, there is one network, administered centrally, consumed by many projects. A network team holds the host project and grants each application team the specific subnets they may use. The application teams get isolated projects, their own IAM, and their own bill — but they cannot create a subnet, edit a route, or write a firewall rule.
Central control of addressing and policy with decentralised workload ownership. Internal traffic between service projects is just VPC-internal traffic — no peering, no gateway, no per-GB peering charge. One place to audit every firewall rule in the estate.
A network team becomes a dependency for every team's subnet request. Quotas are shared, so one noisy project can exhaust addresses for everyone. And the IAM model needs care: compute.networkUser on a specific subnet, not on the whole host project, or you have granted more than you meant.
Shared VPC versus peering, on GCP: peering exists and works the same way it does elsewhere — non-transitive, no overlapping ranges — but on GCP you reach for it far less often, because the two problems it usually solves (multi-region and multi-team) are already solved by the global network and by Shared VPC respectively. If you find yourself designing a peering mesh on GCP, check first whether a Shared VPC would remove the need.
The global model tempts you toward one network for everything. Three situations where you should deliberately create more than one:
The middle path most large GCP estates settle on: one Shared VPC per environment, each global and consumed by many service projects, with hierarchical firewall policies at the folder level enforcing the rules that must hold everywhere. You keep the multi-region simplicity within an environment and keep a hard wall between environments.
Cloud NAT deserves its own note because it is architecturally unlike its AWS and Azure counterparts. It is not an appliance in your subnet. There is no instance, no interface, no bandwidth chokepoint to size, and nothing sitting in a path that could fail. It is configuration applied to the Cloud Router in a region, and the translation happens in the software-defined network itself, distributed across the fabric.
No single-appliance throughput ceiling; capacity scales with your instances rather than with a gateway you provisioned. No per-AZ deployment decision, because it is regional by nature. Nothing to fail over.
You still pay. Roughly $0.044 per hour per VM using the gateway plus about $0.045/GB processed — so the per-gigabyte economics of pulling model weights through NAT are essentially the same as AWS. And you still pay egress on top if the destination is outside Google.
Which means the same fix applies: turn on Private Google Access and pull weights from Cloud Storage over internal paths instead of through NAT. Free, one flag, and it removes the largest per-GB line item from a GPU node's lifecycle.
| Option | What it is | Throughput / SLA | Reach for it when |
|---|---|---|---|
| HA VPN | IPsec over the internet with two interfaces and two external addresses, using BGP via Cloud Router | ~3 Gbps per tunnel; 99.99% SLA when configured with two interfaces | The default on-ramp for hybrid. Also the standard backup path for an Interconnect. |
| Classic VPN | The single-interface predecessor | 99.9%, legacy | Don't. HA VPN is the current product; classic exists for migration only. |
| Dedicated Interconnect | A physical 10 or 100 Gbps circuit into a Google colocation facility | Up to 99.99% with the right redundancy topology | Sustained multi-gigabit, predictable latency, or a compliance rule against internet transit. |
| Partner Interconnect | A circuit through a service provider, from 50 Mbps upward | 99.9–99.99% depending on topology | You need private connectivity but are not in a Google colo facility, or you need less than 10 Gbps. |
| Cross-Cloud Interconnect | A Google-provisioned circuit directly to another cloud provider | 10 or 100 Gbps | Genuine multi-cloud data paths — the AWS-to-GCP link that does not traverse the internet. Directly relevant if your training runs on one cloud and serving on another. |
Cloud Router is the piece that makes all of these work: it speaks BGP (Border Gateway Protocol, the routing protocol of the internet) with your on-premises router and exchanges routes dynamically. Add a subnet in GCP and your datacenter learns about it automatically; add a range on-premises and GCP learns it. Static routing across a hybrid boundary is a maintenance burden you do not need to accept.
Google renamed the entire load-balancing family. If §2 says "HTTP(S) Load Balancing", "SSL Proxy" or "TCP/UDP Network Load Balancing", it is describing today's products under yesterday's names. The current taxonomy is two families, each in external/internal and global/regional variants:
The structural difference from AWS and Azure follows directly from the global VPC: the global external Application Load Balancer has one anycast IP address announced worldwide. A client in Sydney and a client in Dublin connect to the same address, enter Google's network at their nearest edge, and are carried over Google's backbone to whichever region has healthy capacity. There is no separate global product to bolt on — no CloudFront, no Front Door, no Traffic Manager — because the load balancer already is that thing.
Practical consequences: no DNS-based failover latency (the address never changes, only the routing behind it does), automatic cross-region overflow when a region saturates, and Cloud Armor available in front of all external load balancers for DDoS protection and WAF rules.
Both taught from documentation — neither appears in §4.
Private Service Connect (PSC) is rung 2 of the connectivity ladder on GCP, and it is the mechanism that matters most for your work. It creates an endpoint with an internal address in your VPC that forwards to a service outside it — a Google API, a managed service, a third party's service, or your own Vertex AI inference endpoint. The service producer publishes a service attachment; you create a forwarding rule pointing at it. Traffic never touches the public internet, and on-premises callers reach it over VPN or Interconnect like any other internal address.
You deploy your model to a dedicated endpoint with Private Service Connect enabled. Vertex creates a service attachment — a published handle for the service, living in Google's producer network, not yours. It has no address in your VPC and nothing can reach it yet.
In your VPC you create a Private Service Connect endpoint: a forwarding rule with an internal address you choose from one of your subnets, targeting that service attachment. The service is now an IP address inside your own network.
Google's guidance: when several prediction endpoints share one service attachment, create one PSC endpoint and reuse it, rather than one per prediction endpoint.
Raw addresses in client code age badly. A private zone visible only to this VPC maps inference.internal to the endpoint address, so your application config looks the same as it did when the endpoint was public. The zone does not exist outside the networks you link it to.
Your client opens TLS to the internal address; the forwarding rule carries the request across the service-attachment boundary to the model. No external address is involved at either end, and the traffic stays on Google's network the whole way. Firewall rules targeting the caller's service account decide which workloads may use the endpoint — network path and workload identity enforcing the same policy.
Because the endpoint is an ordinary internal address, anything already connected to the VPC over HA VPN or Interconnect can reach it too — you advertise the range over BGP through Cloud Router and a datacenter client dials the same name. That is the payoff of rung 2: one private address serves in-cloud and on-premises callers identically, with no second integration to build.
Cloud DNS provides public and private managed zones with a 100% availability SLA. The pattern to know: a private zone visible only to specified VPC networks, so inference.internal resolves to a PSC endpoint's internal address inside your network and does not exist outside it. Cloud DNS also supports routing policies (weighted, geolocation, failover) for the cases the global load balancer does not already cover.
| GCP object | What it is | AWS | Azure |
|---|---|---|---|
| VPC network | Global. Subnets are regional; routes and firewall rules are network-wide | VPCregional; peering needed between regions | VNetregional; peering needed between regions |
| Subnet | Regional, spans all zones, 4 reserved, expandable in place | Subnetone AZ, 5 reserved, fixed size | Subnetregional, 5 reserved, resizable |
| Firewall rule / policy | Network-scoped; targets by tag or service account; hierarchical policies at org and folder | Security groupattached to the interface | NSGattached to NIC or subnet |
| Service account as target | Firewall scoped by workload identity — IAM-governed | Security group referencenot IAM-governed | ASGnot IAM-governed |
| Shared VPC | One host project's network consumed by many service projects | RAM-shared subnets / TGW | Hub-and-spoke + VNet Manager |
| Cloud NAT | Distributed SDN translation — no appliance, no chokepoint | NAT Gatewaya real per-AZ resource | NAT Gatewaya real resource |
| Private Google Access | Subnet flag; internal path to Google APIs; free | Gateway VPC endpointfree, S3/DynamoDB only | Service endpointfree |
| Private Service Connect | Internal address for a service — including your own Vertex endpoint | PrivateLink | Private Link |
| Global external Application LB | One anycast IP worldwide, L7, Cloud Armor | CloudFront + ALBtwo products to get the same shape | Front Door + App Gatewaytwo products |
| Passthrough Network LB | L4, no proxy, direct server return | NLB | Load Balancer (Standard) |
| HA VPN | Two interfaces, BGP, 99.99% SLA | Site-to-Site VPN | VPN Gateway |
| Cloud Interconnect | Dedicated, Partner, and Cross-Cloud to another provider | Direct Connect | ExpressRoute |
| Cloud DNS | Public and private zones, 100% availability SLA | Route 53 | Azure DNS + Private DNS |
This is the calculation most directly relevant to you, and the one that most reliably goes wrong. A GKE cluster in VPC-native mode gives every pod a real routable address from a secondary range on the node subnet — the "alias IP" model. Three ranges are consumed, not one.
| Primary range — node addresses, one per node | 1 per node |
Secondary range: pods — a whole /24 block is allocated per node when max-pods-per-node is 110 (the default) | 256 per node |
| Secondary range: services — one per ClusterIP service | 1 per service |
| The second row is the trap: 110 pods requires 256 addresses, because GKE rounds up to the next power of two and allocates the block whether or not the node is full. | |
Today: 6 g2-standard-8 nodes with L4 GPUs, roughly 4 pods per node (one vLLM replica plus sidecars). Plausibly in a year: 60 nodes. Plan the ranges.
Node range /24 — 252 usable | fine |
Pod range /20 (4,096) with max-pods 110 → 256 per node | 16 nodes MAX |
Service range /24 — 252 services | fine |
| Cluster stops scaling at 16 nodes, with a node quota that says 60 | hard ceiling |
Set --max-pods-per-node=16 — GKE allocates a /27 (32 addresses) per node instead of a /24 | 8× denser |
Node range /24 — 252 usable | 252 nodes |
Pod range /20 (4,096) ÷ 32 per node | 128 nodes |
Service range /22 — 1,020 services | generous |
Same /20, 8× the headroom, for one flag at cluster creation | 128 nodes |
Two more notes for the same subnet: the pod range cannot be expanded once the cluster exists (unlike a primary subnet range, which can), and internal Application Load Balancers need their own proxy-only subnet per region — a /26 shared by every internal L7 load balancer in that region. Allocate it in the plan; discovering it during a deployment is an irritating detour.
Scenario. A 40 GB model, 4 nodes, recycling twice daily (240 pulls/month = 9,600 GB), plus 500 GB/month of inference responses to internet clients. us-central1, list price.
| A Weights pulled from Cloud Storage through Cloud NAT 9,600 GB × ~$0.045 NAT processing + gateway hours | ~$464 |
| B Weights pulled with Private Google Access on, same-region bucket internal path to Google APIs, no NAT, no egress | $0.00 |
| Inference responses, 500 GB internet egress, Premium Tier (the default) @ $0.12/GB | $60.00 |
| Same 500 GB on Standard Tier @ ~$0.085/GB — public internet transit instead of Google's backbone | $42.50 |
| Design A total vs Design B total (Premium egress in both) | ~$524 → $60 |
Worth having in mind when comparing clouds: at 10 TB/month of internet egress the base cost is roughly $913 on AWS, $882 on Azure, and $1,137 on GCP Premium — GCP is the most expensive at list on Premium and the cheapest of the three on Standard. If your workload moves bulk data out, the tier setting is a first-class architecture decision, not a footnote.
Google's own Cloud Threat Horizons report for H1 2026 records that through 2025 its engineers observed threat actors exploiting misconfigured applications and deploying secondary payloads — cryptominers most often — into Compute Engine and GKE instances in under one hour from instance creation. Initial access via misconfiguration fell from 29.4% to 21% of cases across the year as automated guardrails improved, which is encouraging and still one in five. The operational implication for a GPU cluster is blunt: the window between "I'll open this port temporarily" and "someone is mining on my L4" is measured in minutes.
Vertex AI Prediction Dedicated Endpoints are now the standard serving method for self-deployed models from Model Garden, and they come in public and private (PSC) flavours. Google publishes an end-to-end codelab that takes Gemma 3 from Model Garden, deploys it to a private dedicated endpoint, and reaches it from a VPC through a Private Service Connect endpoint targeting the auto-generated service attachment — with TLS on the private path. That is precisely your Gemma-on-vLLM workload, on the managed path, with no public endpoint.
Codelab: TLS + private endpoint for a Model Garden model Vertex AI PSC endpoints — docs
Google's documentation is explicit that Private Service Connect backends are not supported with Vertex AI online prediction endpoints — the supported pattern is a PSC endpoint (a forwarding rule to the service attachment), not a PSC backend behind a load balancer. It also recommends creating one PSC endpoint per service attachment and reusing it when several prediction endpoints share that attachment, rather than one endpoint each. Both details are the kind of thing that turns a two-hour task into a two-day one if you learn them by discovery.
Accessing Vertex AI through Private Service Connect endpoints
1 · The L4 VM running the vLLM tutorial stack. Check it today. If it has an external IP and a firewall rule allowing tcp:8000 from 0.0.0.0/0, it is on the same list as the 175,000 exposed servers from S1, and the Threat Horizons finding above says the discovery window is under an hour. The fix takes five minutes: delete the external address, enable Private Google Access on the subnet so package and weight pulls still work, add Cloud NAT if it needs general outbound, and reach the machine with gcloud compute ssh --tunnel-through-iap — Identity-Aware Proxy tunnels SSH through Google's front end with no inbound rule and no public address at all, gated by IAM. It is the GCP counterpart of AWS Session Manager and it is free.
2 · The Vertex AI deployment script. Your endpoint is public by default — reachable from the internet, protected by IAM rather than by topology. That is layer one doing its job, and it is rung 1 of the ladder. To move it to rung 2: deploy to a dedicated endpoint with Private Service Connect enabled, take the service attachment Vertex generates, and create a forwarding rule for it in your VPC. Add a Cloud DNS private zone so your client code keeps using a name. Your autoscaling 1–3 on spot T4s does not change at all — this is purely a networking change in front of the same compute.
3 · The GKE cluster, when you build it. Set --max-pods-per-node at creation (the arithmetic above), turn on Private Google Access, make it a private cluster so nodes have no external addresses, and expose the model service through a Gateway or Ingress that provisions a global external Application Load Balancer with Cloud Armor in front. Scope the node pool's service account tightly and use it as the target of your firewall rules — that is where C2's identity work and this stage's network work become one control instead of two.
About 40 minutes. Use the free tier and the smallest machine: an e2-micro in a free-tier-eligible US region. No GPU. Run any small HTTP server as a stand-in for vLLM — the network topology is what you are proving, and it is identical whether the process behind it is python -m http.server or a 70B model.
10.50.0.0/24, with Private Google Access enabled. Notice there is nothing regional about the network itself.sa-inference with no roles beyond the default. This is going to be a firewall target, not just an identity.e2-micro in that subnet, with no external IP address, running as sa-inference.gcloud compute ssh vm-name --tunnel-through-iap. You will need one firewall rule allowing tcp:22 from 35.235.240.0/20 — Google's IAP range, and a good example of a narrow source range doing real work. No public address anywhere.curl ifconfig.me from the VM and watch it succeed through a gateway that is not an appliance. Then turn NAT off and gsutil ls a public bucket — it still works, because Private Google Access is on. That contrast is the lesson.tcp:8000 to targets running as sa-inference, from source range 10.50.0.0/24 only. Start a server on 8000 and confirm it is reachable from inside and invisible from outside.e2-micro in it. Ping the first one by internal address. It works, with no peering and no configuration. That is Fig 4.1, on your own account, in ninety seconds.Carve a /16 into tiers and zones with reserved headroom, do the usable-address arithmetic for each cloud's reservations, and allocate from a registry so a future peering cannot collide. Plus the GKE secondary-range calculation that catches most people once.
L4 or L7, regional or global, public or private — on all three clouds, with the vendor-specific traps: Azure's four products, GCP's renamed family, and the timeout and health-check settings that specifically break streamed generation.
Security groups referencing groups, NSGs with ASGs and service tags, GCP firewall policies targeting service accounts. The stateful/stateless distinction and how to spot the ephemeral-port bug in a flow log in three seconds.
NAT hourly plus per-GB plus egress, the double-metered byte, Azure peering billed in both directions, and the single highest-leverage move in cloud networking: a free private path to object storage instead of a metered one.
And the translation habit that carries into the exams: every question is one of eight primitives wearing one of three vocabularies, with exactly one genuine structural difference — GCP's global VPC — that you can now explain in a sentence and design around.
C4 gave you the second layer of defence in depth and the topology the rest of the stack sits on. The natural next layers are data (where storage services attach to these private paths, and where the encryption and residency decisions land) and observability (flow logs were a preview — the full picture joins them to metrics and traces). Both build directly on what you can now draw.