The Whole Platform, End to End
Every pattern from the series, assembled into one picture: environments, identity & secrets, agent runtime, the LLM gateway's layered edge defense, and the observability fork.
ENVIRONMENTS
DEV
Control Plane
Model Registry
Budget Layer (soft allocation)
STAGING
Control Plane
Model Registry
Budget Layer (soft allocation)
PROD
Control Plane
Model Registry
Budget Layer (hard cap enforced)
IDENTITY & SECRETS LAYER
Federated Identity (workload / on-behalf-of)
Secrets Store (Key Vault / Secret Manager / Vault)
AGENT RUNTIME ARCHITECTURE
Agent Identity
(config, created once,
no live infra)
decoupled
Agent Instance
(deployable runtime,
redeployed freely)
Memory
tiered: ephemeral session
scratch vs. durable, shareable
Tool Servers
own resource type,
not "a mode" of an agent
agents call the gateway like any client
LLM GATEWAY — LAYERED EDGE DEFENSE
Client
WAF / Bot Mgmt
(network layer)
Payload Guardrails
(PII, injection, cost limits)
LLM Gateway
Model
budget check at request time
fork (non-blocking)
Trace Collector
buffers & retries independently
Governed Analytics Store
audit, cost, model behavior review
READING THE PICTURE
1. Environments (green) isolate blast radius — each holds its own control plane, model registry, and budget enforcement.
2. Identity & Secrets (red) underpins every environment and every agent — nothing authenticates with a long-lived static credential.
3. Agent Runtime (purple) splits identity from instance so redeploys never destroy configuration or accumulated memory.
4. The Gateway path (green/blue/orange) layers network-edge and payload-aware defense before any request reaches the model.
5. The Observability fork (blue/orange, bottom) copies telemetry sideways — a slow or down analytics store never touches live traffic.
This is one way to assemble these pieces, not the only one — see the capstone post for where we're still not sure we got the boundaries right.