4. Securing Agentic AI: Authentication, Authorization, and PII

🔍 The Gap That Wasn’t Visible

After the 57-gap audit described in Article 3, one category of finding stood out as the most operationally dangerous: the authorization and privacy architecture looked correct from the outside and was almost entirely absent from the inside. The credit agent had OPA policy enforcement wired in. The orchestrator, the sharia agent, the risk agent, and the simulation agent had none. Every decision those four made was completely unguarded by policy.

The gap crept in quietly: credit was built first, OPA got wired in during that sprint, and every agent after it was scaffolded from a template that predated the OPA node. Four agents making regulated decisions — credit risk, Sharia compliance, portfolio risk — with no policy layer at all.

Sitting right alongside it, almost as an afterthought in the gap register, was a PII problem with the exact same shape. Presidio-backed PII masking was available as a tool to every agent. It was documented, tested, registered. Not a single agent graph called it.

None of this was obvious from reading the code. The credit agent’s graph had a check_authorization node, the architecture diagram showed policy enforcement, the PII tool existed in the registry, and the tests passed. The gaps were structural — they lived in what was absent, not in what was broken. That’s exactly the failure mode systematic audits exist to catch.

---

🛡️ The Three-Layer Security Model

Security in a multi-agent system isn’t a single control — it’s three layers, each answering a different question. Conflating them, or implementing one and assuming it covers the others, is the most common security mistake in agentic AI today.

• Layer 1 — Authentication: Who are you? Enforced at the HTTP boundary via API middleware and per-agent identity tokens through OpenBao.
• Layer 2 — Authorization: Are you allowed to do this? Enforced inside each agent graph with OPA — on every agent, not just the first one you build.
• Layer 3 — Least-privilege tool access: Are you limited to only the tools you need? Enforced at the MCP server with a per-agent tool allowlist.

🔐 Authentication at the Boundary

Our MCP server — the shared FastAPI tool layer exposing credit scoring, vector search, Sharia validation, and more to all five agents — had no auth middleware at all. Any process that could reach the port could call any endpoint. The fix: a FastAPI dependency that rejects any request without a valid API key before any tool logic runs. Boundary enforcement, not business-logic enforcement, so it can’t be bypassed by a later refactor.

The shared MCP_API_KEY problem took more work. Each agent now authenticates to OpenBao (the MPL-2.0 fork of HashiCorp Vault we moved to during the license audit) using its own AppRole — a machine identity with a unique role ID and secret ID — and the MCP server validates the resulting token on every request. Before: an audit log entry that says “an agent called credit_score.” After: “agent-credit called credit_score at 14:32:07, token valid.” Per-agent identity isn’t just a security control, it’s an accountability mechanism — when something goes wrong, the audit trail needs to name the actor.

One more thing we caught: the OpenAPI docs endpoint (/docs) was unauthenticated on all eight services. Handy in development, but in production it’s a free enumeration surface — every tool name and route, no credential required. We disabled it via ENVIRONMENT=production.

⚖️ Authorization at Every Decision Point

Authentication answers “who are you?” Authorization answers “are you allowed to do this specific thing, right now?” Treating API key validation as access control is the most common mix-up in AI security writing, and it’s wrong. We enforce authorization with OPA: a decision request — “can agent-credit approve this amount for this customer tier?” — gets evaluated against a Rego policy bundle and returns an allow/deny with a reason. Deterministic, auditable, and the policy logic lives in version-controlled files instead of agent code.

Only the credit agent had a check_authorization node. The sharia, risk, and simulation agents were issuing verdicts, scores, and projections with no OPA check at all — and neither was the orchestrator, which decides which agents to invoke and in what order. That last gap matters more than it sounds: an attacker who manipulates routing, say through prompt injection, can make the system skip the credit agent’s checks entirely. Adding the node to all five graphs closed it. The node packages agent identity, requested action, and context into an OPA input, calls the decision endpoint, and either proceeds or terminates the graph on denial — before any business logic runs.

OPA runs as a sidecar polling a MinIO bundle store every thirty seconds — we started at five, until a synthetic load test showed the polling itself was the bottleneck across five agents at once. A policy change takes effect everywhere within that window, no restart required. That’s the difference between policy baked into Python config and policy actually served as code.

🔑 Least-Privilege Tool Access

Even authenticated and authorized, an agent could still call any tool on the MCP server — nothing stopped the simulation agent from calling credit_score. This maps directly to two MITRE ATLAS entries: AML.T0056 (unsafe tool use) and AML.T0060 (tool confusion), where a manipulated agent is induced to call a tool outside its intended scope. Per-agent tool allowlists in Postgres, enforced at the MCP server on every invocation, close it — an off-allowlist call is rejected with a named denial and an audit event.

Agent	Permitted MCP Tools	Rationale
agent-orchestrator	credit_score, vector_search, sharia_validate	Coordinates all three decision types
agent-credit	credit_score, vector_search	No Sharia tool access needed
agent-sharia	vector_search, sharia_validate	No credit data access needed
agent-risk	vector_search	No financial tools needed
agent-simulation	vector_search	Aggregate patterns only, no live customer data

The simulation agent has no route to credit_score — not because it’s blocked mid-request, but because it was never on the list. It can’t be talked into it, no matter how the prompt is crafted.

---

⏱️ Replay Attacks and Session Integrity

Replay attacks — capturing a valid request and resending it later — sound theoretical until you look at your own agent-to-agent traffic. It’s a bigger attack surface than a single-service API and harder to monitor continuously.

Our A2A calls used a static X-Agent-Key header — same key every time, no timestamp, no nonce, no expiry. A key captured from a log or an exposed env file could replay any agent call indefinitely, and the receiving agent had no way to tell a legitimate call from a replayed one.

The fix is the standard pattern: every A2A call now carries an X-Request-Timestamp, validated within a ±5 minute window of the receiver’s clock. Too old, too new, or already-seen (tracked via a nonce in Valkey, keyed by minute) — all rejected. Five minutes is narrow enough to shrink a captured request’s usable window to almost nothing, while still tolerant of ordinary clock skew.

A related, easy-to-miss problem: the integrity of what agents read back from shared caches. Agents write session state to Valkey — decision context, near-miss counters for HITL escalation, cached OPA results. If that’s tamperable, an attacker with Valkey write access can steer agent behavior without ever touching the authentication boundary — this is MITRE ATLAS AML.T0058, memory manipulation. The near-miss counter is the sharpest example: reset it to zero and an agent that should be escalating to human review just keeps deciding autonomously, which in a banking system turns a security gap into a compliance failure.

The fix is a ValkeySigner wrapper: security-critical values get an HMAC-SHA256 signature on write, verified on every read, with a mismatch treated as an incident rather than ignored. We apply it selectively — near-miss counters and cached OPA results, not routine cache lookups — because encryption and HMAC solve different problems, and here we only needed to know if a value had been altered, not hide it from anyone.

---

🗺️ PII: Three Threat Vectors, Three Controls

PII masking was available — every agent had access to a Presidio-backed redaction tool — and not one agent graph called it. That wasn’t carelessness so much as a mistake that’s easy to make in a system with five independent graphs evolving on their own schedules: conflating “the capability exists” with “the capability is applied.” A masking call inside a graph node is one refactor away from silently disappearing; masking in middleware that intercepts every request before the graph starts can’t vanish without removing the middleware itself — which shows up in git history, not as a quiet omission.

Once we treated PII as a boundary problem rather than a tooling problem, three distinct attack surfaces fell out, each needing its own control:

• Input to the LLM: a customer note with a name or account number gets sent, unmasked, to a cloud LLM provider — who now has data the bank has lost control of. Fix: a FastAPI middleware on all five services intercepts every POST /tasks request before the LangGraph graph starts, runs free-text fields through Presidio, and forwards a masked body. The graph never sees the original text. As a second line of defense, a pii_guard flag on our shared call_llm() scans the fully assembled prompt — including anything vector search or the database adds later — and hard-fails on detection. It auto-disables for on-prem Ollama calls, since nothing leaves the machine there.
• Output back to consumers: the LLM’s narrative — a risk report, a decision explanation — can surface a name or account number it inferred from context, and an unescaping UI renders it, or worse, executes injected HTML/JS. Fix: every narrative field is passed through html.escape() before it leaves the service. We stopped trusting downstream consumers to do it right and did it once, at the boundary. Separately, we added an injection scanner for user-controlled text entering prompts, and wrapped vector search results in explicit delimiters (<fatwa_context>...</fatwa_context>) so the LLM treats them as retrieved context, not instructions.
• Lateral access between agents: the credit agent has legitimate access to income and DBR data; the Sharia agent doesn’t, but nothing stopped it from receiving those fields if called. This is invisible to conventional security tooling. Fix: a LateralMovementGuard middleware with an ALLOWED_CALLERS set per agent — an unlisted caller gets HTTP 403 and a LATERAL_MOVEMENT_DETECTED audit event in production (it logs-but-allows in development, so local testing isn’t blocked). The intended caller relationships — orchestrator calls credit/risk/simulation, credit calls sharia, nothing else calls anything — had existed only in documentation and team memory. Relying on convention isn’t a control; it’s an honor system that fails quietly.

---

🎭 Obfuscation-Aware Scanning

Our injection scanner worked fine against plain ASCII and fell apart against obfuscated variants — techniques catalogued under MITRE ATLAS AML.T0038.

• Homoglyphs: Cyrillic е (U+0435) looks identical to Latin e (U+0065) but doesn’t match a Latin regex.
• Zero-width characters: invisible Unicode characters inserted between keywords break pattern matching while the text looks clean.
• Encoding variants: Base64 or ROT13 an injection payload and it reads as noise to a regex scanner — but a capable model can still decode it.

The fix is a normalization pass before any pattern matching: unicodedata.normalize('NFKC') collapses homoglyphs, zero-width and BiDi characters get stripped by code point, and long substrings get a speculative Base64/ROT13 decode-then-scan. Existing regex patterns run unchanged after that. It’s a pre-processing layer, not a replacement — independently testable against a library of known bypass strings — and it reduces the bypass surface rather than eliminating it. Red-team probing, covered in Article 5, is what closes the rest.

---

📊 The Four-Output Standard

Across every one of these controls we kept hitting the same gap: the control fired, detected the event, and wrote a log entry nobody was watching. So we standardized: any security control that fires — PII intercepted, injection detected, lateral movement blocked — must produce four outputs at once.

• Structured log (Loki): machine-readable JSON — event type, agent ID, timestamp, context.
• Prometheus counter: a spike in pii_interceptions_total is visible on a dashboard without anyone querying logs.
• Audit event (Postgres → outbox → Redpanda): written in the same transaction as the detection, delivered durably to downstream compliance systems.
• Alert to the monitoring topic: the trigger that actually wakes someone at 2am.

---

📈 What Changed in Production Posture

Before: an unauthenticated MCP server, static A2A keys with no replay protection, OPA on one agent, any agent could call any tool, no inter-agent caller checks, PII masking available but unused, unsanitized narrative output, a homoglyph-bypassable injection scanner, and tamperable Valkey state. After:

• Every MCP tool call requires a valid per-agent OpenBao token, verified via introspection
• Every A2A call is timestamped, nonce-checked, and rejected outside a 5-minute window
• Every agent graph runs an OPA check before any business logic executes
• Each agent is limited to its own tool allowlist — cross-agent calls are rejected and logged
• Security-critical Valkey values are HMAC-signed and verified on read (AML.T0058 mitigated)
• PII masking runs as middleware before every graph, not as an optional in-graph tool call
• All LLM narrative output is HTML-escaped before leaving the service boundary
• Injection scanning includes NFKC normalization, zero-width stripping, and decode-then-scan
• Lateral movement between agents is technically blocked, not just documented as convention
• /docs is disabled in production across all eight services

---

---