TL;DR
Once you decide to build a multi-agent AI system, you face three engineering choices that will determine whether the platform is governable or just functional: what orchestration framework controls the agent graph, how you structure the codebase so five services don’t drift into five incompatible interpretations of the same standard, and which infrastructure components you can actually deploy in a regulated bank without creating compliance exposure. LangGraph won the orchestration decision because it puts deterministic gates before LLM narrative — not the other way around. A monorepo with a shared SDK is used because compliance logic that exists in only one service will eventually be missing from the service that needs it most (this needs to be evaluated for the actual production deployment purposes). Four components failed the license and sovereignty audit and had to be replaced. These are not implementation details. They are governance decisions. Getting them right before writing application logic is what separates a platform that can be audited from one that only looks like it can.
Previous: From Monolith to Multi-Agent — Why One AI Is Not Enough | Next: The 57-Gap Audit — What “Done” Actually Means in Production AI
In This Article
- Why LangGraph: The Framework That Puts Determinism First
- Graph Topology as a Compliance Artifact
- Typed State Schema and the Vendor-Neutral LLM Caller
- The Honest Limitations: What LangGraph Costs You
- The Five-Services-Five-Opinions Problem
- The Four-Area Directory Structure
- The Shared SDK as Governance Enforcement Layer
- Three-Layer Configuration Without Drift
- Service Boundary Decision Rules
- The Real Costs of Monorepo Architecture
- The License Problem Nobody Talks About
- Four Components That Had to Be Replaced
- The Replacement Decisions: What Won and Why
- Data Sovereignty as a Compliance Requirement
- What Stayed: The Parts That Passed the Audit
- Key Takeaways
What You Need to Know
- Basic understanding of graph theory (nodes, edges, directed graphs) — used to explain LangGraph’s execution model
- Familiarity with Python — code patterns referenced but not required to follow in detail
- General awareness of what a secrets manager and a vector database do — explained briefly when introduced
- Understanding of why regulated industries (banking, healthcare) have data residency requirements
The previous article in this series explained why five specialised agents are better than one general AI for regulated finance. This article explains how those agents are orchestrated, structured, and deployed.
🏛️ Why LangGraph: The Framework That Puts Determinism First
The orchestration framework decision is not primarily a developer experience decision. It is a governance decision. The framework you choose determines whether your agent workflows can be audited, whether their execution paths can be explained to a regulator, and whether the deterministic business rules that must never be overridden by an LLM are enforced structurally or just hoped for.
Before settling on LangGraph, I evaluated three alternatives. Each was ruled out for a specific reason.
- LangChain agents: The ReAct loop is LLM-driven — the agent decides what tool to call next by asking the model. In a regulated banking context, this is architecturally unacceptable. The LLM cannot decide whether a Sharia compliance check happens. That gate is mandatory. A framework where the LLM can reason its way around a mandatory gate is not a compliant framework.
- CrewAI: The role-based crew model is an intuitive mental model for multi-agent collaboration. But the execution order is determined at runtime by the framework’s own coordination logic. When a regulator asks “show me the exact execution path for decision X,” the answer cannot be “it depends on what the framework decided at runtime.”
- Raw FastAPI with manual orchestration: Fully auditable because the code is fully explicit. But the operational overhead of implementing retry logic, state management, conditional routing, and error recovery from scratch on five services is prohibitive. The framework exists for a reason.
LangGraph won because it models agent workflows as explicit directed graphs. Every node is a Python function. Every edge is a conditional routing decision expressed in code. The execution path for any given input is deterministic and inspectable. A regulator can ask for the execution path of a specific credit decision, and the answer is the graph — a specific sequence of named nodes with the state at each transition logged.
LangGraph is not magic. It is a graph execution engine with state management and streaming support. The value is not that it does something you couldn’t do without it — it is that it provides a standardised structure for doing it, which makes the system’s behaviour legible to anyone who reads the graph definition.
Back to top . Next: Graph Topology as Compliance
📋 Graph Topology as a Compliance Artifact
The most important architectural principle in the platform’s agent design is this: deterministic gates execute before LLM narrative generation. The LLM never decides whether a transaction is approved. The deterministic gate decides. The LLM explains the decision the gate already made.
This distinction matters because of what happens when the LLM gets it wrong. Language models can generate plausible-sounding explanations for incorrect conclusions. If the LLM is both the decision-maker and the narrator, a hallucination in the decision can be dressed in convincing language. If the LLM is only the narrator, and the decision was already made by a deterministic rule, a hallucination in the narrative does not change the outcome — it only affects the explanation. The platform detects narrative contradictions (where the LLM’s explanation conflicts with the gate’s decision) and escalates them to human review rather than passing them silently.
Consider the Sharia compliance agent’s graph. The node sequence is explicit:
check_authorization— OPA policy gate. Fails here if the agent is operating above its permitted autonomy tier or the request lacks required attributes.retrieve_fatwas— vector search against the Sharia fatwa collection. Returns the most similar scholarly rulings to the transaction structure.evaluate_compliance— deterministic compliance gate. Checks the transaction’s structure against Islamic finance rules.is_compliantis set here based on explicit logic, not LLM output.generate_sharia_reasoning— LLM call. Takes the deterministicis_compliantvalue and the retrieved fatwas, generates a narrative explanation citing the specific scholarly basis. The LLM cannot changeis_compliant.check_hitl_required— conditional routing. Novel structures (where no fatwa similarity exceeds a threshold) force human review regardless of theis_compliantvalue.finalize_decision— writes to the Sharia decision audit table, emits an audit event, returns the result.
The graph topology is itself a compliance document. The sequence of nodes shows a regulator exactly which gates a transaction passes through, in what order, and what data is available at each gate. When a Sharia Board member asks “why was this transaction approved despite having no close fatwa match?” the answer is in the graph: it was approved by the deterministic gate, then flagged as a novel structure, then escalated to the HITL queue — and the board review record shows what happened next. Every one of those steps is a named node in the graph, not an implicit consequence of LLM reasoning.
The governance principle embedded in graph topology: if you cannot draw the graph on a whiteboard and explain every node and edge to a regulator, the agent is not auditable. If the execution path depends on what the LLM decided to do next, it is not a governed workflow. It is an autonomous agent with a compliance label.
Back to top . Next: Typed State and call_llm()
📝 Typed State Schema and the Vendor-Neutral LLM Caller
Two implementation decisions in the orchestration layer have governance consequences that are not immediately obvious: how you define agent state, and how you call the LLM. Both decisions compound over the lifetime of the platform.
📊 The Typed State Schema as Implicit Contract
LangGraph’s state is defined as a Python TypedDict. Every field in the state schema is explicitly declared with its type. This feels like a simple code quality practice, but in a compliance context it is more than that: the state schema is the implicit contract between every node in the graph.
Consider what happens without a typed schema. Node A sets state["is_eligible"] = True. Node B reads state.get("is_eligible"). If Node A is refactored and the field is renamed to murabaha_eligible, Node B silently reads None. The workflow continues with a missing value and may produce an incorrect decision. The bug is invisible at runtime because Python dictionaries do not complain about missing keys when you use .get().
With a typed schema, the state is a TypedDict with murabaha_eligible: bool declared explicitly. Mypy catches the rename at type-check time. The state schema is also the documentation of what data is available at each graph node — a developer reading the state definition immediately sees every field that could influence a decision, what type it should be, and (via field comments) what governance meaning it carries. The typed state schema is an audit trail of what the graph knows at any point in its execution.
🔧 The Vendor-Neutral call_llm()
The LLM caller is a single function in the shared SDK: call_llm(messages, model, api_key, base_url, max_tokens, ...). It makes a direct httpx POST to the /v1/chat/completions endpoint — the standard OpenAI-compatible API format that Ollama, the Claude API via proxy, and most modern inference providers all support.
There is no LangChain. There is no provider SDK. The only runtime dependency is httpx and the target URL. When the target URL is http://ollama:11434/v1, the call goes to the local Ollama container. When it is https://api.anthropic.com/v1 via a proxy adapter, it goes to the cloud API. The application code does not change. The environment variable changes.
This has a practical consequence that compounded over the build: the platform was tested entirely on Ollama running open-weight models for months. When the decision was made to use the Claude API for production narrative generation (because the quality of explanations for novel Sharia structures was significantly better), the migration was one environment variable change. No service needed to be redeployed. No imports needed to be changed. No tests broke because the mock LLM interface matched the real one.
The vendor neutrality is also a sovereignty feature. Every environment variable in the platform describes function, not vendor: LLM_BASE_URL not ANTHROPIC_API_URL, CACHE_URL not REDIS_URL, SECRETS_ADDR not VAULT_ADDR. When you name a variable after a vendor, you create implicit coupling to that vendor in every configuration file, every deployment script, and every runbook that references the variable name. Vendor-neutral naming is refactoring debt you never have to pay.
Orchestration Layer Components
LangGraph
Explicit directed graph execution. Deterministic gates before LLM calls. Graph topology is the compliance artifact.
call_llm()
Single vendor-neutral httpx POST. Ollama in dev, Claude API in production. One environment variable separates them.
YAML Prompt Templates
All prompt strings in YAML files, zero in Python. Version-controlled alongside policies. Prompt changes are code changes — reviewable, auditable.
Langfuse
LLM trace storage, self-hosted in Docker. Every prompt and response captured. The observability layer for the LLM calls the orchestration layer makes.
OPA
Policy-as-code, hot-reloadable bundles from MinIO. Every agent checks authorization before executing. The gate runs before the graph node proceeds.
Structured Audit Events
TypedDict audit event schema, dual-written to Postgres and Redpanda outbox. Every gate decision is an immutable record before the next node executes.
Back to top . Next: Honest Limitations
⚠️ The Honest Limitations: What LangGraph Costs You
An honest assessment of a technology choice includes what it costs. LangGraph is the right choice for this use case, but it comes with specific trade-offs that every team should understand before committing to it.
- Graph complexity compounds: A five-node graph is readable. A fifteen-node graph with four conditional branches requires careful documentation to remain comprehensible. As agent capabilities expand, the graph grows. The governance benefit of explicit topology becomes a maintenance cost if the topology becomes too complex to explain at a whiteboard. The discipline of keeping graphs as simple as the requirements allow is a real ongoing effort.
- Error handling is your responsibility: LangGraph provides the execution engine; it does not provide the error recovery strategy. If a node raises an exception, the graph stops. Building robust error handling (catch, log, set error state, route to a recovery node) requires deliberate design for each failure mode. This is not a weakness unique to LangGraph — it is a consequence of the explicitness that makes it auditable. Explicit graphs have explicit failure points.
- Async/sync mixing requires discipline: LangGraph supports both async and sync nodes. Mixing them in the same graph can produce subtle execution order bugs. The platform standardised on fully async nodes throughout, which eliminates the mixing problem but requires that every node function, including simple ones that do not make network calls, be declared
async. - State migration is a manual operation: When the state schema changes — adding a new field, changing a field’s type — existing serialised states in progress do not automatically migrate. For long-running workflows interrupted mid-graph, this can produce deserialization errors. The platform’s approach is to keep in-flight states short-lived (complete within a single request cycle) and treat state schema changes as breaking changes that require versioned deployment.
None of these limitations are disqualifying. They are manageable with the right discipline. The point is that every framework choice involves trade-offs, and teams that adopt LangGraph purely because it is fashionable will be surprised by these costs. Teams that adopt it deliberately, understanding the trade-offs, will build better systems because of the explicit structure it enforces.
Back to top . Next: The Five-Services Problem
🤔 The Five-Services-Five-Opinions Problem
Five separate services, each with its own team member or sprint of development, will independently develop five interpretations of how to implement PII masking. Five interpretations of how to emit audit events. Five interpretations of what a properly structured LLM response looks like. In a regulated system where the compliance requirement is uniform across all services, divergence between interpretations is not a code quality problem — it is a compliance gap.
The discovery that triggered the monorepo approach came during an audit of the five services’ PII handling. The original design had each service independently import Presidio and call the PII masking function. The services were nominally doing the same thing. But when the actual implementations were compared, four distinct patterns had emerged:
- Service A masked all detected PII entities in the prompt text before the LLM call
- Service B masked PII in the response text but not the prompt
- Service C only masked national ID numbers (not names or account numbers) because those were the only entities the developer was aware of
- Service D called the masking function but did not check its return value, silently continuing with the original unmasked text if Presidio returned an error
All four services were “implementing PII masking.” None of them were doing it the same way. One was doing it in a way that actively failed silently. A compliance requirement that each service implements independently will eventually be implemented incorrectly by at least one service. The only question is which service and how soon.
Back to top . Next: Directory Structure
📁 The Four-Area Directory Structure
The monorepo is organised into four areas. The organisation is not arbitrary — it reflects the governance boundary of each area: who owns it, what it can change, and who depends on it.
services/— The five agent services and supporting APIs. Each is a complete Python service with its ownpyproject.toml,Dockerfile,tests/, and graph definition. Services are the consumers ofshared/sdk. They contain domain-specific logic (how the credit eligibility calculation works, what fatwas are relevant to a given product type) but not compliance logic (how PII is masked, how audit events are structured).shared/— The shared SDK that all five services import. Contains:call_llm(), theAuditLogger, PII guard middleware, injection guard, replay guard, theValkeySignerfor session key integrity, OPA client wrapper, and agent/model registry clients. Any logic that must be identical across services lives here. This is the governance enforcement layer.infra/— Infrastructure definitions and governance documentation. Postgres init SQL, OPA policies, Docker Compose, Airflow DAG configurations, OpenMetadata ingestion configs, MLflow setup, threat model documents, runbooks, governance policy documents. Nothing ininfra/is a Python import — it is the operational and governance substrate that the services run on.tests/— Platform-level tests that span multiple services. End-to-end test suites, contract tests (verifying the message schemas between services are compatible), and validation scenario suites. Individual service unit tests live inservices/{name}/tests/; cross-service integration tests live here.
The Docker Compose file, the Makefile, and the environment files all live at the repository root. A developer starting the platform for the first time runs one command from the repository root. The monorepo does not require understanding the internal structure of any individual service to get a running local environment.
Back to top . Next: The Shared SDK
📦 The Shared SDK as Governance Enforcement Layer
The shared SDK is not a utility library. It is the mechanism by which compliance requirements are enforced uniformly across all five services without requiring any service to re-implement them correctly.
Consider the PII guard middleware. In the SDK, PiiGuardMiddleware is a FastAPI middleware that intercepts every POST /tasks request, extracts any free-text fields in the request body, scans them with Presidio, and masks detected PII entities before the request reaches the graph. Every service that mounts this middleware gets identical PII masking behaviour. The divergence problem described above cannot occur — there is no alternative implementation to diverge to.
The same principle applies to audit events. The AuditLogger in the SDK writes an AuditEvent TypedDict to both the Postgres audit_events table and the audit_outbox in the same database transaction. A service that calls audit_logger.record(event) is guaranteed to get dual-write behaviour — the event is either in both Postgres and the outbox, or in neither. A service that implemented audit logging independently might write to the log but forget the outbox, producing incomplete compliance records. The SDK makes the correct behaviour the path of least resistance.
What Lives in the Shared SDK vs. What Lives in Each Service
In the SDK — compliance logic that must be identical everywhere
call_llm() with PII guard and injection guard. AuditLogger with dual-write to Postgres and Redpanda outbox. PiiGuardMiddleware, InjectionGuardMiddleware, ReplayGuardMiddleware, LateralMovementGuard. OPA client wrapper that builds the correct input shape. Agent registry client. Model registry client with Postgres LISTEN/NOTIFY cache invalidation. ValkeySigner for HMAC-protected session keys.
In each service — domain-specific logic that belongs to that agent
Graph node logic: how the credit eligibility formula works, which Sharia rules apply to a given product structure, how risk tiers are computed from component scores, how Murabaha stress tests are parameterised. Prompt YAML templates specific to that agent’s domain. Service-specific Pydantic settings models that validate environment variables at startup.
The decision rule for what belongs in the SDK is explicit: if the logic creates a compliance gap when implemented differently by different services, it belongs in the SDK. PII masking creates a compliance gap if implemented differently — SDK. Audit event structure creates a compliance gap if implemented differently — SDK. Credit score calculation is domain-specific to the credit agent — service. The boundary is not about code reuse. It is about which differences between service implementations would represent a compliance risk.
Back to top . Next: Three-Layer Configuration
⚙️ Three-Layer Configuration Without Drift
Configuration drift is the silent enemy of monorepo consistency. With five services sharing a repository, the temptation to hardcode a value directly in a service file rather than plumbing it through the configuration hierarchy is real and constant. Each hardcoded value is a future compliance gap — a threshold that needs to change when a regulatory requirement changes but only changes in one service because the developer who needs to change it does not know where all the instances are.
The platform uses a three-layer configuration hierarchy that eliminates hardcoded values from application code:
- Layer 1 — Platform defaults (
.env.devat repository root): Values that apply to all services in all environments. LLM base URL, Postgres DSN, Redpanda brokers, MinIO endpoint. These are the values a developer needs to start a local environment without configuring anything. - Layer 2 — Service Pydantic Settings (
config.pyin each service): Each service declares a PydanticBaseSettingsmodel that validates its required environment variables at startup. If a required variable is missing, the service fails immediately with a descriptive error rather than starting successfully and failing later when the variable is first accessed. Service-specific values (the agent’s name, its autonomy tier, which tools it is permitted to call) are declared here. - Layer 3 — Secrets (OpenBao /
.secrets): Values that must not appear in version-controlled files. API keys, database passwords, signing secrets. In development,.secretsis a gitignored file. In production, values come from OpenBao, injected as environment variables at container startup. The application code does not know which layer provided a value — it reads from the environment, which is assembled from all three layers.
Business thresholds — the debt-burden ratio ceiling, the HITL escalation score threshold, the fairness audit disparity threshold — live in the orchestrator_config Postgres table, not in environment variables or code. This distinction matters: environment variables require container restarts to change. A Postgres table with LISTEN/NOTIFY triggers can deliver configuration changes to running services within seconds, without restarts, with a full audit trail of who changed what and when. When a regulator asks “what was the DBR ceiling in effect on the date of this decision?”, the answer is in the configuration audit table — not reconstructed from deployment history.
Back to top . Next: Service Boundary Rules
🔎 Service Boundary Decision Rules
In a monorepo, every piece of logic must live somewhere — and the wrong place compounds into architectural debt faster than in a polyrepo, because the temptation to import across boundaries is low-friction. Having explicit rules prevents the gradual erosion of the boundary.
The boundary rules I applied throughout the build:
- Logic that creates a compliance gap when implemented differently across services → SDK
- Logic that is specific to one agent’s regulatory domain (credit eligibility calculation, Sharia product validation, risk tier assignment) → service
- Infrastructure definitions and governance documentation →
infra/ - Tests that span multiple services or validate cross-service contracts →
tests/ - Anything that a compliance auditor would want to review independently of the application code →
infra/
The hardest boundary cases are the ones that look domain-specific but are actually compliance-generic. The narrative consistency check — verifying that the LLM’s output does not contradict the deterministic gate’s decision — looks like an orchestrator concern. But it is actually a compliance concern that applies to every agent that generates LLM narratives. It belongs in the SDK. The injection guard looks like a security concern. It is also a compliance concern — an injected prompt that bypasses a Sharia check is a compliance violation, not just a security incident. It belongs in the SDK.
The test: if you removed this logic from one service but not the others, would a compliance auditor notice? If yes, it belongs in the SDK where removal from one service is impossible without removing it from all. If no, it is domain-specific and belongs in the service.
Back to top . Next: Real Costs
💸 The Real Costs of Monorepo Architecture
An honest assessment of monorepo architecture acknowledges the costs. These are real and should not be minimised for the sake of a cleaner narrative.
- SDK changes are breaking changes: When the shared SDK changes a function signature or a TypedDict field name, all five services must update at the same time. In a fast-moving development environment, this creates coordination overhead. The solution is SDK versioning and careful backwards-compatible evolution — but this requires discipline that slows down SDK development.
- CI/CD is more complex: A change to one service should ideally only rebuild and deploy that service. Detecting which services are affected by a change to the shared SDK requires a more sophisticated CI pipeline than a simple per-service build. Tools like Nx or Turborepo solve this problem for monorepos, but they add operational complexity.
- Developers need more context: A developer working on the risk agent needs to understand the shared SDK’s interfaces to work effectively. In a polyrepo, a developer can understand one service in isolation. In a monorepo, they need to understand the shared layers.
- Merge conflicts in shared files:
infra/docker/postgres/init/01_schema.sqlanddocker-compose.ymlare modified by every sprint that adds new infrastructure. These files become frequent sources of merge conflicts.
For this specific use case — a regulated banking platform where compliance uniformity across services is mandatory — the governance benefits of the shared SDK outweigh these operational costs. For a less compliance-constrained platform, the calculus might be different. The monorepo choice is not universally correct. It is correct when the cost of compliance divergence between services exceeds the cost of the coordination overhead the monorepo imposes.
Back to top . Next: The License Problem
⚠️ The License Problem Nobody Talks About
The early months of the build used the obvious choices. Redis for caching and session state. HashiCorp Vault for secrets management. A popular hosted API for LLM inference. ChromaDB for vector search. These are the tools that appear in almost every AI stack tutorial. They are well-documented, well-understood, and have large communities. What the tutorials do not discuss — and what became clear by reading license files rather than README files — is that several of them have terms that are incompatible with self-hosted commercial deployment at scale.
This matters for a banking platform in a specific way. Banks operate under data residency rules that require customer financial data to remain within specific geographic boundaries. The UAE PDPL (Personal Data Protection Law), for example, restricts the transfer of personal data outside the UAE without specific conditions. The CBUAE additionally requires that core banking system data remain on UAE-hosted infrastructure. A tool that requires data to flow through the vendor’s cloud infrastructure to function is not just a cost concern — it is a compliance violation in a regulated Gulf banking context.
The license audit is also a transparency requirement. When a bank deploys an AI system that makes credit decisions, it must be able to explain to regulators exactly what software components are involved in that decision. If a component’s license requires a commercial agreement that is not in place, the bank is in breach of the vendor’s terms. That breach could require disclosure to regulators depending on the jurisdiction. A license violation in production infrastructure is not a legal technicality. It is a material risk that belongs in the bank’s risk register.
Back to top . Next: Four Replacements
🔄 Four Components That Had to Be Replaced
Four components failed the self-hosting and license review. Each was replaced. The reasons are specific and instructive because they represent categories of risk that appear repeatedly in AI stack decisions.
🔴 Redis — Business Source License 1.1
In early 2024, Redis Labs changed the license of Redis from BSD 3-Clause to Business Source License 1.1 (BSL 1.1). The BSL 1.1 is a “delayed open source” model: the software is source-available but not open source under the OSI definition, and the commercial use restrictions are significant. Specifically, BSL 1.1 prohibits providing the software as a commercial service without a commercial agreement.
For a bank using Redis internally — not reselling it as a service — the commercial restriction may not directly apply. But the license change introduced uncertainty that is unacceptable in a regulated deployment. The risk is not the current terms. The risk is vendor-controlled terms that can change again. A dependency whose license terms changed once can change again. A bank cannot build governance documentation for a system component whose compliance status can be retroactively altered by a corporate decision at the vendor.
🔴 HashiCorp Vault — Business Source License 1.1
HashiCorp made the same move: Vault shifted from Mozilla Public License 2.0 to Business Source License 1.1. Secrets management is arguably the most sensitive component in the stack — it holds the credentials, API keys, and signing certificates that protect the entire system. Having vendor-controlled license terms on your secrets manager is a governance risk that is categorically different from having them on a caching layer. If a dispute with HashiCorp required migrating away from Vault under time pressure, the migration would touch every service that reads a secret — which is every service in the platform.
🟡 ChromaDB — Operational Maturity at Scale
ChromaDB uses Apache 2.0, which is a permissive and commercially compatible license. The replacement was not primarily a license issue — it was a combination of operational maturity and specific banking platform requirements. At the time of evaluation, ChromaDB’s production deployment story (persistent storage, access control, multi-collection management, backup and restore) was less mature than alternatives. Qdrant, with Apache 2.0 and a significantly more complete production operation story, was the stronger fit for multi-collection isolation at the scale of a banking platform.
🔴 Groq API — Data Sovereignty and Vendor Lock-In
Groq is a fast LLM inference provider with excellent developer experience. But it has two properties that make it unsuitable for this platform’s primary LLM inference path. Customer financial data would leave the bank’s infrastructure in LLM prompts, and there is a free-tier policy that restricts commercial production use. An LLM prompt in this platform contains context derived from the customer’s credit score, risk tier, debt-burden ratio, and product terms. That data must not leave the bank’s infrastructure unless the customer has explicitly consented to it under UAE data protection law.
Back to top . Next: What Won and Why
✅ The Replacement Decisions: What Won and Why
The replacements were not chosen to be contrarian or to avoid popular tools. They were chosen because they pass all three audit criteria: permissive commercial license, full self-hosting capability, and data sovereignty by default. In each case, the replacement is at least as capable as the original for the specific use cases in this platform.
✅ Valkey — Redis Fork, BSD 3-Clause
Valkey is the Linux Foundation-hosted fork of Redis, created immediately after the BSL 1.1 license change. It maintains full protocol and API compatibility with Redis — the Redis client library works unchanged. Valkey is governed by the Linux Foundation, which provides institutional independence from any single corporate entity’s business decisions. The migration from Redis to Valkey required changing exactly two things: the Docker Compose service definition (image name) and the constructor call in the client wrapper. No application code changed.
✅ OpenBao — Vault Fork, Mozilla Public License 2.0
OpenBao is the Linux Foundation-hosted fork of HashiCorp Vault, also created in response to the BSL 1.1 change. It maintains full API compatibility with Vault. The Mozilla Public License 2.0 is a copyleft license at the file level: modifications to MPL-licensed files must be shared, but combining MPL code with proprietary code is permitted. For a bank using OpenBao as-is without modifying its source, MPL 2.0 is commercially viable. Using an institutionally-governed tool with a stable open license means the bank’s ability to operate does not depend on a commercial relationship with any particular vendor.
✅ Qdrant — Apache 2.0, Full Self-Host
Qdrant is a vector database written in Rust, licensed under Apache 2.0. The self-hosted version is identical to the cloud version in functionality — there is no feature-limited community edition with advanced features locked to a cloud tier. The platform uses three separate collections: Sharia fatwas, product catalogue, and customer profiles. Each collection has its own access controls. The privacy-by-design implication is direct: vector embeddings generated from customer financial profiles are stored entirely within the bank’s infrastructure, protected by the bank’s own access controls — not sent to a cloud vector database vendor.
✅ Ollama + Claude API — Self-Host First, Cloud API Second
The LLM inference architecture uses a two-tier approach. Ollama runs open-weight models entirely on the bank’s own infrastructure during development and for use cases where the prompt contains sensitive customer data. The vendor-neutral call_llm() function means switching from Ollama to the Claude API or any other provider requires changing one environment variable, not any application code.
For cloud LLM API calls — when used for low-sensitivity narrative generation only — the key design decision is that sensitive customer data never reaches the cloud LLM API. The PII guard middleware in the shared SDK enforces this at the call site: if a Presidio scan of the assembled prompt detects PII entities, the call is blocked when LLM_BASE_URL points to a cloud endpoint. Sovereignty by design means the enforcement mechanism is in the code, not in a policy document that developers might not have read.
Back to top . Next: Data Sovereignty
🏛️ Data Sovereignty as a Compliance Requirement
Data sovereignty is the property of a system where data does not leave the boundary you define for it unless you explicitly choose to send it outside that boundary. For a regulated banking platform in the Gulf, that boundary is the bank’s own data centre or a UAE-region deployment. Every component in the stack must be evaluated against this requirement: does running this component create data flows that cross the boundary by default?
The evaluation matrix considers data categories and their permitted boundaries. Customer personal data (names, national IDs, dates of birth) must never leave the bank’s infrastructure under any circumstances — a hard requirement under UAE PDPL. Derived financial data (credit scores, risk tiers, DBR ratios) must also remain within the bank’s infrastructure under the CBUAE’s data localisation requirements. Anonymised analytical aggregates can potentially be processed by external tools if necessary, though the preference is to keep them local.
Every component is evaluated against this matrix. Presidio runs on-premises for PII detection — to detect PII, you must handle PII, so the detector must be local. Langfuse runs in Docker for LLM trace storage — LLM traces contain prompt context derived from customer financial data, so trace storage must be local. MLflow runs in Docker for model artifact storage — model training data provenance contains references to customer segments, so it must be local. The stack is not evaluated component by component. It is evaluated as a system where data flows from one component to the next, and every node in that flow must be within the sovereignty boundary.
Privacy by design in stack selection means choosing components that are sovereign by default. If a component requires data to leave your infrastructure to function, it is not sovereign by design. It is sovereign by configuration — and configuration can change, be misconfigured, or be overridden. Structural sovereignty is stronger than configured sovereignty.
Back to top . Next: What Stayed
✅ What Stayed: The Parts That Passed the Audit
Not everything was replaced. The components that passed the license and sovereignty audit stayed because they genuinely earned their place. Understanding why they passed is as instructive as understanding why the others did not.
Stack Components That Passed the Full Audit
FastAPI
MIT license. Self-hosted. No data leaves the process. Zero governance concerns.
Postgres
PostgreSQL License (permissive BSD-style). Fully self-hosted. No vendor dependency. The governance database of record for every platform table.
Apache Spark
Apache 2.0. Self-hosted. No cloud dependency required. Bronze/Silver/Gold pipeline processing entirely within the bank’s infrastructure.
Apache Airflow
Apache 2.0. Self-hosted. No data sent to external services. Fully open, fully sovereign DAG scheduler.
Open Policy Agent
Apache 2.0. Self-hosted. Rego policies version-controlled, hot-reloaded via MinIO bundles. Policy-as-code with no external dependency for evaluation.
Prometheus + Grafana
Apache 2.0 (Prometheus), AGPL 3.0 (Grafana). Both self-hosted. Metrics and dashboards within the bank’s observability boundary.
Redpanda
BSL 1.1 — internal use only, not as a managed service for others. Kafka-compatible, single binary. Read the specific restriction, not just the category label.
Presidio
MIT license. Runs entirely on-premises. No PII sent to external services. The local PII detection engine that makes cloud LLM calls safe.
Evidently AI
Apache 2.0. Self-hosted. No data sent to external services. Drift detection, fairness reports, and data quality monitoring all remain local.
Redpanda’s BSL 1.1 license deserves a note. The BSL 1.1 restriction for Redpanda specifically prohibits using it to build a managed Kafka-compatible service for third parties. Using Redpanda internally for the platform’s own event streaming — not offering it as a service to other organisations — does not trigger the commercial restriction. Every component deserves a reading of the actual license text, not just the category label.
The cost discipline of this stack is itself a fairness mechanism. A platform built entirely on self-hostable, license-free infrastructure can afford to run all governance controls continuously, because running them costs nothing additional. The fairness audit runs after every pipeline completion because it costs nothing to run it. Drift monitoring, red-team scanning, and embedding integrity checks are standard pipeline steps, not optional add-ons that get cut when budgets tighten. Governance features that are structurally free to run are governance features that actually run.
Back to top . Next: Key Takeaways
Key Takeaways
- Determinism before LLM narrative — the LLM explains a decision already made by a deterministic gate; it never makes the decision. This principle is what makes agent workflows auditable rather than just functional.
- Graph topology is a compliance artifact — an explicit directed graph with named nodes and logged state at each transition is a record an auditor can follow. An LLM-driven loop with emergent execution paths is not.
- The shared SDK is governance enforcement, not code reuse — compliance logic that lives in the SDK cannot be implemented differently by different services. That is its value. The goal is not reducing duplication — it is eliminating the possibility of compliant-and-non-compliant interpretations coexisting in the same platform.
- Configuration drift is a compliance gap — business thresholds hardcoded in application files will eventually be out of sync. Thresholds in a governed Postgres table with an audit trail of changes are compliant. The data residency of configuration values matters as much as the data residency of customer data.
- The license audit is the risk audit — a dependency with vendor-controlled license terms is a dependency where your compliance posture can be altered by a corporate decision you have no influence over. Read license files, not README files.
- Vendor-neutral naming protects future flexibility — when environment variables describe the function (LLM_BASE_URL, SECRETS_ADDR) not the vendor (ANTHROPIC_API_URL, VAULT_ADDR), migrating providers is a one-variable change rather than a codebase search-and-replace.
- Institutional governance is stronger than vendor promises — Linux Foundation-governed tools (Valkey, OpenBao) have a governance model for license changes that corporate vendors do not. The institutional structure is part of the compliance posture, not a nice-to-have property.
- Sovereignty by design beats sovereignty by configuration — a component that structurally cannot send data outside your boundary is safer than one configured not to, but physically capable of it. Structural constraints do not drift. Configuration does.
- Zero-cost infrastructure enables continuous governance — when running drift detection, fairness audits, and red-team scans costs nothing, they run every time the pipeline runs. When they carry per-API-call costs, they get deferred when budgets tighten. Governance features should be structurally free to run.
Thank You, Reader
This article covers three decisions that were made before writing a single line of application logic: how to orchestrate the agents, how to structure the codebase, and which infrastructure components can be deployed in a regulated banking environment. None of these decisions are purely technical. Each one has governance implications that compound over the lifetime of the platform. LangGraph enforces explicit execution paths. The shared SDK enforces compliance uniformity. The license audit enforces data sovereignty. Taken together, they create a foundation where every subsequent capability — PII masking, audit events, fairness monitoring — can be implemented once and trusted everywhere. The next article is where the foundation gets tested: a systematic audit that found 57 gaps in a platform that looked done.
Connect With Me
- LinkedIn: Connect on LinkedIn
- GitHub: github.com/neerajg5
- Blog: learnwithneeraj.com
Enjoyed this article?
Get notified when the next one is published.
We send one email per new article — no spam, unsubscribe any time.
⚠️ Disclaimer: The information provided on LearnWithNeeraj.com regarding Astrology, Numerology, and other topics is for educational and guidance purposes only.
Not Professional Advice: This content should not be used as a substitute for professional medical, legal, or financial advice. Always consult a certified professional for specific concerns.
Guest Authors: This site features articles by various contributors. The views and interpretations expressed are those of the individual authors and do not necessarily reflect the views of the website administrator.
Your destiny is in your hands. Use this information as a map, not a mandate.