Skip to main content

Platform

Reference architecture

Our reference architecture is arranged in six layers within a single trust boundary and operates without any outbound network dependency. The technologies used within each layer vary according to your environment, while the overall structure remains consistent across engagements.

On-premises reference architectureSix layers inside the client security boundary. From the bottom: infrastructure (GPU nodes, block and object storage, network boundary); serving (inference engines, model router, KV and prompt cache); data and retrieval (vector index, document store, connectors); agent runtime (planner, tool broker, memory, policy engine, state store); governance (identity, authorisation, audit log, evaluation, observability), which spans the full width; and interfaces (APIs, internal UI, existing line-of-business systems). Nothing crosses the boundary outward.client security boundary, no egressINTERFACESInternal APIsOIDC-protectedOperator consoletraces, approvalsLine-of-business systemsITSM, CRM, EDRMBatch / scheduled jobsAGENT RUNTIMEPlannergraph executionTool brokertyped contracts, MCPPolicy engineleast privilegeMemoryworking / episodicState storecheckpoint + replayDATA & RETRIEVALVector indexhybrid + rerankDocument storeACL-awareFeature storeSource connectorsSERVINGInference enginevLLM / SGLang / NIMModel routersize-to-taskPrompt + KV cacheEmbedding + rerankINFRASTRUCTUREGPU nodes (Kubernetes, MIG-aware)Block + object storageSegmented network, no outbound routeGovernance plane · identity (AD/LDAP, OIDC) · authorisation · immutable audit log · evaluation · OpenTelemetry
Figure 1. The full stack. Model weights, execution traces and logs all remain inside the boundary, and the governance plane spans every layer.

01Infrastructure

GPU nodes under Kubernetes with the device plugin and topology awareness configured, block and object storage, and a network segment with no outbound route.

Instead, when · On smaller estates we have run single-node serving under systemd with no orchestrator at all. If you have two GPUs and one model, Kubernetes is overhead you will pay for at 03:00.

02Serving

An inference engine (vLLM or SGLang, or NVIDIA NIM where its curated model profiles and validated configurations are worth the packaging), a router that sends each task to the smallest model that handles it, and prompt and KV caching.

Instead, when · Multi-node coordination through Ray Serve when a model genuinely does not fit on one node, or when one cluster must host several models. Not before. It is a distributed system you will have to operate.

03Data & retrieval

Hybrid search with a reranking stage, a document store that carries entitlement metadata, and connectors that record provenance on ingest.

Instead, when · Where the corpus is small and stable, a well-tuned lexical index with no vectors at all. We have replaced vector search with BM25 and a reranker more than once, to the client’s benefit.

04Agent runtime

A planner executing a declared graph over typed state, a tool broker exposing typed contracts over MCP, a policy engine enforcing least privilege, tiered memory, and a state store that checkpoints at node boundaries.

Instead, when · For single-step tasks, no agent runtime at all. A retrieval-grounded call with a good evaluation set is often the correct architecture and is much easier to operate.

05Governance

Identity from your existing directory, authorisation per agent rather than per system, an append-only audit log in a framework-independent format, the evaluation harness, and OpenTelemetry throughout.

Instead, when · Nothing. This layer is not optional and it cannot be added later without rebuilding the ones above it.

06Interfaces

APIs protected by your identity provider, an operator console for traces and approvals, integrations into the systems people already use, and scheduled batch paths.

Instead, when · Where a workflow already exists in an established tool, we integrate into it rather than building another place for people to look.

Inside the agent runtime

The planner executes a control-flow graph that can be inspected directly. Any step requiring a capability is routed through the tool broker, which consults the policy engine before granting access. Every transition is written to the audit log, which also provides the input replayed by the evaluation harness.

Agent runtime internalsA request enters the planner, which executes a directed graph of steps. Each step that needs a capability calls the tool broker, which checks the policy engine before invoking a typed tool. Working and episodic memory are read and written through the state store. Every transition is written to an append-only audit log, which feeds both the operator console and the evaluation harness. Human approval gates interrupt the graph and resume it from a checkpoint.Requestuser or schedulePlannerexplicit graph, typed stateTool brokerMCP servers, typed I/OPolicy enginescopes, quotas, redactionToolssearch, systems, codeApproval gateinterrupt / resumeMemory + statecheckpointsAppend-only audit log. Every step, tool call and decisionEvaluation harnessreplays tracesObservabilityOpenTelemetry
Figure 2. Agent runtime. The audit tap is part of the primary execution path rather than a secondary logging function.

Where the protocols sit

MCP standardises how an agent reaches tools and context. A2A standardises how agents delegate to one another. They are complementary layers rather than competing choices, and most systems need the first without the second.

MCP and A2A operate at different layersTwo agents sit side by side. Each agent uses the Model Context Protocol downwards to reach tools, files and databases in a standard way. The two agents talk to each other sideways using the Agent-to-Agent protocol to discover capabilities and delegate tasks. The protocols are complementary: MCP standardises an agent's access to context and tools; A2A standardises delegation between agents.Agent Aplanner + toolsAgent Bspecialist, other teamA2A, discovery, task delegation, statusMCP, one standard way to reach context and capabilityDocument storeSearch indexTicketing systemData warehouseDifferent layers, not competitors. Most systems that need both end up using both.A single agent with three tools needs MCP and no A2A. Two teams' agents cooperating need A2A on top.
Figure 3. MCP governs access downwards to tools and context. A2A governs delegation sideways between agents.

Discuss how this applies to your estate

Every environment contains conditions that a reference design has to be adapted to, whether that is existing hardware, a fixed latency budget or a network with no outbound route. Tell us about yours and we will work through the implications with you in detail.

Talk to an engineer Security model