Building a Secure Agent Runtime: Combining Harness and Sandbox

Recap: Two Problems, One System

The first two articles in this series covered two separate but related problems:

• Sandboxing AI Agents — how to safely execute code an agent generates, so a bad decision or a prompt injection can't reach beyond a tightly controlled boundary.
• The Agent Harness — the broader system around the LLM that decides when to act, what it's allowed to do, what it remembers, and whether it actually worked.

This article puts both pieces together into a single, coherent architecture — and builds a minimal, working version of it.

The Combined Architecture

             User
               │
               ▼
              LLM
               │
               ▼
            Harness
               │
      ┌────────┴────────┐
      │                 │
   Policies          Orchestrator
      │                 │
      └────────┬────────┘
               ▼
             Tools
               │
               ▼
            Sandbox
               │
      ┌────────┼────────┐
      ▼        ▼        ▼
    Python   Files    Network
               │
               ▼
            Result

Reading this top to bottom as a request flows through it:

1. The user sends a request.
2. The LLM reasons about it and proposes an action.
3. The harness intercepts that proposal before anything happens.
4. Policies (guardrails) check whether the action is allowed at all, and whether it needs approval.
5. The orchestrator manages the step-by-step loop and decides which tool to invoke.
6. The tool translates the model's intent into a concrete operation.
7. The sandbox executes that operation in an isolated, resource-limited environment.
8. The result flows back up to the LLM, which decides the next step — or that the task is complete.

Every step is logged for observability, and every completed run is a candidate for evaluation against known-good outcomes.

Design Principles Worth Stating Explicitly

Before the implementation, a few principles that should hold across the whole runtime, not just individual pieces:

• Deny by default. New tools and new destinations (files, network hosts, database tables) start blocked and get explicitly allow-listed — never the other way around.
• Every execution is ephemeral. No sandbox instance should outlive a single tool call unless there's a specific, reviewed reason for it to persist.
• Policy is enforced outside the model. Never rely on a system prompt instruction like "don't delete files" as your actual security boundary — the harness's guardrails and the sandbox's isolation are the real boundary; the prompt is just a hint about intent. This is the same reasoning behind the infrastructure-level LLM firewall in the Zero-Trust backend.
• Everything is logged. If a step isn't observable, treat it as if it didn't happen safely.
• The loop has a hard stop. Maximum steps, maximum tokens, and a timeout, independent of whether the model "thinks" it's done.

A Minimal Reference Implementation

The following is a simplified but functional pattern for a Python-based agent runtime: an orchestrator loop, a guardrail check, and a Docker-backed sandboxed tool. It's intentionally small — the goal is to make the architecture concrete, not to be production-ready as-is.

import subprocess
import json
import uuid

# --- Guardrails -------------------------------------------------

ALLOWED_TOOLS = {"run_python"}

def check_guardrails(tool_name: str, args: dict) -> tuple[bool, str]:
    if tool_name not in ALLOWED_TOOLS:
        return False, f"Tool '{tool_name}' is not permitted."
    code = args.get("code", "")
    forbidden = ["import os", "import socket", "subprocess", "open("]
    if any(term in code for term in forbidden):
        return False, "Code contains a disallowed operation."
    return True, "ok"


# --- Sandbox (Docker-backed execution) ---------------------------

def run_in_sandbox(code: str, timeout_seconds: int = 10) -> dict:
    run_id = uuid.uuid4().hex[:8]
    cmd = [
        "docker", "run", "--rm",
        "--network", "none",
        "--memory", "256m",
        "--cpus", "0.5",
        "--pids-limit", "32",
        "--read-only",
        "--tmpfs", "/tmp",
        "--name", f"agent-sbx-{run_id}",
        "python:3.12-slim",
        "python", "-c", code,
    ]
    try:
        result = subprocess.run(
            cmd, capture_output=True, text=True, timeout=timeout_seconds
        )
        return {
            "stdout": result.stdout,
            "stderr": result.stderr,
            "exit_code": result.returncode,
        }
    except subprocess.TimeoutExpired:
        return {"stdout": "", "stderr": "Execution timed out.", "exit_code": -1}


# --- Orchestrator loop --------------------------------------------

def run_agent(task: str, max_steps: int = 5):
    trace = []
    for step in range(max_steps):
        # In a real system, this calls the LLM with `task` + `trace`
        # and gets back a structured tool call. Simplified here:
        tool_call = decide_next_action(task, trace)

        if tool_call is None:
            break  # model signaled the task is complete

        allowed, reason = check_guardrails(tool_call["tool"], tool_call["args"])
        trace.append({"step": step, "tool_call": tool_call, "allowed": allowed})

        if not allowed:
            trace[-1]["result"] = {"error": reason}
            continue

        if tool_call["tool"] == "run_python":
            result = run_in_sandbox(tool_call["args"]["code"])
            trace[-1]["result"] = result

    return trace

Even at this scale, every principle from earlier shows up in the code:

• ALLOWED_TOOLS is a deny-by-default allow-list, not a block-list.
• run_in_sandbox creates a fresh, ephemeral, resource-capped, network-isolated container per call, and it's the only path to execution.
• check_guardrails runs before the sandbox ever sees the code — policy is enforced outside the model.
• trace gives you a full observability log of every step, allowed or not, for free.
• max_steps is the loop's hard stop, independent of what the model thinks.

A real implementation would replace decide_next_action with an actual LLM call, expand the tool set, and add richer guardrail policies — but the shape of the system stays the same.

Tracing a Request Through the Runtime, Step by Step

To make the code above concrete, here's exactly what happens for a task like "Calculate the average of this list of numbers and tell me if it's above 50."

1. run_agent("Calculate the average...") starts the loop with an empty trace.
2. decide_next_action calls the LLM with the task and (empty) trace. The model responds with a structured tool call: {"tool": "run_python", "args": {"code": "print(sum([...])/len([...]))"}}.
3. check_guardrails runs first — the tool is in ALLOWED_TOOLS, and the code contains none of the forbidden strings, so it passes.
4. run_in_sandbox launches a fresh, network-isolated, memory-capped container, runs the code, captures stdout, and destroys the container — all before the function returns.
5. The result ({"stdout": "62.5\n", ...}) is appended to trace and would, in a full implementation, be fed back into the next LLM call as the observation for that step.
6. On the next iteration, the model sees the result, reasons that 62.5 is above 50, and returns None — signaling the task is complete.
7. The loop exits and returns the full trace, which is both the answer and a complete audit log of everything that happened.

Seven steps, and only two of them (step 2 and step 6) actually involved the LLM "thinking." Everything else was deterministic infrastructure — which is exactly the point of separating the harness from the model.

Scaling Beyond a Single Machine

The reference implementation above runs Docker on the same host as the orchestrator, which is fine for prototypes and low-volume internal tools. Once an agent runtime needs to serve many concurrent users, a few things typically change:

• A dedicated execution service. Instead of shelling out to docker run directly from the orchestrator process, teams move sandboxed execution behind its own service — often backed by a pool of pre-warmed, disposable execution environments that requests are routed to. This removes the cold-start latency of spinning up a fresh container per call.
• Per-tenant isolation, not just per-request isolation. In a multi-tenant product, it's not enough that Tenant A's code runs in its own container — you also want to ensure Tenant A's execution requests can't starve Tenant B's, typically via separate resource quotas or queues per tenant.
• Centralized logging and tracing, rather than an in-memory trace list, so observability data survives process restarts and can be queried across many concurrent agent runs.
• Async orchestration. Real agent tasks often involve I/O-bound waits (LLM calls, tool calls, human approval steps that might take hours). Production orchestrators are typically built on an async event loop or a durable workflow engine, so a runtime can have thousands of tasks in flight without thousands of blocked threads.
• Cost and rate-limit awareness. At scale, the LLM calls themselves — not the sandboxed execution — usually become the dominant cost and the tightest bottleneck. Understanding how prefill and decode consume a GPU differently is what makes harness-level caching, batching and step budgets tractable.

None of this changes the architecture described earlier in this series — it's still LLM → harness → guardrails → orchestrator → tools → sandbox. What changes is that each box becomes its own scalable service instead of a function call within one process. The containerisation and CI/CD patterns in MLOps in Practice apply directly to each of those services.

What This Doesn't Solve

It's worth being direct about the gaps a runtime like this still has, so you don't oversell it in a portfolio piece or a production pitch:

• Prompt injection is still possible. A well-designed runtime limits the damage injected instructions can do; it doesn't prevent the model from being influenced by adversarial content it reads.
• Evaluation is separate work. This architecture makes agent behavior safe and observable — it doesn't tell you whether the agent's answers are correct.
• Container isolation has limits. As discussed in the sandboxing article, Docker alone isn't sufficient isolation for adversarial, multi-tenant workloads — that's where gVisor, Firecracker, or full VMs come in.
• Guardrail policy needs to evolve. A static block-list of dangerous strings, as in the example above, is a starting point, not a long-term security model; real systems move toward capability-scoped tools instead of trying to filter arbitrary code.

Where to Go From Here

This series has walked through the problem in order: why execution needs to be isolated, what system has to exist around the model to make it an agent at all, and how to combine both into a working runtime. Two natural next steps:

• Agent observability — proper tracing, cost tracking, and debugging tools for multi-step agent runs, beyond a simple in-memory trace list.
• Agent evaluation — building a repeatable test suite that measures whether an agent actually completes tasks correctly, not just safely.

Conclusion

Sandboxing answers "how do we contain what the agent executes?" The harness answers "what decides when, what's allowed, and how do we know it worked?" Neither is complete on its own — an agent with a perfect sandbox but no guardrails will safely execute actions it never should have taken, and an agent with perfect guardrails but no sandbox has no real containment once execution starts.

A secure agent runtime is what you get when you stop treating those as separate problems and design them together, from the start, as one system.

The full series: Sandboxing AI Agents → The Agent Harness → Building a Secure Agent Runtime.