Lex Series

LexVerdict

Open Source

Post-execution verification API. POST a tool call, goal, and result - get back pass or steer with a reason.

Quick start

Start with the primary command, then continue with the full workflow below.

curl -X POST https://your-lexverdict.workers.dev/v1/verify \
  -H "Content-Type: application/json" \
  -d '{

Problem

Every agentic workflow has the same blind spot: tool calls that succeed technically but produce wrong results. The agent writes a config with the wrong environment, queries an API and gets unexpected data, or runs a script that exits 0 but outputs garbage. Pre-execution checks can't catch these because the call itself is valid - a correct call with a bad result is the most common agent failure. Post-hoc evals (LangSmith, Braintrust) catch errors but run after the agent has already moved on, and manual review doesn't scale. Teams need real-time result validation that fits inside the agent loop.

Solution

LexVerdict is a simple HTTP API that verifies tool results in real time: POST a tool call, goal, and result and get back pass or steer with a reason - with no SDK or code changes to your agent. The architectural choice is a Cloudflare Worker that fronts a fast verifier model via a service binding (JIMMY_SERVICE, default) or an HTTPS OpenAI-compatible fallback URL (JIMMY_URL), using a two-stage reasoning prompt - restate the goal/result, run six systematic checks, then emit a verdict. The worker exposes standalone verification (POST /v1/verify), a proxy mode (POST /v1/chat/completions that verifies upstream responses with steer injection), and GET /health. The backend is MIT open source and self-hostable with any model endpoint; the hosted SaaS uses a 10,000+ TPS dedicated model for sub-100ms verification and is invite-only.

How it works

Deploy your own copy: git clone https://github.com/LatticeAG/LexVerdict.git && cd LexVerdict && npm install, then npm run dev locally or npm run deploy:staging / npm run deploy:production to ship the Worker.

Configure the verifier: either a JIMMY_SERVICE Cloudflare service binding (takes precedence) or an HTTPS JIMMY_URL OpenAI-compatible endpoint, plus optional UPSTREAM_API_KEY secrets (npx wrangler secret put UPSTREAM_API_KEY --env <environment>) for proxy mode.

In your agent loop, after each tool execution capture the triple: the tool call itself, the goal it was meant to achieve, and the result it returned.

POST the triple to LexVerdict: curl -X POST https://your-lexverdict.workers.dev/v1/verify with a JSON body of tool_call, goal, result - or the alias form { decision, context: { tool_call, goal } }.

The worker sends the request to the fast verifier (15k TPS model, Llama 3.1 8B), which runs six checks: Match (does result satisfy the goal?), Environment (wrong staging vs prod?), Data (wrong values, file, or target?), Security (weak passwords, secrets, excessive perms?), Failure (command failed, file not found, 0 tests?), Drift (contradicting goal or going off course?).

LexVerdict returns the verdict: pass means the result looks correct and the agent continues; steer means the result is off-course and the message is injected into the agent's context - a JSON response like { verdict: steer, confidence: 0.92, message: "[LexVerdict] Config references 'staging' but goal was 'production'..." }.

In a standalone agent loop, check the verdict and append the steering message on steer - a Python integration via httpx.post("https://your-lexverdict.workers.dev/v1/verify", json={...}) fits in a few lines, with no SDK required.

Optionally run the bundled accuracy suite against your configured endpoint: python3 test/accuracy_test.py (60 cases, 3 scored runs + 1 diagnostic pass = 240 requests).

Technical architecture

Each handoff carries structured context through the product's execution path. Hover a node to inspect its role.

Tool Execution

Step 1: Tool Execution receives structured context from the prior stage and prepares it for the next handoff.

Result Capture

Step 2: Result Capture receives structured context from the prior stage and prepares it for the next handoff.

Verify Request

Step 3: Verify Request receives structured context from the prior stage and prepares it for the next handoff.

Goal Restatement

Step 4: Goal Restatement receives structured context from the prior stage and prepares it for the next handoff.

Systematic Checks

Step 5: Systematic Checks receives structured context from the prior stage and prepares it for the next handoff.

Verdict Emission

Step 6: Verdict Emission receives structured context from the prior stage and prepares it for the next handoff.

Steering Injection

Step 7: Steering Injection receives structured context from the prior stage and prepares it for the next handoff.

When to use

You need real-time validation of tool results - pre-execution blockers can't catch a correct call with a wrong result.
You want to inject corrective steering back into an agent loop without changing agent code.
You're building a CI/CD pipeline that should fail or flag deployments with off-goal configs.
You want a self-hosted, MIT-licensed verifier with any OpenAI-compatible model endpoint.
You're already on Axion and need a fast-model judgment layer for post-execution checks.
You need per-result verdicts with confidence scores and human-readable reasons.

Not for

Pre-execution call blocking - LexVerdict verifies results after the tool ran (pair it with LexShield for that).
Very high accuracy on security and content-mismatch cases with an 8B model - measured ~33% security / ~38% wrong-content accuracy; a 70B+ model would likely exceed 90% on the same prompt.
Batch APIs - POST /v1/verify/batch is explicitly not supported in v1; use one call per observed result.
Production SaaS without an invite - the hosted 10,000+ TPS endpoint is invite-only.

Features

Exact endpoint POST /v1/verify - takes { tool_call, goal, result }, returns { verdict: pass | steer, confidence, message }

Runs on a 15k TPS model (Llama 3.1 8B) - verdict in under 100ms on the hosted endpoint

Six systematic failure-mode checks - Match, Environment, Data, Security, Failure, Drift

Two-stage reasoning prompt - Restate, Systematic checks, Verdict; won by 10-15 points across every category vs. 6 alternative prompt designs

Proxy mode - POST /v1/chat/completions verifies upstream responses, supports streaming and non-streaming, buffers and injects steering

GET /health - liveness plus verifier configuration status (ok when a Jimmy service binding or fallback URL is configured, degraded otherwise)

Alias request form accepted - { decision, context: { tool_call, goal } } alongside the canonical triple

Input hardening - 128 KB max request body, 8,000-char max per verification field, 24,000-char prompt cap, 64-message chat window, NUL-stripping, 15s verifier timeout

Open source (MIT) backend - deploy your own with any model endpoint

Zero code changes to your agent - callable from any agent loop or from CI/CD

Drop-in as the verification model behind Axion (post-execution check proxy)

Bundled accuracy harness - 60 diverse scenarios (20 pass, 40 steer) across 3 scored runs = 180 scored API calls

Request logging toggle - REQUEST_LOGGING_ENABLED metadata-only logging

Install and usage

usage.sh

# Verify a tool result after execution (CI/CD pattern)
curl -X POST https://your-lexverdict.workers.dev/v1/verify \
  -H "Content-Type: application/json" \
  -d '{
    "tool_call": "write_file deploy-config.yml",
    "goal": "Configure staging deployment",
    "result": "'"$(cat deploy-config.yml)"'"
  }'

# Response
# { "verdict": "steer", "confidence": 0.92, "message": "[LexVerdict] Config references 'staging' but goal was 'production'. Verify and correct before continuing." }

# Deploy your own copy
git clone https://github.com/LatticeAG/LexVerdict.git
cd LexVerdict
npm install
npm run dev
npm run deploy:staging
npm run deploy:production

# Standalone Python integration
import httpx
resp = httpx.post("https://your-lexverdict.workers.dev/v1/verify", json={
    "tool_call": tool_call,
    "goal": goal,
    "result": result,
})
verdict = resp.json()

Architecture explorer

Problem

Solution

View on GitHub

LexVerdict

Quick start

Problem

Solution

How it works

Technical architecture

When to use

Not for

Features

Install and usage

Architecture explorer

Problem

Solution

Related products