THE BLOG
Notes from building.
Experiments, decisions, and lessons from backend engineering.
Whole articles made a local model accurate, and 15× too slow
On a local model, every 1,000 tokens of context costs ~2 s. Whole articles reach 0.59 correctness at ~160 s per answer; a context budget gets 0.49 at ~11 s, up from 0.36 with compact context. A frontier model scores 0.65 in 5 s.
MCP tool over a durable agent
An MCP tool backed by Temporal workflows: agent runs survive worker crashes, stay within a hard USD cost budget, and work with OpenAI, Claude, or local models. End-to-end OpenTelemetry tracing with token and cost tracking.