Harsh Verma
Harsh Verma

BACKEND SYSTEMS · SEARCH · AI EVALUATION

Hi, I’m Harsh.

Senior software engineer specializing in retrieval systems, AI evaluation, and production reliability.

Previously at BenchSci, where I owned evaluation for agentic retrieval services, implemented hybrid search, and established the company’s first SRE function. Now building and writing about backend engineering work in backend-lab.

View my résumé

Writing

All posts
Whole articles made a local model accurate, and 15× too slow
·5 min read

On a local model, every 1,000 tokens of context costs ~2 s. Whole articles reach 0.59 correctness at ~160 s per answer; a context budget gets 0.49 at ~11 s, up from 0.36 with compact context. A frontier model scores 0.65 in 5 s.

  • Latency
  • Evals
  • Local models
  • Qwen3.8
  • Retrieval
  • RAG
  • Langfuse
MCP tool over a durable agent
·2 min read

An MCP tool backed by Temporal workflows: agent runs survive worker crashes, stay within a hard USD cost budget, and work with OpenAI, Claude, or local models. End-to-end OpenTelemetry tracing with token and cost tracking.

  • MCP
  • Budget constraint
  • Temporal
  • Elasticsearch
  • OpenTelemetry
  • Langfuse