Mission control for 350 agents

The Agent Control Panel was my answer at Decasonic: one login where every agent is observable, measurable and remotely controllable. It runs as a modular monolith, one Node process serving the API and the React app, with 9 tables and about 60 endpoints. No message broker, no worker fleet. The interesting decisions were about who it serves and what it refuses to hide.

Four people, four questions

OperatorsIs anything broken or burning money?Outages and runaway spend, at a glance

Prompt engineersCan I change it safely?Edit, version and roll back without a deploy

FinanceWhat does each agent cost?Token and dollar cost per agent, per app, per day

DevelopersHow do I connect a new app?A five-step integration wizard

The four audiences the control plane serves, and the question each one asks first

A health score anyone can read

Every connected app reports heartbeats and runs. Each agent gets a status (running, idle, paused, stopped or error) and a rolling health score recomputed on every run and heartbeat. I kept the formula deliberately simple, so an operator can check it in their head.

health = 100 − (errors in the last 20 runs ÷ 20 × 100)

Example: 3 errors in 20 runshealth 85

The rolling health score, with an example window of 20 runs (orange marks an error)

Control without redeploying

Pause, resume, stop or restart an agent, swap its model, or push a new system prompt, while the target app keeps running. Commands travel as HMAC-SHA256-signed webhooks with exponential-backoff retry, and each one has a visible lifecycle so nobody wonders whether it landed.

Commandpause · model · prompt

Pendingqueued and signed

Sentwebhook delivered

Acknowledgedthe app confirms

Failedretry with backoff

Every remote command moves through the same visible lifecycle

Prompts are versioned assets with one-click rollback. Feedback folds straight into a new version through tags like [INSTRUCTION], [IMPROVEMENT], [BUG FIX] and [KEEP DOING], and an AI reviewer scores answers from 0 to 100 with a written reason.

The platform watches itself

The panel’s own AI features are modelled as eight real internal agents, so the platform’s LLM spend is tracked with exactly the same machinery as everyone else’s.

Integration AnalyzerCode GeneratorQuality ReviewerPricing Searcher APricing Searcher BPricing JudgeError ExplainerError Solver

The eight internal agents, each with its own editable prompt and model

Prices that check their own sources

Cost tracking is only as good as the price table behind it, and model prices change. So pricing discovery follows a protocol that spends tokens only when it has to.

Public pricing APIfree and deterministic

Two searchersrun in parallel, only on failure

Disagree?each re-checks against the other

Judgea third model decides

The pricing discovery protocol. Every step goes to a capped audit log, with per-model cooldowns and a global concurrency cap

Small things that matter in production

A heartbeat can never silently un-pause an agent someone stopped.
Token usage is upserted atomically, so concurrent heartbeats can’t lose counts.
When a provider fails, the fallback chain answers and the UI shows an amber badge, so an operator is never misled about which model replied.
Prompt and feedback bodies are base64-encoded on the way in, because a production firewall kept reading legitimate prompt text as an attack.

You can’t improve agents you can’t see. Visibility first, then control, then cost.