Saad Al-TamariAgent Skills OS is a content system and test lab for agent instructions. A skill is written, versioned, A/B tested against a real agent, judged, improved if it loses, and only then deployed into production prompts.
Test on the real thing
The test lab runs the same prompt through a real, synced production agent twice: once with its normal system prompt, once with the skill added, using that agent’s actual model. An LLM judge compares the two outputs and returns a structured verdict. Results reflect how the agent really behaves, not a simulation of it.
Losing is useful
If the skill loses, a frontier model rewrites it into a new version, using both test outputs and the judge’s reasoning. That version is tested again. Skills improve through evaluation rather than through whoever last edited them.
Fail open, on purpose
Agents pull their enabled skills at prompt-fetch time. If this service is unreachable, agents fall back to their base prompts instead of breaking. It is a deliberate availability trade: an outage in the skills service must never take down the agent fleet.
Skills service down, so every agent that depends on it errors. One service takes down the fleet.
Skills service down, so agents run on their base prompts. Quality dips a little; nothing breaks.
Suggestions you can trust
A suggestions engine reads the real skills, agents, tests and assignments and proposes new skills. Any agent or skill identifier it hallucinates is filtered out by schema validation before it reaches the screen.
If you can’t measure a skill against the agent that will use it, you are guessing.