[All work](https://boiko.ai/work/)

Agent evaluation · Instruction enforcement · Created by [Andrii Boiko](https://boiko.ai/about/)

# obey: Coding Agent Instruction Compliance

obey by Andrii Boiko evaluates coding agent instruction compliance, generates enforcement hooks, and compares behavior with and without repository rules.

[Explore the repository](https://github.com/B0yko/obey)

System map

## [Turn an instruction into a check.](https://boiko.ai/work/obey/#system-map-obey)

1. Extract

   ### Repository rules

   Review supported instructions and mark judgment-based rules uncheckable.

2. Probe

   ### Recorded behavior

   Compare actions with and without the rule on designed tasks.

3. Enforce

   ### Hooks and retesting

   Use the same rule engine in hooks, then measure violations and evasions.

A measured guardrail with explicit support limits; hooks are not a security boundary.

## [Test the instructions against the work.](https://boiko.ai/work/obey/#test-the-instructions-against-the-work)

Repository instructions can say which package manager to use, which paths to leave alone, or when to run tests. obey turns those rules into checks and small tasks that give a coding agent an opportunity to break them. The report compares recorded actions with the rule, using commands, file changes, commits, and pushed refs.

The extractor reads AGENTS.md, CLAUDE.md, GEMINI.md, Cursor rules, and Copilot instructions. It maps supported instructions to twelve deterministic checker types; rules that need judgment remain uncheckable, and optional LLM suggestions require review.

## [Measure before and after enforcement.](https://boiko.ai/work/obey/#measure-before-and-after-enforcement)

By default, each probe runs in a fresh container, alongside a baseline with the relevant instruction removed. A model gateway keeps the provider key outside the agent container and meters spending; a separate proxy limits network access. Tracked repository files enter the run and may be sent to the chosen model provider.

The same Python rule engine scores recorded runs and powers generated hooks. Reports separate violations from cases where the rule was never exercised, show confidence intervals, and count attempts that still break a rule after a hook blocks the direct route.

## [A study that also tested the checkers.](https://boiko.ai/work/obey/#a-study-that-also-tested-the-checkers)

The main experiment recorded 2,241 runs of Claude Haiku 5.5 through Claude Code and OpenCode on two authored repositories. After checker corrections, the agent followed a present instruction in all 1,474 exercised runs; without the instruction, it complied in 397 of 488.

Reviewing the traces exposed 136 checker false positives, all corrected from violated to obeyed after the outcomes were known. The project publishes both analyses and the deviations. In a separate exploratory comparison on selected rules broken by the baseline, hooks raised compliance from 47 of 138 exercised runs to 137 of 137; none of 123 blocked runs ended in a measured evasion.

[Recorded study and deviations](https://github.com/B0yko/obey/blob/a5f81afb7611bbd450169801842dfa15c1ce0cd3/study/REPORT.md)[Pre-registered methodology](https://github.com/B0yko/obey/blob/a5f81afb7611bbd450169801842dfa15c1ce0cd3/study/PREREGISTRATION.md)

## [Guardrails with explicit limits.](https://boiko.ai/work/obey/#guardrails-with-explicit-limits)

These are results on designed probes, one model, and two agent harnesses. They do not establish everyday compliance, universal phrasing advice, or a security boundary. Hooks allow the action with a warning if they encounter an internal error, and some edit routes can bypass a pre-tool check.

Hook generation covers six agents plus Git pre-commit checks, with different capabilities and levels of validation. The committed study exercised generated hooks in Claude Code and OpenCode; Gemini and Cursor support relies on contract tests, and Cursor has no headless runner in this release.

[Agent support and validation](https://github.com/B0yko/obey/blob/a5f81afb7611bbd450169801842dfa15c1ce0cd3/docs/agents.md)[Enforcement behavior](https://github.com/B0yko/obey/blob/a5f81afb7611bbd450169801842dfa15c1ce0cd3/docs/enforce.md)

## [Related work](https://boiko.ai/work/obey/#related-work-heading)

- [proof-of-done: checking verification evidence](https://boiko.ai/work/proof-of-done/)

- [Agent Cheat Index: inspecting agent behavior](https://boiko.ai/work/cheat-index/)

Author: [Andrii Boiko](https://boiko.ai/about/)

## Contact and profiles

- [Andrii Boiko on GitHub (B0yko)](https://github.com/B0yko)
- [Andrii Boiko on LinkedIn](https://www.linkedin.com/in/andriyboyko)
- [Andrii Boiko on Hugging Face](https://huggingface.co/aboiko)
- [Email](mailto:andrewvboyko@gmail.com)

Source: [https://boiko.ai/work/obey/](https://boiko.ai/work/obey/)