Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on LiveCodeBench Hard
Five small models solve a programming problem together: a planner, two ideators, a solver and a manager, plus a repair worker. Each starts from a fresh context and talks only through files on disk. A script that runs the samples decides success, not a model. It is the method of GVS5H (Lee, 2026), written as a Memdoor workflow: eight YAML files, no code, no model named.
The authors’ harness rebuilt on the same model, Qwen3.8-27B, with their prompts, loop and 128K reply cap. The first ten problems of their LiveCodeBench Hard list, five passes each, graded on every hidden test with their corrected grader:
The authors’ two columns are the regraded result files in their repository, the numbers in their paper. Our column is bench/results/parity-5pass/: all 50 runs, each with every request and reply, the program, the grader’s output and the cost. A run took a median of 40 minutes and $0.39; the 50 cost $35.79.
What this does and does not show:
The parity check (every request byte for byte against their code), the hosts, the changes it took and the first run’s findings are in NOTES.md. The same result, on one page: memdoor.ai/docs/team-solve.
For your own problem, replace task.md and put paired tests/N.in / tests/N.out beside it. The program is solution.py, read from stdin, compared to the samples with diff; it needs Python 3 and bash. /workflow in memdoor tui draws the run live.
The steps name no model: whatever /model is pinned to answers, a local one included (how to connect one). To change a role, change its prompt in steps/.
Every agent prompt says “fresh context, read the files, do not rely on chat”. The paper credits that ledger discipline for the gain. Each step is one YAML file in steps/, in the format of mario, the engine underneath; requires: is the DAG.