Evaluator-Optimizer Refinement Loop

One model generates, another grades against a rubric, and the loop repeats until the rubric passes or a limit is hit.

Useful when quality criteria are clear and checkable (translations, code that must pass tests, copy with constraints).

Steps

1. Generator produces a draft.
2. Evaluator scores it against an explicit rubric and returns structured feedback.
3. If pass is false, feed the feedback to the generator and regenerate.
4. Stop at pass: true or after N rounds; return the best-scoring draft.

Evaluator prompt

You are a strict reviewer. Grade the DRAFT against each criterion.
Return JSON only:
{"scores":{"accuracy":0-5,"constraints":0-5,"clarity":0-5},
 "pass": true|false,
 "fixes": ["specific, actionable change", ...]}
pass is true only if every score >= 4.
CRITERIA: {{rubric}}
DRAFT: {{draft}}

Config

max_rounds: 3
keep: best_score
prefer_programmatic_checks: true   # run tests/linters before the LLM judge

Failure modes

• Evaluator and generator share blind spots when they are the same model.
• Loops oscillate without converging; enforce a max-iteration limit.
• Vague rubrics give inconsistent scores; use concrete pass/fail criteria.

Markdown version: /blueprints/evaluator-optimizer.md

Related blueprints