Complexity-Based Cheap/Strong Model Router

Route easy requests to a cheap model and hard ones to a strong model using a lightweight classifier plus escalation.

Strategy

1. Heuristic pre-filter (length, presence of code, math, multi-step words).
2. Cheap classifier call returns simple or complex.
3. simple → cheap model; complex → strong model.
4. Escalate: if the cheap model's answer fails validation (schema error, low self-reported confidence, tests fail), retry on the strong model.

Router

def route(prompt: str) -> str:
    if len(prompt) > 6000 or "```" in prompt:
        return "strong"
    label = classify(prompt)        # cheap model, max_tokens=1, temperature=0
    return "strong" if label == "complex" else "cheap"

CLASSIFIER_PROMPT = """Label the task as 'simple' (lookup, rewrite, short answer)
or 'complex' (multi-step reasoning, code, math, analysis). Reply with one word.
Task: {prompt}"""

Config

models: {cheap: small-fast-model, strong: frontier-model}
escalate_on: [json_parse_error, validator_fail]
log: [route, model, tokens_in, tokens_out, latency_ms, escalated]

Measure on your own traffic: compare quality on a sample before trusting the savings.

Failure modes

• Classifier misroutes hard requests to the cheap model; escalation on validation failure is required.
• Escalation doubles latency and cost for misrouted requests.
• Routing thresholds drift as models change; re-check against logged traffic.

Markdown version: /blueprints/complexity-based-routing.md

Related blueprints