Skip to content

Skill Forge rewriter/gate ignore agents.defaults.model, fall back to region-blocked provider default #155

Description

@arelchan

Summary

Skill Forge's auxiliary LLM calls (the query rewriter and the LLM gate)
do not use the configured agent model (agents.defaults.model). When no
Skill-Forge-specific model is set, they fall through to the provider's
hardcoded default_model, which for the openrouter provider is
openrouter/anthropic/claude-sonnet-4-5. If that default model is unavailable
(e.g. region-blocked), the rewrite/gate step hard-fails even though the agent is
configured with a working model -- so the main turn succeeds while
skill.rewrite / skill.gate error out.

Evidence (from tracing)

Config: agents.defaults.model = "minimax/minimax-m3", provider openrouter.

In one turn, the trace shows:

  • llm.call with invocation_source = session.turn (main reasoning) ->
    llm.model = minimax/minimax-m3, status OK.
  • llm.call with invocation_source = skill.rewrite ->
    llm.model = openrouter/anthropic/claude-sonnet-4-5, status ERROR:
    OpenrouterException - {"error":{"message":"This model is not available in your region.","code":403}}.

So the rewriter used a different model than the configured minimax/minimax-m3,
and that model is region-blocked. The rewriter then falls back to
need_retrieval = True with an empty rewritten query.

Root cause

  • raven/context_engine/factory.py: QueryRewriter(provider, max_tokens=...)
    is constructed with no model; LLMGateFilter(..., model=skill_forge.llm_gate_model or None, ...)
    is None unless llm_gate_model is set.
  • QueryRewriter.analyze / LLMGateFilter.filter call
    provider.chat_with_retry(...) without a model, so they use
    provider.default_model.
  • raven/providers/registry.py: the openrouter provider spec hardcodes
    default_model = "openrouter/anthropic/claude-sonnet-4-5".

The main agent loop passes model=agents.defaults.model explicitly on every
call, so it is unaffected; only the auxiliary Skill-Forge calls hit the
hardcoded default.

Impact

  • Query rewrite and skill gate fail whenever the provider's hardcoded default
    model is unavailable/region-blocked, regardless of the configured agent model.
  • The rewriter has no config override (unlike the gate's llm_gate_model),
    so this cannot be worked around by configuration alone.

Proposed fix

Make the Skill-Forge auxiliary calls inherit agents.defaults.model when no
Skill-Forge-specific model is configured:

  • Thread the configured agent model into QueryRewriter / LLMGateFilter in
    factory.py (add a model parameter to QueryRewriter).
  • Pass that model on the chat_with_retry calls, so rewrite/gate use the same
    working model as the main loop by default; a Skill-Forge override (e.g.
    llm_gate_model) still wins when set.

Workarounds (until fixed)

  • Gate: set skill_forge.llm_gate_model to a model available in your region
    (e.g. minimax/minimax-m3).
  • Rewriter: no config knob; either disable it (skill_forge.rewrite_enabled = false)
    or apply the code fix above.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions