Two related integrity gaps in the think channel, filed together because they break the same contract: downstream consumers should be able to trust that final_answer is an actual answer and that think markup is well-formed.
1. Salvage promotes think fragments into final_answer with no marker
When a run hits the context limit, the salvage path can take a truncated reasoning fragment and emit it as final_answer. The result is an "answer" containing an orphan </think> (or trailing off mid-reasoning), indistinguishable from a genuine answer by any flag in the payload. Consumers only discover this by scanning answer text for stray think markup — we lost a full data-cleaning pass to this before knowing to look.
Proposal: salvage output must be marked (is_salvaged: true, or finish_reason: context_limit_salvaged); a think fragment should never be promoted to final_answer silently. Ideally the salvaged text goes into a separate field and final_answer stays null.
2. Think-tag misalignment when the backend has no reasoning parser
When the serving backend is launched without a reasoning parser (e.g. sglang without --reasoning-parser), the chat template emits the opening <think> tag inside the generation prompt, so the completion contains only bare reasoning text plus a closing </think>. Raven records this as-is: trajectories carry a broken protocol (closing tag with no opening tag), and every consumer ends up hand-writing the same canonicalization.
The root cause is arguably serving-side configuration — but Raven is the layer that actually sees the malformed stream, and is best positioned to either normalize it (re-attach the opening tag) or at minimum emit a loud warning that the backend's reasoning parsing looks misconfigured.
Proposal: detect unbalanced think tags in assistant messages at ingestion time; normalize, or warn loudly.
Two related integrity gaps in the think channel, filed together because they break the same contract: downstream consumers should be able to trust that
final_answeris an actual answer and that think markup is well-formed.1. Salvage promotes think fragments into
final_answerwith no markerWhen a run hits the context limit, the salvage path can take a truncated reasoning fragment and emit it as
final_answer. The result is an "answer" containing an orphan</think>(or trailing off mid-reasoning), indistinguishable from a genuine answer by any flag in the payload. Consumers only discover this by scanning answer text for stray think markup — we lost a full data-cleaning pass to this before knowing to look.Proposal: salvage output must be marked (
is_salvaged: true, orfinish_reason: context_limit_salvaged); a think fragment should never be promoted tofinal_answersilently. Ideally the salvaged text goes into a separate field andfinal_answerstays null.2. Think-tag misalignment when the backend has no reasoning parser
When the serving backend is launched without a reasoning parser (e.g. sglang without
--reasoning-parser), the chat template emits the opening<think>tag inside the generation prompt, so the completion contains only bare reasoning text plus a closing</think>. Raven records this as-is: trajectories carry a broken protocol (closing tag with no opening tag), and every consumer ends up hand-writing the same canonicalization.The root cause is arguably serving-side configuration — but Raven is the layer that actually sees the malformed stream, and is best positioned to either normalize it (re-attach the opening tag) or at minimum emit a loud warning that the backend's reasoning parsing looks misconfigured.
Proposal: detect unbalanced think tags in assistant messages at ingestion time; normalize, or warn loudly.