Skip to content

Commit 980ff9b

Browse files
authored
feat(ai-openrouter): surface per-request cost on RUN_FINISHED (#654)
* feat(openrouter): surface per-request cost on RUN_FINISHED OpenRouter reports the actual cost of each request inline on the chat response. Forward it on the terminal RUN_FINISHED event as usage.cost, with OpenRouter's per-request breakdown under usage.costDetails. This is the cost OpenRouter itself reports, so it accounts for routing, fallback providers, BYOK, and cached-token pricing rather than being computed from token counts. Both the Chat Completions (openRouterText) and Responses (openRouterResponsesText) adapters populate it. A shared UsageTotals type in @tanstack/ai carries the optional cost/costDetails fields, so they're also available on the middleware onUsage and onFinish hooks. Cost-detail keys are normalized to camelCase so the SDK-parsed and raw fallback paths stay consistent. The fields are optional and additive; adapters that don't report cost are unaffected. * test(e2e): drain request body in OpenRouter cost mount Match the other aimock mounts (drainBody) so the keep-alive socket has no unread request bytes before the SSE response is written. * refactor(ai): close UsageCostDetails to known breakdown fields Replace the open `Record<string, number | null | undefined>` shape on `UsageTotals.costDetails` with a typed `UsageCostDetails` interface enumerating the five fields OpenRouter actually reports across its Chat Completions and Responses endpoints. Consumers get autocomplete on the breakdown and a typo on a key becomes a compile error; unknown keys are dropped at extraction time so the public surface stays closed. The extractor swaps generic `toCamelCase` for a snake↔camel allowlist keyed on the known fields, and treats `null` as absent rather than forwarding it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump @tanstack/ai as patch, not minor The @tanstack/ai changes are additive type plumbing only (new exported UsageTotals/UsageCostDetails interfaces, optional fields on existing shapes) — no runtime change, no new callable surface. The feature lives in @tanstack/ai-openrouter. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(ai): normalize cost breakdown onto a canonical provider-neutral shape UsageCostBreakdown becomes three concrete fields (upstreamCost, upstreamInputCost, upstreamOutputCost) that every adapter maps its provider-specific wire keys onto at extraction time. Consumer code reads the same fields regardless of which gateway populated them, so swapping adapters is a one-line change with no consumer rewrites. The OpenRouter adapter collapses its two endpoint naming styles (Chat Completions' prompt/completions and Responses' input/output) onto the same canonical input/output split — they bill against the same tokens. Replaces the prior declaration-merging approach, which leaked OpenRouter vocabulary into every consumer site that read costDetails. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: apply automated fixes
1 parent d5645cf commit 980ff9b

16 files changed

Lines changed: 853 additions & 20 deletions

File tree

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
---
2+
'@tanstack/ai-openrouter': minor
3+
'@tanstack/ai': minor
4+
---
5+
6+
Surface OpenRouter's per-request cost on `RUN_FINISHED.usage`.
7+
8+
OpenRouter reports the actual cost of each request inline on the chat response.
9+
The `openRouterText` and `openRouterResponsesText` adapters now forward that
10+
value on the terminal `RUN_FINISHED` event as `usage.cost`, with OpenRouter's
11+
per-request breakdown under `usage.costDetails`. This is the cost OpenRouter
12+
itself reports — it is not computed locally from token counts, so it accounts
13+
for routing, fallback providers, BYOK, and cached-token pricing.
14+
15+
`@tanstack/ai` adds a shared `UsageTotals` type with optional `cost` and
16+
`costDetails` fields, plus a provider-neutral `UsageCostBreakdown` interface
17+
with three canonical fields (`upstreamCost`, `upstreamInputCost`,
18+
`upstreamOutputCost`). Each adapter's extractor normalizes its provider's
19+
wire-shape onto this canonical form, so consumer code reads the same fields
20+
regardless of which gateway populated them — swapping adapters is a one-line
21+
change with no consumer rewrites. The OpenRouter adapter collapses its two
22+
endpoint naming styles (Chat Completions' `prompt`/`completions` and
23+
Responses' `input`/`output`) onto the same canonical input/output split, since
24+
they bill against the same tokens. `RunFinishedEvent.usage`, the middleware
25+
`UsageInfo` (`onUsage`), and `FinishInfo.usage` (`onFinish`) all use
26+
`UsageTotals`. The fields are optional and additive — adapters that do not
27+
report cost are unaffected.

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -186,7 +186,7 @@ Official adapters include:
186186

187187
| Package | Use it for |
188188
| ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------ |
189-
| [`@tanstack/ai-openrouter`](https://tanstack.com/ai/latest/docs/adapters/openrouter) | 300+ models through one OpenRouter API |
189+
| [`@tanstack/ai-openrouter`](https://tanstack.com/ai/latest/docs/adapters/openrouter) | 300+ models through one OpenRouter API, with per-request cost tracking |
190190
| [`@tanstack/ai-openai`](https://tanstack.com/ai/latest/docs/adapters/openai) | OpenAI chat, image, video, speech, transcription, realtime, and provider tools |
191191
| [`@tanstack/ai-anthropic`](https://tanstack.com/ai/latest/docs/adapters/anthropic) | Anthropic Claude chat, thinking, tools, and structured outputs |
192192
| [`@tanstack/ai-gemini`](https://tanstack.com/ai/latest/docs/adapters/gemini) | Google Gemini chat, image, speech, and audio generation |

docs/adapters/openrouter.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -169,6 +169,38 @@ Caveats while the Responses adapter is in beta:
169169
- If in doubt, prefer `openRouterText`. The Chat Completions endpoint has
170170
broader provider coverage and feature parity today.
171171

172+
## Cost Tracking
173+
174+
OpenRouter reports the actual cost of each request inline on the streamed
175+
response. When present, the adapter forwards it on the terminal `RUN_FINISHED`
176+
event under `usage.cost`, with OpenRouter's per-request breakdown under
177+
`usage.costDetails`. This is the cost OpenRouter itself reports for the
178+
request — it is **not** computed locally from token counts, so it already
179+
accounts for routing, fallback providers, BYOK, and cached-token pricing. See
180+
OpenRouter's [Usage Accounting](https://openrouter.ai/docs/use-cases/usage-accounting)
181+
docs for the meaning and units of these fields.
182+
183+
```typescript
184+
import { chat } from "@tanstack/ai";
185+
import { openRouterText } from "@tanstack/ai-openrouter";
186+
187+
for await (const chunk of chat({
188+
adapter: openRouterText("openai/gpt-5"),
189+
messages: [{ role: "user", content: "Hello!" }],
190+
})) {
191+
if (chunk.type === "RUN_FINISHED") {
192+
console.log("cost:", chunk.usage?.cost);
193+
console.log("breakdown:", chunk.usage?.costDetails);
194+
}
195+
}
196+
```
197+
198+
The same `usage` (including `cost` / `costDetails`) is passed to middleware via
199+
the `onUsage` and `onFinish` hooks. When OpenRouter does not report a cost, the
200+
fields are simply absent and the stream completes normally. Both
201+
`openRouterText` and `openRouterResponsesText` populate cost when OpenRouter
202+
returns it.
203+
172204
## Next Steps
173205

174206
- [Getting Started](../getting-started/quick-start) - Learn the basics
Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
/**
2+
* Helpers for extracting OpenRouter's provider-reported per-request cost from the
3+
* SDK usage object and shaping it for `RUN_FINISHED.usage`.
4+
*
5+
* OpenRouter returns an authoritative per-request `cost` plus an optional
6+
* `cost_details` breakdown. We forward `cost` verbatim and normalize the
7+
* breakdown onto `@tanstack/ai`'s canonical `UsageCostBreakdown` shape — so
8+
* consumer code reads the same three fields regardless of which adapter (or
9+
* which OpenRouter endpoint) produced them. OpenRouter exposes the breakdown
10+
* under two naming families (Chat Completions: `prompt`/`completions`,
11+
* Responses: `input`/`output`); both map onto the same canonical input/output
12+
* split, because they bill against the same tokens.
13+
*
14+
* Input is intentionally typed `unknown`: callers pass usage objects whose static
15+
* types are narrowed to token-only fields (notably the Responses adapter), and the
16+
* Responses usage normalizer can leave `cost_details` in snake_case. Reading both
17+
* `costDetails` and `cost_details` and narrowing here keeps every call site simple.
18+
*/
19+
20+
import type { UsageCostBreakdown } from '@tanstack/ai'
21+
22+
export interface ExtractedCost {
23+
cost?: number
24+
costDetails?: UsageCostBreakdown
25+
}
26+
27+
/**
28+
* Wire-key → canonical-key mapping. Snake_case keys come from the raw/UNKNOWN
29+
* `response.completed` fallback in the Responses adapter; camelCase keys come
30+
* from the SDK-parsed path. Both Chat Completions' prompt/completions naming
31+
* and Responses' input/output naming collapse onto `upstreamInputCost` /
32+
* `upstreamOutputCost`.
33+
*/
34+
const KNOWN_DETAIL_KEYS: Record<string, keyof UsageCostBreakdown> = {
35+
upstream_inference_cost: 'upstreamCost',
36+
upstreamInferenceCost: 'upstreamCost',
37+
upstream_inference_prompt_cost: 'upstreamInputCost',
38+
upstreamInferencePromptCost: 'upstreamInputCost',
39+
upstream_inference_input_cost: 'upstreamInputCost',
40+
upstreamInferenceInputCost: 'upstreamInputCost',
41+
upstream_inference_completions_cost: 'upstreamOutputCost',
42+
upstreamInferenceCompletionsCost: 'upstreamOutputCost',
43+
upstream_inference_output_cost: 'upstreamOutputCost',
44+
upstreamInferenceOutputCost: 'upstreamOutputCost',
45+
}
46+
47+
function asRecord(value: unknown): Record<string, unknown> | undefined {
48+
return typeof value === 'object' && value !== null
49+
? (value as Record<string, unknown>)
50+
: undefined
51+
}
52+
53+
/**
54+
* Narrow a raw `cost_details`/`costDetails` map to the canonical fields of
55+
* `UsageCostBreakdown`. Negative values (e.g. discounts) are preserved; `null`,
56+
* non-finite numbers, non-numeric values, and unknown keys are dropped.
57+
*/
58+
function extractCostDetails(details: unknown): UsageCostBreakdown | undefined {
59+
const record = asRecord(details)
60+
if (!record) return undefined
61+
62+
const out: UsageCostBreakdown = {}
63+
for (const [rawKey, value] of Object.entries(record)) {
64+
const key = KNOWN_DETAIL_KEYS[rawKey]
65+
if (!key) continue
66+
if (typeof value === 'number' && Number.isFinite(value)) {
67+
out[key] = value
68+
}
69+
}
70+
71+
return Object.keys(out).length > 0 ? out : undefined
72+
}
73+
74+
/**
75+
* Extract `cost`/`costDetails` from a provider usage object.
76+
*
77+
* - `cost` is attached only when it is a finite number — this preserves `cost === 0`
78+
* and rejects `NaN`/`Infinity`, and does not clamp negative values.
79+
* - `costDetails` is attached only alongside a valid `cost` (an orphan breakdown
80+
* without a total cannot be reconciled and is dropped). Both camelCase
81+
* `costDetails` and snake_case `cost_details` are read.
82+
*
83+
* Returns an empty object when no usable cost is present, so call sites can spread
84+
* the result unconditionally.
85+
*/
86+
export function extractUsageCost(usage: unknown): ExtractedCost {
87+
const record = asRecord(usage)
88+
if (!record) return {}
89+
90+
const cost = record.cost
91+
if (typeof cost !== 'number' || !Number.isFinite(cost)) return {}
92+
93+
const costDetails = extractCostDetails(
94+
record.costDetails ?? record.cost_details,
95+
)
96+
97+
return {
98+
cost,
99+
...(costDetails && { costDetails }),
100+
}
101+
}

packages/ai-openrouter/src/adapters/responses-text.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ import { convertFunctionToolToResponsesFormat } from '../internal/responses-tool
99
import { isWebSearchTool } from '../tools/web-search-tool'
1010
import { isWebFetchTool } from '../tools/web-fetch-tool'
1111
import { getOpenRouterApiKeyFromEnv } from '../utils'
12+
import { extractUsageCost } from './cost'
1213
import type { SDKOptions } from '@openrouter/sdk'
1314
import type { ResponsesFunctionTool } from '../internal/responses-tool-converter'
1415
import type {
@@ -623,6 +624,7 @@ export class OpenRouterResponsesTextAdapter<
623624
promptTokens: usage.inputTokens ?? 0,
624625
completionTokens: usage.outputTokens ?? 0,
625626
totalTokens: usage.totalTokens ?? 0,
627+
...extractUsageCost(usage),
626628
},
627629
}),
628630
}
@@ -1433,6 +1435,7 @@ export class OpenRouterResponsesTextAdapter<
14331435
promptTokens: responseObj.usage?.inputTokens || 0,
14341436
completionTokens: responseObj.usage?.outputTokens || 0,
14351437
totalTokens: responseObj.usage?.totalTokens || 0,
1438+
...extractUsageCost(responseObj.usage),
14361439
},
14371440
finishReason,
14381441
}

packages/ai-openrouter/src/adapters/text.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ import { extractRequestOptions } from '../internal/request-options'
77
import { makeStructuredOutputCompatible } from '../internal/schema-converter'
88
import { convertToolsToProviderFormat } from '../tools'
99
import { getOpenRouterApiKeyFromEnv } from '../utils'
10+
import { extractUsageCost } from './cost'
1011
import type { SDKOptions } from '@openrouter/sdk'
1112
import type {
1213
ChatContentItems,
@@ -549,6 +550,7 @@ export class OpenRouterTextAdapter<
549550
promptTokens: lastUsage.promptTokens,
550551
completionTokens: lastUsage.completionTokens,
551552
totalTokens: lastUsage.totalTokens,
553+
...extractUsageCost(lastUsage),
552554
},
553555
}),
554556
}
@@ -1076,6 +1078,7 @@ export class OpenRouterTextAdapter<
10761078
promptTokens: lastUsage.promptTokens || 0,
10771079
completionTokens: lastUsage.completionTokens || 0,
10781080
totalTokens: lastUsage.totalTokens || 0,
1081+
...extractUsageCost(lastUsage),
10791082
},
10801083
}),
10811084
finishReason,
Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
import { describe, expect, it } from 'vitest'
2+
import { extractUsageCost } from '../src/adapters/cost'
3+
4+
describe('extractUsageCost', () => {
5+
it('extracts a finite cost', () => {
6+
expect(extractUsageCost({ cost: 0.0123 })).toEqual({ cost: 0.0123 })
7+
})
8+
9+
it('preserves cost === 0 (not treated as absent)', () => {
10+
expect(extractUsageCost({ cost: 0 })).toEqual({ cost: 0 })
11+
})
12+
13+
it('returns empty object when cost is absent', () => {
14+
expect(extractUsageCost({ promptTokens: 5 })).toEqual({})
15+
})
16+
17+
it('returns empty object for non-number / non-finite cost', () => {
18+
expect(extractUsageCost({ cost: '0.5' })).toEqual({})
19+
expect(extractUsageCost({ cost: NaN })).toEqual({})
20+
expect(extractUsageCost({ cost: Infinity })).toEqual({})
21+
expect(extractUsageCost({ cost: null })).toEqual({})
22+
})
23+
24+
it('returns empty object for non-object input', () => {
25+
expect(extractUsageCost(undefined)).toEqual({})
26+
expect(extractUsageCost(null)).toEqual({})
27+
expect(extractUsageCost(42)).toEqual({})
28+
})
29+
30+
it('reads costDetails (camelCase) and normalizes to canonical keys', () => {
31+
expect(
32+
extractUsageCost({
33+
cost: 0.01,
34+
costDetails: { upstreamInferenceCost: 0.008 },
35+
}),
36+
).toEqual({ cost: 0.01, costDetails: { upstreamCost: 0.008 } })
37+
})
38+
39+
it('reads cost_details (snake_case) and normalizes to canonical keys', () => {
40+
expect(
41+
extractUsageCost({
42+
cost: 0.01,
43+
cost_details: { upstream_inference_cost: 0.008 },
44+
}),
45+
).toEqual({ cost: 0.01, costDetails: { upstreamCost: 0.008 } })
46+
})
47+
48+
it('collapses Chat Completions prompt/completions onto canonical input/output', () => {
49+
expect(
50+
extractUsageCost({
51+
cost: 0.0042,
52+
cost_details: {
53+
upstream_inference_completions_cost: 0.0026,
54+
upstream_inference_cost: 0.0038,
55+
upstream_inference_prompt_cost: 0.0012,
56+
},
57+
}),
58+
).toEqual({
59+
cost: 0.0042,
60+
costDetails: {
61+
upstreamOutputCost: 0.0026,
62+
upstreamCost: 0.0038,
63+
upstreamInputCost: 0.0012,
64+
},
65+
})
66+
})
67+
68+
it('collapses Responses input/output onto the same canonical input/output', () => {
69+
expect(
70+
extractUsageCost({
71+
cost: 0.0042,
72+
cost_details: {
73+
upstream_inference_cost: 0.0038,
74+
upstream_inference_input_cost: 0.0012,
75+
upstream_inference_output_cost: 0.0026,
76+
},
77+
}),
78+
).toEqual({
79+
cost: 0.0042,
80+
costDetails: {
81+
upstreamCost: 0.0038,
82+
upstreamInputCost: 0.0012,
83+
upstreamOutputCost: 0.0026,
84+
},
85+
})
86+
})
87+
88+
it('prefers camelCase costDetails when both are present', () => {
89+
expect(
90+
extractUsageCost({
91+
cost: 0.01,
92+
costDetails: { upstreamInferenceCost: 1 },
93+
cost_details: { upstream_inference_cost: 2 },
94+
}),
95+
).toEqual({ cost: 0.01, costDetails: { upstreamCost: 1 } })
96+
})
97+
98+
it('preserves negative detail values (e.g. cache discount)', () => {
99+
expect(
100+
extractUsageCost({
101+
cost: 0.01,
102+
costDetails: { upstreamInferenceCost: -0.002 },
103+
}),
104+
).toEqual({ cost: 0.01, costDetails: { upstreamCost: -0.002 } })
105+
})
106+
107+
it('drops null, non-finite, and non-numeric detail entries', () => {
108+
expect(
109+
extractUsageCost({
110+
cost: 0.01,
111+
costDetails: {
112+
upstreamInferenceCost: 0.5,
113+
upstreamInferenceInputCost: null,
114+
upstreamInferenceOutputCost: Infinity,
115+
upstreamInferencePromptCost: NaN,
116+
upstreamInferenceCompletionsCost: 'x',
117+
},
118+
}),
119+
).toEqual({ cost: 0.01, costDetails: { upstreamCost: 0.5 } })
120+
})
121+
122+
it('drops unknown breakdown keys', () => {
123+
expect(
124+
extractUsageCost({
125+
cost: 0.01,
126+
costDetails: {
127+
upstreamInferenceCost: 0.008,
128+
futureUnknownField: 0.001,
129+
},
130+
}),
131+
).toEqual({ cost: 0.01, costDetails: { upstreamCost: 0.008 } })
132+
})
133+
134+
it('omits costDetails entirely when no known entries remain', () => {
135+
expect(
136+
extractUsageCost({ cost: 0.01, costDetails: { unknownKey: 1 } }),
137+
).toEqual({ cost: 0.01 })
138+
})
139+
140+
it('drops an orphan costDetails when cost is absent', () => {
141+
expect(
142+
extractUsageCost({ costDetails: { upstreamInferenceCost: 0.008 } }),
143+
).toEqual({})
144+
})
145+
})

0 commit comments

Comments
 (0)