* perf: stop re-parsing the whole tool-argument buffer on every streamed chunk
Converting an OpenAI stream to Anthropic events buffers each tool call's arguments and, to find out when the JSON is complete, parsed the entire buffer again on every chunk. A tool call with large arguments pays that parse thousands of times, and the cost grows with the square of the argument size.
The parse now runs only when the buffer could actually be complete. A JSON object can only close on its final brace, so a chunk that does not end there cannot complete it. Arguments that are not an object, or that start with whitespace, keep parsing on every chunk exactly as before.
Measured on CPython 3.12 with 130 KB of tool arguments over 7648 chunks:
| | before | after |
|---|---|---|
| parses | 7648 | 1 |
| time | 382 ms | 0.82 ms |
The block closes on exactly the same chunk as before, verified by replaying randomized fragmentations of objects with braces inside strings, escaped characters, unicode escapes, arrays, bare scalars, leading and trailing whitespace and a buffer that never completes, against both JSON backends.
* refactor: read tool['arguments'] directly in the JSON completion guard
Restores the pre-existing comment above the guard to its original wording and drops the `buffered` local, so the guard and the parse call both read `tool['arguments']`, the name the rest of the file already uses for that buffer. Behaviour is unchanged: same three conditions in the same order, same short-circuit result.
* perf: strip whitespace in the tool-argument completion guard
The character guard only looked at the first and last byte of the buffer, so a
chunk that ended in a space still triggered a full parse and a tool argument
with leading whitespace fell back to parsing on every chunk. Stripping first
collapses both cases to a single parse at the end of the stream.
Measured on a streamed tool call, parses and wall time for the whole stream,
orjson on the left of the slash and stdlib json on the right:
| argument shape | before | after |
|---|---|---|
| 20 KB string, char-by-char deltas | 3678 parses, 56 / 28 ms | 1 parse, 3.1 / 3.1 ms |
| 200 KB, 20-char deltas | 1473 parses, 176 / 63 ms | 1 parse, 3.1 / 2.8 ms |
| 8 KB prose, leading whitespace | 715 parses, 5.5 / 2.5 ms | 1 parse, 0.18 ms |
| 8 KB of spaces inside a value | 713 parses, 6.0 / 2.9 ms | 1 parse, 0.83 / 0.72 ms |
The strip costs about 20 ns per delta on arguments that have no whitespace at
either end, which is where the old form was already optimal: a 20 KB compact
argument goes from 191 to 216 us over 1786 deltas. Soundness is unchanged, the
guard can still only skip a parse that would have failed: 2660892 buffers
(exhaustive to length 6 over a JSON-lexical alphabet, every prefix of 26 named
cases with a trailing byte appended, and every codepoint below U+3000 after a
complete document) with zero cases where a parse would have succeeded.
* fix: use the pooled client timeout for the Anthropic Messages passthrough
The native `/api/v1/messages` passthrough still referenced `openai.AIOHTTP_CLIENT_TIMEOUT`, which stopped existing when `routers/openai.py` moved onto `session_pool.get_client_timeout()`. Every passthrough request therefore raised `AttributeError: module 'open_webui.routers.openai' has no attribute 'AIOHTTP_CLIENT_TIMEOUT'` before it was sent, and the surrounding handler turned that into a 502 "Open WebUI: Server Connection Error", so Anthropic-format clients such as Cline could not reach any model at all.
Use `get_client_timeout(stream=...)` like the OpenAI and Ollama proxies do, so the configured `AIOHTTP_CLIENT_TIMEOUT` applies and streaming requests additionally get the idle-read timeout.
Fixes#27595
* fix: authenticate native Anthropic requests with x-api-key
The Anthropic Messages passthrough and the token-count forwarding both build their upstream request through `get_anthropic_request_target`, which sends the connection key as `Authorization: Bearer <key>`. Anthropic's OpenAI-compatible `/chat/completions` endpoint accepts that, which is why the model works in the chat UI, but the native `/v1/messages` and `/v1/messages/count_tokens` endpoints do not: they require the key in `x-api-key` and reject a bearer token with 401 `Invalid bearer token` (and `jwt auth is not yet supported on count_tokens`). They also require an `anthropic-version` header, which was never sent.
For `api.anthropic.com` connections, send `anthropic-version` and move the key into `x-api-key`, dropping the bearer header. Connections using session, OAuth or Entra ID auth keep their token untouched, LiteLLM passthrough connections are unaffected, and admin-configured custom headers still win over both defaults.
Fixes#27695
