Commit graph

14 commits

Author SHA1 Message Date
Yujong Lee
868cd06ea9 ci: drop the uv sync retry loop and call uv sync directly
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
uv already retries network failures, so the wrapper only repeated deterministic uv sync failures five times. Cargo network settings it exported already live in .cargo/config.toml
2026-10-19 10:46:32 -07:00
devin-ai-integration[bot]
48050d9646
chore(ci): drop litellm_internal_staging and litellm_oss_staging references, main is the only trunk (#42745) 2026-09-23 08:14:11 -07:00
devin-ai-integration[bot]
0abd9267c1
feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174)
* ci: benchmark and gate an installed release wheel

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: simplify installed-wheel benchmark check

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(rust): add native tokenizer codec

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(tokenizer): route Python tokenization through the Rust extension

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(lint): format tokenizer call

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(packaging): restore runtime dependencies and native images

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(tokenizer): preserve Python SDK behavior with Rust tokenizers

* fix(tokenizer): restore compatibility paths

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(tokenizer): count custom tokenizers directly

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(tokenizer): preserve caller-supplied Python tokenizer counts

* fix(tokenizer): reuse packaged vocabularies in the native wheel

* refactor(rust_bridge): route token counting through the catalog as RUST_OPT_IN

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(spend_tracking): compare tokenizer groups by value

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(deps): re-resolve filelock under the <4.0 pin

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(llms): align transformation override signatures with base configs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* build(rust): use fat LTO to keep the native wheel under the 35 MB limit

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(tokenizer): preserve Python defaults with opt-in Rust dispatch

* test(proxy): tolerate missing litellm.utils.Tokenizer when patching it

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): patch the tokenizer dispatch function instead of the removed alias

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(tokenizer): give the Rust wrappers the tiktoken and tokenizers surface

Callers of litellm.encoding and litellm.create_tokenizer must see the same
read-only API whichever backend the catalog selects.

- OpenAIEncoding mirrors tiktoken.Encoding: n_vocab, max_token_value,
  token_byte_values, encode_single_token, encode_with_unstable,
  encode_to_numpy, decode_with_offsets, is_special_token, repr; the Rust
  tiktoken crate keeps a Vocabulary beside each CoreBPE and reports the
  requested encoding name (gpt2 stays gpt2).
- HuggingFaceTokenizer mirrors the read-only tokenizers.Tokenizer surface
  (token_to_id, id_to_token, get_vocab, get_vocab_size,
  get_added_tokens_decoder, num_special_tokens_to_add, padding, truncation,
  encode_special_tokens, from_buffer); HuggingFaceEncoding gains the
  char/word/token lookups, pad, truncate, set_sequence_id and merge.
  Mutators stay on the Python tokenizer.
- from_json/from_pretrained claim the fork gate only when the huggingface
  feature is compiled in; the surrogate fallback matches on the Codec.
- Tokenizer caching is keyed on the same catalog Context the dispatch runs
  on; rust_tokenizer reads the encoding name without loading an encoding;
  LITELLM_RUST parsing is cached.
- Drop the unused tiktoken_encoding_for_model export and Error::Download.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(tokenizer): close the exhaustive matches with assert_never

CodeQL reads a `match` over a Literal with no default arm as an implicit
`None` return. `assert_never` makes the exhaustiveness explicit for both the
HuggingFace tokenizer loader and the Rust token-counter factory.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(tokenizer): derive the fast counter from the shared tokenizer

The count-only counter (`fast` feature) and the codec each parsed the same
artifact: TokenCounter took the Anthropic JSON and the tiktoken rank files
from Python while Tokenizer loaded them again. One parse now serves both.

- FastTokenizer builds from a model another loader holds: `from_shared`
  takes the Arc<tokenizers::Tokenizer> the HF codec keeps, and
  `from_*_pairs` take the ranks the tiktoken vocabulary already parsed.
- `FastCounter::fast_counter` in the core crate derives it from either codec;
  encodings the fast scanner does not reproduce are refused.
- Native `Tokenizer.count(text, fast=False)` opts into that counter, built
  once per tokenizer on first use; `TokenCounter.from_tokenizer(tokenizer,
  fast=False)` replaces the JSON and rank-file constructors.
- The Python route counts over the native tokenizers the codec path shares
  (`native_encoding`, `native_anthropic`) and no longer reads rank files;
  the packaged Anthropic tokenizer has one loader, `tokenizer_dispatch.anthropic`.
- Public wrappers gain `count(text, fast=False)`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-22 04:41:11 +00:00
joshua
4a7d8bbffa fix(mcp): resolve SDK2 wire-shape regressions in guardrail, arize, and benchmark paths
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:36:03 +00:00
devin-ai-integration[bot]
194a3cc202
ci: build the benchmark environment outside the CodSpeed runner (#38426)
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 13:58:04 -07:00
devin-ai-integration[bot]
c255f53bfb
ci: only run CodSpeed on backend changes (#34345)
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-23 16:58:55 -07:00
mateo-berri
bb1b3dc937 fix(ci): retry setup-uv installs to survive transient manifest fetch failures 2026-07-14 14:41:36 -07:00
Yassin Kortam
7d15f2fc68
ci(codspeed): re-enable benchmarks on litellm_internal_staging (#32340) 2026-07-07 20:35:51 +03:00
yuneng-jiang
ae8084de74
ci(codspeed): pin benchmark runner to ubuntu-24.04 (#31746)
* ci(codspeed): pin benchmark runner to ubuntu-24.04

ubuntu-latest resolves to different runner images between the BASE
(main/staging) and HEAD (PR) runs, so CodSpeed reports 'Different
runtime environments detected' and emits false-positive regressions
(e.g. a -25.2% swing on test_completion_multi_turn in #31684, an MCP
auth fix with no LLM code changes). Pinning the runner to a fixed
image keeps BASE and HEAD on the same hardware so 1 ms swings on a
~3 ms benchmark stop blocking unrelated PRs.

Fixes #31738

* ci(codspeed): stop running benchmarks on litellm_internal_staging

The CodSpeed check flip-flops on internal staging and on PRs targeting
it (e.g. "+11.75% improvement" on one run, "-25.36% regression" on the
next) because the comparison flags "different runtime environments" and
the benchmarks are only 3-4 ms, so sub-millisecond runner noise swings
the result by 25-30%. Pinning the runner to ubuntu-24.04 in this PR
helps the head side, but the internal_staging base is still recorded on
the old unpinned runner, so comparisons keep flapping until the pin
merges and the base is re-baselined.

Until that settles, the red X's on internal staging make the OSS
project look unhealthy and confuse contributors, so drop the
litellm_internal_staging push and pull_request triggers and keep
CodSpeed running on main only.

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-06-30 15:25:55 -07:00
Yassin Kortam
1eb7122465
test(benchmarks): add CodSpeed benchmarks for inference, MCP and A2A hot paths (#31716)
Guard the per-request CPU cost of the chat completion, MCP tool and A2A
message transforms against regressions on every commit. All benchmarks are
pure in-process work with no network I/O so they stay deterministic under
CodSpeed's simulation mode, and they import under the base dependency set the
benchmark job installs.

Inference covers the full SDK overhead via mock_response (simple, multi-turn,
tools, streaming) plus convert_to_model_response_object as a deterministic
anchor. MCP covers the client-side tool translation and the proxy server-side
tool-name prefix round-trip. A2A covers the client request/response transforms
and the proxy server-ingress message conversion.

Adds the mcp and a2a-sdk packages to the benchmark run since those transform
modules need them, and broadens the workflow triggers to litellm_internal_staging
so the internal branch flow is benchmarked too.
2026-06-30 10:27:12 -07:00
stuxf
a6c30b30bf
build: migrate packaging, CI, and Docker from Poetry to uv (#25007)
* build: migrate packaging metadata to uv

* ci: move automation and local tooling to uv

* docker: migrate image builds and runtime setup to uv

* docs: update install and deployment guidance for uv

* chore: align auxiliary scripts and tests with uv

* test: harden test_litellm isolation

* fix: keep release and health check images self-contained

* build: pin uv tooling and health check deps

* test: isolate bedrock image request formatting from suite state

* test: cover sandbox executor requirements flow

* ci: fix circleci no-op command steps

* ci: fix circleci publish workflow parsing

* fix: stabilize remaining uv migration CI checks

* ci: increase matrix test timeout headroom

* fix: restore published docker and license coverage

* fix: restore proxy runtime build parity

* fix: restore proxy extras parity and venv migrations

* ci: persist uv path across circleci steps

* fix: keep psycopg binary in default test env

* docker: preserve prisma cache across stages

* test: run local proxy checks through uv python

* build: restore runtime deps moved into ci

* build: refresh uv lock after upstream merge

* fix: restore module import in test_check_migration after merge

The conflict resolution imported only the function but the test body
references check_migration as a module throughout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert dependency promotions, remove nodejs-wheel-binaries, fix Docker layer caching

- Move google-generativeai, Pillow, tenacity back to ci group (they are
  lazily imported and bloat the base SDK install needlessly)
- Remove nodejs-wheel-binaries from extra_proxy and proxy-dev (redundant
  in Docker where system Node.js is already installed via apk)
- Remove all nodejs-wheel node replacement and venv npm patching blocks
  from Dockerfiles since the wheel is no longer installed
- Add --no-default-groups to CodSpeed benchmark workflow so the benchmark
  environment matches the old minimal pip install footprint
- Apply standard uv two-phase Docker pattern: copy metadata first, install
  deps (cached layer), then copy source and install project
- Replace CircleCI enterprise no-op with proper uv sync command

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate uv.lock after removing nodejs-wheel-binaries

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): use cache/restore instead of cache to prevent cache poisoning

The old workflow used actions/cache/restore (read-only). The uv migration
changed it to actions/cache (read-write), which zizmor flags as a cache
poisoning risk. Restore the safer read-only variant.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): disable setup-uv built-in cache to silence cache-poisoning alert

The setup-uv action enables caching by default, which zizmor flags as a
cache poisoning risk. Disable it since we already use a read-only
cache/restore step.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): disable setup-uv cache in publish workflow

Silences zizmor cache-poisoning alert. Publishing workflow runs
infrequently on protected branches so caching adds no real benefit.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(test): remove duplicate verbose_logger mock in test_check_migration

The logger was patched twice — first via mocker.patch() then via
mocker.patch.object(autospec=True). The second call fails because
autospec cannot inspect an already-mocked attribute. Remove the
redundant first patch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ci): free disk space before Docker build in test-server-root-path

The Dockerfile.non_root build ran out of disk on the CI runner. Remove
Android SDK, .NET, Boost, and GHC toolchains (~12GB) to free space.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 11:46:23 -07:00
Krrish Dholakia
dfb543369b fix: address zizmor comments 2026-03-26 21:09:01 -07:00
Yuneng Jiang
f86b240d7e pin github scripts + remove unused 2026-03-25 17:38:36 -07:00
codspeed-hq[bot]
be20a8a93d
Add CodSpeed performance benchmarks (#23676)
Co-authored-by: codspeed-hq[bot] <117304815+codspeed-hq[bot]@users.noreply.github.com>
2026-03-14 18:44:36 -07:00