Commit graph

39980 commits

Author SHA1 Message Date
Yuneng Jiang
ec4273ed8b
[Infra] Improve CodeQL scanning coverage and schedule
Switch query suite from security-extended to security-and-quality to
match the default GitHub Advanced Security setup. Run scheduled scans
daily instead of weekly. Remove paths-ignore for _experimental/out so
build artifacts are also scanned.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 12:04:09 -07:00
yuneng-jiang
1b111d23f3
Merge pull request #24688 from Sameerlite/litellm_litellm_team-model-group-name-routing-fix
fix(team-routing): preserve sibling deployment candidates for team public models
2026-03-27 12:00:34 -07:00
Krrish Dholakia
fa23218e75 docs: add faq to docs 2026-03-27 11:53:45 -07:00
Sameer Kankute
c4159a2ade
Fix codeql 2026-03-28 00:01:33 +05:30
Yuneng Jiang
ca3457b091
Pin nodejs-wheel-binaries in CI workflows running prisma generate
prisma generate internally runs `npm install prisma@5.4.2` against the
npm registry at runtime. Without a bundled Node.js, this causes
ECONNRESET failures on flaky GitHub Actions network and leaves the
npm transitive dependency tree unpinned.

Pre-install nodejs-wheel-binaries==24.13.1 (matching the Dockerfiles)
so prisma uses the bundled Node/npm instead of fetching from the
registry.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 11:25:03 -07:00
yuneng-jiang
8c2c6a40a6
Merge pull request #24689 from Sameerlite/litellm_litellm_remove-200k-pricing-opus-sonnet-46
fix(pricing): remove above_200k_tokens price tiers for claude-opus-4-6 and claude-sonnet-4-6
2026-03-27 10:26:17 -07:00
Sameer Kankute
b4d0e3213f
Fix the Pricing changes for claude models 2026-03-27 22:44:40 +05:30
Sameer Kankute
453fc75ee9
fix(pricing): remove above_200k_tokens price tiers for claude-opus-4-6 and claude-sonnet-4-6
These models include the full 1M token context at standard pricing with no 2x surcharge above 200k tokens.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 22:44:40 +05:30
yuneng-jiang
2ac1efdc0d
Merge pull request #24603 from Sameerlite/litellm_openrouter-wildcard-strip-prefix
fix(openrouter): strip routing prefix for wildcard proxy deployments
2026-03-27 10:11:01 -07:00
yuneng-jiang
b6506bf40f
Merge pull request #24610 from Sameerlite/litellm_lyria-3-cost-map-doc
feat(gemini): Lyria 3 preview models in cost map and docs
2026-03-27 10:10:03 -07:00
yuneng-jiang
4bf5e66dbf
Merge pull request #24624 from Sameerlite/litellm_sanitize-proxy-inputs
fix(proxy): sanitize user_id input and block dangerous env var keys
2026-03-27 10:08:32 -07:00
Krrish Dholakia
412fd469e2
Merge pull request #24692 from BerriAI/litellm_security_townhall_blog
Litellm security townhall blog
2026-03-27 10:00:54 -07:00
yuneng-jiang
53ac4c5459
Merge pull request #24661 from Sameerlite/litellm_filter-metadata-user-id
fix(anthropic): strip undocumented keys from metadata before sending to API
2026-03-27 10:00:36 -07:00
yuneng-jiang
695304d758
Merge pull request #24662 from Sameerlite/litellm_gemini-retrieve-file-url-normalize
feat(gemini): normalize AI Studio file retrieve URL
2026-03-27 09:59:46 -07:00
yuneng-jiang
210e30f055
Merge pull request #24665 from Sameerlite/litellm_gemini-3-1-flash-live-preview
feat(gemini): add gemini-3.1-flash-live-preview to model cost map
2026-03-27 09:58:44 -07:00
yuneng-jiang
f3fe6d1c0a
Merge pull request #24687 from Sameerlite/litellm_litellm_azure-finetuning-fixes
feat(fine-tuning): fix Azure OpenAI fine-tuning job creation
2026-03-27 09:58:12 -07:00
yuneng-jiang
3e7ee3e887
Merge pull request #24690 from Sameerlite/litellm_litellm_openai-reasoning-items-chat-completions
feat(openai): round-trip Responses API reasoning_items in chat completions
2026-03-27 09:54:23 -07:00
yuneng-jiang
bda8c9d8ac
Merge pull request #24691 from Sameerlite/litellm_litellm_batch_metrics_4
feat(prometheus): add metrics for managed batch lifecycle
2026-03-27 09:52:59 -07:00
Krrish Dholakia
198a0e8438 docs: cleanup docs 2026-03-27 09:13:20 -07:00
Krrish Dholakia
ff63df25a2
Merge pull request #24663 from BerriAI/litellm_test_branch_03_26_2026_p1
Add zizmor to ci/cd
2026-03-27 08:59:07 -07:00
Sameer Kankute
931c88f567
Fix test 2026-03-27 21:21:43 +05:30
Sameer Kankute
09675ef205
Fix test 2026-03-27 21:12:37 +05:30
Sameer Kankute
e4a1e529c5
Merge pull request #2 from Sameerlite/litellm_litellm_health-check-driven-routing2
feat(router): health-check-driven routing
2026-03-27 21:05:56 +05:30
Krrish Dholakia
9c049ded65 docs: draft townhall doc
compressing slides into a deck
2026-03-27 08:35:38 -07:00
Sameer Kankute
8210fd7e1d
fix: revert accidental _litellm_uuid import back to _uuid
The isort hook picked up a stale rename from the working directory.
Both router.py and proxy_server.py need litellm._uuid, not _litellm_uuid.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 20:57:08 +05:30
Sameer Kankute
f784beb74f
fix: re-attach model_id after endpoint cleaning, bump log level
- model_id is now added after _clean_endpoint_data() so it survives
  health_check_details: False (MINIMAL_DISPLAY_PARAMS filtering)
- Health state write failures logged at warning instead of debug

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 20:57:08 +05:30
Sameer Kankute
7675488640
feat(router): add health-check-driven routing behind opt-in flag
Background health checks now feed deployment health state into the
router candidate-filtering pipeline. Unhealthy deployments are excluded
proactively instead of waiting for request failures to trigger cooldown.

Gated by `enable_health_check_routing: true` in general_settings.
Off by default — zero behavior change for existing users.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 20:57:08 +05:30
Sameer Kankute
b212b340ab
feat(gemini): normalize AI Studio file retrieve URL and harden tests
Made-with: Cursor
2026-03-27 20:43:19 +05:30
Sameer Kankute
38e8003297
fix(anthropic): strip undocumented keys from metadata before sending to API 2026-03-27 20:42:16 +05:30
Conductor
cdb3037c4e
Saving uncommitted changes before archiving 2026-03-27 20:38:37 +05:30
Sameer Kankute
92a07e2d6e
fix(proxy): address Greptile review feedback
- Remove HTTP_PROXY/HTTPS_PROXY from blocklist (legitimately used in corporate envs)
- Add NO_PROXY/no_proxy to blocklist (prevents bypassing proxy monitoring)
- Remove dead code in _is_valid_user_id (space exception was unreachable)
- Update tests accordingly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 20:38:36 +05:30
Sameer Kankute
8112fbf274
fix(proxy): sanitize user_id input and block dangerous env var keys
Add input validation to get_user_id_from_request (length limit, control char rejection) and a blocklist of dangerous environment variable keys in _load_environment_variables to prevent PATH/LD_PRELOAD/PYTHONPATH override via config.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 20:38:36 +05:30
Sameer Kankute
cdc1dd5c37
Fix the tests 2026-03-27 20:36:01 +05:30
Sameer Kankute
1aa90f9bd1
Fix model map 2026-03-27 20:36:01 +05:30
Sameer Kankute
3bbc694461
Fix model map 2026-03-27 20:36:00 +05:30
Sameer Kankute
cc73ae776a
feat(gemini): add Lyria 3 preview models to cost map and docs
Made-with: Cursor
2026-03-27 20:36:00 +05:30
Sameer Kankute
9d7fc307b8
fix(openrouter): strip LiteLLM prefix when proxy sets custom_llm_provider
Wildcard openrouter/* deployments pass custom_llm_provider=openrouter with
the full openrouter/provider/model id; OpenRouter expects provider/model.
Strip the outer openrouter/ only when the remainder contains a slash so
native ids like openrouter/auto stay intact.

Adds regression test for proxy wildcard path.

Made-with: Cursor
2026-03-27 20:35:17 +05:30
Sameer Kankute
bbd8ca3b3d
feat(prometheus): add metrics for managed batch lifecycle
- Add Prometheus metrics for managed batch and file operations
- Track batch creation, file size, duration, and deletion events
- Add CheckBatchCost polling metrics (jobs polled/processed, errors)
- Record metrics in managed_files hook and check_batch_cost utility
- Metrics include labels for model, provider, user, and status

Made-with: Cursor
2026-03-27 20:30:09 +05:30
Sameer Kankute
00a810e92d
feat(openai): round-trip Responses API reasoning_items in chat completions
Made-with: Cursor
2026-03-27 20:25:08 +05:30
yuneng-jiang
d3568efad0
Merge pull request #24611 from Sameerlite/Sameerlite/order-fallback2
feat(router): order-based fallback across deployment priority levels
2026-03-27 20:15:30 +05:30
Sameer Kankute
15f5dc38c4
Fix tests 2026-03-27 20:11:28 +05:30
Sameer Kankute
1fac58abb3
fix(tests): reset module-level cache in stale alias bypass tests
Reset _ENABLE_TEAM_STALE_ALIAS_BYPASS to None in both test functions
to ensure test isolation and prevent ordering-dependent failures

Made-with: Cursor
2026-03-27 20:11:28 +05:30
Sameer Kankute
7436f889ca
fix(router): address final Greptile P1/P2 comments
- Reorder team_public_model_name assignment to happen before model_name mutation for clarity
- Add comment explaining no-rename fast-exit case in _update_existing_team_model_assignment
- Add comment explaining final patch_data.model_name = None applies to all code paths

Made-with: Cursor
2026-03-27 20:11:28 +05:30
Sameer Kankute
2321d77599
fix(router): address remaining Greptile review comments
- Cache LITELLM_ENABLE_TEAM_STALE_ALIAS_BYPASS at module level to avoid hot-path secret lookups
- Add clarifying comments for should_include_deployment team isolation logic
- Add negative assertion for update_team.assert_not_called() in test
- Add docstring clarification for _get_team_deployments helper pattern
- Add explicit assertion message in test_get_model_list_alias_optimization

Made-with: Cursor
2026-03-27 20:11:28 +05:30
Sameer Kankute
592ac98ddc
fix(router): address Greptile P1/P2 review comments
- Add deduplication guard in _update_team_model_index to prevent duplicate indices
- Add wildcard comment in map_team_model for clarity
- Add monkeypatch to test_team_alias_stale_bypass_disabled_by_default for determinism
- Extract _get_team_deployments helper to centralize DB access pattern
- Add clarifying comments for team_public_model_name assignment ordering

Made-with: Cursor
2026-03-27 20:11:28 +05:30
Sameer Kankute
1a0b30aaac
Fix greptile reviews and mock test 2026-03-27 20:11:28 +05:30
Sameer Kankute
fb8d9c2e9a
Fix greptile reviews and mock test 2026-03-27 20:11:28 +05:30
Sameer Kankute
c6cc0341f6
Fix greptile reviews and mock test 2026-03-27 20:11:28 +05:30
Sameer Kankute
9a0a216195
Fix code qa issues 2026-03-27 20:11:28 +05:30
Sameer Kankute
316a742945
Fix greptile comments 2026-03-27 20:11:28 +05:30