Commit graph

40072 commits

Author SHA1 Message Date
yuneng-jiang
2ac1efdc0d
Merge pull request #24603 from Sameerlite/litellm_openrouter-wildcard-strip-prefix
fix(openrouter): strip routing prefix for wildcard proxy deployments
2026-03-27 10:11:01 -07:00
yuneng-jiang
b6506bf40f
Merge pull request #24610 from Sameerlite/litellm_lyria-3-cost-map-doc
feat(gemini): Lyria 3 preview models in cost map and docs
2026-03-27 10:10:03 -07:00
yuneng-jiang
4bf5e66dbf
Merge pull request #24624 from Sameerlite/litellm_sanitize-proxy-inputs
fix(proxy): sanitize user_id input and block dangerous env var keys
2026-03-27 10:08:32 -07:00
Krrish Dholakia
412fd469e2
Merge pull request #24692 from BerriAI/litellm_security_townhall_blog
Litellm security townhall blog
2026-03-27 10:00:54 -07:00
yuneng-jiang
53ac4c5459
Merge pull request #24661 from Sameerlite/litellm_filter-metadata-user-id
fix(anthropic): strip undocumented keys from metadata before sending to API
2026-03-27 10:00:36 -07:00
yuneng-jiang
695304d758
Merge pull request #24662 from Sameerlite/litellm_gemini-retrieve-file-url-normalize
feat(gemini): normalize AI Studio file retrieve URL
2026-03-27 09:59:46 -07:00
yuneng-jiang
210e30f055
Merge pull request #24665 from Sameerlite/litellm_gemini-3-1-flash-live-preview
feat(gemini): add gemini-3.1-flash-live-preview to model cost map
2026-03-27 09:58:44 -07:00
yuneng-jiang
f3fe6d1c0a
Merge pull request #24687 from Sameerlite/litellm_litellm_azure-finetuning-fixes
feat(fine-tuning): fix Azure OpenAI fine-tuning job creation
2026-03-27 09:58:12 -07:00
yuneng-jiang
3e7ee3e887
Merge pull request #24690 from Sameerlite/litellm_litellm_openai-reasoning-items-chat-completions
feat(openai): round-trip Responses API reasoning_items in chat completions
2026-03-27 09:54:23 -07:00
yuneng-jiang
bda8c9d8ac
Merge pull request #24691 from Sameerlite/litellm_litellm_batch_metrics_4
feat(prometheus): add metrics for managed batch lifecycle
2026-03-27 09:52:59 -07:00
Krrish Dholakia
198a0e8438 docs: cleanup docs 2026-03-27 09:13:20 -07:00
Krrish Dholakia
ff63df25a2
Merge pull request #24663 from BerriAI/litellm_test_branch_03_26_2026_p1
Add zizmor to ci/cd
2026-03-27 08:59:07 -07:00
Sameer Kankute
931c88f567
Fix test 2026-03-27 21:21:43 +05:30
Sameer Kankute
09675ef205
Fix test 2026-03-27 21:12:37 +05:30
Sameer Kankute
e4a1e529c5
Merge pull request #2 from Sameerlite/litellm_litellm_health-check-driven-routing2
feat(router): health-check-driven routing
2026-03-27 21:05:56 +05:30
Krrish Dholakia
9c049ded65 docs: draft townhall doc
compressing slides into a deck
2026-03-27 08:35:38 -07:00
Sameer Kankute
8210fd7e1d
fix: revert accidental _litellm_uuid import back to _uuid
The isort hook picked up a stale rename from the working directory.
Both router.py and proxy_server.py need litellm._uuid, not _litellm_uuid.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 20:57:08 +05:30
Sameer Kankute
f784beb74f
fix: re-attach model_id after endpoint cleaning, bump log level
- model_id is now added after _clean_endpoint_data() so it survives
  health_check_details: False (MINIMAL_DISPLAY_PARAMS filtering)
- Health state write failures logged at warning instead of debug

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 20:57:08 +05:30
Sameer Kankute
7675488640
feat(router): add health-check-driven routing behind opt-in flag
Background health checks now feed deployment health state into the
router candidate-filtering pipeline. Unhealthy deployments are excluded
proactively instead of waiting for request failures to trigger cooldown.

Gated by `enable_health_check_routing: true` in general_settings.
Off by default — zero behavior change for existing users.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 20:57:08 +05:30
Sameer Kankute
b212b340ab
feat(gemini): normalize AI Studio file retrieve URL and harden tests
Made-with: Cursor
2026-03-27 20:43:19 +05:30
Sameer Kankute
38e8003297
fix(anthropic): strip undocumented keys from metadata before sending to API 2026-03-27 20:42:16 +05:30
Conductor
cdb3037c4e
Saving uncommitted changes before archiving 2026-03-27 20:38:37 +05:30
Sameer Kankute
92a07e2d6e
fix(proxy): address Greptile review feedback
- Remove HTTP_PROXY/HTTPS_PROXY from blocklist (legitimately used in corporate envs)
- Add NO_PROXY/no_proxy to blocklist (prevents bypassing proxy monitoring)
- Remove dead code in _is_valid_user_id (space exception was unreachable)
- Update tests accordingly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 20:38:36 +05:30
Sameer Kankute
8112fbf274
fix(proxy): sanitize user_id input and block dangerous env var keys
Add input validation to get_user_id_from_request (length limit, control char rejection) and a blocklist of dangerous environment variable keys in _load_environment_variables to prevent PATH/LD_PRELOAD/PYTHONPATH override via config.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 20:38:36 +05:30
Sameer Kankute
cdc1dd5c37
Fix the tests 2026-03-27 20:36:01 +05:30
Sameer Kankute
1aa90f9bd1
Fix model map 2026-03-27 20:36:01 +05:30
Sameer Kankute
3bbc694461
Fix model map 2026-03-27 20:36:00 +05:30
Sameer Kankute
cc73ae776a
feat(gemini): add Lyria 3 preview models to cost map and docs
Made-with: Cursor
2026-03-27 20:36:00 +05:30
Sameer Kankute
9d7fc307b8
fix(openrouter): strip LiteLLM prefix when proxy sets custom_llm_provider
Wildcard openrouter/* deployments pass custom_llm_provider=openrouter with
the full openrouter/provider/model id; OpenRouter expects provider/model.
Strip the outer openrouter/ only when the remainder contains a slash so
native ids like openrouter/auto stay intact.

Adds regression test for proxy wildcard path.

Made-with: Cursor
2026-03-27 20:35:17 +05:30
Sameer Kankute
bbd8ca3b3d
feat(prometheus): add metrics for managed batch lifecycle
- Add Prometheus metrics for managed batch and file operations
- Track batch creation, file size, duration, and deletion events
- Add CheckBatchCost polling metrics (jobs polled/processed, errors)
- Record metrics in managed_files hook and check_batch_cost utility
- Metrics include labels for model, provider, user, and status

Made-with: Cursor
2026-03-27 20:30:09 +05:30
Sameer Kankute
00a810e92d
feat(openai): round-trip Responses API reasoning_items in chat completions
Made-with: Cursor
2026-03-27 20:25:08 +05:30
yuneng-jiang
d3568efad0
Merge pull request #24611 from Sameerlite/Sameerlite/order-fallback2
feat(router): order-based fallback across deployment priority levels
2026-03-27 20:15:30 +05:30
Sameer Kankute
15f5dc38c4
Fix tests 2026-03-27 20:11:28 +05:30
Sameer Kankute
1fac58abb3
fix(tests): reset module-level cache in stale alias bypass tests
Reset _ENABLE_TEAM_STALE_ALIAS_BYPASS to None in both test functions
to ensure test isolation and prevent ordering-dependent failures

Made-with: Cursor
2026-03-27 20:11:28 +05:30
Sameer Kankute
7436f889ca
fix(router): address final Greptile P1/P2 comments
- Reorder team_public_model_name assignment to happen before model_name mutation for clarity
- Add comment explaining no-rename fast-exit case in _update_existing_team_model_assignment
- Add comment explaining final patch_data.model_name = None applies to all code paths

Made-with: Cursor
2026-03-27 20:11:28 +05:30
Sameer Kankute
2321d77599
fix(router): address remaining Greptile review comments
- Cache LITELLM_ENABLE_TEAM_STALE_ALIAS_BYPASS at module level to avoid hot-path secret lookups
- Add clarifying comments for should_include_deployment team isolation logic
- Add negative assertion for update_team.assert_not_called() in test
- Add docstring clarification for _get_team_deployments helper pattern
- Add explicit assertion message in test_get_model_list_alias_optimization

Made-with: Cursor
2026-03-27 20:11:28 +05:30
Sameer Kankute
592ac98ddc
fix(router): address Greptile P1/P2 review comments
- Add deduplication guard in _update_team_model_index to prevent duplicate indices
- Add wildcard comment in map_team_model for clarity
- Add monkeypatch to test_team_alias_stale_bypass_disabled_by_default for determinism
- Extract _get_team_deployments helper to centralize DB access pattern
- Add clarifying comments for team_public_model_name assignment ordering

Made-with: Cursor
2026-03-27 20:11:28 +05:30
Sameer Kankute
1a0b30aaac
Fix greptile reviews and mock test 2026-03-27 20:11:28 +05:30
Sameer Kankute
fb8d9c2e9a
Fix greptile reviews and mock test 2026-03-27 20:11:28 +05:30
Sameer Kankute
c6cc0341f6
Fix greptile reviews and mock test 2026-03-27 20:11:28 +05:30
Sameer Kankute
9a0a216195
Fix code qa issues 2026-03-27 20:11:28 +05:30
Sameer Kankute
316a742945
Fix greptile comments 2026-03-27 20:11:28 +05:30
Sameer Kankute
d02a70ab4e
Fix greptile comments 2026-03-27 20:11:28 +05:30
Sameer Kankute
fc6865c3a3
Fix greptile comments 2026-03-27 20:11:28 +05:30
Sameer Kankute
303072dc44
Fix greptile comments 2026-03-27 20:11:28 +05:30
Sameer Kankute
173695f5e0
Fix greptile comments 2026-03-27 20:11:27 +05:30
Sameer Kankute
8db867c51c
fix(routing): address state consistency and type safety issues
- Check alias target pattern to detect stale team aliases
- Fix PrismaClient type annotation to Optional
- Eliminate in-place mutation in index update logic

Made-with: Cursor
2026-03-27 20:11:27 +05:30
Sameer Kankute
e8fb7762b3
perf(routing): optimize team model checks and improve test coverage
- Use O(1) team index lookup instead of map_team_model in alias guard
- Fix MockPrismaClient to validate where clause filters
- Add comment explaining DB query trade-off for team deployments

Made-with: Cursor
2026-03-27 20:11:27 +05:30
Sameer Kankute
8aa58bdcaa
fix(routing): prevent stale model_aliases from interfering with team routing
- Skip model_aliases rewrite if model resolves to team deployments
- Add test coverage for sibling-preservation branch
- Update MockPrismaClient to support sibling deployment scenarios

Made-with: Cursor
2026-03-27 20:11:27 +05:30
Sameer Kankute
298df75066
fix(router): guard None model_info and deduplicate team index logic
- Guard against None model_info in sibling deployment check
- Extract _update_team_model_index helper to eliminate duplication

Made-with: Cursor
2026-03-27 20:11:27 +05:30