Commit graph

32003 commits

Author SHA1 Message Date
Sameer Kankute
1971c22b43 Add documentation for this new feat 2026-02-11 12:36:10 +05:30
Sameer Kankute
6f2de0542d Add filtering at handler level 2026-02-11 12:35:57 +05:30
Sameer Kankute
2ec0072008 Update code to handle anthropic beta headers mapping 2026-02-11 12:35:22 +05:30
Ishaan Jaffer
f29165561f fix linting 2026-02-10 19:28:04 -08:00
ken
2913db783e
feat: add dashscope/qwen3-max model with tiered pricing (#20919)
Add support for Alibaba Cloud's Qwen3-Max model with:
- 258K input tokens, 65K output tokens
- Tiered pricing based on context window usage (0-32K, 32K-128K, 128K-252K)
- Function calling and tool choice support
- Reasoning capabilities enabled

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-10 18:53:32 -08:00
Ishaan Jaff
3407006120
[Docs] Add docs guide for using policies (#20914)
* init schema with TAGS

* ui: add policy test

* resolvePoliciesCall

* add_policy_sources_to_metadata + headers

* types Policy

* preview Impact

* def _describe_match_reason(

* match based on TAGs

* TestTagBasedAttachments

* test fixes

* add policy_resolve_router

* add_guardrails_from_policy_engine

* TestMatchAttribution

* refactor

* fix

* fix: address Greptile review feedback on policy resolve endpoints

- Track unnamed keys/teams as separate counts instead of inflating
  affected_keys_count with duplicate "(unnamed key)" placeholders.
  Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
  converts exact patterns to Prisma "in" and suffix wildcards to
  "startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
  force_sync query param (default false) to avoid 2 DB round-trips
  on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
  time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: eliminate duplicate DB queries and fix header delimiter ambiguity

- Fetch teams table once in estimate_attachment_impact and reuse for
  both tag-based and alias-based lookups (was querying teams twice when
  both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
  filters that operate on pre-fetched data (_filter_keys_by_tags,
  _filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
  as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
  (e.g. "tag:healthcare+team:health-team") to avoid conflict with
  header delimiters.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs v1 guide with UI imgs

* docs fix

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 18:52:31 -08:00
Harshit Jain
bc0622692d
fix: type error & better error handling (#20689) 2026-02-10 18:33:57 -08:00
Ishaan Jaff
f83620157e
[Feat] Policies - Allow connecting Policies to Tags, Simulating Policies, Viewing how many keys, teams it applies on (#20904)
* init schema with TAGS

* ui: add policy test

* resolvePoliciesCall

* add_policy_sources_to_metadata + headers

* types Policy

* preview Impact

* def _describe_match_reason(

* match based on TAGs

* TestTagBasedAttachments

* test fixes

* add policy_resolve_router

* add_guardrails_from_policy_engine

* TestMatchAttribution

* refactor

* fix

* fix: address Greptile review feedback on policy resolve endpoints

- Track unnamed keys/teams as separate counts instead of inflating
  affected_keys_count with duplicate "(unnamed key)" placeholders.
  Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
  converts exact patterns to Prisma "in" and suffix wildcards to
  "startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
  force_sync query param (default false) to avoid 2 DB round-trips
  on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
  time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: eliminate duplicate DB queries and fix header delimiter ambiguity

- Fetch teams table once in estimate_attachment_impact and reuse for
  both tag-based and alias-based lookups (was querying teams twice when
  both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
  filters that operate on pre-fetched data (_filter_keys_by_tags,
  _filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
  as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
  (e.g. "tag:healthcare+team:health-team") to avoid conflict with
  header delimiters.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update litellm/proxy/policy_engine/policy_resolve_endpoints.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-10 17:50:37 -08:00
Alexsander Hamir
b7993b14cf
Add semgrep & Fix OOMs (#20912) 2026-02-10 17:50:14 -08:00
shin-bot-litellm
26d561081b
fix(cloudzero): update CBF field mappings per LIT-1907 (#20906)
* fix(cloudzero): update CBF field mappings per LIT-1907

Phase 1 field updates for CloudZero integration:

ADD/UPDATE:
- resource/account: Send concat(api_key_alias, '|', api_key_prefix)
- resource/service: Send model_group instead of service_type
- resource/usage_family: Send provider instead of hardcoded 'llm-usage'
- action/operation: NEW - Send team_id
- resource/id: Send model name instead of CZRN
- resource/tag:organization_alias: Add if exists
- resource/tag:project_alias: Add if exists
- resource/tag:user_alias: Add if exists

REMOVE:
- resource/tag:total_tokens: Removed
- resource/tag:team_id: Removed (team_id now in action/operation)

Fixes LIT-1907

* Update litellm/integrations/cloudzero/transform.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: define api_key_alias variable, update CBFRecord docstring

- Fix F821 lint error: api_key_alias was used but not defined
- Update CBFRecord docstring to reflect LIT-1907 field mappings
- Remove unused Optional import

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-10 17:33:16 -08:00
yuneng-jiang
c68383068a
Merge pull request #20803 from BerriAI/litellm_ui_e2e_02
[Infra] UI - E2E Tests: Key Delete, Regenerate, and Update TPM/RPM Limits
2026-02-10 17:02:37 -08:00
yuneng-jiang
a8b4b4ccba
Merge pull request #20894 from BerriAI/litellm_ui_chart_fix_02
[Fix] UI - Usage: Request Chart stack variant
2026-02-10 16:32:04 -08:00
yuneng-jiang
2730e91356
Merge pull request #20898 from BerriAI/litellm_config_pt_endpoints
[Feature] Include Config Defined Pass Through Endpoints
2026-02-10 16:31:36 -08:00
yuneng-jiang
df37bc1900
Merge pull request #20796 from BerriAI/litellm_guardrail_list_sec
[Fix] /v2/guardrails/list Returns Sensitive Values
2026-02-10 16:31:16 -08:00
yuneng-jiang
cfd261d679 Split e2e ui testing for browser 2026-02-10 16:30:31 -08:00
Alexsander Hamir
ebce0e5f8c
[Release - 02/10/2026] v1.81.10-nightly 2026-02-10 16:26:30 -08:00
michelligabriele
8507df483c
fix(router): propagate model-level tags from config to SpendLogs (#20769) 2026-02-10 15:52:52 -08:00
yuneng-jiang
39bf5b780b addressing comments 2026-02-10 15:29:07 -08:00
Ishaan Jaffer
f311fba194 fix 2026-02-10 15:24:46 -08:00
Ishaan Jaff
f8619e2000
[Stability] Investigate + fix issue where model cost map became poorly formatted (#20895)
* init: GetModelCostMap

* fix

* docs

* docs fix

* docs fixes

* docs fix

* test model cost map resilience

* MODEL_COST_MAP_MIN_MODEL_COUNT

* validate_model_cost_map

* test_should_have_minimum_models_in_backup

* docs fix

* docs fix

* fix

* dos fix

* docs fix

* docs fix

* docs fix

* docs fix

* validate_model_cost_map

* fix

* cleanup
2026-02-10 15:17:01 -08:00
yuneng-jiang
e002d6afe8 addressing comments 2026-02-10 15:16:18 -08:00
Krish Dholakia
10d891a365
Guardrails - add logging to all unified_guardrails + link to custom code guardrail templates (#20900)
* feat(guardrail_hooks/): add guardrail logging to all unified guardrails

ensures unified guardrails use the 'log_guardrail_information' decorator for logging

* fix(custom_guardrail.py): don't log inputs on guardrail response - just emit state

* refactor: don't double log bedrock guardrail information

* feat: add in-product nudges for contributing + trying community custom code guardrails

allows users to contribute / share custom code guardrails
2026-02-10 15:13:54 -08:00
yuneng-jiang
fc0563fab3 get pass through include config defined pass through 2026-02-10 14:55:37 -08:00
Emerson Gomes
a6f90586ac
feat(model-db): add azure_ai/kimi-k2.5 pricing entry (#20896) 2026-02-10 14:46:40 -08:00
yuneng-jiang
79b24c8f25 remove stack from charts 2026-02-10 14:11:55 -08:00
yuneng-jiang
9f8878ee17
Merge pull request #20893 from BerriAI/pypi_fix_feb10
[Infra] CI/CD - Fix PyPI CI Step
2026-02-10 14:01:05 -08:00
yuneng-jiang
7d2c874434 Fixing ci pypi build 2026-02-10 13:59:48 -08:00
yuneng-jiang
ea38630e7c bump: version 0.4.33 → 0.4.34 2026-02-10 13:58:57 -08:00
yuneng-jiang
b7107ab803
Merge pull request #20892 from BerriAI/litellm_ui_spend_logs_model
[Feature] UI - Spend Logs: Paginated Searchable Model Select
2026-02-10 13:51:53 -08:00
yuneng-jiang
3fe1c1ba24 fixing build 2026-02-10 12:53:51 -08:00
yuneng-jiang
7fd8c0e160 Searchable Paginated Model Select For Spend Logs 2026-02-10 12:44:38 -08:00
michelligabriele
3bbc25a3f0
fix(aiohttp): respect ssl_verify with shared sessions (#20349)
* fix(aiohttp): respect ssl_verify with shared sessions

* fix(aiohttp): resolve mypy error for ssl parameter type

Pass ssl kwarg conditionally to aiohttp request() only when explicitly
configured, since None is not a valid value for the ssl parameter
(expected SSLContext | bool | Fingerprint).
2026-02-10 10:17:35 -08:00
michelligabriele
1afe3032fd
fix(otel): auto-infer otlp_http exporter when endpoint is configured (#20438)
When OpenTelemetry is configured via the UI, only OTEL_ENDPOINT and
OTEL_HEADERS are set, but OTEL_EXPORTER is not specified. This caused
the exporter to default to "console", meaning traces were printed to
stdout instead of being sent to the configured endpoint.

This fix adds logic in OpenTelemetryConfig.__post_init__ to automatically
infer "otlp_http" as the exporter when an endpoint is specified but the
exporter is still the default "console".

Fixes issue reported by Elastic team where traces weren't being sent
to their OTEL endpoint when configured through the LiteLLM UI.
2026-02-10 09:33:16 -08:00
Sameer Kankute
0f01802dde
Merge pull request #20845 from BerriAI/litellm_gemini_image_handling
Handle image in assitant message for gemini
2026-02-10 18:24:09 +05:30
Sameer Kankute
3de892b8ca
Merge pull request #20860 from BerriAI/litellm_perplexity_research_api_support
[Feat] Perplexity research api support
2026-02-10 18:22:30 +05:30
Sameer Kankute
cdab87dec0
Merge pull request #20838 from BerriAI/litellm_managed_error_file
Add support managed error file
2026-02-10 18:20:26 +05:30
Sameer Kankute
f6228fda3e Fix mypy issues 2026-02-10 18:18:41 +05:30
Sameer Kankute
0b4c511576
Merge pull request #20863 from BerriAI/litellm_moonshotai.kimi-k2.5
Add moonshotai.kimi-k2.5
2026-02-10 18:12:36 +05:30
Sameer Kankute
7166ca8c59 Add moonshotai.kimi-k2.5 2026-02-10 18:10:37 +05:30
Sameer Kankute
63eedc26b0 Fix test_async_post_call_success_hook_for_unified_finetuning_job 2026-02-10 18:08:23 +05:30
Sameer Kankute
fda64caa28
Merge pull request #20855 from BerriAI/litellm_bedrock_kimi2
Add Kimi model pricing by region
2026-02-10 18:05:25 +05:30
Sameer Kankute
5222fd4795
Merge pull request #20783 from BerriAI/litellm_oss_staging_02_09_2026
litellm oss staging 09/02/2026
2026-02-10 18:02:14 +05:30
Sameer Kankute
45133fab53
Merge pull request #20854 from BerriAI/litellm_oss_staging_02_05_2026_3
Litellm oss staging 02 05 2026 3
2026-02-10 17:59:56 +05:30
Sameer Kankute
7d5141c28c Fix mypy issues 2026-02-10 17:58:15 +05:30
Sameer Kankute
19628f6187 Fix mypy issues 2026-02-10 17:52:17 +05:30
Sameer Kankute
f4ae6ed181 Fix mypy issues 2026-02-10 17:48:41 +05:30
Sameer Kankute
2eb52db3e9 Add documentation for perplexity 2026-02-10 17:44:00 +05:30
Sameer Kankute
9c1bf84729 Fix mypy issues 2026-02-10 17:41:03 +05:30
Sameer Kankute
ac65524d9f Use openai base config 2026-02-10 17:37:08 +05:30
Sameer Kankute
be0ebb153e Add perplexity response api routing 2026-02-10 17:01:08 +05:30