Commit graph

28729 commits

Author SHA1 Message Date
yuneng-jiang
26b8e7f889 UI build 2025-12-13 09:07:43 -08:00
Alexsander Hamir
5de9bfde53
[Fix] CI/CD - mypy & check_code_and_doc_quality & mcp_testing (#17920)
* Fix duplicate imports in SAP embedding transformation

* fix: add missing prompt_spec parameter to HumanloopLogger.get_chat_completion_prompt

- Add prompt_spec: Optional[PromptSpec] = None parameter to match base class signature
- Import PromptSpec from litellm.types.prompts.init_prompts
- Pass prompt_spec to super().get_chat_completion_prompt() call
- Fixes mypy type error: Signature incompatible with supertype CustomLogger

* fix: add missing parameters to AnthropicCacheControlHook.async_get_chat_completion_prompt

- Add ignore_prompt_manager_model and ignore_prompt_manager_optional_params parameters
- Change litellm_logging_obj type from Any to LiteLLMLoggingObj using TYPE_CHECKING pattern
- Pass all parameters including prompt_spec to get_chat_completion_prompt call
- Fixes mypy type errors: Signature incompatible with supertype CustomLogger and PromptManagementBase

* fix: add missing parameters to DotpromptManager.async_get_chat_completion_prompt

- Add ignore_prompt_manager_model and ignore_prompt_manager_optional_params parameters
- Change litellm_logging_obj type from Any to LiteLLMLoggingObj using TYPE_CHECKING pattern
- Pass all parameters including ignore flags to PromptManagementBase.async_get_chat_completion_prompt
- Fixes mypy type errors: Signature incompatible with supertype CustomLogger and PromptManagementBase

* fix: document envs

* fix: add missing parameters to LangfusePromptManagement.async_get_chat_completion_prompt

- Add ignore_prompt_manager_model and ignore_prompt_manager_optional_params parameters
- Pass all parameters including prompt_spec and ignore flags to get_chat_completion_prompt
- Fixes mypy type errors: Signature incompatible with supertype CustomLogger and PromptManagementBase

* fix: add missing parameters to prompt management async methods (Category 1)

- vector_store_pre_call_hook: add ignore_prompt_manager_model, ignore_prompt_manager_optional_params, prompt_spec
- gitlab_prompt_manager: add ignore parameters, fix litellm_logging_obj type
- bitbucket_prompt_manager: add ignore parameters, fix litellm_logging_obj type
- proxy/custom_prompt_management: add prompt_spec parameter
- Fixes mypy type errors: Signature incompatible with supertype

* fix: fix arize_phoenix_prompt_manager and custom_prompt_management (Category 2)

- arize_phoenix_prompt_manager: add prompt_spec to all methods, fix prompt_id types, implement async_compile_prompt_helper
- custom_prompt_management: implement async_compile_prompt_helper abstract method
- Fixes mypy type errors: Signature incompatible with supertype and abstract method errors

* fix: fix obvious type errors (Category 3 - Quick Wins)

- langfuse: change 'callable' to 'Callable' type annotation
- presidio: add type narrowing check for Choices vs StreamingChoices
  - StreamingChoices doesn't have .message attribute, only Choices does
  - Add hasattr check before accessing choice.message
- Fixes mypy type errors: callable? not callable and union-attr errors

* fix: handle expires_after None in Azure files handler (Todo 14)

- Extract logic to _prepare_create_file_data helper method
- Remove expires_after from dict if None to match SDK's Omit pattern
- Add type ignore for FileExpiresAfter -> file_create_params.ExpiresAfter mismatch
- Fixes mypy error: Argument expires_after has incompatible type

* fix: change purpose parameter type to OpenAIFilesPurpose (Todo 18)

- Import OpenAIFilesPurpose in storage_backend_service.py
- Change upload_file_to_storage_backend purpose parameter from str to OpenAIFilesPurpose
- Change _create_file_object_with_storage_metadata purpose parameter from str to OpenAIFilesPurpose
- Fixes mypy error: Argument purpose has incompatible type str; expected Literal type
- Purpose is already validated in files_endpoints.py before reaching these functions

* fix: handle UploadFile | str type for expires_after form fields (Todo 19)

- Validate expires_after[anchor] and expires_after[seconds] are strings, not UploadFiles
- Validate anchor equals 'created_at' before using literal in TypedDict
- Use literal 'created_at' (not variable) in FileExpiresAfter to satisfy Literal type
- Add proper error handling for invalid anchor values and int conversion
- Fixes mypy errors: Incompatible types for anchor and seconds in FileExpiresAfter

* fix: add type narrowing for expires_after_seconds_str to fix mypy error

- Add assert statement after UploadFile validation to help mypy narrow type
- Use validated variable with explicit str type annotation
- Fixes: Argument of type 'UploadFile | str' cannot be assigned to int()

* fix: trigger async_success_handler for MCP tool calls to enable cost tracking and logging

- Set call_type to CallTypes.call_mcp_tool.value before calling async_success_handler
- Update mcp_tool_call_metadata with cost info when server is found
- Call async_success_handler to build standard_logging_object and trigger callbacks
- Fixes test_mcp_cost_tracking by ensuring standard_logging_payload is populated

* refactor: use positive isinstance check for safer type narrowing

- Replace assert with positive isinstance(..., str) check
- Matches codebase pattern (see pass_through_endpoints.py)
- Safer than assert: assertions can be disabled with -O flag
- Mypy properly narrows type after positive isinstance check
- More explicit and readable than assert statement

* fix: add missing REDIS_DAILY_AGENT_SPEND_UPDATE_QUEUE to ServiceTypes enum (Todo 17)

- Add REDIS_DAILY_AGENT_SPEND_UPDATE_QUEUE enum value following the pattern of other daily spend queues
- Add corresponding entry to DEFAULT_SERVICE_CONFIGS with GAUGE metrics
- Fixes mypy error: 'type[ServiceTypes]' has no attribute 'REDIS_DAILY_AGENT_SPEND_UPDATE_QUEUE'
- This enum value is already used in redis_update_buffer.py for agent spend tracking
2025-12-13 08:18:43 -08:00
Ishaan Jaff
2f82c223d3
Litellm docs a2a cost tracking (#17913)
* docs - a2a cost tracking

* docs fix

* docs a2a cost trackign

* docs langgraph agents
2025-12-12 18:23:25 -08:00
yuneng-jiang
ab9ec11ba8
Merge pull request #17907 from BerriAI/litellm_theme_mermaid_fix
[Infra] bumping docusaurus/theme-mermaid to 3.9.0
2025-12-12 18:06:50 -08:00
Ishaan Jaffer
802a343fdb ui new build 2025-12-12 17:15:40 -08:00
Ishaan Jaffer
17ce970d99 ui fix 2025-12-12 17:10:36 -08:00
Ishaan Jaffer
2027ee7f85 UI new build 2025-12-12 17:04:48 -08:00
Ishaan Jaffer
c57f0665db ui buld fix 2025-12-12 16:58:47 -08:00
Ishaan Jaff
3054b6ea60
[Feat] A2A Gateway - allow adding Azure Foundry Agents on UI (#17909)
* add CostConfigFields

* add CostConfigFields

* add output_cost_per_token

* refactor table

* add agent cost view

* add azure foundry fields

* add foundry logo

* fix: clean error

* fix utils

* fix agent edi

* add easter egg

* fix order

* test_handle_streaming_forwards_api_key

* fix forward api key down

* fix a2a send msg

* add A2a comparison on compare playground

* fix chat ui

* fix bedrock agentcore stream
2025-12-12 16:38:04 -08:00
Ishaan Jaff
bede40a90d
[A2a gateway] Add agent cost tracking on UI (#17899)
* add CostConfigFields

* add CostConfigFields

* add output_cost_per_token

* refactor table

* add agent cost view
2025-12-12 16:37:31 -08:00
YutaSaito
b93a5f0ca6
Merge pull request #17908 from BerriAI/litellm_fix-mcp-tool-name-prefix
Litellm fix mcp tool name prefix
2025-12-13 08:23:47 +09:00
Yuta Saito
bcb26cc55a fix: format 2025-12-13 07:56:05 +09:00
Yuta Saito
864b61b433 fix: support ResponseFunctionToolCall in follow-up input 2025-12-13 07:55:13 +09:00
yuneng-jiang
e3f1ce0138 bumping docusaurus/theme-mermaid to 3.9.0 2025-12-12 14:42:31 -08:00
yuneng-jiang
11926ea2d4
Merge pull request #17901 from BerriAI/litellm_usage_date_pick_fix
[Fix] UI - Agent Usage Page Minor Issues
2025-12-12 14:34:15 -08:00
yuneng-jiang
64aaba6c58
Merge pull request #17902 from BerriAI/litellm_ui_default_user
[Fix] Add All Proxy Models To Default User Settings
2025-12-12 14:33:42 -08:00
Yuta Saito
48579eafe3 fix: strip mcp server prefixes in responses 2025-12-13 07:17:50 +09:00
yuneng-jiang
d090b4ad3e Add All Proxy Models To Default User Settings 2025-12-12 13:15:25 -08:00
yuneng-jiang
94a0ff08b7 Merge remote-tracking branch 'origin' into litellm_usage_date_pick_fix 2025-12-12 13:00:59 -08:00
Cesar Garcia
ed28818f76
feat(bedrock): add EU Claude Opus 4.5 model (#17897)
Add eu.anthropic.claude-opus-4-5-20251101-v1:0 to support
AWS Bedrock cross-region inference in EU regions.

Fixes #17867
2025-12-12 12:56:17 -08:00
yuneng-jiang
5d01f14f94 Fixing build 2025-12-12 12:55:15 -08:00
yuneng-jiang
b2695379d1
Merge pull request #17896 from BerriAI/litellm_agent_usage_2
[Feature] Usage Entity labels
2025-12-12 12:53:16 -08:00
yuneng-jiang
19504adb9d Agent Usage small issues 2025-12-12 12:52:51 -08:00
yuneng-jiang
1037dc18b9 Adding test 2025-12-12 12:23:11 -08:00
yuneng-jiang
5b0bf86ee5 Usage Entity labels 2025-12-12 12:13:51 -08:00
Ishaan Jaff
92d71a3ed4
[UI] - Re organize leftnav to have correct categories + get agents on root (#17890)
* left nav

* fix order

* fix

* fix bad new usage

* fix new
2025-12-12 12:10:13 -08:00
YutaSaito
8899b63fa4
Merge pull request #17747 from BerriAI/litellm_feat_mcp-chat-completions
feat: add support for using MCPs on /chat/completions
2025-12-13 05:05:21 +09:00
Ishaan Jaff
b651012fdc
[fix] UI playground - allow custom model name as option[0] (#17892)
* fix: move custom model to top

* fix test
2025-12-12 12:00:23 -08:00
Cesar Garcia
1531b58493
feat(openai): add reasoning_effort='xhigh' support for gpt-5.2 models (#17875)
Add support for the 'xhigh' reasoning effort level on all gpt-5.2 model
variants, not just gpt-5.2-pro. This enables deeper reasoning capabilities
for the base gpt-5.2 model.

Changes:
- Add is_model_gpt_5_2_model() method to detect gpt-5.2 variants
- Update xhigh validation to allow gpt-5.2 models
- Update documentation with gpt-5.2 reasoning_effort support
- Update tests to reflect new behavior
2025-12-12 11:40:35 -08:00
Ishaan Jaff
700a1bb574
[Feat] UI - show UI version on top left near logo (#17891)
* left nav

* fix order

* fix

* add ui version on ui

* fix link
2025-12-12 11:39:02 -08:00
Alexsander Hamir
b635f92d90
Add benchmark_proxy_vs_provider.py script to scripts directory with usage examples (#17889) 2025-12-12 11:26:34 -08:00
Alexsander Hamir
f4db4b6f0e
feat(otel): add latency metrics (TTFT, TPOT, Total Generation Time) to OTEL logging (#17888)
- Add time_to_first_token_histogram using api_call_start_time for accurate measurement
- Add time_per_output_token_histogram for average time per output token
- Add response_duration_histogram for total LLM API generation time
- Extract latency metric recording into dedicated helper methods
- Fix parent span double-ending bug when reused as primary span
- Use api_call_start_time for TTFT to exclude LiteLLM overhead (matches Prometheus)
- Support both streaming and non-streaming requests
- Handle both datetime and float timestamp formats
2025-12-12 11:19:17 -08:00
Ishaan Jaff
d38f241032
[Feat] JWT Auth - auth allow selecting team_id from request header (#17884)
* feat: add get_team_id_from_header for JWT Auth

* fix Auth builder JWT Auth

* test_get_team_id_from_header

* test_auth_builder_uses_team_from_header_e2e

* Select Team via Request Header
2025-12-12 10:18:20 -08:00
yuneng-jiang
25a38b0f27
Merge pull request #17883 from BerriAI/litellm_ui_new_badge
[Feature] New Badge for Agent Usage
2025-12-12 10:02:31 -08:00
yuneng-jiang
1829e58d25 New Badge for Agent Usage 2025-12-12 09:47:08 -08:00
Sameer Kankute
d5fcb6fce6
Merge pull request #17882 from BerriAI/litellm_target_storage_documentation
Add documentation for target storage
2025-12-12 22:46:04 +05:30
Sameer Kankute
6f0efff28b Add documentation for target storage 2025-12-12 22:44:53 +05:30
Sameer Kankute
da81ab6d97
Merge pull request #17758 from BerriAI/litellm_managed_files_target_storage
Add v0 support for target storage
2025-12-12 22:26:34 +05:30
Sameer Kankute
afda51d476
Merge pull request #17873 from BerriAI/litellm_rerank_foraward_headers
Add support for forwarding client headers in /rerank endpoint
2025-12-12 22:25:57 +05:30
Sameer Kankute
d98ee8a448
Merge pull request #17872 from BerriAI/litellm_embedding_header_forwarding
fix: bedrock header forwarding with cutom api
2025-12-12 22:25:45 +05:30
Sameer Kankute
abbf8be07b
Merge pull request #17864 from BerriAI/litellm_fix_x-litellm-key-spend
Fix x-litellm-key-spend header update
2025-12-12 22:23:42 +05:30
Sameer Kankute
e0428388a7
Merge pull request #17860 from BerriAI/litellm_openai_files_expire_after_support
Add support for expires after param in Files endpoint
2025-12-12 22:23:11 +05:30
Alexsander Hamir
d196e0b9c8
refactor(router): replace time.perf_counter() with time.time() for timing measurements (#17881) 2025-12-12 08:51:18 -08:00
AlexsanderHamir
1ad6763500 fix: add PROMETHEUS_MULTIPROC_DIR to docs 2025-12-12 08:33:38 -08:00
Alexsander Hamir
d9cf53b555
fix: remove dependency on database and redis from health test (#17880) 2025-12-12 08:21:48 -08:00
Alexsander Hamir
9c39539d78
revert CI changes (#17879) 2025-12-12 08:13:18 -08:00
Alexsander Hamir
5f2f823d44
fix: use docker executor (#17878) 2025-12-12 07:56:03 -08:00
Alexsander Hamir
c9063d13b1
Add health endpoint tests to CI with database and Redis support (#17877)
- Add database and Redis setup to litellm_mapped_tests_proxy job in CircleCI
- Create shared test helpers in tests/test_litellm/proxy/conftest.py for proxy test setup
- Refactor health endpoint tests to use shared helpers from conftest
- Support automatic Redis cache configuration when REDIS_HOST is set
- Ensure minimal config is created when Redis/database is needed
2025-12-12 07:35:50 -08:00
Alexsander Hamir
762b429d6c
enhance: create_litellm_branch tool to be more robust (#17874) 2025-12-12 05:35:50 -08:00
Krish Dholakia
eab5bca583
Add Milvus REST client and update examples (#17736)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-12-12 04:38:28 -08:00