Commit graph

31319 commits

Author SHA1 Message Date
Yuneng Jiang
ddfc37a965
chore: fixes 2026-04-04 23:34:07 -07:00
shin-bot-litellm
7d8ce4100e litellm_refactor(a2a): remove unused handler.py, use BaseLLMHTTPHandler pattern
The A2A provider was already correctly using BaseLLMHTTPHandler through main.py:
- main.py routes a2a_agent requests to base_llm_http_handler.completion()
- A2AAgentConfig provides request/response transformation
- A2AStreamingIterator handles SSE streaming

The handler.py file was redundant and not being used:
- It defined A2AAgentChatCompletion(BaseLLM) which duplicated HTTP handling
- This pattern is deprecated in favor of BaseLLMHTTPHandler
- Removing it eliminates confusion about which code path is used

This aligns the A2A provider with the standard LiteLLM HTTP handler pattern
used by other providers like Anthropic, DeepSeek, etc.
2026-01-31 07:57:17 +00:00
shin-bot-litellm
d09a144d4e litellm_feat: add A2A (Agent-to-Agent) protocol as a provider
This PR adds support for calling A2A-compliant agents through the standard
LiteLLM completion API by transforming OpenAI chat completion requests to
A2A protocol and vice versa.

Features:
- Full transformation between OpenAI chat completion format and A2A protocol
- Support for both streaming and non-streaming completions
- Proper handling of A2A message/task/artifact responses
- Integration with LiteLLM's provider infrastructure

Usage:
  response = litellm.completion(
      model='a2a_agent/my-agent',
      messages=[{'role': 'user', 'content': 'Hello!'}],
      api_base='http://localhost:9999',  # A2A agent endpoint
  )

For streaming:
  response = litellm.completion(
      model='a2a_agent/my-agent',
      messages=[{'role': 'user', 'content': 'Hello!'}],
      api_base='http://localhost:9999',
      stream=True,
  )

A2A Protocol Reference: https://github.com/a2aproject/A2A
2026-01-31 07:51:18 +00:00
Shivam Rawat
3f1bda57e2
Merge pull request #20101 from BerriAI/ui-mcp-list-tools-instructions-fix
fixed mcp tools instructions on ui to show comma seprated str instead…
2026-01-30 19:40:37 -08:00
Krrish Dholakia
0edd50fe3c docs: cleanup docs 2026-01-30 19:39:17 -08:00
shivam
d01267707e fixed mcp tools instructions on ui to show comma seprated str instead of list 2026-01-30 19:25:30 -08:00
Ishaan Jaffer
87c1e6fd68 remove md 2026-01-30 18:57:06 -08:00
Ishaan Jaff
4abae44006
[Feat] UI - Add Pretty print view of request/response (#20096)
* v1 - tool viewer in logs page

* add preview for tool sections

* ui fixes

* new tool view

* v1 - new pretty view

* clean ui

* polish fixes

* nice view input/output

* working i/o cards

* fixes for log view

---------

Co-authored-by: Warp <agent@warp.dev>
2026-01-30 18:56:34 -08:00
Ishaan Jaff
1cd83e9022
[Feat] UI - New View to render "Tools" on Logs View (#20093)
* v1 - tool viewer in logs page

* add preview for tool sections

* ui fixes

* new tool view

* Refactor: Address code review feedback - use Antd components

Changes:
- Use Antd Space component instead of manual flex layouts
- Use Antd Text.copyable prop instead of custom clipboard utilities
- Extract helper functions to utils.ts for testability
- Remove clipboardUtils.ts (replaced with Antd built-in)
- Update DrawerHeader, LogDetailsDrawer, and constants

Benefits:
- Cleaner code using standard Antd patterns
- Better testability with separated utils
- Consistent UX with Antd's copy tooltips
- Reduced custom code maintenance

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-30 18:48:52 -08:00
Ishaan Jaff
5345a763c2
[Feat] v2 - Logs view with side panel and improved UX (#20091)
* init: azure_ai/azure-model-router

* show additional_costs in CostBreakdown

* UI show cost breakdown fields

* feat: dedicated cost calc for azure ai

* test_azure_ai_model_router

* docs azure model router

* test azure model router

* fix transfrom

* Add transform file

* fix:feat: route to config

* v0 - looks decen view

* refactored code

* fix ui

* fixes ui

* complete v2 viewer

* address feedback

* address feedback
2026-01-30 18:34:13 -08:00
Ishaan Jaffer
e2f7b10c8d address feedback 2026-01-30 18:20:50 -08:00
Ishaan Jaffer
2ce87a3d62 new utils 2026-01-30 18:17:45 -08:00
yuneng-jiang
267993b388
Merge pull request #20095 from BerriAI/litellm_user_dropdown_v2
[Refactor] UI - Navbar: User Dropdown
2026-01-30 18:12:25 -08:00
yuneng-jiang
0942f98b4d refactoring user dropdown 2026-01-30 17:18:06 -08:00
yuneng-jiang
b64cd318d8
Merge pull request #20092 from BerriAI/litellm_user_dropdown
[Fix] UI - Navbar: Fixed Default Logo + Bound Logo Box
2026-01-30 16:47:24 -08:00
yuneng-jiang
3e8e1ded54 update image and bounded logo in navbar 2026-01-30 16:35:24 -08:00
Ishaan Jaff
ad1b48a008
Revert logs view commits to recreate with clean history (#20090)
This reverts commits:
- 437e9e23bd fix drawer
- 61bb51dc3d complete v2 viewer
- 2014bcf9d8 fixes ui
- 5f07635310 fix ui
- f07ef8af00 refactored code
- 8b7a9250ce v0 - looks decen view

Will create a new clean PR with the original changes.
2026-01-30 16:02:58 -08:00
Ishaan Jaffer
437e9e23bd fix drawer 2026-01-30 15:53:37 -08:00
Ishaan Jaffer
61bb51dc3d complete v2 viewer 2026-01-30 15:53:05 -08:00
Ishaan Jaffer
2014bcf9d8 fixes ui 2026-01-30 15:53:05 -08:00
Ishaan Jaffer
5f07635310 fix ui 2026-01-30 15:53:05 -08:00
Ishaan Jaffer
f07ef8af00 refactored code 2026-01-30 15:53:05 -08:00
Ishaan Jaffer
8b7a9250ce v0 - looks decen view 2026-01-30 15:53:05 -08:00
Alexsander Hamir
a11b043f33
fix(proxy): resolve high CPU when router_settings in DB by avoiding REGISTRY.collect() in PrometheusServicesLogger (#20087) 2026-01-30 14:01:45 -08:00
Ishaan Jaff
6897d5f59e
[Feat] Add async_post_call_response_headers_hook to CustomLogger (#20083)
* Add async_post_call_response_headers_hook to CustomLogger (#20070)

Allow CustomLogger callbacks to inject custom HTTP response headers
into streaming, non-streaming, and failure responses via a new
async_post_call_response_headers_hook method.

* async_post_call_response_headers_hook

---------

Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
2026-01-30 12:44:44 -08:00
Ishaan Jaff
974837c4e1
fix: working claude code with agent SDKs (#20081) 2026-01-30 11:58:05 -08:00
Harshit Jain
481bb4b6ce
fixes: ci pipeline router coverage failure (#20065) 2026-01-30 11:55:49 -08:00
Ishaan Jaff
3910161a02
Realtime API benchmarks (#20074)
* Add /realtime API benchmarks to Benchmarks documentation

- Added new section showing performance improvements for /realtime endpoint
- Included before/after metrics showing 182× faster p99 latency
- Added test setup specifications and key optimizations
- Referenced from v1.80.5-stable release notes

Co-authored-by: ishaan <ishaan@berri.ai>

* Update /realtime benchmarks to show current performance only

- Removed before/after comparison, showing only current metrics
- Clarified that benchmarks are e2e latency against fake realtime endpoint
- Simplified table format for better readability

Co-authored-by: ishaan <ishaan@berri.ai>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
2026-01-30 11:55:29 -08:00
Alexsander Hamir
0acfcb494b
Add mock client factory pattern and mock support for PostHog, Helicone, and Braintrust integrations (#19707)
* Add LangSmith mock client support

- Create langsmith_mock_client.py following GCS and Langfuse patterns
- Add mock mode detection via LANGSMITH_MOCK environment variable
- Intercept LangSmith API calls via AsyncHTTPHandler.post patching
- Add verbose logging throughout mock implementation
- Update LangsmithLogger to initialize mock client when mock mode enabled
- Supports configurable mock latency via LANGSMITH_MOCK_LATENCY_MS

* Add Datadog mock client support

- Create datadog_mock_client.py following GCS, Langfuse, and LangSmith patterns
- Add mock mode detection via DATADOG_MOCK environment variable
- Intercept Datadog API calls via AsyncHTTPHandler.post and httpx.Client.post patching
- Add verbose logging throughout mock implementation
- Update DataDogLogger and DataDogLLMObsLogger to initialize mock client when mock mode enabled
- Supports both async and sync logging paths
- Supports configurable mock latency via DATADOG_MOCK_LATENCY_MS

* refactor: consolidate mock client logic into factory pattern

- Create mock_client_factory.py to centralize common mock HTTP client logic
- Refactor GCS, Langfuse, LangSmith, and Datadog mock clients to use factory
- Improve GET/DELETE mock accuracy for GCS (return valid StandardLoggingPayload)
- Fix DELETE mock to return empty body (204 No Content) instead of JSON
- Reduce code duplication across integration mock clients

* feat: add PostHog mock client support

- Create posthog_mock_client.py using factory pattern
- Integrate mock client into PostHogLogger with mock mode detection
- Add verbose logging for mock mode initialization and batch operations
- Enable mock mode via POSTHOG_MOCK environment variable

* Add Helicone mock client support

- Created helicone_mock_client.py using factory pattern (similar to GCS)
- Integrated mock mode detection and initialization in HeliconeLogger
- Mock client patches HTTPHandler.post to intercept Helicone API calls
- Uses factory pattern for should_use_mock and MockResponse utilities
- Custom HTTPHandler.post patching required since HTTPHandler uses self.client.send()

* Add mock support for Braintrust integration and extend mock client factory

- Add braintrust_mock_client.py with mock HTTP client for Braintrust integration testing
- Integrate mock client into BraintrustLogger with mock mode detection
- Refactor Helicone mock client to fully utilize factory's HTTPHandler.post patching
- Extend mock_client_factory to support patching HTTPHandler.post for sync calls
- Enable endpoint-specific mock responses for Braintrust (/project vs /project_logs)
- All mock clients now properly handle both async (AsyncHTTPHandler) and sync (HTTPHandler) calls

* Fix linter errors: remove unused imports and suppress complexity warning

- Remove unused imports from gcs_bucket_mock_client.py (httpx, json, timedelta, Dict, Optional)
- Remove unused Callable import from mock_client_factory.py
- Add noqa comment to suppress PLR0915 complexity warning for create_mock_client_factory function

* Document mock environment variables for PostHog, Helicone, Braintrust, Datadog, and Langsmith integrations

- Add POSTHOG_MOCK and POSTHOG_MOCK_LATENCY_MS documentation
- Add HELICONE_MOCK and HELICONE_MOCK_LATENCY_MS documentation
- Add BRAINTRUST_MOCK and BRAINTRUST_MOCK_LATENCY_MS documentation
- Add DATADOG_MOCK and DATADOG_MOCK_LATENCY_MS documentation
- Add LANGSMITH_MOCK and LANGSMITH_MOCK_LATENCY_MS documentation

All mock env vars follow the same pattern: enable mock mode for integration testing by intercepting API calls and returning mock responses without making actual network calls.

* Fix security issue
2026-01-30 09:52:53 -08:00
yuneng-jiang
c301339412
Merge pull request #20039 from BerriAI/litellm_ui_key_model_bd
[Feature] UI - Usage: Model Breakdown Per Key
2026-01-30 09:16:23 -08:00
Sameer Kankute
6e9210381c
Merge pull request #20058 from BerriAI/litellm_vertex_ai_prompt-caching-scope-2026-01-05,
Fix: remove unsupported prompt-caching-scope-2026-01-05 header for vertex ai
2026-01-30 20:32:00 +05:30
Sameer Kankute
8363a26d2e Fix: remove unsupported prompt-caching-scope-2026-01-05 header for vertex ai 2026-01-30 17:33:40 +05:30
Sameer Kankute
1877483deb
Merge pull request #20056 from BerriAI/litellm_vllm_embedding
Fix: vllm embedding format
2026-01-30 17:01:57 +05:30
Sameer Kankute
5277a0cf82
Merge pull request #20053 from BerriAI/litellm_gemini_edit_jan_30
fix aspectRatio mapping in image edit
2026-01-30 17:01:46 +05:30
Sameer Kankute
8b9d5018b6
Merge pull request #20052 from BerriAI/litllm_disable_cache_jan_29
Add disable flag for anthropic gemini cache translation
2026-01-30 17:01:34 +05:30
Sameer Kankute
404c33eca6
Merge pull request #20051 from BerriAI/litellm_xai_web_search_routing
Add routing of xai chat completions to responses when web search options is present
2026-01-30 17:01:19 +05:30
Sameer Kankute
8d485f2403
Merge pull request #19986 from BerriAI/litellm_batch_cost_tracking_jan29
[Feat]Add cost tracking and usage object in aretrieve_batch call type
2026-01-30 17:00:42 +05:30
Sameer Kankute
cfc9c77347
Merge pull request #19981 from BerriAI/litellm_fix_batches_permissions
Fix: Batch and File user level permissions
2026-01-30 17:00:10 +05:30
Sameer Kankute
51747ee534
Merge pull request #20055 from BerriAI/litellm_files_gemini_endpoints_support
feat: add /delete endpoint support for gemini
2026-01-30 16:58:59 +05:30
Sameer Kankute
a8054264ae
Merge pull request #19975 from BerriAI/litellm_oss_staging_01_29_2026
Litellm oss staging 01 29 2026
2026-01-30 16:58:28 +05:30
Sameer Kankute
55348dd9c5 Fix: vllm embedding format 2026-01-30 16:42:59 +05:30
Sameer Kankute
fdb4b54add
Merge pull request #20009 from genga6/fix/#20006-update-max-input-tokens-for-gpt-5.2-codex
Fix `max_input_tokens` for `gpt-5.2-codex`
2026-01-30 16:18:53 +05:30
Sameer Kankute
ab8b459dfc
Merge pull request #20046 from nht1206/bugfix/input-tokens-details-not-correctly-extracted
fix(ResponseAPILoggingUtils): extract input tokens details as dict
2026-01-30 16:13:35 +05:30
Sameer Kankute
ed4c71ecf5
Merge pull request #20018 from varunsripad123/fix/google-files-retrieval
fix(gemini): support file retrieval in GoogleAIStudioFilesHandle
2026-01-30 16:09:19 +05:30
Sameer Kankute
78dd4b2b2f feat: add /delete endpoint support for gemini 2026-01-30 16:06:07 +05:30
Sameer Kankute
3c451e945a fix aspectRatio mapping 2026-01-30 15:25:18 +05:30
Sameer Kankute
5ac3f75996 Add disable flahg for anthropic gemini cache translation 2026-01-30 14:58:10 +05:30
Sameer Kankute
11c2160379 Add web search tests 2026-01-30 14:56:11 +05:30
Sameer Kankute
dbc8006190 Add routing of xai chat completions to responses when web search options is present 2026-01-30 14:15:35 +05:30
Sameer Kankute
b0b4362dd7
Merge pull request #19989 from BerriAI/litellm_add_passthrough_openai_route
Add /openai_passthrough route for openai passthrough requests:
2026-01-30 11:10:52 +05:30