Commit graph

22916 commits

Author SHA1 Message Date
Alessandro Marrella
0e10ae1cf1
change Message init type annotation to support other roles (#11942) 2025-06-27 18:12:13 -07:00
ohmeow
9ab7d97b85
adds model to metadata to estimated cost is calculated; corrects how to pass metadata in braintrust in the docs (#12022) 2025-06-27 18:11:07 -07:00
Ishaan Jaff
723f24ef17 ci/cd new release 2025-06-27 18:07:44 -07:00
Ishaan Jaff
080372399f test_mock_router_testing_params_str_to_bool_conversion 2025-06-27 18:07:33 -07:00
Ishaan Jaff
0db60009b5 ci/cd new release 2025-06-27 18:06:06 -07:00
Ishaan Jaff
d8dc84ee96 test fix - router utils 2025-06-27 18:05:37 -07:00
Ishaan Jaff
c76f9be729 ui - new build 2025-06-27 18:00:59 -07:00
Ishaan Jaff
01b9e6efb1
Revert "fix: set logger levels based on LITELLM_LOG environment variable" (#12122) 2025-06-27 17:55:44 -07:00
Ishaan Jaff
ebf6395bc1
[Feat] Add Eleven Labs - Speech To Text Support on LiteLLM (#12119)
* add ELEVENLABS as a provider

* add deepgram to main.py

* add ElevenLabsException

* add ElevenLabsAudioTranscriptionConfig

* add transform_audio_transcription_response

* TestElevenLabsAudioTranscription

* add elevenlabs/scribe_v1 to model cost map

* add ElevenLabsAudioTranscriptionConfig

* add AudioTranscriptionRequestData

* add ElevenLabs transform

* use AudioTranscriptionRequestData

* refactoring fixes

* add ProcessedAudioFile util for reading audio files

* test_elevenlabs_diarize_parameter_passthrough

* docs eleven labs

* docs fixes

* fix code qa checks

* fixes - audio transcription

* ui - add ElevenLabs logo

* add elevenlabs logo

* docs - ElevenLabs

* test fix elevenlabs
2025-06-27 17:50:49 -07:00
Ishaan Jaff
041db0268c
[Bug fix] Router - handle cooldown_time = 0 for deployments (#12108)
* fix get cooldown time

* fixes for _should_run_cooldown_logic

* test_cooldown_time_zero_uses_zero_not_default

* Update litellm/router_utils/cooldown_cache.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update litellm/router_utils/cooldown_handlers.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-27 17:50:35 -07:00
Cole McIntosh
cdb5c6fb6b
Merge pull request #12035 from colesmcintosh/add-gemini-md
docs(GEMINI.md): add development guidelines and architecture overview
2025-06-27 17:30:52 -06:00
Ishaan Jaff
f04808e293
[Bug Fix] Exception mapping for context window exceeded - should catch anthropic exceptions (#12113)
* fix is is_error_str_context_window_exceeded

* test_is_error_str_context_window_exceeded

* fix is_error_str_context_window_exceeded
2025-06-27 15:49:07 -07:00
Jason Roberts
b57cfb8bff
Adding Feature: Palo Alto Networks Prisma AIRS Guardrail (#12116)
* feat: Add PANW Prisma AIRS guardrail integration

- Add PANW_PRISMA_AIRS to SupportedGuardrailIntegrations enum
- Update guardrail registry and initializers
- Add complete documentation with curl examples and response formats
- Support pre_call, post_call, and during_call modes
- Include fail-safe error handling and comprehensive logging
- Integration with official PANW Prisma AIRS API

* feat: Add PANW Prisma AIRS guardrail integration

- Update to test file
- Fail-closed security on API errors

* update fail closed behavior

* fix: Update PANW Prisma AIRS guardrail integration pattern

* fix: Remove unused import

* fix: addressed MyPy error
2025-06-27 15:17:36 -07:00
Ishaan Jaff
a42a058b0c
[Bug Fix] Fix handling str, bool types formock_testing_fallbacks on router using /audio endpoints (#12117)
* use a dataclass for managing mock params

* fixes for mock testing
2025-06-27 15:11:09 -07:00
Ishaan Jaff
f85cd9c54a
[Docs] - Show how to use fallbacks with audio transcriptions endpoints (#12115)
* add overview

* docs audio transcriptions

* docs on fallbacks with audio endpoints
2025-06-27 12:43:55 -07:00
Ishaan Jaff
7a5e7f454d bump: version 1.73.3 → 1.73.4 2025-06-27 11:10:14 -07:00
Ishaan Jaff
f1c7024e70
[Feat] Add Bridge from generateContent <> /chat/completions (#12081)
* add GenerateContentToCompletionHandler

* working - non streaming bridge

* add GoogleGenAIAdapter

* add google gen ai adapter

* working streaming bridge

* working streaming usage for adapter

* tool calling transform for generate content to openai

* fixes for accumulating tool calls

* fix code qa checks

* Best Practices for Production

* fix code qa checks

* test_streaming_partial_tool_calls_accumulation

* linting fixes

* add supported_openai_chat_completion_params

* fix translate_generate_content_to_completion

* test_google_genai_adapter.py
2025-06-27 11:08:55 -07:00
Cole McIntosh
575d25ab16
Merge pull request #12111 from colesmcintosh/fix/logger-level-initialization
fix: set logger levels based on LITELLM_LOG environment variable
2025-06-27 11:58:05 -06:00
Cole McIntosh
cce2656376
Merge pull request #12112 from colesmcintosh/fix/cost-calculation-log-level
fix: change cost calculation logs from INFO to DEBUG level
2025-06-27 11:57:55 -06:00
Cole McIntosh
bc41cee0a1 fix: change cost calculation logs from INFO to DEBUG level
- Change verbose_logger.info() to verbose_logger.debug() for cost calculation messages
- Ensures cost calculation details are only shown when DEBUG logging is enabled
- Reduces log noise in production environments

Part 2/2 of fix for #9815
2025-06-27 11:36:58 -06:00
Cole McIntosh
9d58d872ac fix: set logger levels based on LITELLM_LOG environment variable
- Initialize verbose_logger, verbose_proxy_logger, and verbose_router_logger with the correct log level
- Ensures loggers respect the LITELLM_LOG environment variable setting
- Fixes part of issue #9815 where logs were shown regardless of log level setting

Part 1/2 of fix for #9815
2025-06-27 11:34:38 -06:00
Kishan
fc17da0aef
[Bug Fix] Anthropic - Token Usage Null Handling in calculate_usage (#12068)
* [Bug Fix] Anthropic - Token Usage Null Handling in calculate_usage (BerriAI/litellm#11920)

* [Fix] Missed a null check and used a cast instead by error
2025-06-27 10:00:23 -07:00
Krish Dholakia
8bd1f8f6ab
Add o3 and o4-mini deep research models (#12109)
* build(model_prices_and_context_window.json): add o3-deep-research models

* build(model_prices_and_context_window.json): add o4-deep-research model

* build(model_prices_and_context_window.json): add o4-mini-deep-research versioned model
2025-06-27 09:58:53 -07:00
Cole McIntosh
a56c28ab2f
fix: handle provider_config type error in passthrough error handler (#12101)
- Fix type error where _handle_error was called with provider_config=None
- Add proper provider config retrieval logic similar to llm_passthrough_route
- Handle cases where provider config may not be available
- Ensure type safety while maintaining backward compatibility
2025-06-27 09:11:28 -07:00
Ishaan Jaff
e3c9318bd4
[Fix] MCP - Ensure internal users can access /mcp and /mcp/ routes (#12106)
* fixes for MCP route checks

* test is is_llm_api_route for MCP
2025-06-27 09:05:38 -07:00
Cole McIntosh
60bf3c36bf
fix(docs): Remove unused dotenv dependency from docusaurus config (#12102)
* fix(docs): Remove unused dotenv dependency from docusaurus config

The dotenv package was being required in docusaurus.config.js but was listed as
a devDependency, causing build failures. Since no environment variables are
actually used in the config, removed the unnecessary import.

* fix(docs): Remove reference to non-existent spending_monitoring doc

The sidebars.js file was referencing proxy/spending_monitoring which was deleted
in commit ba7463b9c. This was causing the documentation build to fail with missing
document errors.
2025-06-27 08:20:44 -07:00
Ishaan Jaff
c994c22c13
[Bug Fix] Bedrock Guardrails - Ensure PII Masking is applied on response streaming or non streaming content when using post call (#12086)
* test_bedrock_guardrail_response_pii_masking_streaming

* fix - apply masking for response on bedrock
2025-06-27 07:41:32 -07:00
Davis Featherstone
51074ffcae
Fix Azure-OpenAI Vision API Compliance (#12075)
* Fix Azure-OpenAI Vision API Compliance

* Linting Fix
2025-06-26 22:54:15 -07:00
Krish Dholakia
dd92b1a0ac
Refactor: bedrock passthrough fixes - migrate to Passthrough SDK (#12089)
* feat: initial commit adding bedrock support via the new sdk passthrough logic

ensures correct sequencing of tasks (pre call checks etc. can run before signing request)

* fix(route_llm_requests.py): passthrough to allm_passthrough_route if no model found

* feat(bedrock/passthrough): working bedrock passthrough via sdk support

* fix(passthrough/main.py): re-add data and json

* feat(passthrough/main): support async passthrough calls to bedrock

* feat(passthrough/main.py): async streaming + completion support

* feat(llm_passthrough_endpoints.py): migrate bedrock passthrough calls to to new bedrock passthrough sdk

Enables calls to work correctly

* fix: fix linting errors

* test: update test
2025-06-26 22:51:35 -07:00
Krish Dholakia
e8d7537b57
Raise clearer error on Anthropic Unified route + allow setting new_key on /key/regenerate
* fix(llm_http_handler.py): raise clearer error on anthropic unified route

Fixes https://github.com/BerriAI/litellm/issues/12063

* fix(key_management_endpoints.py): add new param `new_key` for setting the regenerated key value

user request

* test: add unit tests

* fix(pass_through_endpoints.py): use data instead of json for passthrough requests

fixes bedrock latency issue

* Revert "fix(pass_through_endpoints.py): use data instead of json for passthrough requests"

This reverts commit 021dfd9165.
2025-06-26 21:56:03 -07:00
Ishaan Jaff
ba7463b9c6 docs - spend monitoring 2025-06-26 17:39:13 -07:00
Ishaan Jaff
edf416c24f docs - Use LiteLLM with Gemini CLI 2025-06-26 17:31:19 -07:00
Ishaan Jaff
72ecac8962 docs - user agent 2025-06-26 16:23:59 -07:00
Ishaan Jaff
16126f0d6b docs Gemini Text-to-Speech 2025-06-26 15:59:48 -07:00
Ishaan Jaff
80acf6bc97 docs */speech to /chat/completions Bridge** 2025-06-26 15:53:25 -07:00
Ishaan Jaff
fc39343599 docs update 2025-06-26 15:32:33 -07:00
Ryan Castner
ac11bfabcb
Responses API - Add reasoning content support for non-OpenAI providers (#12055)
Add reasoning content support when Responses API falls
back to completions API, enabling reasoning content for
all LLM providers (Anthropic, Vertex AI, etc.) since
OpenAI is currently the only native Responses API
provider.

* Add ReasoningSummaryTextDeltaEvent for streaming
  reasoning deltas
* Update streaming iterator to detect and transform
  reasoning content
* Extract reasoning content as separate output items in
  transformations
* Support reasoning content alongside regular message
  content

Closes https://github.com/BerriAI/litellm/issues/11302
2025-06-26 14:21:49 -07:00
Cole McIntosh
98ba5c8ffe
Merge pull request #12042 from colesmcintosh/fix-today-selector-date-mutation-bug
Fix today selector date mutation bug in dashboard components
2025-06-26 14:43:02 -06:00
Ishaan Jaff
e556071544 docs - Use LiteLLM with Gemini CLI 2025-06-26 13:35:33 -07:00
Cole McIntosh
8f376a60fa fix(ui): fix Today filter not showing usage data in dashboard
- Fix date formatting in API calls from ISO format to YYYY-MM-DD
- Update userDailyActivityCall, teamDailyActivityCall, and tagDailyActivityCall
- Prevent date mutation by creating new Date objects before API calls
- Set proper time boundaries (00:00:00 to 23:59:59) for same-day selections

The API expects dates in YYYY-MM-DD format but the UI was sending
full ISO timestamps, causing the Today filter to return empty results.
2025-06-26 13:59:40 -06:00
Cole McIntosh
92948b33b1
Merge branch 'BerriAI:main' into fix-today-selector-date-mutation-bug 2025-06-26 12:29:44 -06:00
Krrish Dholakia
1de2e4ce28 fix(anthropic_endpoints.py): publicly expose anthropic v1/messages endpoint 2025-06-26 10:47:37 -07:00
Krrish Dholakia
0e96f412a1 build(model_prices_and_context_window.json): mark azure o3-pro as responses api model
Fixes https://github.com/BerriAI/litellm/issues/12059
2025-06-26 10:41:03 -07:00
Ishaan Jaff
3c43b0f8ad
add _pretty_print_redis_config (#12073) 2025-06-26 10:26:57 -07:00
tanjiro
bd0a5f7387
Inkeep searchbar and chat added to the Docs (#12030)
* inkeep chat added

* add prod key
2025-06-26 10:13:42 -07:00
Cole McIntosh
7efb2c0f3c
Merge pull request #12041 from colesmcintosh/ci-artifact
Enhance CircleCI integration in LLM translation testing workflow
2025-06-26 11:12:04 -06:00
Cole McIntosh
bd77321c6d
Merge branch 'BerriAI:main' into ci-artifact 2025-06-26 10:59:17 -06:00
Cole McIntosh
7b7472d1f1
Merge pull request #12072 from colesmcintosh/fix/test-mock-create-audio-file
fix(proxy): Fix test_mock_create_audio_file by adding managed_files hook
2025-06-26 10:58:58 -06:00
Cole McIntosh
e4bc213bfe fix(proxy): Add managed_files hook to test_mock_create_audio_file
The test was failing because the managed_files hook was not configured.
This fix adds a DummyManagedFiles class and mocks the create_file
function as an AsyncMock to ensure proper test execution.
2025-06-26 10:42:25 -06:00
Cole McIntosh
c4f3cc6de2 Enhance CircleCI integration in LLM translation testing workflow. Updated commit SHA retrieval method, improved pipeline search logic, and refined artifact downloading process. Added checks for test workflows and job statuses, with placeholder results creation if no artifacts are found. Updated artifact upload step for clarity. 2025-06-26 10:06:30 -06:00