Ishaan Jaffer
8c1dd78c13
fix imports
2025-11-08 18:01:32 -08:00
Krish Dholakia
06906534b3
feat(audio_transcriptions/): calculate duration of audio file for cost calculation + feat (image_generations): cost tracking accuracy improved with output_format, quality, size values fixed per openai model
...
* feat(audio_transcriptions/): calculate duration of audio file for cost calculation
Fixes https://github.com/BerriAI/litellm/issues/11846
Closes https://github.com/BerriAI/litellm/issues/14605
* fix(cost_calculator.py): correctly use base model, when set
Fixes issue where azure base model was being ignored
* feat(cost_calculator.py): fix default cost tracking quality param for image generation
* feat(image_generations/): return output_format, quality, size
aligns response to openai spec and improves cost tracking accuracy
* fix(cost_calculator.py): refactor cost calculation for image generation to use image response instead of hidden params
* build: update build
* fix: fix cost calculation
* build: update poetry lock
* fix: fix ruff checks
* fix: fix aembedding
* fix: fix ruff errors
* fix: modify to catch errors
* fix: test
* fix: loosen test to handle openai lib out of sync
* fix: fix base models
* fix: fix usage object
2025-11-08 16:24:31 -08:00
Sameer Kankute
86d73c918c
Adds support for returning Azure Content Policy error information when exceptions from Azure OpenAI occur ( #16231 )
...
* add provider_specific_fields to ContentPolicyViolationError
* use provider_specific_fields in ProxyException
* update openai_exception_handler
* fix use exception checker for content policy violation azure
* add AzureOpenAIExceptionMapping
* test_azure_with_content_safety_error
* Accessing Provider-Specific Error Details
* TestExceptionCheckers
* unit test got provider_specific_fields=
* add clear types for error dict
* fix test_azure_with_content_safety_error
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-11-08 16:04:36 -08:00
Sameer Kankute
e037d9315d
Add Vertex and Gemini Videos API with Cost Tracking + UI support ( #16323 )
...
* Use video id for videos api
* remove mock code
* Potential fix for code scanning alert no. 3630: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* remove print statements
* Update video prefix for 'video_'
* Add veo with openai videos unified specs
* Add videos testing to UI
* remove mock code
* Remove not need ui changes:
* Fix mypy errors related to gemini
* fix test_transform_video_create_request
* Add vertex ai veo config
* Add vertex ai veo config
* Add cost tracking for gemini and add optional param passing
* fix bugs related to vertex ai veo
* Add Gemini Veo Video Generation in Openai Videos Unified Spec (#16229 )
* Add veo with openai videos unified specs
* Add videos testing to UI
* remove mock code
* Remove not need ui changes:
* Fix mypy errors related to gemini
* fix test_transform_video_create_request
* Add contant video duration for gemini and vertex
* Fix litellm_mapped_tests tests
* fix azure videos issue
* Added doc for videos vertex ai
* fix seconds param error
* fix lint errors
* test_transform_video_create_response_cost_tracking_no_duration
---------
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
2025-11-08 16:03:51 -08:00
Krish Dholakia
202eaeb1a2
Revert "(feat) Audio transcription - cost tracking + (feat) image generation …" ( #16409 )
...
This reverts commit c96da44265
2025-11-08 15:38:16 -08:00
Krish Dholakia
c96da44265
(feat) Audio transcription - cost tracking + (feat) image generation - accurate cost tracking based on output_format/quality/size
...
* feat(audio_transcriptions/): calculate duration of audio file for cost calculation
Fixes https://github.com/BerriAI/litellm/issues/11846
Closes https://github.com/BerriAI/litellm/issues/14605
* fix(cost_calculator.py): correctly use base model, when set
Fixes issue where azure base model was being ignored
* feat(cost_calculator.py): fix default cost tracking quality param for image generation
* feat(image_generations/): return output_format, quality, size
aligns response to openai spec and improves cost tracking accuracy
* fix(cost_calculator.py): refactor cost calculation for image generation to use image response instead of hidden params
* build: update build
* fix: fix cost calculation
* build: update poetry lock
* fix: fix ruff checks
* fix: fix aembedding
* fix: fix ruff errors
* fix: modify to catch errors
* fix: test
* fix: loosen test to handle openai lib out of sync
2025-11-08 15:30:46 -08:00
Krish Dholakia
bce8a5d6b7
Revert "(feat) audio transcriptions cost tracking (for azure/other non-openai…" ( #16402 )
...
This reverts commit 9a88fe0861
2025-11-08 14:24:51 -08:00
Ishaan Jaffer
e4c31add58
test_dd_llms_obs_redaction
2025-11-08 14:23:42 -08:00
Ishaan Jaffer
1bf284abca
test_delete_callbacks_in_db
2025-11-08 14:05:44 -08:00
Ishaan Jaffer
a4c008c433
test_datadog_message_redaction
2025-11-08 14:02:24 -08:00
Krish Dholakia
9a88fe0861
(feat) audio transcriptions cost tracking (for azure/other non-openai models) + (fix) image generations - accurate cost tracking for dalle3/gpt-image-1 - uses the correct max image quality ( #16076 )
...
* feat(audio_transcriptions/): calculate duration of audio file for cost calculation
Fixes https://github.com/BerriAI/litellm/issues/11846
Closes https://github.com/BerriAI/litellm/issues/14605
* fix(cost_calculator.py): correctly use base model, when set
Fixes issue where azure base model was being ignored
* feat(cost_calculator.py): fix default cost tracking quality param for image generation
* feat(image_generations/): return output_format, quality, size
aligns response to openai spec and improves cost tracking accuracy
* fix(cost_calculator.py): refactor cost calculation for image generation to use image response instead of hidden params
* build: update build
* fix: fix cost calculation
* build: update poetry lock
* fix: fix ruff checks
2025-11-08 13:54:37 -08:00
Ishaan Jaffer
993c75d10b
test_redaction_responses_api
2025-11-08 13:43:22 -08:00
Ishaan Jaffer
c43fcf2ded
test_redaction_responses_api
2025-11-08 13:39:40 -08:00
Sameer Kankute
6fb0a8fc58
Added xai responses support ( #16391 )
...
* Added xai responses support
* add the xai provider config above
* remove init file
* remove init file
* Fix f string lint error
---------
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
2025-11-08 12:30:09 -08:00
Ishaan Jaffer
eef8c01d4e
test_redaction_responses_api
2025-11-08 12:29:38 -08:00
Ishaan Jaffer
1d2bdaebb6
test_openai_streaming_logging
2025-11-08 11:49:36 -08:00
Ishaan Jaffer
9b4e32c880
Revert "Added support for desabling thoughts by setting budget to 0 ( #16347 )"
...
This reverts commit b6f792f301
2025-11-08 11:34:05 -08:00
Ishaan Jaffer
4fb521c251
test_basic_openai_responses_api_non_streaming_with_logging
2025-11-08 11:12:35 -08:00
Ishaan Jaffer
c60a13c919
fixes
2025-11-08 11:11:22 -08:00
Ishaan Jaffer
da7ecb3ea9
test_gemini_reasoning_effort_zero_budget_disables_thoughts
2025-11-08 11:10:54 -08:00
Ishaan Jaffer
bbcdf6f996
test_basic_openai_responses_api_non_streaming_with_logging
2025-11-08 10:47:48 -08:00
Ishaan Jaffer
0148b8d2f7
test_basic_openai_responses_api_non_streaming_with_logging
2025-11-08 10:36:47 -08:00
Ishaan Jaffer
89157f4b5c
test_basic_openai_responses_api_streaming_with_logging
2025-11-08 10:30:41 -08:00
Ishaan Jaffer
6bb963dca2
test_basic_openai_responses_api_non_streaming_with_logging
2025-11-08 10:13:27 -08:00
Ishaan Jaffer
68d24b40c6
test_gemini_25_implicit_caching_cost
2025-11-08 10:13:04 -08:00
Ishaan Jaffer
f9d95b71bb
fix _get_assembled_streaming_response
2025-11-08 10:08:03 -08:00
Ishaan Jaffer
7bf9d69007
test_gemini_reasoning_effort_env_override
2025-11-07 22:09:59 -08:00
Ishaan Jaffer
45323dd537
test_patch_guardrail_endpoint
2025-11-07 22:09:52 -08:00
Alan Ponnachan
2408e09f6a
feat: Add support for Anthropic Memory Tool ( #16115 )
...
* add memory tool in anthropic.py
* add memory tool test
* make format
* update transformation
* adding memory to hosted tools
* add test
* make format
2025-11-07 19:27:28 -08:00
YutaSaito
6eb74bd62a
Feat/persist mcp credentials in db ( #16308 )
...
* feat: persist mcp credentials in db
* feat: remove Auth Value field from MCP Tool Testing Playground
* fix: test
2025-11-07 19:22:49 -08:00
Sameer Kankute
b6f792f301
Added support for desabling thoughts by setting budget to 0 ( #16347 )
2025-11-07 19:19:16 -08:00
Jack Cherng
2ab34f9a52
Fix HostedVLLMRerankConfig will not be used ( #16352 )
...
* Fix HostedVLLMRerankConfig will not be used
Signed-off-by: Jun-Fei Cherng <jfcherng@realtek.com>
* Fix no usage statistics in rerank with hosted_vllm
Signed-off-by: Jun-Fei Cherng <jfcherng@realtek.com>
* Revise typo in comment
Signed-off-by: Jun-Fei Cherng <jfcherng@realtek.com>
---------
Signed-off-by: Jun-Fei Cherng <jfcherng@realtek.com>
2025-11-07 19:11:59 -08:00
Ishaan Jaffer
a8533dc5c4
Revert "Added xai responses support ( #16310 )"
...
This reverts commit ee50f09e73
2025-11-07 18:39:38 -08:00
Ishaan Jaffer
736b6b3e1e
TestVertexAIRerankTransform
2025-11-07 18:18:48 -08:00
Ishaan Jaff
674d4b4cab
[Feat] Guardrails - LiteLLM Content Filter, Allow Viewing/Editing Content Filter Settings ( #16383 )
...
* fix safe dumps
* add patterns.json
* add PrebuiltPattern
* add test patterns
* fix edit and view
* fix backend handling
* fix CF ui edit
* fix init
2025-11-07 18:15:09 -08:00
Ishaan Jaff
ca229fe030
[Feat] LiteLLM Guardrail - UI Fix, ensure you can see UI Friendly name for PII Patterns ( #16382 )
...
* fix safe dumps
* add patterns.json
* add PrebuiltPattern
* add test patterns
2025-11-07 18:14:58 -08:00
Sameer Kankute
faae0ff0dc
Fix Azure DALL-E-3 health check content policy violation by using safe default prompt ( #16329 )
...
* Add custom health check prompt support
* Add constant for health check prompt
* Add constant for health check prompt
2025-11-07 15:30:56 -08:00
Ishaan Jaff
2bd85dc455
[Feat] Add DD Agent Host support for datadog callback ( #16379 )
...
* add DD_AGENT_HOST
* docs DD Agent
* test_datadog_agent_configuration
* DD_AGENT_HOST
2025-11-07 15:18:23 -08:00
yuneng-jiang
3a96c700b4
Adjusted based on comments
2025-11-07 15:02:41 -08:00
Ishaan Jaff
a6b0993405
[Feat] Secret Manager - Hashicorp, add auth via approle ( #16374 )
...
* add _verify_required_credentials_exist and _auth_via_approle
* test_hashicorp_secret_manager_approle_auth
* docs hcorp auth
2025-11-07 14:39:33 -08:00
Alan Ponnachan
5b01fe0a81
fix(vertex_ai): Correctly map 429 Resource Exhausted to RateLimitError ( #16363 )
2025-11-07 14:36:20 -08:00
Emerson Gomes
940a72ceb0
Add Vertex MiniMAX m2 ( #16373 )
2025-11-07 14:27:42 -08:00
Xingyao Wang
4860cdbfd5
Fix: Azure GPT-5 incorrectly routed to O-series config (temperature parameter unsupported) ( #16246 )
...
* Fix Azure GPT-5 incorrectly routing to O-series config
GPT-5 models support reasoning but are NOT O-series models and DO support
temperature parameter. The previous routing logic in get_provider_responses_api_config()
was incorrectly sending Azure GPT-5 requests to AzureOpenAIOSeriesResponsesAPIConfig
which removes temperature from supported params.
This fix explicitly excludes GPT-5 models from O-series routing, ensuring they
use the standard AzureOpenAIResponsesAPIConfig which properly supports temperature.
Fixes: Azure GPT-5 throwing UnsupportedParamsError for temperature parameter
Tested: Added comprehensive unit tests for GPT-5 and O-series routing
* Apply suggestion from @xingyaoww
* Apply suggestion from @xingyaoww
* Improve Azure routing logic to use broader 'gpt' check for temperature support
Based on feedback from @krrishdholakia, updated the routing logic to check
for 'gpt' in model name instead of specifically 'gpt-5'. This approach is:
- More future-proof: covers all GPT models (gpt-3.5, gpt-4, gpt-5, future models)
- Simpler: single check for all GPT variants
- More maintainable: won't need updates for each new GPT model
Changes:
- litellm/utils.py: Changed from is_gpt5 to is_gpt_model check
- tests: Added comprehensive test for all GPT model variants (gpt-3.5 through gpt-5)
All tests pass:
- GPT models (gpt-3.5-turbo, gpt-4, gpt-4o, gpt-5) -> AzureOpenAIResponsesAPIConfig (supports temperature)
- O-series models (o1, o3) -> AzureOpenAIOSeriesResponsesAPIConfig (no temperature)
Co-authored-by: openhands <openhands@all-hands.dev>
---------
Co-authored-by: openhands <openhands@all-hands.dev>
2025-11-07 10:24:27 -08:00
Sameer Kankute
94e1a1ecac
Use vertex creds passed via arguments ( #16266 )
2025-11-06 19:35:22 -08:00
Jason Roberts
5733f711fd
feat(guardrails): panw prisma airs guardrail deduplication and enhanced session tracking ( #16273 )
...
* feat(guardrails): Add deduplication and session tracking
- Implement deduplication logic to prevent duplicate scans (via call_id; add _check_and_mark_scanned) caused by LiteLLM callback system
- Add session tracking using litellm_trace_id as AI Session ID for Prisma AIRS SCM logging
- Extract helper methods _extract_prompt_from_request maintainability
- Use httpxSpecialProvider import (LoggingCallback -> GuardrailCallback)
- Add comprehensive tests for deduplication and session tracking (7 new tests)
- Update documentation with multi-turn conversation tracking examples
* docs: update PANW Prisma AIRS multi-turn conversation example to use industry-standard terminology
- Clearer example for conversation tracking
- Updated terminology from 'AI Session ID' to 'Prisma AIRS AI Session ID' for clarity
* fix: remove unused asyncio import
* fix: correct mypy type ignore comment
2025-11-06 19:34:37 -08:00
Andrii Kislitsyn
c497b6f239
Add retry-after header support for errors 502, 503, 504 ( #16288 )
...
* retry-after-header-support-for-502-503-504-initial
* retry-after-header-support-for-502-503-504-tests-and-linters
2025-11-06 19:33:29 -08:00
Cesar Garcia
16325024df
fix: Use valid CallTypes enum value in embeddings endpoint ( #16328 )
...
* Fix embeddings endpoint call_type to use valid CallTypes enum value
Fixed bug where the `/embeddings` endpoint was passing `call_type="embeddings"`
to guardrail hooks, but "embeddings" is not a valid value in the CallTypes enum.
Changed to use `call_type="aembedding"` (async embedding) which is the correct
CallTypes enum value and matches the route_type used in the same function.
Added unit tests to verify:
- "embeddings" is not a valid CallTypes enum value
- "aembedding" is the correct valid value
- The fix prevents ValueError when guardrails are enabled
Fixes #16240
* Inline embeddings call type regression check
* Ensure embedding test preserves proxy metadata
2025-11-06 19:25:00 -08:00
Sameer Kankute
83998d3573
Update the fireworks url in tests and doc ( #16346 )
2025-11-06 19:22:21 -08:00
Ishaan Jaff
bd7d653bae
Revert "Update perplexity cost tracking ( #15743 )" ( #16345 )
...
This reverts commit ad6a0f4d44
2025-11-06 19:00:45 -08:00
Ishaan Jaffer
5157078e0a
test_delete_callbacks_in_db
2025-11-06 18:37:36 -08:00
