* docs(prompt_management.md): document how to onboard prompts to litellm
* feat(arize_phoenix_prompt_manager.py): support new prompt management integration
allows users to connect arize phoenix prompt manager to litellm
* fix(proxy/utils.py): remove prompt variables to avoid re-processing prompt
* docs(arize_phoenix_prompts.md): document new prompt management integration
- Reset mock call counts at start of test to ensure clean state
- Add span method to mock trace to handle log_provider_specific_information_as_span calls
- Re-establish mock chain before test call to ensure fresh state
- Add exception handling to catch and report errors during test execution
- Add verification that trace was called before checking generation
This should fix the flaky test that was failing intermittently with
'Expected generation to have been called once. Called 0 times.'
Fixed three flaky tests that were intermittently failing in CI:
1. test_no_duplicate_spend_logs (test_litellm/responses/test_no_duplicate_spend_logs.py)
Problem: Used await asyncio.sleep(1) to wait for async logging completion,
which created race conditions. The async logging worker queues tasks
in the background, and sleep() doesn't guarantee completion.
Fix: Replaced sleep() with GLOBAL_LOGGING_WORKER.flush() which properly waits
for the logging queue to empty, ensuring all async logging tasks complete
before assertions run.
2. test_log_langfuse_v2_handles_null_usage_values (test_litellm/integrations/test_langfuse.py)
Problem: Used datetime.datetime.now() twice for start_time and end_time, which
could cause timing inconsistencies between test runs, especially in
CI environments with variable execution speeds.
Fix: Use fixed timestamps instead of datetime.now() to ensure consistent timing
across all test runs, eliminating timing-related flakiness.
3. test_watsonx_gpt_oss_prompt_transformation (test_litellm/llms/watsonx/test_watsonx.py)
Problem: Directly accessed mock_post.call_args without checking if it exists,
which could be None if the mock wasn't called or if an exception
occurred before the POST request. The test catches exceptions and
continues, making this a potential failure point.
Fix: Added proper assertions and use call_args_list[0] for safer access:
- Assert that call_args_list has at least one call
- Assert that call_args is not None
- Assert that 'data' key exists in kwargs
This ensures the test fails with clear error messages rather than
intermittent AttributeError exceptions.
All fixes maintain the original test intent while making them deterministic
and reliable in CI environments.
Reapplies the fix from commit a885e21543 that was
reverted in 6c9556be67.
The original revert was done because the test was flaky and giving false
negatives. This fix properly mocks the Langfuse client to ensure the test
can correctly verify that _log_langfuse_v2 converts None usage values to 0.
Changes:
- Add mock_langfuse_client.client attribute to prevent errors during init
- Add trace_id to mock_langfuse_generation for proper return value handling
- Remove redundant mock setup code
- Explicitly set logger.Langfuse to mock client after initialization
- Set logger.langfuse_sdk_version to ensure _supports_* methods work correctly
* Fix test_log_langfuse_v2_handles_null_usage_values test failure
The test was failing because the logger's Langfuse client wasn't properly
mocked. Even though sys.modules was mocked, the logger's __init__ method
creates its own Langfuse client instance that wasn't using the test's mock.
Changes:
- Explicitly set logger.Langfuse to the mock client after initialization
- Set logger.langfuse_sdk_version to ensure _supports_* methods work correctly
- Added mock_langfuse_client.client attribute to prevent errors during init
- Added trace_id to mock_langfuse_generation for proper return value handling
- Removed redundant mock setup code
This ensures the test can properly verify that _log_langfuse_v2 correctly
converts None usage values to 0 by allowing the mock's generation method
to be called and asserted.
Fixes: AssertionError: Expected 'generation' to have been called once. Called 0 times.
* feat(teams.py): param for disabling guardrails by team
allows use-case where you don't run global guardrails for team - only run team-specific guardrails
* feat(custom_guardrail.py): add support for disabling global guardrails
only run guardrails requested for in the request/key/team
* feat: support adding disable_global_guardrails to metadata if present in key/team metadata
* feat(create_key_button.tsx): new disable global guardrails field
* feat(key_edit_view.tsx): support disabling global guardrails on key edit
* feat(teams.tsx): add disable global guardrails on create team on UI
* feat(team_info.tsx): allow disabling global guardrails on team update
* Address a bug where cloudzero spend is not sent to cloudzero if a spend
update happens
* revert change unrelated to PR
* use polars for mocking instead of sqlite
* feat: change guardrail_information to list type to support displaying multiple guardrails
* fix: add missing commit and revert auto-format changes in utils.py
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* streaming support in langfuse otel
* Added testing for Langfuse Otel tracing in the response API
---------
Co-authored-by: eycjur <eycjur@example.com>
* fix(opentelemetry.py): fix issue where headers were not being split correctly
* feat(bedrock/image): Support bedrock titan image generation
Closes https://github.com/BerriAI/litellm/issues/361
* build(model_prices_and_context_window.json): track titan image gen pricing
enables cost tracking per request
* feat(amazon_titan_transformation.py): support titan image generation cost tracking
* docs: document new model
* docs: update docs to indicate cost tracking + refactor rerank into separate doc
* fix: fix mypy linting error
* fix: fix type ignore
* fix(presidio.py): handle content as a list of texts
covers openai + anthropic messages api
* fix(presidio.py): safe get messages
* test: add unit testing for presidio guardrails
* fix(unified_guardrail.py): initial commit
* fix(enkryptai.py): implement apply_guardrail to enkrypt guardrail
* fix(unified_guardrail.py): support unified guardrail on input
* feat(unified_guardrail.py): add post call success hook implementation
allows us to just have 1 place to handle llm translation to guardrail api spec
* refactor: refactor initial unified guardrail component
* refactor: more refactoring
* feat(responses/): add guardrails to responses api
allows existing guardrails to work for new llm endpoints
* docs(adding_guardrail_support.md): document new guardrail endpoint support
* test: add unit tests
* feat(image_generation/): add guardrail support for image generation endpoint
* feat(openai/text_completion): support guardrails on `/v1/completions` API
* docs: document guardrails support on new endpoints
* docs: clarify when guardrails run
* feat(openai/speech): add guardrail support for input
* docs(rerank/): add guardrail support on input query
* fix: fix ruff check
* Addd v2/chat support for cohere
* fix streaming
* Use v2_transformation for logging passthrough:
* Use v2_transformation for logging passthrough:
* Add test for checking if document and citation_options is getting passed
* Update the cohere model
* Add cost tracking for vertex ai passthrough batch jobs
* Add full passthrough support
* refactor code according to the comments
* Add passthrough handler
* remove invalid params
* Updated documentation
* Updated documentation
* Updated documentation
* Correct the import
* Add openai videos generation and retrieval support
* add retrieval endpoint
* Add docs
* Add imports
* remove orjson
* remove double import
* fix openai videos format
* remove mock code
* remove not required comments
* Add tests
* Add tests
* Add other video endpoints
* Fix cost calculation and transformation
* Fixed mypy tests
* remove not used imports
* fix documentation for get batch req (#15742)
* Add grounding info to responses API (#15737)
* Add grounding info to responses API
* fix lint errors
* Use typed objects for annotations
* Use typed objects for annotations
* fix mypy error
* Litellm fix json serialize alreting 2 (#15741)
* fix json serializable error for alerts
* Add test
* fix mypt errors
* fix mypt errors
* Add Qwen3 imported model support for AWS Bedrock (#15783)
* Add qwen imported model support
* fix mypy errors
* fix empty user message error (#15784)
* fix typed dict for list
* Add azure supported videos endpoint
* fix mapped tests
* add azure sora models to model map
* Add OpenAI video generation and content retrieval support (#15745)
* Add openai videos generation and retrieval support
* add retrieval endpoint
* Add docs
* Add imports
* remove orjson
* remove double import
* fix openai videos format
* remove mock code
* remove not required comments
* Add tests
* Add tests
* Add other video endpoints
* Fix cost calculation and transformation
* Fixed mypy tests
* remove not used imports
* fix typed dict for list
* fix mypy errors
* move directory
* make v2 chat default
* Fix mypy tests
* Fix mypy tests
* Fix mypy tests
* Fix mypy tests
* Revert "Add Azure Video Generation Support with Sora Integration"
* refactor videos repo
* add test
* Add azure openai videos support
* Add azure openai videos support
* Add router endpoint support for videos
* fix mypy error
* add azure models
* fix mapped test
* fix mypy error
* Add proxy router test
* Add proxy router test
* remove deprecated model name from tests
* fix import error
* fix import error
* Add gaurdrail integration in videos endpoint
* Add logging support for videos endpoint
* Add final documentation supporting videos integration
* fix model name and document input
* Update literals to avoid mypy errors
* Remove unused imports and print statements
* revert guardrail support for video generation and video remix
* revert guardrail support for video generation and video remix
* Fix failing mapped and llm translation tests
