* Update langfuse.py
Fixing issue with input_tokens and cache_read_tokens
* Clarify input token calculation in langfuse.py
Add comment to clarify input token calculation based on Langfuse documentation.
* fix(redis): handle float redis_version from AWS ElastiCache Valkey
AWS ElastiCache Valkey returns redis_version as a float (7.0) instead
of a string ('7.0.0'), causing AttributeError: 'float' object has no
attribute 'split' in async_lpop when parsing version for LPOP count.
Changes:
- Extract version parsing into _parse_redis_major_version() helper
- Add DEFAULT_REDIS_MAJOR_VERSION constant (replaces magic number)
- Support multiple version formats: string, float, int, malformed
- Add comprehensive test coverage for all version format edge cases
Fixes: 'LiteLLM Redis Cache LPOP: - Got exception from REDIS' error
during db_spend_update_job cronjobs
* refactor: move DEFAULT_REDIS_MAJOR_VERSION to constants.py
* Fix: Remove automatic summary field from reasoning_effort transformation
Problem:
The _map_reasoning_effort() function was automatically adding
reasoning.summary field when users specified reasoning_effort parameter,
causing 400 errors for users with unverified OpenAI organizations.
Root Cause:
According to OpenAI's official documentation, the summary field is opt-in
and requires organization verification:
"Reasoning summary output [...] will not be included unless you explicitly
opt in to including reasoning summaries."
"Before using summarizers with our latest reasoning models, you may need
to complete organization verification"
Source: https://platform.openai.com/docs/guides/reasoning#reasoning-summaries
Solution:
Remove the automatic inclusion of summary field from all reasoning_effort
levels (high, medium, low, minimal). Users who want reasoning summaries
can explicitly pass reasoning={"effort": "high", "summary": "auto"} in
their requests.
Impact:
- Fixes#16032
- Works for all organizations (verified and unverified)
- Maintains backward compatibility for users passing reasoning object directly
- Follows OpenAI's recommended opt-in approach
Testing:
- All existing tests pass (4/4 tests in transformation suite)
- Manual verification confirms only effort field is included
* test: Fix MockResponse missing headers attribute in test_openai_responses_api
The MockResponse class was missing the 'headers' attribute which caused
APIConnectionError when processing the mock response. Added headers={}
to fix the test.
* feat: Add dict support to reasoning_effort parameter
Allow users to pass reasoning_effort as either:
- String: reasoning_effort="high" (no summary, safe default)
- Dict: reasoning_effort={"effort": "high", "summary": "detailed"} (opt-in)
This preserves backward compatibility while giving users flexibility
to explicitly opt-in to the summary field when needed (for verified
OpenAI organizations).
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
This change enables HTTP client session reuse by:
- Adding shared_session parameter to all responses API methods (responses, delete_responses, get_responses, list_input_items, cancel_responses)
- Passing shared_session to get_async_httpx_client() for connection pooling
- Adding debug logging to track shared session usage
This helps reduce memory overhead by reusing HTTP connections instead of creating new clients for each request, which is particularly important for high-throughput proxy scenarios.
This commit fixes two bugs in Responses API streaming tests:
1. **Usage field naming bug**: Tests were using `input_tokens` and
`output_tokens` but the Usage object uses `prompt_tokens` and
`completion_tokens`.
2. **Missing cost in streaming usage**: When `include_cost_in_streaming_usage`
was enabled, the cost was calculated and added to ResponseAPIUsage, but was
lost during the transformation to the Usage object.
Changes:
- Updated test assertions to use correct field names (prompt_tokens, completion_tokens)
- Added cost preservation logic in FakeStreamerResponsesAPIIterator
- Modified _transform_response_api_usage_to_chat_usage() to preserve cost attribute
All streaming tests now pass successfully.
Fixes TypeError when sorting daily spend transactions with missing entity IDs (e.g., null tags).
Changed x[1].get(entity_id_field) to x[1].get(entity_id_field) or "" to provide a sortable
default value when the field is None, preventing comparison errors between NoneType and str.
* add helper functions
* update generic_cost_per_token function
* add test
* formatting
* add examples in docstring for _calculate_tiered_cost
* Restore files to upstream/main version
* dashscope specific calculation
* improve for different costs
* remove _calculate_flat_cost function
* fix(anthropic-adapter): properly translate Anthropic image format to OpenAI
Fixed bug where images were stripped during Anthropic Messages API to Azure
OpenAI translation. Image source data was being stringified instead of having
fields properly extracted.
- Added _translate_anthropic_image_to_openai() helper method
- Support both base64 and URL image formats per Anthropic API spec
- Refactored user message and tool result image handling
* test(anthropic-adapter): add comprehensive image translation tests
Add 5 unit tests covering image translation from Anthropic to OpenAI format:
- User messages with base64 images
- User messages with URL images
- Tool results with base64 images
- Tool results with URL images
- Mixed content with multiple images
* Add v1 cut of container api
* fix lint errors
* Add proxy support to container apis & logging support (#16049)
* Add proxy support to container apis
* Add logging support
* Add cost tracking support for containers and documentation
* Add new constant documentation
* Add container cost in model map
* fix failing azure tests
* Update tests based on model map changes
* fix model map tests
* fix model map tests
* Container modeshould be container
* Container tests fix
* Merge branch 'main' into litellm_sameer_oct_staging_2
* Add Prometheus metric to track callback logging failures in S3 (#16102)
* Add proxy support to container apis
* Add logging support
* prometheus metric measures how often s3_v2 is failing
* remove not needed files
* remove not needed files
* remove not needed files
* fix mypy errors
* Use logging_callback_manager to get all the callbacks
---------
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
* feat(llm_passthrough_endpoints.py): support milvus passthrough api
* fix(llm_passthrough_endpoints.py): move streaming request value to the top of the function
* docs: document new milvus vector store passthrough flow
* feat: change guardrail_information to list type to support displaying multiple guardrails
* fix: add missing commit and revert auto-format changes in utils.py
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* streaming support in langfuse otel
* Added testing for Langfuse Otel tracing in the response API
---------
Co-authored-by: eycjur <eycjur@example.com>
