Fixes RuntimeError "Session is closed" by:
- Checking session.closed before use and recreating if needed
- Catching RuntimeError during requests and retrying with new session
- Validating newly created sessions aren't already closed
Adds tests for both proactive detection and reactive retry scenarios.
* Fix: Add shared_session to all_litellm_params to prevent JSON serialization error
The shared_session parameter (aiohttp.ClientSession) was being passed through
to provider API calls, causing "Object of type ClientSession is not JSON
serializable" errors during embedding requests.
Added shared_session to the all_litellm_params list so it's properly filtered
out as a LiteLLM-internal parameter and not passed to the provider's API.
* Fix: Add shared_session support for embedding calls with connection pooling
The shared_session parameter was not being properly handled in embedding calls,
causing it to be passed through to provider API requests where it's not needed.
Changes:
- Added shared_session to all_litellm_params to filter it from provider API request body
- Extract shared_session in main embedding() function and pass it explicitly
- Updated OpenAI embedding handlers (embedding() and aembedding()) to accept shared_session
- Pass shared_session to _get_openai_client for HTTP client creation
This enables proper connection pooling for embedding requests when shared_session
is provided, improving performance for high-throughput scenarios.
* test: add regression test for shared_session in embedding calls
Add comprehensive test to prevent JSON serialization errors when using
shared_session.
The test verifies two critical aspects:
1. shared_session is in all_litellm_params to prevent "Object of type
ClientSession is not JSON serializable" errors
2. shared_session flows through the complete call chain across 6 layers:
- litellm.embedding()
- OpenAI.embedding/aembedding()
- _get_openai_client()
- AsyncHTTPHandler.create_client()
- _create_async_transport()
- _create_aiohttp_transport()
Similar to test_acompletion_session_reuse_e2e.py but focused on
embedding endpoints. Uses inspect.getsource() to verify the parameter
is not only accepted but actually passed through each layer.
* fix: preserve lists in SensitiveDataMasker to prevent string conversion
Added 'list' to allowed primitive types in mask_dict() to prevent lists like
tags from being converted to string representations in API responses.
Before: {"tags": "['East US 2', 'production', 'test']"}
After: {"tags": ["East US 2", "production", "test"]}
* add: unit test
Fixes#15263
This PR fixes the cost calculation for Bedrock Anthropic models with prompt caching.
**Root Cause:**
PR #9838 incorrectly removed adding `cacheWriteInputTokens` to `prompt_tokens`
for Bedrock, based on the assumption that it would cause double counting (similar
to an Anthropic API issue). However, Bedrock's token structure is different:
- **Bedrock API**: `inputTokens`, `cacheReadInputTokens`, and `cacheWriteInputTokens`
are ALL separate values that should be summed for total input tokens
- **Anthropic API**: Same structure - all three token types are separate
The fix in #9838 was later reverted for Anthropic (correctly re-adding
`cache_creation_input_tokens` to `prompt_tokens`), but Bedrock was never fixed.
**Changes:**
1. Re-add `cacheWriteInputTokens` to `input_tokens` in Bedrock transformation
2. Update test assertions to reflect correct behavior
3. Add regression test for prompt caching cost calculation
4. Fix typo in Anthropic transformation where `cache_creation_tokens` was
incorrectly set to `cache_read_input_tokens`
**Testing:**
- All existing Bedrock transformation tests pass
- New test validates correct cost calculation with prompt caching
- Verified costs are non-negative and accurate
