* Fix: Add shared_session to all_litellm_params to prevent JSON serialization error
The shared_session parameter (aiohttp.ClientSession) was being passed through
to provider API calls, causing "Object of type ClientSession is not JSON
serializable" errors during embedding requests.
Added shared_session to the all_litellm_params list so it's properly filtered
out as a LiteLLM-internal parameter and not passed to the provider's API.
* Fix: Add shared_session support for embedding calls with connection pooling
The shared_session parameter was not being properly handled in embedding calls,
causing it to be passed through to provider API requests where it's not needed.
Changes:
- Added shared_session to all_litellm_params to filter it from provider API request body
- Extract shared_session in main embedding() function and pass it explicitly
- Updated OpenAI embedding handlers (embedding() and aembedding()) to accept shared_session
- Pass shared_session to _get_openai_client for HTTP client creation
This enables proper connection pooling for embedding requests when shared_session
is provided, improving performance for high-throughput scenarios.
* test: add regression test for shared_session in embedding calls
Add comprehensive test to prevent JSON serialization errors when using
shared_session.
The test verifies two critical aspects:
1. shared_session is in all_litellm_params to prevent "Object of type
ClientSession is not JSON serializable" errors
2. shared_session flows through the complete call chain across 6 layers:
- litellm.embedding()
- OpenAI.embedding/aembedding()
- _get_openai_client()
- AsyncHTTPHandler.create_client()
- _create_async_transport()
- _create_aiohttp_transport()
Similar to test_acompletion_session_reuse_e2e.py but focused on
embedding endpoints. Uses inspect.getsource() to verify the parameter
is not only accepted but actually passed through each layer.
This commit implements a complete, end-to-end fix for the native Gemini API translation feature, allowing requests to be correctly routed to other model providers via `model_group_alias`.
The original implementation was broken, causing `systemInstruction` and `tools` to be dropped from requests. This was resolved by refactoring the Gemini endpoint to use a dedicated translation path, similar to the Anthropic adapter.
Additionally, this commit hardens the streaming response adapter to correctly handle tool calls generated by the newly-fixed request path. Key improvements to the response handling include:
- Replaced the fragile `id`-based tool call tracking with a robust `index`-based accumulation logic.
- Fixed a memory leak and improved logging in the stream finalization process.
- Prevented empty, non-compliant chunks from being sent to the client during tool call streaming.
- Optimized the accumulator to skip and log superfluous empty chunks sent by some models.
* fix: use fastuuid helper across the codebase
First batch of changes, simple drop in replacement.
* second batch of changes
* fixed: script mistake on helper file
Allow passing aiohttp.ClientSession to acompletion() calls for better
performance and resource management. Includes debug logging, tests,
and documentation. Backward compatible.
- Add missing provider_config parameter in main.py for proper HTTP handler integration
- Update tests to use correct respx mocking pattern with litellm.disable_aiohttp_transport
- Add get_error_class method to CompactifAI transformation for proper error handling
- Fix authentication error test to expect APIConnectionError instead of AuthenticationError
- All 8 CompactifAI tests now pass successfully
- Keep CompactifAI provider detection logic
- Include new OVHCloud provider from main branch
- Both providers now work correctly with model prefix detection
- Add COMPACTIFAI to LlmProviders enum for type safety
- Register CompactifAIChatConfig in ProviderConfigManager
- Import CompactifAIChatConfig in main __init__.py
- Add 'compactifai/' model prefix detection in get_llm_provider()
- Wire CompactifAI completion handler in main.py routing logic
- Support COMPACTIFAI_API_KEY environment variable
- Enable base_llm_http_handler for OpenAI-compatible requests
- Maintain consistency with existing provider integration patterns
