* Enhance proxy CLI with Rich formatting and improved user experience
- Integrated Rich library for better console output in `proxy_cli.py`, including version display, health check results, and test completion responses.
- Updated health check and test completion methods to provide progress indicators and formatted tables.
- Refactored feedback display in `proxy_server.py` to use Rich for a more visually appealing user interface.
- Adjusted tests in `test_proxy_cli.py` to mock console output instead of using print statements, ensuring compatibility with Rich formatting.
* fix linting error
* refactor(proxy_cli.py): simplify DB setup logging
- Removed progress indicators for IAM token generation and environment variable decryption to simplify the code.
- Consolidated the logic for generating the database URL and setting environment variables.
- Enhanced error handling for configuration loading and database setup, ensuring clearer feedback
* Update test-linting workflow to include proxy-dev dependencies in Poetry installation
* Enhance proxy server initialization with Rich console for improved model display. Added support for loading model parameters from environment variables and refined provider identification logic. Fallback to original print formatting if Rich is not available.
* Refactor feedback handling: Moved feedback message generation and custom warning display to utils.py. Enhanced feedback box with rich formatting and fallback to ASCII for environments without rich. Cleaned up proxy_server.py by removing obsolete code.
* fix linting error
* Refactor model initialization display: Moved model initialization logic to a new utility function `display_model_initialization` for improved readability and maintainability. Enhanced model provider extraction with a dedicated function. Fallback to basic logging if Rich console is unavailable.
* Refactor model provider extraction: Replace the `_extract_provider_from_model` function with a more robust approach using `get_llm_provider`. Implement fallback logic for provider identification and improve error handling. Ensure compatibility with Rich console for model initialization display.
* Refactor get_end_user_id_from_request_body to support user ID retrieval from custom headers and multiple request body formats. Enhance tests to cover various scenarios including header precedence and fallback mechanisms.
* Refactor get_end_user_id_from_request_body function to accept request_body as the first parameter, improving clarity and flexibility. Update tests for compatibility and add new cases to ensure correct functionality across various request body formats.
* Update _user_api_key_auth_builder and user_api_key_auth to pass request object to get_end_user_id_from_request_body, enhancing user ID retrieval from request data.
* refactor(auth_utils.py): update get_end_user_id_from_request_body to accept request_headers instead of request, and adjust related function calls in user_api_key_auth and tests
* refactor(tests): update mock request handling in LLM pass-through endpoint tests
- Replaced the Request object with a Mock for better flexibility in testing.
- Enhanced mock setup to include user API key handling and virtual key retrieval.
- Updated test calls to reflect changes in mock request structure and added necessary patches for new dependencies.
* refactor(vertex_and_google_ai_studio_gemini.py): remove redundant variable declaration for url_context_metadata, linting error
* Handle file content type transformation in responses api (#11310)
* Handle file content type transformation in responses api
* change to use input_file
* -
* TestLiteLLMCompletionResponsesConfig
* test: TestLiteLLMCompletionResponsesConfig
* fix: fix linting
---------
Co-authored-by: Jayme Gordon <jayme_gordon@icloud.com>
* fix(vertex_and_google_ai_studio_gemini.py): add web search request tracking
Enables cost calculation for google web search
* fix(vertex_and_gemini): use common processing logic across stream / non-stream calls
* fix(vertex_And_google_ai_studio_Gemini.py): fix initial choice
* fix: fix linting error
* fix: add initial support for google search cost tracking
* fix(tool_call_cost_tracking.py): working tool cost tracking for gemini
* fix(vertex_ai/gemini/cost_calculator.py): add google web search tool cost tracking for vertex ai
Closes LIT-210
* fix: fix check
* build(model_prices_and_context_window.json): fix amazon nova max output tokens
Closes https://github.com/BerriAI/litellm/issues/11441
* fix: fix ruff check
* fix: using litellm with claude code bedrock
* fix: usage for bedrock with /messages
* fix: bedrock_sse_wrapper
* tests: test for test_chunk_parser_usage_transformation
* test fix
* fix(huggingface): use get() instead of pop() for input_type parameter
Fixes embedding generation for HuggingFace models where input_type override
is required (e.g. BAAI/bge-m3). The pop() method was mutating optional_params
and removing input_type before downstream functions could access it.
* Add unit tests to catch regression
* Move tests around
- Renamed SSOSettingsResponse to inherit from a new base class SettingsResponse for better structure.
- Introduced InternalUserSettingsResponse and DefaultTeamSettingsResponse models for internal user and default team settings.
- Updated endpoint responses to use field_schema instead of schema for consistency.
- Enhanced test cases to validate the new response structure and ensure proper functionality of SSO settings.
* fix(anthropic/): fix regression when passing file url's to the 'file_id' parameter
add test and ensure anthropic file url's are correctly sent as 'document' blocks
* fix(vertex_and_google_ai_studio.py): Use same usage calculation function as non-streaming
Closes https://github.com/BerriAI/litellm/issues/10667
* test(test_vertex_and_google_ai_studio_gemini.py): update test
* fix(ui_sso.py): update user as proxy admin in db table, when checking for proxy_admin_id
Fixes issue where existing internal user, unable to make calls when set as proxy admin id
* fix(utils.py): fix custom base path
* Fixes issue with team_endpoints on member budget update
* refactored location of budget membership fix
* added test for _upsert_budget_membership func
* fix(proxy_server.py): working swagger on custom base
removes the swagger monkey patch - this seems to render the swagger on custom base paths
* fix(ui/): working custom auth uptil login success event
* fix(ui/): working custom server root path for login
* fix(proxy_server.py): create typed dict for ui returned token
allows better documentation of expected params
* refactor(proxy_server.py): refactor all ui login endpoints to use same returned ui token object
* feat(ui_sso.py): add server root path to ui token
* feat(ui_sso.py): allows ui to call correct endpoint
* fix(networking.tsx): update proxy base url with custom root path
* fix(networking.tsx): handle updating proxy base url for non-local instances
* refactor: remove uneccessary references to proxybaseurl in ui code - reduce potential for errors
* fix: fix linting error
* fix(onboarding_link.tsx): fix onboarding link when custom server path is set
* feat(ui_discovery_endpoints.py): add new public .well-known/ route for litellm ui config
returns the server root path and proxy base url for constructing api calls
* feat(_types.py): add litellm well known config as public route
allows ui to query it
* fix(/_types.py): add .well-known config to as public route
* fix(page.tsx): create pattern for loading in ui config before making network requests
ensures requests are formatted correctly
* fix(page.tsx): call credential endpoint once ui config is loaded
* fix(page.tsx): route correctly to litellm dashboard from new user login
* fix(page.tsx): remove hardcoded `/litellm` for /sso/key/generate request
* fix(proxy_server.py): re-add moderations endpoint
* fix(proxy_server.py): mount __next__ at / and /litellm
allows it to work when proxy is mounted on root
* docs(contributing.md): remove /ui on ui doc - it will now run on root
* docs(custom_root_ui.md): add docs on custom root path
* Add support for DataRobot as a router in LiteLLM
* Updates to logic
* Changes to make things work better
* Capitalize bearer
* Revert change
* Undo and simplify things
* Add basic testing
* Add some extra handling
* More tests
* Lowercase
* Fix
* Comment
* Add local test_completion for datarobot
* Sync repo to main
* Update get_complete_url to accept deployments
* Migrate to OpenAILike
* Mock datarobot in test
* Migrate everything to OpenAI LIke
* Apply fixes and changes from review
* Update docs
* Update env vars
* Move tests
* fix(base_routing_strategy.py): compress increments to redis - reduces write ops
* fix(base_routing_strategy.py): make get and reset in memory keys atomic
* fix(base_routing_strategy.py): don't reset keys - causes discrepency on subsequent requests to instance
* fix(parallel_request_limiter.py): retrieve values of previous slots from cache
more accurate rate limiting with sliding window
* fix: fix test
* fix: fix linting error
* feat: Add Gemini TTS audio parameter support
- Add is_model_gemini_audio_model() method to detect TTS models
- Include 'audio' parameter in supported params for TTS models
- Map OpenAI audio parameter to Gemini speechConfig format
- Add _extract_audio_response_from_parts() method to transform audio
output to openai format
* updated unit-test to use pcm16
* - created typedict for speechconfig
- simplified gemini tts model detection
- moved gemini_tts test to test_litellm
* simplified is_model_gemini_audio_model more
* feat(parallel_request_limiter_v2.py): add sliding window logic
allows rate limiting to work across minutes
* fix(parallel_request_limiter_v2.py): decrement usage on rate limit error
* fix(base_routing_strategy.py): fix merge from redis - preserve values in in-memory cache during gap b/w push to redis and read from redis
* fix(base_routing_strategy.py): catch the delta change during redis sync
ensures values are kept in sync
* fix(parallel_request_limiter_v2.py): update tpm tracking to use slot key logic
* fix: fix linting error
* test: update testing
* test: update tests
* test: skip on rate limit or internal server errors
* test: use pytest fixture instead
* test: bump mistral model
* fix(utils.py): support non default params for audio transcription
allows passing provider specific params straight through on transcription calls
* fix(gpt_transformation.py): fix o_series model routing
call _transform_request on async event
* refactor: refactor tests
* test(test_azure_chat_o_series_transformation.py): add unit test for azure o series error
* test: update test
* test: update json
* fix: fix mutiple keyword error
* fix(helicone.py): add helicone api base support
Fixes https://github.com/BerriAI/litellm/issues/10825
* test: add unit test for cache hit response on embedding calls
* fix(caching_handler.py): fix handling cache hit on embedding when input is string
Fixes LIT-197
* docs(helicone_integration.md): document new helicone api base param
* fix: init commit for object permissions
* fix: init commit for object permissions
* fix: add vector_store_id to permissions
* fix vector store selector
* feat:add vector store permission mgmt
* feat: ui add allowed vector stores dropdown
* feat: add new vector store object permissions
* testing: key mgmt
* fix: stor vector store permissions on team
* ui select vector store for teams
* ui add vector store settings for orgs
* feat: allow setting org vector store permissions
* test: adding team permissions for vector stores
* fix(anthropic/chat/handler.py): Fixes https://github.com/BerriAI/litellm/issues/10328
Adopts changes from https://github.com/BerriAI/litellm/pull/10329
* fix(vertex_and_google_ai_studio.py): don't set 'include thoughts' if thinking budget = 0
VertexAI raises errors
* fix(vertex_llm_base.py): new function for deciding the api base, handles 'global' api base
Fixes https://github.com/BerriAI/litellm/issues/11190
* fix(vertex_ai/partner_models): fix instrumentation for custom api base check
* refactor(vertex_ai/partner): refactor function to keep below 50 LOC
* fix(vertex_ai/gemini): remove parallel tool calls error for >1 tool - just ignore (prevent call from failing)
* fix: fix linting error
