* feat(parallel_request_limiter_v2.py): add sliding window logic
allows rate limiting to work across minutes
* fix(parallel_request_limiter_v2.py): decrement usage on rate limit error
* fix(base_routing_strategy.py): fix merge from redis - preserve values in in-memory cache during gap b/w push to redis and read from redis
* fix(base_routing_strategy.py): catch the delta change during redis sync
ensures values are kept in sync
* fix(parallel_request_limiter_v2.py): update tpm tracking to use slot key logic
* fix: fix linting error
* test: update testing
* test: update tests
* test: skip on rate limit or internal server errors
* test: use pytest fixture instead
* test: bump mistral model
* feat: Make gemini accept the openai parameter parallel_tool_calls
When mapping, allow the parameter: True because that is the
intrinsic behavior of Gemini. Allow False, but reject if there
are multiple tools because there's no actual equivalent in Gemini.
fixes#9686
ref: issues/9686
* chore: cleanup and move test_vertex.py down to tests/litellm
as suggested in https://github.com/BerriAI/litellm/pull/11125#discussion_r2105905871
* fix(main.py): use processed non-default-params as standard input params for langfuse
Fixes https://github.com/BerriAI/litellm/issues/11072
Fixes https://github.com/BerriAI/litellm/issues/11096
* fix(main.py): rename variable to be more accurate
* test(test_langfuse_e2e_test.py): add router unit test for langfuse e2e testing
Prevent https://github.com/BerriAI/litellm/issues/11072 from happening again
* build: update lock
* fix(utils.py): refactor optional params function
make it easier to get the standardized non default params
* fix(utils.py): improve process non default params function
* fix(main.py): include provider specific params in processed non default params used in logging
ensures user can see any provider specific params on langfuse
ensures user can see any provider specific params on langfus e
* fix: add flag for disabling use_aiohttp_transport
* feat: add _create_async_transport
* feat: fixes for transport
* add httpx-aiohttp
* feat: fixes for transport
* refactor: fixes for transport
* build: fix deps
* fixes: test fixes
* fix: ensure aiohttp does not auto set content type
* test: test fixes
* feat: add LiteLLMAiohttpTransport
* fix: fixes for responses API handling
* test: fixes for responses API handling
* test: fixes for responses API handling
* feat: fixes for transport
* fix: base embedding handler
* test: test_async_http_handler_force_ipv4
* test: fix failing deepeval test
* fix: add YARL for bedrock urls
* fix: issues with transport
* fix: comment out linting issues
* test fix
* test: XAI is unstable
* test: fixes for using respx
* test: XAI fixes
* test: XAI fixes
* test: infinity testing fixes
* docs(config_settings.md): document param
* test: test_openai_image_edit_litellm_sdk
* test: remove deprecated test
* bump respx==0.22.0
* test: test_xai_message_name_filtering
* test: fix anthropic test after bumping httpx
* use n 4 for mapped tests (#11109)
* fix: use 1 session per event loop
* test: test_client_session_helper
* fix: linting error
* fix: resolving GET requests on httpx 0.28.1
* test fixes proxy unit tests
* fix: add ssl verify settings
* fix: proxy unit tests
* fix: refactor
* tests: basic unit tests for aiohttp transports
* tests: fixes xai
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
* fix(openai/gpt_transformation.py): handle missing filename for openai file data call
* fix(openai/gpt_transformation.py): clean handling for sync + async pdf url transformation flows
Fixes https://github.com/BerriAI/litellm/issues/10820
* build(model_prices_and_context_window.json): add 'supports_pdf_input' for all openai models which have 'vision' support
Follows openai guidelines
* feat(bedrock/chat): support cache pointing tool calls on Bedrock
Closes https://github.com/BerriAI/litellm/pull/10613
* fix: fix linting error
* fix(duration_parser.py): support `mo` unit
* test(test_key_management_endpoints.py): add test confirming generate_key_helper_fn uses predictable budgets
Closes https://github.com/BerriAI/litellm/issues/10800
* fix(anthropic/chat/transformation.py): add tool use cost tracking
* fix(anthropic/): refactor how hosted tool usage tracking is done
keep it separate from prompt / completion token details
* fix(anthropic/): add web search tool cost tracking
accurate cost tracking
* feat(anthropic/chat/transformation.py): map openai 'web_search_options' param to anthropic hosted tool
Allows calling anthropic web search in same format as openai
* feat(anthropic/chat/transformation.py): support unified anthropic 'web_search_options' param
Allows calling anthropic's web search tool in the openai format
* feat(anthropic/chat/transformation.py): map openai 'search_context_size' to anthropic 'max_uses' param
Translate search effort across both providers
* fix: mark web_search_options param as supported by openai + azure
* fix: fix linting error
* fix: fix linting errors
* fix: fix linting error
* fix: check if usage hasattr
* fix: pass web search options param
* feat(cohere/embed): v2 embed api support
adds output_dimensions param support
* fix(cohere/embed): migrate to v2 embedding
Adds output dimension support
* fix: maintain /v1/embedding compatibility for bedrock cohere
Bedrock cohere is still using /v1 endpoints
* fix: fix linting error
* fix: fix passing extra headers
* test: update tests
* add test_function_calling_with_tool_response to base llm tests
* run test suite for nova
* update test_function_calling_with_tool_response
* allowed ToolJsonSchemaBlock keys
* fix ToolJsonSchemaBlock
* add back pytest fixture
* test: test_prompt_caching
* fix(main.py): use base model instead of user model if given
Fixes https://github.com/BerriAI/litellm/issues/10760
* feat(azure/image_generation/__init__.py): make azure image gen check more robust
Fixes https://github.com/BerriAI/litellm/issues/10760
* fix(user_api_key_auth.py): support bearer token auth for `x-litellm-api-key` header
Fixes earlier regression on vertex ai passthrough auth
* fix(user_api_key_auth.py): refactor get api key into separate function
enables easier testing
* fix: cleanup
* fix: fix linting error
* fix: cleanup
* test: update tests
* Support pdf url's to openai (#10640)
* fix(gpt_transformation.py): support pdf url input to openai
pass as base64 as openai doesn't support image url's
* fix(openai.py): support async message transformation
allows async get request to convert url to base64
* fix(gpt_transformation.py): fix linting errrors and use common components across sync + async flows
* fix: fix linting errors
* fix(openai.py): pop correct var
* Fix sagemaker chat calls - content length error (#10607)
* fix(sagemaker_chat/): support passing dynamic aws params
previously being ignored
* refactor(sagemaker/chat): more refactoring
* fix(sagemaker_chat/): make sure streaming is correctly handled post-refactor
* refactor: more refactoring to support using signed json str
* fix(sagemaker/chat): working sync streaming post refactor
* fix(sagemaker/chat): support async streaming post refactor
* fix(llm_http_handler.py): await async function
* fix: remove print statements
* test: update test
* test: update test
* fix(llm_http_handler.py): retain passing in data as json str
* test: update test
* fix(base_model_iterator.py): fix linting error
* test: test auth
* fix: fix linting error
* test: update test
* test: update translation test
* fix(gpt_transformation.py): handle awaitable/non-awaitable object
* fix: handle async flow for message transformation on openai compatible api's
* test: cleanup testing
* test: update test
* test(test_router.py): use model with higher quota
* test: simplify test
* test: update test
* test(base_llm_unit_tests.py): return '<thinking>' tag in response content
* fix(converse_transformation.py): extract `<thinking>` block from nova tool use response
Fixes https://github.com/BerriAI/litellm/issues/9063
* fix(factory.py): handle non-signature reasoning blocks to bedrock
pass as text input - bedrock raises ""User messages cannot contain reasoning content. Please remove the r
easoning content and try again." otherwise
* fix(main.py): Add drop params support for gpt
Fixes https://github.com/BerriAI/litellm/issues/10501
* fix(converse_transformation.py): fix linting error
* fix(utils.py): fix linting error
* test: cleanup test
* test: skip test until we have bedrock prompt caching permission
* build(model_prices_and_context_window.json): add bedrock llama4 models to model cost map
* fix template conversion for Llama 4 models in Bedrock (#10557)
* test: add testing to repro https://github.com/BerriAI/litellm/pull/10557
* test: add unit testing
* test(test_main.py): refactor where test is kept
---------
Co-authored-by: aswny <87371411+aswny@users.noreply.github.com>
* fix(vertex_and_google_ai_studio.py): fix finish reason to be 'tool_calls' when tool call returned
Vertex returns 'Stop', openai format is 'tool calls'
* test(base_llm_unit_tests.py): bump test to assert tool calls in finish reason
* test(test_auth_checks.py): add unit tests for ExperimentalUIJWTToken
* test: add appropriate flag
* fix: fix ruff check
* test: add autouse fixture to test salt key
* fix(user_api_key_auth.py): fix auth flow logic
* test: skip flaky test - anthropic does not reliably return 'redacted_thinking'
* fix(triton/completion/transformation.py): remove bad_words / stop words from triton call
parameter 'bad_words' has invalid type. It should be either 'int', 'bool', or 'string'.
* fix(proxy_track_cost_callback.py): add debug logging for track cost callback error
* feat(fireworks_ai/chat): handle tool calling with fireworks ai correctly
Fixes https://github.com/BerriAI/litellm/issues/7209
* fix(utils.py): handle none type in message
* fix: fix model name in test
* fix(utils.py): fix validate check for openai messages
* fix: fix model returned
* fix(main.py): fix text completion routing
* test: update testing
* test: skip test - cohere having RBAC issues
* build(model_prices_and_context_window.json): add vertex ai gemini-2.5-flash pricing
* build(model_prices_and_context_window.json): add gemini reasoning token pricing
* fix(vertex_and_google_ai_studio_gemini.py): support counting thinking tokens for gemini
allows accurate cost calc
* fix(utils.py): add reasoning token cost calc to generic cost calc
ensures gemini-2.5-flash cost calculation is accurate
* build(model_prices_and_context_window.json): mark gemini-2.5-flash as 'supports_reasoning'
* feat(gemini/): support 'thinking' + 'reasoning_effort' params + new unit tests
allow controlling thinking effort for gemini-2.5-flash models
* test: update unit testing
* feat(vertex_and_google_ai_studio_gemini.py): return reasoning content if given in gemini response
* test: update model name
* fix: fix ruff check
* test(test_spend_management_endpoints.py): update tests to be less sensitive to new keys / updates to usage object
* fix(vertex_and_google_ai_studio_gemini.py): fix translation
* fix(azure/common_utils.py): check for azure tenant id, client id, client secret in env var
Fixes https://github.com/BerriAI/litellm/issues/9598#issuecomment-2801966027
* fix(azure/gpt_transformation.py): fix passing response_format to azure when api year = 2025
Fixes https://github.com/BerriAI/litellm/issues/9703
* test: monkeypatch azure api version in test
* test: update testing
* test: fix test
* test: update test
* docs(config_settings.md): document env vars
