mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
d76ec3a47e
* fix(proxy): answer 400 for malformed tools on count_tokens instead of crashing A non-list tools value crashed the vector store lookup during auth and the built-in tool cost lookup in the failure logging path, so the proxy answered an unhandled 500 and closed the keep-alive connection. count_tokens now answers a 400 invalid_request_error for a malformed body, drops malformed tools when drop_params is on, and both lookups skip values of the wrong shape. * fix(proxy): skip non-list tools in the tool allowlist check Also validate the count_tokens body without recursion, which the recursion detector rejects. * test(integration): restore the int tools case on count_tokens The proxy now answers a non-list tools value with a 400, so the case is deterministic again. * fix(proxy): reject malformed tools at the LLM route entry and keep vector store checks on malformed ids * fix(proxy): apply drop_params like core when dropping malformed tools, and reject malformed tool vector store ids Check tools after request setup so the Claude Code default and the logged call type apply, read the request, global, and model group drop_params the way core does, drop tool_choice and friends with tools, cover Gemini generateContent, and answer 400 for a malformed tools[].vector_store_ids with the strict flag off * fix(proxy): keep the request's call type when logging a gate rejection after request setup The body-shape fallback relabeled a /v1/messages rejection as acompletion and a list-input /v1/responses rejection as aembedding, so it now only applies when the logging object carries no known call type * fix(proxy): share the malformed tools check across tool routes and honor drop_params per entry * fix(proxy): accept null vector_store_ids, read every named store id in auth, check the responses websocket * refactor(proxy): leave the responses websocket out of the request-level tool check * fix(proxy): let a request's additional_drop_params replace the deployment list and keep compact's call type on rejections * fix(proxy): read router default drop lists and deployment ids in the malformed tools check * fix(proxy): leave malformed tools to the router when it can pick another deployment Fallbacks, auto-routers, model strings, comma batches and team model names let the router pick a deployment after the proxy checks the request, so the proxy now forwards a malformed tools field there as main does, and only rejects it when the deployment it checks is the one that serves. Token counting never reroutes, so count_tokens and input_tokens keep reading the group they count with. additional_drop_params names are read the way core reads them, so a list with a non-string entry still names its strings. * test(integration): cover malformed tools on reroutable models, router defaults and deployment ids * fix(proxy): resolve malformed tools at the deployment the router picks * fix(proxy): keep provider-repaired tool schemas, honor fallback_on_unsupported_params, skip cooldown on malformed tools Empty schemas and JSON-string schemas pass the tools check the way litellm's provider paths already read them. The rejection is a BadRequestError subclass carrying the deployment's provider, so fallback_on_unsupported_params: false keeps the 400, and the router never cools a deployment down for a request litellm rejected before calling it. Adds the audit's pre_call guardrail and cross-provider bridge cells. * test(integration): expect a 400 for an empty-string tools field on /v1/messages Real Anthropic rejects tools "" with a 400 on main too, so the case joins the malformed tools test * fix(router): keep web search routing for tools passed as a tuple * fix(router): keep a deployment's own tools when drop_params drops a malformed request tools * test(router): list _merged_tools as covered through the deployment tools merge tests * fix(router): drop the request's tool settings with its malformed tools when deployment tools replace them * fix(router): skip a tool whose type is not a string in the web search filter instead of raising TypeError * fix(prometheus): keep deployment health metrics clean on a malformed tools rejection * test: type the new malformed tools tests * test: mark new test locals Final and type kwargs dicts |
||
|---|---|---|
| .. | ||
| _types | ||
| agentops | ||
| arize | ||
| azure_sentinel | ||
| azure_storage | ||
| bitbucket | ||
| clickhouse | ||
| cloudzero | ||
| code_interpreter_interception | ||
| compression_interception | ||
| datadog | ||
| deepeval | ||
| dotprompt | ||
| email_templates | ||
| focus | ||
| gcs_bucket | ||
| gcs_pubsub | ||
| generic_api | ||
| generic_prompt_management | ||
| gitlab | ||
| langfuse | ||
| levo | ||
| litellm_agent | ||
| mavvrik_focus | ||
| newrelic | ||
| opentelemetry_utils | ||
| opik | ||
| otel | ||
| pointfive | ||
| prometheus_helpers | ||
| SlackAlerting | ||
| ternary | ||
| vantage | ||
| vector_store_integrations | ||
| weave | ||
| websearch_interception | ||
| zerobus | ||
| __init__.py | ||
| adaptive_concurrency.py | ||
| additional_logging_utils.py | ||
| anthropic_cache_control_hook.py | ||
| argilla.py | ||
| athina.py | ||
| batch_line_item_objects.py | ||
| batch_utils.py | ||
| braintrust_logging.py | ||
| braintrust_mock_client.py | ||
| callback_configs.json | ||
| custom_batch_logger.py | ||
| custom_guardrail.py | ||
| custom_logger.py | ||
| custom_prompt_management.py | ||
| custom_secret_manager.py | ||
| custom_sso_handler.py | ||
| dynamodb.py | ||
| email_alerting.py | ||
| galileo.py | ||
| greenscale.py | ||
| helicone.py | ||
| helicone_mock_client.py | ||
| humanloop.py | ||
| lago.py | ||
| langsmith.py | ||
| langsmith_mock_client.py | ||
| langtrace.py | ||
| literal_ai.py | ||
| logfire_logger.py | ||
| lunary.py | ||
| mlflow.py | ||
| mock_client_factory.py | ||
| openmeter.py | ||
| opentelemetry.py | ||
| posthog.py | ||
| posthog_mock_client.py | ||
| prometheus.py | ||
| prometheus_metrics_endpoint.py | ||
| prometheus_services.py | ||
| prompt_layer.py | ||
| prompt_management_base.py | ||
| Readme.md | ||
| rubrik.py | ||
| s3.py | ||
| s3_v2.py | ||
| shadow_eval_logger.py | ||
| sqs.py | ||
| supabase.py | ||
| test_httpx.py | ||
| traceloop.py | ||
| weights_biases.py | ||
Integrations
This folder contains logging integrations for litellm
eg. logging to Datadog, Langfuse, Prometheus, s3, GCS Bucket, etc.