scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-16 04:29:31 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 138
litellm/tests/unit/litellm_core_utils
History
ryan-crabbe-berri 38f3f8a1a9
fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344) 
* fix(router): carry a 429 status on cooldown and no-deployment errors

RouterRateLimitError and RouterRateLimitErrorBasic had no status_code, so the
proxy answered an all-deployments-cooled-down request as a generic 500 class
error and logged the bug report notice for every cooldown. Both now carry
status_code 429, which the proxy maps to a rate_limit_error and which the
retry policy already treats as retryable

* fix(proxy): start Sentry at config load and never attach request bodies

Sentry only came up inside set_callbacks on the first LLM call, so boot and
background errors were never reported, and general_settings.alerting: [sentry]
always raised Missing SENTRY_DSN because the alerting path read a dead
litellm.utils pin instead of the live SDK. The proxy now initializes Sentry
once while loading the config whenever sentry is a configured callback or
alerting target, the alerting and failure handlers read the live instance, and
max_request_body_size is pinned to never so prompts and completions stay out
of Sentry events

* fix(tracing): log Lens failures at ERROR and export row counters to Prometheus

A tracing store that fails to start, or a Lens service that rejects a batch,
was only logged at WARNING, so operators running at the default ERROR level
never saw tracing silently go dark. Both now log at ERROR and name the lens
store. The exporter's rows_written and rows_dropped tallies were never
exported, so the two counts are now Prometheus counters
litellm_lens_export_rows_written_total and
litellm_lens_export_rows_dropped_total, registered lazily on the default
registry like the other proxy-level counters and charted on the all-metrics
Grafana dashboard

* test(proxy): drop the litellm import the alerting tests no longer use

* fix(proxy): start Sentry once a DSN is set, from any config source

Boot-time init used to lock Sentry in even with no SENTRY_DSN, so a DSN loaded later from DB environment_variables was never picked up. initialize_sentry now skips when no DSN is set and re-inits when the DSN changes. alerting_handler calls it too, so alerting: [sentry] set through the DB works. Drops the litellm_settings.callbacks check since that config fails to load with sentry in it

* fix(sentry): keep stack frame locals out of events unless PII is opted in

Frame locals carried prompts (messages, request bodies, provider options) into every error event. include_local_variables now follows SENTRY_SEND_DEFAULT_PII

* test(router): drop the test that only pinned the 429 class attribute

* fix(proxy): start Sentry at boot when it is only an input callback

* fix(proxy): keep Sentry init and alerting within the strict ruff and basedpyright ceilings

---------

Co-authored-by: yuneng-jiang <yuneng@berri.ai>
2026-10-16 00:53:52 +00:00
..
audio_utils test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
llm_cost_calc fix(pricing): drop duplicate and bare gemini-4-argon rows (#46246) 2026-10-13 17:44:26 +00:00
llm_response_utils fix(ollama): prevent reused response ids from dropping spend logs (#39227) 2026-10-15 17:17:51 -07:00
prompt_templates fix(anthropic): count a replayed server tool call's cache mark the way modify_params sends it (#46591) 2026-10-15 14:46:43 -07:00
specialty_caches test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
__init__.py test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
conftest.py test(ci): repair stale tests and flaky CI infrastructure (#43983) 2026-10-01 17:46:43 +00:00
event_loop_lag.py test: repair stale and polluting tests red on scheduled main CI (#44229) 2026-10-02 21:32:17 +00:00
fake_secret_vault.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
messages_with_counts.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_agentic_followup_kwargs.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_anthropic_dedup_factory.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_api_route_to_call_types.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_app_crypto.py test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
test_audio_utils.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_aws_partition.py perf(ci): isolate and shard integration suites (#44146) 2026-10-10 11:15:19 -07:00
test_bedrock_converse_dedup_factory.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_bug_report.py fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344) 2026-10-16 00:53:52 +00:00
test_chat_completion_agentic_loop.py fix(websearch_interception): drop stream_options when forcing stream=False and replay it on the final stream (#45433) 2026-10-09 22:16:02 -07:00
test_classifier_logging.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_cli_token_utils.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_cloud_storage_security.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_codestral_provider_routing.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_core_helpers.py feat(gemini)!: auto-continue gemini-4-argon long decodes with continuationToken (#46240) 2026-10-15 22:54:23 +00:00
test_coroutine_checker.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_cost_discount.py feat(cost): support provider/model-pattern keys in cost_discount_config (#43335) 2026-10-12 14:59:18 -07:00
test_custom_logger_registry.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_dd_tracing.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_decisions_pricing.py feat(decisions): bill decisions and systemone from _decisions cost map fields (#45904) 2026-10-12 14:31:12 -07:00
test_decode_special_tokens.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_default_encoding.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_dot_notation_indexing.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_duration_parser.py feat(proxy): report when each per-model budget window resets (#45797) 2026-10-15 10:56:45 -07:00
test_env_utils.py fix(config): treat blank env values as unset so the proxy neither crash-loops nor silently drops a guardrail (#46341) 2026-10-14 12:49:31 -07:00
test_error_normalization.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_exception_mapping_utils.py feat(providers): add Inference APIs as OpenAI-compatible provider (#41669) 2026-10-15 08:17:36 -07:00
test_extract_base64_image.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_fallback_generalizations.py fix(responses): translate OpenAI tool_search to Anthropic tool search for Claude models (#45882) 2026-10-14 20:42:22 -07:00
test_fallback_utils.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_get_litellm_params.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_get_llm_provider_endpoint_match.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_get_llm_provider_logic.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_get_model_cost_map.py fix: keep model capability flags out of provider requests (#46647) 2026-10-15 14:30:10 +00:00
test_get_supported_openai_params.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_health_check_helpers.py feat(decisions): bill decisions and systemone from _decisions cost map fields (#45904) 2026-10-12 14:31:12 -07:00
test_hidden_params.py refactor: expose public hidden_params accessors (#44668) 2026-10-07 00:01:33 -07:00
test_image_handling.py feat(decisions)!: send images to Cloudflare Clef through its images field (#45734) 2026-10-14 12:14:48 +00:00
test_initialize_dynamic_callback_params.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_internal_call_metadata.py fix(responses): bill a background response once, on the first read or poll that sees it finished (#45703) 2026-10-15 11:00:14 -07:00
test_json_fragment_accumulator.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_json_schema_validation.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_litellm_logging.py fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344) 2026-10-16 00:53:52 +00:00
test_llm_judge.py fix(guardrails): resolve decision models in caller team scope (#46085) 2026-10-15 11:43:36 -07:00
test_llm_request_utils.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_logging_callback_manager.py test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
test_logging_utils.py fix(logging): bound data URI regex so base64 truncation stays linear (#45132) 2026-10-07 12:32:04 -07:00
test_logging_worker.py fix(router): preserve native baseline identity and accounting (#44960) 2026-10-07 11:00:31 -07:00
test_max_streaming_duration.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_model_param_helper.py test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
test_model_response_utils.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_moderation_standard_logging.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_oauth_token_exchange.py feat(microsoft_365_copilot): add Microsoft 365 Copilot chat provider with OAuth token exchange (#45158) 2026-10-09 15:19:07 -07:00
test_optional_imports.py refactor(sdk): separate core AWS and tokenizer dependencies (#44447) 2026-10-08 19:07:57 -07:00
test_path_utils.py refactor(core): move path_utils from proxy common_utils to litellm_core_utils (#45775) 2026-10-10 19:07:22 +00:00
test_private_json.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_provider_affinity.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_provider_specific_headers.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_ptu_pricing.py feat(proxy): split Azure PTU deployments across teams with per-team PTU ceilings and PTU-hour reporting (#43043) 2026-10-15 18:44:30 +00:00
test_realtime_errors.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_realtime_streaming.py feat(xai): bill grok voice realtime sessions (#46600) 2026-10-15 05:27:02 +00:00
test_redact_messages.py fix(logging): redact nested callback input and provider response fields (#45923) 2026-10-13 12:25:56 -07:00
test_request_timeout_resolver.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_retry_after_headers.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_safe_divide_seconds.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_safe_json_dumps.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_sensitive_data_masker.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_sentry_scrubbing.py fix(proxy): cooldown errors carry 429, Sentry starts at boot without request bodies, tracing failures log at ERROR, exporter counters exported (#46344) 2026-10-16 00:53:52 +00:00
test_served_output_texts.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_stream_usage_logging.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_streaming_chunk_builder_cursor.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_streaming_chunk_builder_server_tool_use.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_streaming_chunk_builder_utils.py fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
test_streaming_handler.py fix(ollama): prevent reused response ids from dropping spend logs (#39227) 2026-10-15 17:17:51 -07:00
test_streaming_overhead.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_text_completion_conversion.py test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
test_thread_pool_executor.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_token_counter.py feat(gemini)!: auto-continue gemini-4-argon long decodes with continuationToken (#46240) 2026-10-15 22:54:23 +00:00
test_token_counter_tool.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_token_counter_tool_data.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_tokenizer.py refactor(sdk): separate core AWS and tokenizer dependencies (#44447) 2026-10-08 19:07:57 -07:00
test_tool_search_spend_logging.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_url_utils.py test(unit): wait on the work itself instead of fixed sleeps (#46261) 2026-10-13 12:41:06 -07:00
test_xai_oauth_routing.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00