scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-14 09:03:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 214
litellm/tests/unit/integrations
History
Exact
quietjules f5ef14308b
fix(bedrock): answer 400 for service tiers Bedrock cannot serve, forward reserved (#50516) 
* fix(bedrock): answer 400 for service tiers Bedrock cannot serve, forward reserved

Converse used to drop any service_tier outside default, flex, and priority
with no error, rewrite auto to default, and drop reserved. Now reserved and
standard_only map to Bedrock tiers, auto sends no tier, and any other value
answers 400 unless drop_params is on. Nova batch rows honor the row's
drop_params.

* fix(bedrock): keep a Responses batch row's drop_params and keep it out of passthrough rows

* test(bedrock): annotate service tier test locals as Final and drop the row helper's kwargs

* fix(bedrock): refuse a service tier on Nova batch rows unless drop_params, and honor the deployment's drop_params

* fix(bedrock): map standard and any-case service tiers, and drop tiers from Claude batch rows under the deployment's drop_params

* fix(bedrock): forward the tiers OpenAI-format batch rows take, accept auto on Claude rows, and label tiers in lowercase

* fix(bedrock): mark the service tier block field read-only

* fix(bedrock): read drop_params given as a string on batch rows, and cover the Nova row mapper's drop_params

* test(prometheus): sort the service tier label test imports

* fix(bedrock): send the Invoke route's service tier as a header and name the accepted tiers in the 400

* fix(bedrock): report and bill the tier Bedrock served on Invoke calls and streams, and send the tier header only to Nova

* fix(bedrock): return copies when stamping the served tier, read its header without re-encoding the others, and pin it per stream chunk

The Nova Invoke mapper and the event-stream decoder now return model_copy results instead of writing service_tier in place. One shared reader finds the X-Amzn-Bedrock-Service-Tier response header case-insensitively, so a non-ASCII request id no longer breaks the stream decoder, and the passthrough uses the same reader. Tests now vary the served tier in both directions, check every chunk, and cover a stream with no tier header or an empty one

* test(bedrock): check the served tier on a stream whose headers come as a plain dict with an upper-case name

The non-ASCII request id stream test now also carries X-AMZN-BEDROCK-SERVICE-TIER in a plain dict and asserts every chunk reports that tier, so a case-sensitive header reader or a lookup by the canonical name fails it

* refactor(bedrock): drop the passthrough's own service tier header reader that main re-added

The passthrough already reads the served tier through the shared case-insensitive reader in common_utils, so the httpx.Headers copy main brought back with the embeddings change has no callers and references a constant this branch removed

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-11-14 01:33:57 +00:00
..
arize refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
azure_storage fix(spend)!: save streamed /v1/responses spend rows under the encrypted resp_ id the client received (#49493) 2026-11-02 14:13:17 -08:00
bitbucket refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
clickhouse fix(router): count a Responses WebSocket turn's tokens before the next turn is admitted (#50730) 2026-11-10 04:13:51 +00:00
cloudzero test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
code_interpreter_interception fix(tool_choice): accept the Responses API's flat allowed_tools layout on chat completions (#50467) 2026-11-10 13:04:06 -08:00
compression_interception fix(tool_choice): accept the Responses API's flat allowed_tools layout on chat completions (#50467) 2026-11-10 13:04:06 -08:00
custom_logger_v2 feat(custom-logger-v2): add types, dispatcher and route callback call sites (step 0) (#51214) 2026-11-13 22:08:05 +00:00
datadog fix(anthropic)!: do not surface server_tool_use as client tool_calls (#48385) 2026-11-06 03:54:34 -08:00
dotprompt refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
email_alerting feat(alerts): add management controls and opt-in daily summaries (#51418) 2026-11-12 10:06:07 -08:00
focus refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
gcs_bucket fix(spend)!: save streamed /v1/responses spend rows under the encrypted resp_ id the client received (#49493) 2026-11-02 14:13:17 -08:00
gcs_pubsub feat(proxy): add customer hierarchy usage analytics (#51495) 2026-11-13 00:32:13 +00:00
generic_api fix(proxy): show committed spend in the remaining budget gauges (#49199) 2026-11-09 20:05:37 +00:00
gitlab refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
helicone test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
langfuse fix(cost): preserve Bedrock cross-region pricing for Messages (#51479) 2026-11-12 15:54:56 -08:00
levo test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
litellm_agent test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
mavvrik_focus refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
newrelic fix(anthropic)!: do not surface server_tool_use as client tool_calls (#48385) 2026-11-06 03:54:34 -08:00
open_telemetry fix(proxy): trace model list requests with the caller's traceparent (#49141) 2026-10-31 01:22:45 -07:00
opik refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
otel refactor(redis): delete the Python RedisBatch now that every request op is native (#51383) 2026-11-13 03:22:28 +00:00
pointfive chore(lint): remove the LIT002 mutable-construction rule (#43971) 2026-10-01 12:24:02 -07:00
SlackAlerting feat(alerts): add management controls and opt-in daily summaries (#51418) 2026-11-12 10:06:07 -08:00
ternary feat(focus): add ternary destination for FOCUS export (#46106) 2026-10-12 22:52:40 -07:00
vantage chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
vector_store_integrations feat(ui): show customer organization usage and budget utilization (#51483) 2026-11-12 17:25:41 -08:00
websearch_interception fix(tool_choice): accept the Responses API's flat allowed_tools layout on chat completions (#50467) 2026-11-10 13:04:06 -08:00
zerobus test(zerobus): move tests into active CI selection (#43235) 2026-09-25 23:35:15 +00:00
__init__.py test: add __init__.py to migrated tests/unit packages 2026-09-20 11:53:49 +00:00
conftest.py fix(spend)!: save streamed /v1/responses spend rows under the encrypted resp_ id the client received (#49493) 2026-11-02 14:13:17 -08:00
rubrik_test_helpers.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_adaptive_concurrency.py fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
test_agentops.py fix(caching): key cached async clients by a per-loop token instead of id(loop) (#48877) 2026-10-29 13:14:48 -07:00
test_anthropic_cache_control_hook.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
test_argilla.py chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
test_athina.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_azure_sentinel.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_braintrust_logging.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_braintrust_span_name.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_custom_callback_input.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_custom_guardrail.py fix(responses): run the guardrails a Responses WebSocket frame selects (#48725) 2026-10-30 14:12:51 -07:00
test_custom_guardrail_post_call_once.py feat(custom-logger-v2): add types, dispatcher and route callback call sites (step 0) (#51214) 2026-11-13 22:08:05 +00:00
test_custom_guardrail_recursion.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_custom_logger.py feat(custom-logger-v2): add types, dispatcher and route callback call sites (step 0) (#51214) 2026-11-13 22:08:05 +00:00
test_custom_prompt_management.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_customer_budget_webhook.py fix(proxy): alert on customer org, customer team, and customer member budgets (#51579) 2026-11-13 09:16:33 -08:00
test_deepeval.py fix(spend)!: save streamed /v1/responses spend rows under the encrypted resp_ id the client received (#49493) 2026-11-02 14:13:17 -08:00
test_dynamodb.py fix(spend)!: save streamed /v1/responses spend rows under the encrypted resp_ id the client received (#49493) 2026-11-02 14:13:17 -08:00
test_email_alerting.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_galileo.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_guardrail_logging_sync.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_helicone.py refactor(rust_bridge): route every Rust-to-Python call through host modules (#50822) 2026-11-11 01:57:54 +00:00
test_humanloop.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_lago.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_langfuse.py fix(logging): mark mirrored silent_model calls with is_silent_experiment in spend logs (#48888) 2026-10-29 05:12:50 -07:00
test_langfuse_otel.py fix(langfuse): reserve span capacity for failure usage attributes (#51538) 2026-11-12 21:14:14 -08:00
test_langsmith.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_langsmith_init.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_langtrace.py fix(otel): serialize pydantic Message objects in langfuse_otel, weave_otel and langtrace (#46845) 2026-11-10 14:50:39 -08:00
test_lunary.py fix(anthropic)!: do not surface server_tool_use as client tool_calls (#48385) 2026-11-06 03:54:34 -08:00
test_mlflow.py test(unit): wait on the work itself instead of fixed sleeps (#46261) 2026-10-13 12:41:06 -07:00
test_openmeter.py fix(spend)!: save streamed /v1/responses spend rows under the encrypted resp_ id the client received (#49493) 2026-11-02 14:13:17 -08:00
test_opentelemetry.py fix(otel): distinguish missing usage from prompt estimates on chat failures (#51443) 2026-11-12 13:48:07 -08:00
test_opentelemetry_dynamic_imports.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_opentelemetry_request_spans.py test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
test_opik_utils.py refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
test_otel_guardrail_violation_spans.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_otel_team_attributes_matrix.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_performance_alert_notifications.py feat(alerts): add management controls and opt-in daily summaries (#51418) 2026-11-12 10:06:07 -08:00
test_posthog.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_prometheus_api_promql_escape.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_budget_metric_guard.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_budget_metrics_db_lookups.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_budget_metrics_timeout.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_cache_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_caller_identity.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_carried_budget_state.py refactor(redis): delete the Python RedisBatch now that every request op is native (#51383) 2026-11-13 03:22:28 +00:00
test_prometheus_client_ip_user_agent.py feat(prometheus): expose per-project per-model rate limit allowed and used gauges (#43561) 2026-10-08 12:03:29 -05:00
test_prometheus_custom_metadata_label_counts.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_deployment_state_proxy_rejects.py fix(proxy)!: answer 400 for malformed tools on count_tokens instead of crashing (#46583) 2026-10-16 14:57:53 -07:00
test_prometheus_end_user_cardinality.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_input_sequence_length_label.py feat(ui): add auto-router usage and savings table (#45152) 2026-10-07 15:04:07 -07:00
test_prometheus_invalid_key_filtering.py fix(azure)!: answer 401 when Entra rejects the deployment's credential (#49863) 2026-11-04 20:44:55 +00:00
test_prometheus_labels.py feat(dynamic_rate_limiter_v3): prometheus share, borrowed and throttle metrics per team and model (#51417) 2026-11-12 15:28:51 -08:00
test_prometheus_mcp_tool_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_media_generation_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_metric_name_consistency.py chore: remove cookbook and preserve live assets (#50944) 2026-11-11 15:38:27 -08:00
test_prometheus_metrics_endpoint.py test(unit): wait on the work itself instead of fixed sleeps (#46261) 2026-10-13 12:41:06 -07:00
test_prometheus_missing_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_none_metadata.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_overhead_with_guardrails.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_queue_guardrail_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_rate_limit_labels.py fix(prometheus): skip custom metadata labels a metric already carries (#46267) 2026-10-13 18:20:52 -07:00
test_prometheus_remaining_tokens_router_fallback.py refactor(router): move router callers onto the entry points and invert the proxy imports (#48847) 2026-10-29 17:26:33 +00:00
test_prometheus_requested_model_cardinality.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_series_cardinality.py feat(prometheus): cap series per metric for every labeled metric (#44420) 2026-10-06 01:28:44 +00:00
test_prometheus_service_tier_label.py fix(bedrock): answer 400 for service tiers Bedrock cannot serve, forward reserved (#50516) 2026-11-14 01:33:57 +00:00
test_prometheus_services.py test: move routing, caching and callback tests out of local_testing (#45761) 2026-10-10 09:33:09 -07:00
test_prometheus_spend_capture_rate.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_spend_logs_metadata.py fix(logging): mark mirrored silent_model calls with is_silent_experiment in spend logs (#48888) 2026-10-29 05:12:50 -07:00
test_prometheus_stream_label.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_token_detail_metrics.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prometheus_user_team_metrics.py fix(proxy): show committed spend in the remaining budget gauges (#49199) 2026-11-09 20:05:37 +00:00
test_prometheus_zero_cost_metric.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_prompt_layer.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_prompt_manager_ssti.py test: move tests/test_litellm integrations and secret_managers into tests/unit (#43194) 2026-09-25 12:57:07 -07:00
test_responses_background_cost.py fix(responses): send the billed background response cost to the creating key's and team's loggers (#47506) 2026-10-20 23:22:21 +00:00
test_rubrik.py fix(guardrails/rubrik): moderate text-completion prompts and forward rac_agent_id (#49912) 2026-11-10 16:30:41 -08:00
test_s3.py feat(s3_v2): add s3_partition_granularity option for hourly S3 folders (#43748) 2026-10-01 11:00:21 -07:00
test_s3_v2.py fix(s3): bound object key filenames (#40825) 2026-10-15 12:59:35 -07:00
test_shadow_eval_logger.py fix(anthropic)!: do not surface server_tool_use as client tool_calls (#48385) 2026-11-06 03:54:34 -08:00
test_sqs.py test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
test_wandb.py refactor(typing): clear about 1,100 basedpyright Any errors across 303 files (#49986) 2026-11-06 13:31:15 -08:00
test_weave_otel.py fix(otel): serialize pydantic Message objects in langfuse_otel, weave_otel and langtrace (#46845) 2026-11-10 14:50:39 -08:00