Commit graph

46618 commits

Author SHA1 Message Date
yucheng-berri
f0fadb7f99
test(e2e): add logging e2e coverage (s3_v2, gcs_bucket, team langfuse callback, datadog failure) (#38552)
* test: add logging e2e coverage (s3_v2, gcs_bucket, team langfuse callback, datadog failure)

Five new live e2e scenarios raising Logging & Guardrails registry coverage:
s3_v2 success and failure objects read back from the real S3 bucket,
gcs_bucket success record read back through the GCS JSON API (with
nextPageToken pagination and per-request bearer minting), team-scoped
Langfuse callback delivery with non-team isolation, and DataDog failure
event delivery queried by indexed model_group. datadog_reader gains
query-based variants of the marker search; the langfuse cell is a new
registry row. Bucket readers settle past a full flush interval so a
late duplicate cannot hide from the exactly-one assertions

* test: cover clock-skew day prefix in gcs read-back and retry team callback propagation

* test: key the s3 failure read-back on the provider error, not payload absence

* chore: rerun ci

* chore: rerun ci after config sync

* chore: rerun ci with pr lane env

* chore: rerun ci

* chore: rerun ci

* chore: rerun ci

* chore: rerun ci

* chore: rerun ci

* test: add guardrail e2e coverage (presidio masking, bedrock post and during call, moderation on messages) (#38553)

* test: add guardrail e2e coverage (presidio masking, bedrock post/during, moderation on messages)

* test: require the phone placeholder positively in the presidio masking predicate

* test: count only the 400 verdict body as a bedrock post_call block

* test(e2e): exempt the guardrail config echo from the post_call leak assertion

* test(e2e): pin the fail-closed contract for an unknown guardrail name (skipped, product gap)

* test(e2e): tolerate the readiness 503 from a transient db blip in the callback-config probes
2026-08-29 09:43:44 -07:00
Rāna(Bass Ver.)
4f630411f0
Merge branch 'litellm_internal_staging' into fix/34379-unblock-customer 2026-08-30 00:17:26 +08:00
Mateo Wang
b2fe72e70c
Merge pull request #35940 from ump45nose/fix_databricks_oauth_url
fix(databricks): derive OAuth token URL from workspace origin
2026-08-29 07:30:14 -07:00
mateo-berri
c09f76a079 refactor(mcp): drop always-true isinstance guard on typed token-endpoint response 2026-08-29 07:15:00 -07:00
Mateo Wang
c42ac262d3
Merge branch 'litellm_internal_staging' into fix_databricks_oauth_url 2026-08-29 07:00:58 -07:00
mateo-berri
c4663a4ae9 chore(budgets): drop stale PLW0133 entry graduated to ruff.toml 2026-08-29 06:51:54 -07:00
mateo-berri
683330e4e8 merge(litellm_internal_staging): resolve typing sweep overlap, keep best-of-both types 2026-08-29 06:48:36 -07:00
mateo-berri
9c653be72c chore(budgets): drop merge headroom above staging ceilings 2026-08-29 06:42:59 -07:00
mateo-berri
163c7033d6 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5 2026-08-29 06:38:33 -07:00
mateo-berri
c37260a2bd Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5
# Conflicts:
#	basedpyright-code-budget.json
#	enterprise/litellm_enterprise/proxy/audit_logging_endpoints.py
#	litellm/_lazy_imports.py
#	litellm/a2a_protocol/litellm_completion_bridge/transformation.py
#	litellm/integrations/bitbucket/bitbucket_client.py
#	litellm/integrations/compression_interception/handler.py
#	litellm/integrations/prometheus_helpers/prometheus_api.py
#	litellm/litellm_core_utils/model_response_utils.py
#	litellm/litellm_core_utils/url_utils.py
#	litellm/llms/anthropic/experimental_pass_through/context_management/dispatcher.py
#	litellm/llms/anthropic/experimental_pass_through/responses_adapters/handler.py
#	litellm/llms/anthropic/skills/transformation.py
#	litellm/llms/azure/files/handler.py
#	litellm/llms/bedrock/realtime/handler.py
#	litellm/llms/chatgpt/chat/streaming_utils.py
#	litellm/llms/compactifai/chat/transformation.py
#	litellm/llms/oci/chat/cohere.py
#	litellm/llms/vertex_ai/vector_stores/rag_api/transformation.py
#	litellm/proxy/agent_endpoints/agent_registry.py
#	litellm/proxy/client/cli/commands/credentials.py
#	litellm/proxy/client/cli/commands/teams.py
#	litellm/proxy/common_utils/get_routes.py
#	litellm/proxy/db/routing_prisma_wrapper.py
#	litellm/proxy/guardrails/guardrail_hooks/custom_code/sandbox.py
#	litellm/proxy/guardrails/guardrail_hooks/hiddenlayer/hiddenlayer.py
#	litellm/proxy/guardrails/guardrail_hooks/llm_as_a_judge/__init__.py
#	litellm/proxy/guardrails/guardrail_hooks/promptguard/promptguard.py
#	litellm/rust_bridge/responses_websocket.py
#	litellm/secret_managers/secret_manager_handler.py
#	ruff-strict-budget.json
#	type-discipline-budget.json
2026-08-29 06:37:10 -07:00
Mateo Wang
e48f8f016f
Merge pull request #38148 from mubashir1osmani/litellm_hosted_vllm_videos
feat(hosted_vllm): add vLLM-Omni videos API
2026-08-29 06:30:35 -07:00
Devin AI
f0849eb0c9 fix(models): xai retirement repricing, bedrock grok-4.6 caching, openai/gemini deprecation dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 13:11:26 +00:00
mateo-berri
8d4620649f Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5
# Conflicts:
#	basedpyright-code-budget.json
#	litellm/caching/valkey_semantic_cache.py
#	litellm/integrations/compression_interception/handler.py
#	litellm/integrations/custom_logger.py
#	litellm/llms/custom_httpx/container_handler.py
#	litellm/llms/infinity/rerank/transformation.py
#	litellm/proxy/agent_endpoints/agent_registry.py
#	litellm/repositories/base_repository.py
#	litellm/repositories/credentials_repository.py
#	litellm/repositories/team_repository.py
#	ruff-strict-budget.json
#	type-discipline-budget.json
2026-08-29 06:03:33 -07:00
Devin AI
401b12e64b Merge remote-tracking branch 'origin/litellm_internal_staging' into devin/1787944648-registry-audit-rolling 2026-08-29 13:02:37 +00:00
Devin AI
bc2b7281e6 chore: ratchet basedpyright reportAny ceiling after merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 11:35:21 +00:00
Devin AI
adf5637fdf chore: merge litellm_internal_staging into litellm_techdebt_20260829
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 11:31:44 +00:00
Mateo Wang
ae2e23bb53
Merge pull request #38501 from BerriAI/litellm_decrease_anys_opus5_0826
refactor(types): replace Any with real types across 178 backend files
2026-08-29 04:29:44 -07:00
mateo-berri
ee86b62b66 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5_0826
# Conflicts:
#	basedpyright-code-budget.json
#	type-discipline-budget.json
2026-08-29 04:18:02 -07:00
Mateo Wang
6a3333d3c8
Merge pull request #38747 from BerriAI/litellm_aws_partition_helper
fix(aws): build every AWS endpoint and ARN from the region's partition (aws-cn, aws-us-gov)
2026-08-29 04:10:59 -07:00
Mateo Wang
39e4f1ae13
Merge pull request #38727 from BerriAI/litellm_aws_external_id_embed_sagemaker
fix(aws): forward aws_external_id in Bedrock embeddings and SageMaker credential loading
2026-08-29 03:30:32 -07:00
mateo-berri
47d8ce6d10 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5_0826
# Conflicts:
#	basedpyright-code-budget.json
#	litellm/llms/soniox/common_utils.py
#	ruff-strict-budget.json
#	type-discipline-budget.json
2026-08-29 03:29:39 -07:00
Mateo Wang
1fbf34e559
Merge pull request #38744 from BerriAI/litellm_fix_bedrock_batch_zero_count_retire
fix(bedrock): map real batch record counts and guard zero-count retire
2026-08-29 03:12:49 -07:00
Devin AI
e535724923 test: cover the bounded Hugging Face config fetch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 10:10:03 +00:00
Mateo Wang
efe51daf5b
Merge pull request #38742 from BerriAI/litellm_batch_id_fallback_pin
fix(router): pin batch, file, and fine-tuning job ids to their owning model group on fallback
2026-08-29 02:57:07 -07:00
Devin AI
1687c65823 fix: hardcode HF config fetch timeout instead of reading an env var
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 09:53:43 +00:00
Devin AI
db1b1e2195 fix: bound Hugging Face config fetch and keep embedding tests off the network
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 09:45:52 +00:00
mateo-berri
850b18d214 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_tag_routing_reads_merged_metadata_tags
# Conflicts:
#	tests/openai_endpoints_tests/test_e2e_openai_responses_api.py
2026-08-29 02:34:51 -07:00
mateo-berri
229970c500 fix(guardrails): unwrap HiddenParamsAsyncIteratorWrapper before deferred dispatch class sniffing 2026-08-29 02:30:34 -07:00
mateo-berri
1947c65081 test(aws): type the new partition test parameters 2026-08-29 02:27:18 -07:00
mateo-berri
664697133b Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_bedrock_guardrail_stream_audit 2026-08-29 02:24:59 -07:00
mateo-berri
f60ccf6234 fix(guardrails): match deferred stream dispatch shape per stream owner and defer passthrough logging until guardrail eos 2026-08-29 02:23:12 -07:00
mateo-berri
d5b5aca498 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_bedrock_batch_zero_count_retire 2026-08-29 02:21:28 -07:00
mateo-berri
2d11870793 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_batch_id_fallback_pin
# Conflicts:
#	tests/openai_endpoints_tests/test_e2e_openai_responses_api.py
2026-08-29 02:19:45 -07:00
mateo-berri
cd5cf84aac Merge branch 'litellm_internal_staging' into litellm_aws_partition_helper 2026-08-29 02:18:16 -07:00
Mateo Wang
3993829a21
Merge pull request #38748 from BerriAI/litellm_fix_gpt55_extra_body_test
test(responses): adapt temperature tests to the gpt-5 reasoning validation
2026-08-29 02:17:01 -07:00
mateo-berri
b0d485568b test(batches): pin the poller against retiring zero-count completed batches 2026-08-29 02:00:11 -07:00
mateo-berri
66947fcf7c test(responses): expect the bad-temperature 400 on a non-reasoning model 2026-08-29 01:58:15 -07:00
mateo-berri
8cf090b368 fix(router): arm the provider-scoped fallback pin only on resource-operating handlers 2026-08-29 01:46:33 -07:00
mateo-berri
7fbcd9c3ed fix(bedrock): treat partial record counts as unknown on batch retrieve 2026-08-29 01:40:07 -07:00
mateo-berri
81eb014e71 test(responses): use a reasoning-legal temperature in the gpt-5.5 extra_body merge test 2026-08-29 01:36:45 -07:00
mateo-berri
2968c246b9 test(responses): repair two tests broken by the gpt-5 temperature gate
PR #38593 stopped forwarding temperature to reasoning models, which left
test_extra_body_merges_with_request_data raising UnsupportedParamsError
and test_bad_request_bad_param_error no longer getting a rejection from
OpenAI because drop_params now eats the param. Both repairs are the same
hunks PR #38739 carries, so the branches merge clean in either order
2026-08-29 01:32:09 -07:00
mateo-berri
64eec53fd8 fix(guardrails): surface post-flush stream blocks as in-stream error frames and keep guardrail_information in spend logs
A guardrail block or failed scan that fires after SSE chunks have been
flushed can no longer set an HTTP status, so raising HTTPException there
silently truncated the stream. _emit_streaming_http_error now routes
post-flush failures through the endpoint translation's
build_stream_error_items, emitting the surface-correct error frame on
chat completions (data: {error}), /v1/messages (event: error), and
/v1/responses (ErrorEvent with the next sequence number). Pre-flush
blocks still raise with a real HTTP status.

Successful flags-on scans also logged metadata.guardrail_information as
null: the chat handler planted litellm_metadata on a route whose bucket
is metadata, flipping the bucket for every later write, and responses
streams fired their spend log before the eos scan ran. The chat handler
now merges user_api_key metadata through get_or_create_metadata_bucket,
and deferred stream-complete logging is armed for aresponses like it
already was for anthropic_messages.
2026-08-29 01:25:08 -07:00
mateo-berri
9da766a1f9 test: use a non-reasoning model for the responses bad-param e2e fixture 2026-08-29 01:22:49 -07:00
mateo-berri
ad8c1457d1 fix(aws): build every AWS endpoint and ARN from the region partition
Adds litellm/litellm_core_utils/aws_partition.py mapping a region to its
AWS partition (aws, aws-cn, aws-us-gov, and the iso partitions), its DNS
suffix, and its ARN prefix, and uses it at every AWS host and ARN build
site: bedrock (runtime, agent, agentcore, legacy client, batches, files,
realtime), sagemaker, polly, secrets manager, s3 log uploads, bedrock
passthrough routes, and rag ingestion. ARN detection now accepts
arn:aws-cn: and arn:aws-us-gov: prefixes.

STS region resolution now falls back to the configured aws_region_name
after the aws_sts_endpoint host and the AWS_REGION/AWS_DEFAULT_REGION env
vars, so cn and gov role assumption no longer silently signs against
us-west-2.

A partition sweep test walks every endpoint builder with cn regions and
asserts no amazonaws.com host or arn:aws: prefix comes out, plus an AST
guard that fails on any new f-string hardcoding either literal.
2026-08-29 01:21:59 -07:00
mateo-berri
1956903c28 refactor(vertex_ai): narrow dict return annotations in gemini transcribe config 2026-08-29 01:21:38 -07:00
mateo-berri
6301a520a6 test: pin reasoning effort none in the responses extra_body fixture 2026-08-29 01:07:24 -07:00
mateo-berri
5d34fb20ff fix(bedrock): map real batch record counts and guard zero-count retire 2026-08-29 01:05:36 -07:00
Devin AI
9bfb332904 refactor: replace fresh getattr/setattr and test type-ignores with typed access
Same-day debt cleanup on code that landed in the last 24 hours. No behavior change.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-29 07:59:27 +00:00
mateo-berri
8fcfaf519c test: drop duplicated callback assertion 2026-08-29 00:53:24 -07:00
mateo-berri
817b5f44f6 fix(anthropic_endpoints): serialize dict-detail HTTPExceptions on /v1/messages like sibling surfaces 2026-08-29 00:48:14 -07:00