Commit graph

51336 commits

Author SHA1 Message Date
mateo-berri
c735cc3db1 test(cost): point dated snapshot tests at a date the cost map cannot carry
The azure row of test_get_model_info_falls_back_from_dated_snapshot_to_undated_entry used gpt-5.6-luna-2026-07-09, which main's cost map carries as an exact azure key, so the lookup returned the dated key and the required misc test job failed on main. All three dated snapshot tests now use a 2099-01-01 snapshot date, so they keep exercising the strip path whatever real snapshots the map gains
2026-09-19 08:00:12 -07:00
Mateo Wang
8c4c394ecc
Merge pull request #41960 from BerriAI/litellm_deepseek_off_peak_pricing
fix(cost): bill DeepSeek V4.1 Flash and V4 Pro at off-peak rates outside peak hours
2026-09-19 06:45:46 -07:00
Mateo Wang
12593788a2
Merge pull request #41949 from BerriAI/litellm_bulk_update_keys_keep_unset_fields
fix(proxy): /key/bulk_update writes only the fields each item carries
2026-09-19 06:08:54 -07:00
Mateo Wang
aa0ffee00f
Merge pull request #41423 from BerriAI/litellm_strip_dated_snapshot_cost_lookup
fix(cost): resolve dated openai/azure snapshots to their undated cost map entry
2026-09-19 05:51:46 -07:00
Mateo Wang
f15dba2e2f
Merge pull request #41943 from BerriAI/litellm_unknown_model_spend_logs_outside_router
fix(proxy): keep the raw client model out of spend logs for rejections outside the router
2026-09-19 05:15:26 -07:00
mateo-berri
dc02e5f5fb test(proxy): stub the existing key's team in the bulk item policy tests 2026-09-19 05:12:12 -07:00
mateo-berri
df6a222cb8 fix(proxy): validate bulk object_permission against the key's team as /key/update does 2026-09-19 04:59:26 -07:00
Mateo Wang
b8d837b2ef
Merge pull request #40147 from abhirup7/fix/azure-image-generation-entra-id-auth
fix(azure): send the resolved Entra ID token on image generation requests
2026-09-19 04:44:07 -07:00
mateo-berri
aadadc647d Merge remote-tracking branch 'origin/main' into litellm_unknown_model_spend_logs_outside_router
# Conflicts:
#	litellm/proxy/openai_files_endpoints/common_utils.py
2026-09-19 04:43:19 -07:00
Mateo Wang
dad8c32d23
Merge pull request #41940 from BerriAI/litellm_rag_ingest_registry_store
fix(rag): resolve registry stores on /v1/rag/ingest and reject providers without ingestion
2026-09-19 04:42:54 -07:00
Mateo Wang
40f51df90f
Merge pull request #41953 from BerriAI/litellm_bridge_drop_tool_search
fix(responses): drop tool_search and local_shell in the chat completions bridge
2026-09-19 04:42:28 -07:00
mateo-berri
e0db862781 fix(cost): bill DeepSeek V4.1 Flash and V4 Pro at their off-peak rates outside peak hours
DeepSeek charges half the listed rate outside 01:00-04:00 and 06:00-10:00 UTC
Monday to Friday, so every deepseek-flash, deepseek-v4-flash,
deepseek-v4-flash-vision-exp, and deepseek-v4-pro entry now carries an
off_peak_pricing block with those windows and the halved input, output, and
cache-hit rates. The generated cost map schema picks up the block, and the
regression tests pin the peak and off-peak cost of one call at fixed moments.
2026-09-19 04:31:55 -07:00
mateo-berri
aef209963a fix(s3_vectors): embed registered-store ingests with the store's embedding model
The S3 Vectors ingestion embedded every chunk with the request's
embedding.model or the default, never the embedding_model the store was
registered with, while search on the same store embeds with the
registered model. A registered store uploaded to by id alone therefore
embedded with the wrong model and AWS rejected the vectors on the
dimension mismatch. The store's embedding model now wins for S3 Vectors
ingestion through a helper next to the one search already uses
2026-09-19 04:31:15 -07:00
Mateo Wang
1f6e5b60b5
Merge pull request #41952 from BerriAI/litellm_masker_memo_depth_fail_closed
fix(masker): memoize shared nodes and fail closed past the depth cap
2026-09-19 04:20:17 -07:00
mateo-berri
e74a5e0c21 test(rag): drop the docstrings from the registered-store ingest tests 2026-09-19 04:10:49 -07:00
Mateo Wang
43e835c3ca
Merge pull request #41950 from BerriAI/litellm_cost_callback_bounded_error_msg
fix(proxy): keep request metadata out of the cost tracking failure alert
2026-09-19 04:08:14 -07:00
mateo-berri
aceae8e566 test: drop the recursive detector allowlist entry for the removed _walk_payload 2026-09-19 04:07:07 -07:00
mateo-berri
ccb48eb528 refactor(s3_vectors): keep the ingest target derivation under llms/s3_vectors
The ingest-side bucket and index precedence now sits next to the shared
store id split instead of under litellm/rag/, where provider-specific
parsing does not belong.
2026-09-19 04:04:59 -07:00
Mateo Wang
c39ec34553
Merge pull request #41234 from BerriAI/litellm_invalid_tool_choice_400
fix(utils): reject an untranslatable tool_choice with a 400 instead of a 500
2026-09-19 04:02:04 -07:00
mateo-berri
093fb78baf fix(masker): cut cycles at the first back-edge and walk pydantic dumps without self-recursion 2026-09-19 03:58:47 -07:00
mateo-berri
d437cd662b fix(proxy): placeholder the metadata copied into a placeholdered row's stored request body 2026-09-19 03:56:44 -07:00
mateo-berri
f982d3e046 docs(proxy): state /key/bulk_update null handling as /key/update parity 2026-09-19 03:55:48 -07:00
mateo-berri
5477dbe74c fix(responses): drop tool_search and local_shell in the chat completions bridge
Hosted Responses API tools with no Chat Completions equivalent were forwarded
verbatim, so Codex 0.140+ got a 400 from the provider on every turn. The bridge
now drops tool_search and local_shell the same way it drops computer_use,
image_generation, and shell, and also drops parallel_tool_calls when no chat
tools remain, since chat completions only accepts it alongside tools
2026-09-19 03:55:23 -07:00
mateo-berri
e4d01d1d78 fix(s3_vectors): reject a store id with an empty bucket or index part
A "bucket:" or ":index" id split into an empty name, so ingestion silently
generated a fresh index and search sent the empty name to AWS. Both sides now
raise the existing format error through the shared helper.
2026-09-19 03:52:48 -07:00
Mateo Wang
db04e7909e
Merge pull request #41934 from BerriAI/litellm_mistral_ocr_batches
feat(batches): support Mistral files/batches and per-page OCR batch cost tracking (internal copy of #40484)
2026-09-19 03:51:27 -07:00
Mateo Wang
aa3e6df086
Merge pull request #41926 from BerriAI/litellm_fix_41515
fix(proxy): register transcribe as a known provider for model grants
2026-09-19 03:47:36 -07:00
mateo-berri
b746ac4456 fix(proxy): accept object_permission on /key/bulk_update items instead of 422 2026-09-19 03:44:07 -07:00
Mateo Wang
165255e671
Merge pull request #41947 from BerriAI/litellm_any_sweep_20260919
refactor(types): replace Any with proven types in 6 files
2026-09-19 03:41:43 -07:00
mateo-berri
e2d118aaf8 fix(rag): read a registered S3 Vectors store's bucket and index from its id
A registered S3 Vectors store usually carries only its "bucket:index" id,
and the previous commit stopped forwarding the caller's bucket and index for
a managed store, so ingesting into one raised KeyError 'vector_bucket_name'.
The ingestion now derives both from vector_store_id with the rule the search
side already uses, explicit keys still winning. The caller's
litellm_credential_name is dropped for a managed store too, since it expands
into api_key and api_base, and max_embedding_requests_per_min joins the
per-upload options a caller may still set.
2026-09-19 03:38:20 -07:00
Mateo Wang
2815d80fa4
Merge pull request #41948 from BerriAI/litellm_unit_shard_per_test_timeout
ci(unit): fail a hung test in 120s with a traceback instead of idling the shard to its step timeout
2026-09-19 03:36:37 -07:00
mateo-berri
7edafd1715 fix(masker): memoize shared nodes and fail closed past the depth cap 2026-09-19 03:28:50 -07:00
mateo-berri
ad4da0f8e6 chore(proxy): regenerate the lazy OpenAPI snapshot on Python 3.12 and drop a test helper docstring 2026-09-19 03:28:42 -07:00
Mateo Wang
825e287f63
Merge pull request #41237 from BerriAI/litellm_internal_copy_36887
fix(cost): carry image and video input tokens through the Responses usage bridge (internal copy of #36887)
2026-09-19 03:27:42 -07:00
mateo-berri
9e8a847c5b fix(proxy): keep request metadata out of the cost tracking failure alert
The cost tracking callback f-stringed chosen_metadata, litellm_metadata,
and old_metadata into the failed_tracking_spend alert on every failure,
at every log level, so one 250-byte request produced a 23 KB alert
carrying the client's metadata, headers, and key-auth reprs four times
over. The alert now carries the exception, the traceback, the model, and
the call type; the metadata keys are logged once at debug level through
lazy formatting, so nothing is built at warning level
2026-09-19 03:21:28 -07:00
mateo-berri
4968e89f3c test(ci): drop the structure-only assertion on the shard script; the parametrized hang test covers both invocations 2026-09-19 03:20:48 -07:00
mateo-berri
a49fbc6272 fix(proxy): keep the raw client model out of the stored request body when a spend row is placeholdered
With store_prompts_in_spend_logs on, the persisted request body kept the client's model string even when the row's model, model_group, and error text had been replaced by the unknown-model placeholder. The body's model now takes the same placeholder on those rows. Also annotates the new test locals with Final and wraps the four test lines that ran past 120 characters.
2026-09-19 03:20:31 -07:00
Mateo Wang
6b8db68fb8
Merge pull request #39102 from gaurav-pandey-zocdoc/litellm_budget_alert_wording
fix(alerting): clarify budget threshold messages
2026-09-19 03:15:16 -07:00
mateo-berri
bdbe265c70 fix(proxy): /key/bulk_update writes only the fields each item carries
A bulk item that carried only tags reached the DB with max_budget, team_id,
and budget_id as explicit nulls, wiping the key's budget and detaching it
from its team. The per-key update is now built from the fields the item
actually set, so a field left out keeps its value and an explicit null still
clears it, the same as /key/update. Items carrying a field the bulk path
cannot apply (object_permission and the like) are rejected with 422 instead
of being silently dropped.
2026-09-19 03:14:39 -07:00
Mateo Wang
385932b4e3
Merge pull request #41941 from BerriAI/litellm_azure_content_safety_api_version_default
fix(guardrails): stop the Javelin api_version default leaking into Azure Content Safety
2026-09-19 03:09:38 -07:00
Devin AI
8ee7591fc8 refactor(types): drop nonessential TypedDict docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 10:05:14 +00:00
mateo-berri
217ff78ae7 fix(mistral): read back files whose purpose Mistral never lets us upload as user_data 2026-09-19 03:04:45 -07:00
mateo-berri
b8c2787bcf Merge remote-tracking branch 'origin/main' into litellm_internal_copy_36887 2026-09-19 02:54:13 -07:00
mateo-berri
343e1eeac8 ci(unit): fail a hung test in 120s with a traceback instead of idling the shard to its step timeout 2026-09-19 02:53:36 -07:00
mateo-berri
e327a6ae76 test(guardrails): drop regression docstrings from the api_version tests 2026-09-19 02:52:42 -07:00
mateo-berri
988bb65aa2 test: require a provider family's rows to reach its wildcard list 2026-09-19 02:50:31 -07:00
mateo-berri
df3a37857c fix(proxy): keep a configured model group in spend logs when it fails before a deployment is picked 2026-09-19 02:48:30 -07:00
mateo-berri
2c2aa5df11 Merge remote-tracking branch 'origin/main' into litellm_invalid_tool_choice_400
# Conflicts:
#	tests/test_litellm/responses/litellm_completion_transformation/test_litellm_completion_responses.py
#	tests/test_litellm/test_main.py
2026-09-19 02:35:29 -07:00
mateo-berri
6e0356a7a0 test: fail a required shard when a cost map provider is unregistered 2026-09-19 02:34:27 -07:00
mateo-berri
24064e3b31 fix(guardrails): treat the stored Javelin api_version default as unset for Azure Content Safety
Guardrails created through POST /guardrails on older releases have api_version "v1" saved in the database, because the writer persists every default. Azure Content Safety never accepts that value, so those guardrails kept answering 404 after the default moved to None. The Azure base now resolves "v1" to 2024-09-01 the same way it resolves a missing value. Also restores the OpenAPI snapshot line that a Python 3.14 regeneration had dedented
2026-09-19 02:29:43 -07:00
Devin AI
f784681bfa refactor(types): replace Any with proven types in 6 files
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 09:28:14 +00:00