scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-30 01:52:18 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 70
52992 commits 2968 branches 1429 tags 3.9 GiB
Commit graph

176 commits

Author SHA1 Message Date
Joshua Valluru
479360ae87 test(mcp): restore scoped execution and credential isolation regressions 2026-09-19 16:41:31 -07:00
kerry
02736e20cc fix(test): repin fireworks fallback cache-read case to the 50% default 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 23:33:29 +00:00
kerry
21a2ed6244 test(integration): drop id diagnostic from fal video wire test 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 23:29:25 +00:00
kerry
9a63e06c63 test(integration): cover fal Seedance video queue wire contract 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 23:28:51 +00:00
kerry
789f0c61bf test(integration): only accept omitted breakdown components when the case expects zero, fix xai output cost 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 23:12:43 +00:00
Mateo Wang
57c757e9d7
Revert "test(mcp): verify scoped execution and OAuth credential isolation" 2026-09-19 16:10:47 -07:00
kerry
c7113f043a test(integration): use per-run request ids, drop unbillable perplexity search case 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 23:01:25 +00:00
kerry
72b007a4ae test(integration): pricing dimension and provider cost cases 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 22:46:43 +00:00
kerry
110d4c2ad1 test(integration): add passthrough cost cases 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 22:05:57 +00:00
kerry
247c4dd68f Merge remote-tracking branch 'origin/litellm_cost_shard_harness_extensions' into litellm_cost_shard_audio_images 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/integration/contracts.json
#	tests/integration/cost_calculation/cost_tracking_case.py
#	tests/integration/cost_calculation/cost_tracking_cases.json
 2026-09-19 21:25:39 +00:00
kerry
5b97d98b7b test(integration): require persisted cost breakdowns unless a case opts out 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 21:24:56 +00:00
kerry
f038a0558c Merge remote-tracking branch 'origin/litellm_cost_shard_harness_extensions' into litellm_cost_shard_embeddings_rerank 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/integration/contracts.json
#	tests/integration/cost_calculation/cost_tracking_case.py
#	tests/integration/cost_calculation/cost_tracking_cases.json
 2026-09-19 21:22:26 +00:00
kerry
c0c5cc84f8 test(integration): audio, image and per-unit cost cases 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 21:19:29 +00:00
kerry
23f5df05f2 test(integration): require a zero cost header to read zero when the case bills nothing 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 21:03:51 +00:00
kerry
8137d878a0 test(integration): drop contract nodes left behind by case renames 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 20:59:05 +00:00
kerry
125bda30e9 test(integration): embeddings, rerank, completions and moderations cost cases 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 20:58:13 +00:00
kerry
ca8d0e500c test(integration): correct responses reasoning and messages tiered expectations 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 20:11:24 +00:00
Joshua Valluru
a41b60cf77 test(mcp): align live regressions with discovery and error contracts 2026-09-19 13:03:00 -07:00
kerry
22d3441789 Merge branch 'litellm_cost_shard_harness_extensions' into litellm_cost_shard_responses_messages 2026-09-19 19:53:49 +00:00
kerry
8907a1d1fc test: add native responses and messages cost cases 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 19:52:43 +00:00
kerry
9bd648baf5 test(integration): price fireworks cached input at the 50% default 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 19:22:25 +00:00
kerry
dccb1b56b4 test(integration): reject out-of-range failure statuses at import 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 19:17:46 +00:00
Joshua Valluru
e0b6bae516 test(mcp): cover scoped execution and OAuth credential isolation 2026-09-19 12:14:32 -07:00
kerry
e42f3f1562 test: fix cost harness review issues 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 19:10:04 +00:00
Joshua Valluru
6f74aa80c0 chore: merge main into MCP regression coverage branch 2026-09-19 12:06:21 -07:00
kerry
c958f9db7e test: extend cost tracking integration harness 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 18:44:51 +00:00
kerry-berri
6718fd67fb
Merge pull request #41328 from BerriAI/litellm_e2e_cost_calculation_scripted_provider 
test(integration): literal request/response cost tracking suite on a test-owned cost map
 2026-09-19 09:40:04 -07:00
kerry
ed0c32cdb0 test(integration): drive cost tracking from literal request/response data 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 04:04:40 +00:00
kerry
57d2fefa8d test(integration): derive scripted shapes from litellm provider configs 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 03:16:16 +00:00
kerry
380ec1a004 docs(integration): keep cost map loading note in README 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 01:43:00 +00:00
kerry
6ccba7fdb5 test(integration): drive scripted wires and provider wiring from data 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 01:41:19 +00:00
kerry
77f6166c39 test(integration): fold scenario client into upstream module 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 01:30:24 +00:00
kerry
a15b0fa6d2 test(integration): tidy xdist collection bookkeeping 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 00:54:45 +00:00
Devin AI
6247b75543 test(mcp): keep the import block as merged on main 
Co-Authored-By: bot_apk <apk@cognition.ai>
 2026-09-19 00:51:51 +00:00
kerry
6eb67a8423 test(integration): run the cost shard with xdist workers 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 00:50:29 +00:00
kerry
e52eea84e6 test(integration): serve scripted wires from the shared upstream 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 00:50:26 +00:00
Devin AI
e3755a88e7 test(mcp): assert the misconfigured credential message on fail-closed rejections 
Co-Authored-By: bot_apk <apk@cognition.ai>
 2026-09-19 00:50:00 +00:00
kerry-berri
92d01fa568
Merge pull request #41901 from BerriAI/litellm_off_peak_pricing_integration_tests 
test(integration): cover off-peak pricing on a live proxy
 2026-09-18 17:26:51 -07:00
kerry
f836bb481d test(integration): keep cost diagnostics and widen shard timeout 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 00:19:11 +00:00
kerry
69f9106759 test(integration): move scripted-provider cost suite into cost shard 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 00:16:12 +00:00
kerry
83d89aa134 test(integration): cover off-peak pricing on a live proxy 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 00:10:22 +00:00
mateo-berri
7cfec730d5 fix(proxy): refuse config-owned keys on POST /config/update 
POST /config/update stored keys the config file owns and answered 200
while the file silently kept winning. Run the config-owned check for
general_settings, litellm_settings, and router_settings before the first
database read, the way /config/field/update already does, so a refused
request stores nothing.

The config reload re-loaded the merged settings as yaml settings, which
turned every saved router setting read-only after one tick. Read the
saved router settings row instead so database-owned values stay writable.

The integration harness seeds num_retries through /config/update instead
of the config file, which is what the effective-settings and observed
routing tests need to keep exercising a database-owned value.
 2026-09-18 12:42:08 -07:00
Joshua Valluru
f83992f786 test(mcp): reject every unauthorized server in health results 2026-09-17 18:44:20 -07:00
Joshua Valluru
daff22a884 test(mcp): grant the guardrail control team its server 2026-09-17 17:50:01 -07:00
Joshua Valluru
640b0e5fa9 test(mcp): discover concrete tools through a catalog key 2026-09-17 17:41:57 -07:00
Joshua Valluru
15b45839e1 test(mcp): enforce security regression contracts through live gateway 2026-09-17 17:37:40 -07:00
yuneng-jiang
efce8b0485
Merge pull request #41373 from BerriAI/litellm_fix_integration_conftest_import 
fix(tests): resolve the integration support package without run.py's PYTHONPATH
 2026-09-17 13:34:05 -07:00
Yuneng Jiang
fa01e2d5b7
Merge branch 'main' into litellm_/buildkite-241-triage-4a48cf 2026-09-17 10:39:37 -07:00
Yuneng Jiang
c3048dcd30
test(http): move the outbound HTTP/2 check into a new integration sdk suite 
The check spins up a hypercorn TLS peer and drives the SDK's own httpx
handlers at it, so it needs litellm importable, hypercorn installed and a
loopback socket. It lived under tests/e2e, whose Buildkite runner image
installs neither litellm nor hypercorn by design (the suite drives a
remote proxy over HTTP), so every scheduled e2e build since #230 failed
to import the module and pytest reported it as a collection error. The
unit tree bans sockets, so it does not belong there either

tests/integration is the CircleCI tier built for real TCP against local
protocol peers. This adds an sdk shard to it for cases that exercise the
SDK's clients with no gateway in the path, registers the two HTTP/2
nodes in the contracts manifest, and adds the shard to the CircleCI
matrix. The test now flips the feature through LITELLM_HTTP2 (the user
surface) instead of patching module attributes, and asserts the version
the peer observed on the wire next to the one the client reports
 2026-09-17 10:37:07 -07:00
Yuneng Jiang
7013204ded
Merge branches 'main' and 'litellm_fix_integration_conftest_import' of github.com:BerriAI/litellm into litellm_fix_integration_conftest_import 
# Conflicts:
#	tests/integration/_support/client.py
#	tests/integration/management/test_partial_update_sequences.py
 2026-09-17 10:33:31 -07:00