scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-10 03:28:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 84
53032 commits 3393 branches 1449 tags 5.1 GiB
Commit graph

182 commits

Author SHA1 Message Date
kerry
8e3bb5daab test(integration): wait for all rollup writes and pin fallback and disconnect rows 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-20 00:49:51 +00:00
kerry
cc93a37322 test(integration): wait for every rollup write and bound the disconnect recount 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-20 00:48:42 +00:00
kerry
2edea0be08 test(integration): assert recount pins and unique fixture request ids 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-20 00:36:18 +00:00
kerry
a841750d46 test(integration): proxy behaviour cost cases 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-20 00:19:56 +00:00
kerry
98ea6dd405 test(integration): drop redundant cost_header defaults 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 23:46:45 +00:00
kerry
a4488fccca test(integration): add provider wire cost cases 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 23:45:51 +00:00
Joshua Valluru
479360ae87 test(mcp): restore scoped execution and credential isolation regressions 2026-09-19 16:41:31 -07:00
kerry
02736e20cc fix(test): repin fireworks fallback cache-read case to the 50% default 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 23:33:29 +00:00
kerry
21a2ed6244 test(integration): drop id diagnostic from fal video wire test 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 23:29:25 +00:00
kerry
9a63e06c63 test(integration): cover fal Seedance video queue wire contract 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 23:28:51 +00:00
kerry
789f0c61bf test(integration): only accept omitted breakdown components when the case expects zero, fix xai output cost 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 23:12:43 +00:00
Mateo Wang
57c757e9d7
Revert "test(mcp): verify scoped execution and OAuth credential isolation" 2026-09-19 16:10:47 -07:00
kerry
c7113f043a test(integration): use per-run request ids, drop unbillable perplexity search case 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 23:01:25 +00:00
kerry
72b007a4ae test(integration): pricing dimension and provider cost cases 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 22:46:43 +00:00
kerry
110d4c2ad1 test(integration): add passthrough cost cases 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 22:05:57 +00:00
kerry
247c4dd68f Merge remote-tracking branch 'origin/litellm_cost_shard_harness_extensions' into litellm_cost_shard_audio_images 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/integration/contracts.json
#	tests/integration/cost_calculation/cost_tracking_case.py
#	tests/integration/cost_calculation/cost_tracking_cases.json
 2026-09-19 21:25:39 +00:00
kerry
5b97d98b7b test(integration): require persisted cost breakdowns unless a case opts out 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 21:24:56 +00:00
kerry
f038a0558c Merge remote-tracking branch 'origin/litellm_cost_shard_harness_extensions' into litellm_cost_shard_embeddings_rerank 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/integration/contracts.json
#	tests/integration/cost_calculation/cost_tracking_case.py
#	tests/integration/cost_calculation/cost_tracking_cases.json
 2026-09-19 21:22:26 +00:00
kerry
c0c5cc84f8 test(integration): audio, image and per-unit cost cases 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 21:19:29 +00:00
kerry
23f5df05f2 test(integration): require a zero cost header to read zero when the case bills nothing 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 21:03:51 +00:00
kerry
8137d878a0 test(integration): drop contract nodes left behind by case renames 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 20:59:05 +00:00
kerry
125bda30e9 test(integration): embeddings, rerank, completions and moderations cost cases 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 20:58:13 +00:00
kerry
ca8d0e500c test(integration): correct responses reasoning and messages tiered expectations 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 20:11:24 +00:00
Joshua Valluru
a41b60cf77 test(mcp): align live regressions with discovery and error contracts 2026-09-19 13:03:00 -07:00
kerry
22d3441789 Merge branch 'litellm_cost_shard_harness_extensions' into litellm_cost_shard_responses_messages 2026-09-19 19:53:49 +00:00
kerry
8907a1d1fc test: add native responses and messages cost cases 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 19:52:43 +00:00
kerry
9bd648baf5 test(integration): price fireworks cached input at the 50% default 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 19:22:25 +00:00
kerry
dccb1b56b4 test(integration): reject out-of-range failure statuses at import 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 19:17:46 +00:00
Joshua Valluru
e0b6bae516 test(mcp): cover scoped execution and OAuth credential isolation 2026-09-19 12:14:32 -07:00
kerry
e42f3f1562 test: fix cost harness review issues 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 19:10:04 +00:00
Joshua Valluru
6f74aa80c0 chore: merge main into MCP regression coverage branch 2026-09-19 12:06:21 -07:00
kerry
c958f9db7e test: extend cost tracking integration harness 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 18:44:51 +00:00
kerry-berri
6718fd67fb
Merge pull request #41328 from BerriAI/litellm_e2e_cost_calculation_scripted_provider 
test(integration): literal request/response cost tracking suite on a test-owned cost map
 2026-09-19 09:40:04 -07:00
kerry
ed0c32cdb0 test(integration): drive cost tracking from literal request/response data 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 04:04:40 +00:00
kerry
57d2fefa8d test(integration): derive scripted shapes from litellm provider configs 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 03:16:16 +00:00
kerry
380ec1a004 docs(integration): keep cost map loading note in README 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 01:43:00 +00:00
kerry
6ccba7fdb5 test(integration): drive scripted wires and provider wiring from data 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 01:41:19 +00:00
kerry
77f6166c39 test(integration): fold scenario client into upstream module 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 01:30:24 +00:00
kerry
a15b0fa6d2 test(integration): tidy xdist collection bookkeeping 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 00:54:45 +00:00
Devin AI
6247b75543 test(mcp): keep the import block as merged on main 
Co-Authored-By: bot_apk <apk@cognition.ai>
 2026-09-19 00:51:51 +00:00
kerry
6eb67a8423 test(integration): run the cost shard with xdist workers 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 00:50:29 +00:00
kerry
e52eea84e6 test(integration): serve scripted wires from the shared upstream 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 00:50:26 +00:00
Devin AI
e3755a88e7 test(mcp): assert the misconfigured credential message on fail-closed rejections 
Co-Authored-By: bot_apk <apk@cognition.ai>
 2026-09-19 00:50:00 +00:00
kerry-berri
92d01fa568
Merge pull request #41901 from BerriAI/litellm_off_peak_pricing_integration_tests 
test(integration): cover off-peak pricing on a live proxy
 2026-09-18 17:26:51 -07:00
kerry
f836bb481d test(integration): keep cost diagnostics and widen shard timeout 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 00:19:11 +00:00
kerry
69f9106759 test(integration): move scripted-provider cost suite into cost shard 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 00:16:12 +00:00
kerry
83d89aa134 test(integration): cover off-peak pricing on a live proxy 
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2026-09-19 00:10:22 +00:00
mateo-berri
7cfec730d5 fix(proxy): refuse config-owned keys on POST /config/update 
POST /config/update stored keys the config file owns and answered 200
while the file silently kept winning. Run the config-owned check for
general_settings, litellm_settings, and router_settings before the first
database read, the way /config/field/update already does, so a refused
request stores nothing.

The config reload re-loaded the merged settings as yaml settings, which
turned every saved router setting read-only after one tick. Read the
saved router settings row instead so database-owned values stay writable.

The integration harness seeds num_retries through /config/update instead
of the config file, which is what the effective-settings and observed
routing tests need to keep exercising a database-owned value.
 2026-09-18 12:42:08 -07:00
Joshua Valluru
f83992f786 test(mcp): reject every unauthorized server in health results 2026-09-17 18:44:20 -07:00
Joshua Valluru
daff22a884 test(mcp): grant the guardrail control team its server 2026-09-17 17:50:01 -07:00