Commit graph

17 commits

Author SHA1 Message Date
kerry
247c4dd68f Merge remote-tracking branch 'origin/litellm_cost_shard_harness_extensions' into litellm_cost_shard_audio_images
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	tests/integration/contracts.json
#	tests/integration/cost_calculation/cost_tracking_case.py
#	tests/integration/cost_calculation/cost_tracking_cases.json
2026-09-19 21:25:39 +00:00
kerry
5b97d98b7b test(integration): require persisted cost breakdowns unless a case opts out
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 21:24:56 +00:00
kerry
c0c5cc84f8 test(integration): audio, image and per-unit cost cases
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 21:19:29 +00:00
kerry
ca8d0e500c test(integration): correct responses reasoning and messages tiered expectations
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 20:11:24 +00:00
kerry
22d3441789 Merge branch 'litellm_cost_shard_harness_extensions' into litellm_cost_shard_responses_messages 2026-09-19 19:53:49 +00:00
kerry
8907a1d1fc test: add native responses and messages cost cases
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 19:52:43 +00:00
kerry
9bd648baf5 test(integration): price fireworks cached input at the 50% default
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 19:22:25 +00:00
kerry
dccb1b56b4 test(integration): reject out-of-range failure statuses at import
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 19:17:46 +00:00
kerry
e42f3f1562 test: fix cost harness review issues
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 19:10:04 +00:00
kerry
c958f9db7e test: extend cost tracking integration harness
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 18:44:51 +00:00
kerry
ed0c32cdb0 test(integration): drive cost tracking from literal request/response data
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 04:04:40 +00:00
kerry
57d2fefa8d test(integration): derive scripted shapes from litellm provider configs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 03:16:16 +00:00
kerry
6ccba7fdb5 test(integration): drive scripted wires and provider wiring from data
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 01:41:19 +00:00
kerry
77f6166c39 test(integration): fold scenario client into upstream module
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 01:30:24 +00:00
kerry
e52eea84e6 test(integration): serve scripted wires from the shared upstream
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:50:26 +00:00
kerry
f836bb481d test(integration): keep cost diagnostics and widen shard timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:19:11 +00:00
kerry
69f9106759 test(integration): move scripted-provider cost suite into cost shard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-19 00:16:12 +00:00