scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-14 23:21:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 156
litellm/tests
History
Exact
ryan-crabbe-berri ae917c5ebf
Some checks failed
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Details
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Details
fix(budget): match prefixed key/end-user entries on bare requests and reject non-enforceable team caps 
The key/end-user matcher normalized only the request side, so an entry keyed
provider/model (e.g. openai/gpt-4) never matched a bare model-group request:
the key cap was silently unenforced, and on team keys the request was charged
to, and blocked by, the shared team counter instead of the key's own cap. The
matcher now delegates to the same both-sides normalization the team path uses.

Team model_max_budget validation now rejects zero, negative, and non-finite
budget_limit values: enforcement skips non-positive caps, so a team admin
could null an imposed cap by lowering it to zero, which the update authority
gate counted as a permitted decrease.

Also brings the branch inside the tightened Final-annotation and lazy-logging
budgets.
2026-08-05 12:15:39 -07:00
..
agent_tests feat(a2a): support a2a-sdk 1.x proxy routing for 0.3 and 1.0 agents (#30950) 2026-06-29 09:32:39 +05:30
audio_tests fix(tests): stabilize image-edit VCR cassettes to stop live gpt-image-1 spend (#28110) 2026-05-18 09:15:39 -07:00
base_sdk_tests fix(deps): move pydantic-settings into the base dependencies 2026-08-01 15:25:26 -07:00
basic_proxy_startup_tests
batches_tests test(batches): run Responses coverage in CI 2026-07-31 10:24:10 -04:00
benchmarks test(benchmarks): run shared logging executor inline to make CodSpeed measurements deterministic (#32435) 2026-07-09 11:14:22 +03:00
code_coverage_tests feat(guardrails): scan and mask MCP tool results via post_mcp_call (#35155) 2026-07-30 14:10:26 -07:00
documentation_tests feat(messages): route native Anthropic /messages through Rust behind LITELLM_RUST env var (#33848) 2026-07-18 18:27:20 -07:00
e2e Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_team_model_max_budget 2026-08-05 11:15:09 -07:00
enterprise test: repair three failing suites on litellm_internal_staging 2026-08-04 16:07:02 -07:00
guardrails_tests fix(proxy): fix zguard httpcode when block input (#31948) 2026-08-05 01:46:05 +00:00
image_gen_tests test(bedrock): switch image gen live test off EOL Titan to Nova Canvas (#31937) 2026-07-01 22:57:29 -07:00
integration feat(sandbox): e2b code execution primitive (#30898) 2026-06-20 16:30:01 -07:00
litellm fix(anthropic): strip all remaining output_format schema keywords rejected by Anthropic 2026-07-22 16:16:04 -07:00
litellm-proxy-extras
litellm_core_utils Merge branch 'litellm_internal_staging' into litellm_staging_03_22_2026 2026-04-20 19:56:00 +05:30
litellm_utils_tests refactor(bedrock): remove the dead BedrockLLM invoke code path 2026-07-29 20:25:36 -07:00
llm_responses_api_testing Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
llm_translation test(bedrock): port the openai-route invoke tests onto the live config 2026-07-29 20:59:08 -07:00
load_tests
local_testing test: repair three failing suites on litellm_internal_staging 2026-08-04 16:07:02 -07:00
logging_callback_tests test(logging): pin routing_decision and internal_call_origin in the gcs pubsub spend log fixture 2026-08-01 14:34:04 -07:00
mcp_tests fix(mcp): keep REST tool listing in step with key/team grant enforcement 2026-07-30 22:13:10 -07:00
multi_instance_e2e_tests
ocr_tests test(ocr): use mistral-document-ai-2512 in azure_ai OCR tests 2026-07-15 18:13:22 -07:00
old_proxy_tests/tests
openai_endpoints_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
otel_tests fix(cli): mint per-session agent credential on lite login (#31072) 2026-06-26 09:05:15 -07:00
pass_through_tests test(pass-through): de-flake vertex spend-log test by routing through the proxy (#31689) 2026-06-30 15:27:48 -07:00
pass_through_unit_tests fix(claude-code): create-only skill registration with a PUT update route (LIT-4110) (#31752) 2026-08-05 01:44:56 +00:00
proxy_admin_ui_tests Revert "chore: remove _experimental/out (#31546)" 2026-07-01 13:25:47 -07:00
proxy_behavior test: repair three failing suites on litellm_internal_staging 2026-08-04 16:07:02 -07:00
proxy_e2e_anthropic_messages_tests Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847) 2026-06-06 14:33:42 -07:00
proxy_migration_tests fix(migrations): recover from an interrupted Prisma toolchain install (#35832) 2026-08-05 10:14:46 -07:00
proxy_security_tests test(proxy): stop running real-DB tests in GitHub Actions unit jobs (#29700) 2026-06-04 14:56:02 -07:00
proxy_unit_tests fix(budget): match prefixed key/end-user entries on bare requests and reject non-enforceable team caps 2026-08-05 12:15:39 -07:00
router_unit_tests fix(auth): resolve managed batch/file deployment model_id to model name for team access checks 2026-07-28 15:35:19 +00:00
scim_tests
search_tests feat(tinyfish): make search provider permissive, attribute errors (#31997) 2026-07-03 10:17:11 -07:00
spend_tracking_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
store_model_in_db_tests feat(team): custom metadata validation hook for team create and update (#33353) 2026-08-03 18:37:45 -07:00
test_litellm fix(budget): match prefixed key/end-user entries on bare requests and reject non-enforceable team caps 2026-08-05 12:15:39 -07:00
unified_google_tests fix(google_genai): preserve complete SSE events in Vertex/Gemini image streaming (#30270) 2026-06-12 07:49:30 -07:00
vector_store_tests Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
windows_tests ci: reproduce default-Windows wheel install to guard MAX_PATH (#29597) 2026-06-03 11:28:08 -07:00
__init__.py
_fake_openai_endpoint_server.py test: point router/completion/triton tests at the local fake OpenAI endpoint (#30900) 2026-06-20 16:20:35 -07:00
_flush_vcr_cache.py tests(vcr): isolate cassette redis to CASSETTE_REDIS_URL 2026-05-01 12:32:59 -07:00
_live_test_helpers.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
_openai_record_replay_proxy.py Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847) 2026-06-06 14:33:42 -07:00
_vcr_conftest_common.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
_vcr_redis_persister.py test(vcr): stop refreshing cassette TTL on read so cassettes lapse after 24h (#29784) 2026-06-05 10:22:41 -07:00
_ws_vcr.py test(realtime): record and replay websocket traffic in redis vcr cassettes (#32390) 2026-07-08 00:19:06 -07:00
eval_swe_bench.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
fake_openai_endpoint.py test: point router/completion/triton tests at the local fake OpenAI endpoint (#30900) 2026-06-20 16:20:35 -07:00
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
README.MD
test_anthropic_compaction_usage.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_budget_management.py
test_callbacks_on_proxy.py test(callbacks): harden flaky proxy callback-leak detector (#28195) 2026-05-18 16:39:02 -07:00
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py
test_end_users.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_entrypoint.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py
test_health.py fix(tests): swap dall-e to gpt-image-1 after openai deprecation 2026-05-12 16:55:18 -07:00
test_keys.py test: harden remaining pass-through CI flakes (image-gen spend poll, ruby assistants timeout) (#30685) 2026-06-17 14:35:47 -07:00
test_litellm_proxy_responses_config.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_logging.conf
test_models.py
test_new_vector_store_endpoints.py
test_openai_endpoints.py Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847) 2026-06-06 14:33:42 -07:00
test_organizations.py
test_otel_thread_leak.py
test_passthrough_endpoints.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_resource_cleanup.py
test_service_logger_otel.py fix(langfuse): send v4 ingestion header for otel callback (#33907) 2026-07-18 20:36:51 -07:00
test_spend_logs.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_team.py
test_team_logging.py
test_team_members.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_users.py Fix: tag budget reset must drop stale management-cache entry (#27568) 2026-05-10 00:18:55 +00:00

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.