scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-06 08:16:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 178
litellm/tests
History
Lucas Rothman b981fddfaa fix(gemini): properly catch context window exceeded errors 
Fixes #18282

This PR fixes two issues with Gemini context window error handling:

1. **Pattern matching for Gemini 2.0 Flash**: The previous pattern
   'input token count exceeds the maximum number of tokens allowed'
   doesn't match Gemini 2.0 Flash errors which include dynamic token
   counts like '(2800010)' in the message. Split into shorter patterns
   that work with both formats.

2. **Add context window check to Gemini block**: The
   is_error_str_context_window_exceeded() check was only called for
   OpenAI-compatible providers, not for Gemini/Vertex AI. Added the
   check to the Gemini-specific error handling block.

Test cases added for both Gemini 2.0 Flash and 2.5/3 error formats.
2025-12-19 23:20:04 -08:00
..
agent_tests [Fix] Security - Remove example API keys with high entropy (#18255) 2025-12-19 10:09:50 -08:00
audio_tests test_runwayml_tts_async 2025-12-13 16:10:48 -08:00
basic_proxy_startup_tests Fix: remove merge markdown (#17586) 2025-12-06 05:38:16 -08:00
batches_tests Add output file id in managed objects for batches 2025-12-16 15:52:07 +05:30
code_coverage_tests code qa check 2025-12-19 19:00:44 +05:30
documentation_tests Litellm docs readme fixes (#16107) 2025-10-30 17:05:32 -07:00
enterprise added extraction of top level metadata for custom lables in prometheus callbacks (#18087) 2025-12-17 11:17:12 +05:30
guardrails_tests [Feat] Guardrails Load Balancing - Allow Platform admins to load balance between guardrails (#18181) 2025-12-19 00:08:03 +05:30
image_gen_tests fix lint error 2025-12-19 23:45:37 +05:30
litellm Merge branch 'main' into litellm_staging_12_16_2025 2025-12-16 21:38:14 +05:30
litellm-proxy-extras Fix: Distinguish permission errors from idempotent errors in Prisma migrations (#17064) 2025-11-24 20:41:44 -08:00
litellm_utils_tests [Fix] Security - Remove example API keys with high entropy (#18255) 2025-12-19 10:09:50 -08:00
llm_responses_api_testing Revert "[Fix] CI/CD - local_testing & mapped tests (#18222)" (#18223) 2025-12-18 14:46:09 -08:00
llm_translation [Fix] Security - Remove example API keys with high entropy (#18255) 2025-12-19 10:09:50 -08:00
load_tests test: test_embedding_performance 2025-05-14 21:31:07 -07:00
local_testing [Fix] Security - Remove example API keys with high entropy (#18255) 2025-12-19 10:09:50 -08:00
logging_callback_tests [Fix] Security - Remove example API keys with high entropy (#18255) 2025-12-19 10:09:50 -08:00
mcp_tests feat: add support for using MCPs on /chat/completions 2025-12-10 10:55:33 +09:00
multi_instance_e2e_tests fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
ocr_tests vertex ai deepseek doesn't support pdf urls, so skip them 2025-12-19 23:53:17 +05:30
old_proxy_tests/tests [Fix] Security - Remove example API keys with high entropy (#18255) 2025-12-19 10:09:50 -08:00
openai_endpoints_tests test_e2e_batches_files 2025-11-27 09:14:43 -08:00
otel_tests [Fix] Security - Remove example API keys with high entropy (#18255) 2025-12-19 10:09:50 -08:00
pass_through_tests fix: use vllm passthrough config for hosted vllm provider instead of raising error (#16537) 2025-11-12 09:12:14 -08:00
pass_through_unit_tests [Fix] Security - Remove example API keys with high entropy (#18255) 2025-12-19 10:09:50 -08:00
proxy_admin_ui_tests Merge remote-tracking branch 'origin' into litellm_new_user_collision_handling 2025-12-06 19:58:35 -08:00
proxy_security_tests test fix 2025-10-04 10:57:02 -07:00
proxy_unit_tests Merge pull request #18236 from BerriAI/litellm_add_background_responses_cost_tracking 2025-12-20 00:14:09 +05:30
router_unit_tests fix(router): add minimum request threshold for error rate cooldown (#17464) 2025-12-12 04:36:10 -08:00
scim_tests [Feat SSO] Add LiteLLM SCIM Integration for Team and User management (#10072) 2025-04-16 19:21:47 -07:00
search_tests [Feat] New Search API Provider - LinkUp Search (#18174) 2025-12-18 14:27:36 +05:30
spend_tracking_tests fix ocr test 2025-10-31 20:32:03 -07:00
store_model_in_db_tests test_edit_mcp_server_redacts_credentials 2025-11-26 11:54:11 -08:00
test_litellm fix(gemini): properly catch context window exceeded errors 2025-12-19 23:20:04 -08:00
unified_google_tests fix: gooogle GenAI route tests 2025-10-04 10:18:25 -07:00
vector_store_tests [Feat] Add new Rag Search API / Query API with rerankers (#18217) 2025-12-19 19:05:07 +05:30
windows_tests [Bug Fix] UnicodeDecodeError: 'charmap' on Windows during litellm import (#10542) 2025-05-03 21:31:05 -07:00
__init__.py [Feat] Add github co-pilot as a new LLM API provider (#12325) 2025-07-04 13:12:16 -07:00
gettysburg.wav
large_text.py
openai_batch_completions.jsonl feat(router.py): Support Loadbalancing batch azure api endpoints (#5469) 2024-09-02 21:32:55 -07:00
README.MD [Feat] MCP Gateway Fine-grained Tools Addition (#15153) 2025-10-03 10:16:29 -07:00
test_budget_management.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_callbacks_on_proxy.py [Fix] Security - Remove example API keys with high entropy (#18255) 2025-12-19 10:09:50 -08:00
test_config.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_debug_warning.py fix(utils.py): fix togetherai streaming cost calculation 2024-08-01 15:03:08 -07:00
test_end_users.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_entrypoint.py (fix) clean up root repo - move entrypoint.sh and build_admin_ui to /docker (#6110) 2024-10-08 11:34:43 +05:30
test_fallbacks.py Ollama Chat - parse tool calls on streaming (#11171) 2025-05-27 16:14:49 -07:00
test_gpt5_azure_temperature_support.py Fix: Azure GPT-5 incorrectly routed to O-series config (temperature parameter unsupported) (#16246) 2025-11-07 10:24:27 -08:00
test_health.py
test_keys.py test: temporarily skip test due to change testing model change - need to update test for new model 2025-05-09 09:02:08 -07:00
test_litellm_proxy_responses_config.py Add native Responses API support for litellm_proxy provider (#15347) 2025-10-08 18:31:26 -07:00
test_logging.conf feat(proxy_cli.py): add new 'log_config' cli param (#6352) 2024-10-21 21:25:58 -07:00
test_models.py test_add_model_run_health 2025-09-27 10:59:25 -07:00
test_openai_endpoints.py test fix claude-sonnet-4-5-20250929 2025-10-28 19:05:13 -07:00
test_organizations.py UI - fix adding vertex models with reusable credentials + fix pagination on keys table + fix showing org budgets on table (#10528) 2025-05-03 08:16:53 -07:00
test_passthrough_endpoints.py fix: update authorization header to use 'Bearer' instead of 'bearer' 2025-09-21 10:44:47 +00:00
test_ratelimit.py test_async_rate_limit 2025-11-15 10:34:24 -08:00
test_resource_cleanup.py Fix: Properly close aiohttp client sessions to prevent resource leaks (#12251) 2025-07-09 09:25:17 -07:00
test_spend_logs.py [Fix] Security - Remove example API keys with high entropy (#18255) 2025-12-19 10:09:50 -08:00
test_team.py [Fix] CI/CD - Add missing documentation for ENVs & Clean Functionality (#18211) 2025-12-18 13:16:15 -08:00
test_team_logging.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_team_members.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
test_users.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.