scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-12 03:48:56 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 57
litellm/tests
History
Shubham Jain 985574f3dc
fix(bedrock): keep GPT tool-result images beside the tool result (#45579) 
* fix(bedrock): keep GPT tool-result images beside the tool result

Bedrock gpt-6.1-sol rejects an image nested in toolResult.content. Put it on the same user turn instead.

* fix(bedrock): read tool-result image support from the price map

OpenAI GPT rows on Bedrock Converse reject an image inside toolResult.content, so those images sit beside the tool result.

* fix: sync price-map test schema and import order

CI validates model_prices_and_context_window.json against an inline schema in test_utils, and ruff checks import order in converse_transformation.

* fix(bedrock): satisfy basedpyright in tool-result image helper

Drop Final assignments inside loops and use TypedDict key checks so the lint type-check gate stays within its merge-base ceiling.

* test(bedrock): cover tool-result image placement helper

Exercise Claude no-op paths, text-only GPT tool results, status preservation, and split text/image tool content for codecov.

* refactor(bedrock): reuse the shared tool-result image placeholder text

---------

Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
2026-10-09 18:02:23 -07:00
..
_support
agent_tests/local_only_agent_tests test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
audio_tests test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
base_sdk_tests refactor(sdk): separate core AWS and tokenizer dependencies (#44447) 2026-10-08 19:07:57 -07:00
basic_proxy_startup_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
batches_tests test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
benchmarks refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
code_coverage_tests chore(greptile): load nested AGENTS.md files as review context scoped to their directories (#45636) 2026-10-09 13:22:29 -07:00
documentation_tests
e2e fix(bedrock): keep tool_reference results as text on converse (#44590) 2026-10-09 17:52:17 -07:00
e2e_harness docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
guardrails_tests test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
harness_e2e feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
image_gen_tests test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
integration fix(bedrock): keep tool_reference results as text on converse (#44590) 2026-10-09 17:52:17 -07:00
litellm_utils_tests test: move offline proxy auth, hook, spend and pass-through tests to tests/unit (#45568) 2026-10-09 09:49:13 -07:00
llm_translation test: fix stale and flaky tests across CircleCI, GHA and Buildkite (#45530) 2026-10-09 11:40:14 -07:00
load_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
local_testing test: move offline anthropic prompt caching tests to tests/unit (#45616) 2026-10-09 10:53:08 -07:00
logging_callback_tests test: move offline proxy auth, hook, spend and pass-through tests to tests/unit (#45568) 2026-10-09 09:49:13 -07:00
mcp_tests feat(mcp): add portable catalog pagination (#44446) 2026-10-06 18:54:50 -07:00
multi_instance_e2e_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
ocr_tests test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
openai_endpoints_tests test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
otel_tests test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
pass_through_tests test: delete 34 legacy tests already covered by e2e (#45340) 2026-10-08 09:43:56 -07:00
pass_through_unit_tests test: move offline proxy auth, hook, spend and pass-through tests to tests/unit (#45568) 2026-10-09 09:49:13 -07:00
proxy_admin_ui_tests test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
proxy_behavior refactor(lens): connect LiteLLM to the independent Lens service (#45529) 2026-10-09 16:17:05 -07:00
proxy_e2e_anthropic_messages_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
proxy_migration_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
proxy_security_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
proxy_unit_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
router_unit_tests test: make 77 legacy live tests offline in litellm_utils, router_unit and responses dirs (#45298) 2026-10-08 02:37:55 -07:00
rust-python-harness docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
search_tests test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
store_model_in_db_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
test_litellm refactor(lens): connect LiteLLM to the independent Lens service (#45529) 2026-10-09 16:17:05 -07:00
test_litellm_rust feat(rust): type the Vertex AI connection params and where Python reads them (#45641) 2026-10-09 16:29:44 -07:00
unified_google_tests test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
unit fix(bedrock): keep GPT tool-result images beside the tool result (#45579) 2026-10-09 18:02:23 -07:00
vector_store_tests refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
windows_tests
__init__.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py
_master_key.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
_openai_record_replay_proxy.py
_process_helpers.py
_vcr_conftest_common.py
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
AGENTS.md docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
capturing_transport.py
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD
test_debug_warning.py
test_keys.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_litellm_proxy_responses_config.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_logging.conf
test_models.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_new_vector_store_endpoints.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_openai_endpoints.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_proxy_server_non_root.py
test_rust_python_harness.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_service_logger_otel.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_spend_logs.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_team_logging.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
test_users.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
white_100x100.png

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/unit

This folder can only run mock tests.