scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-07 07:58:09 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 150
litellm/tests
History
jaideeppyne ab78459097 fix(cohere): handle multimodal inputs in token counting and guardrails 
Response transformation crashed with TypeError when counting tokens for
multimodal (dict-shaped) embedding inputs, since encoding.encode() was
called directly on the dict instead of its text parts. Also removed a
dead, equally-buggy duplicate of the same loop in _transform_response.

Separately, the embeddings guardrail handler silently skipped multimodal
dict inputs instead of scanning them, letting prohibited text/image
content bypass the configured guardrail. It now extracts and scans the
text parts while leaving image parts untouched.
2026-08-13 09:01:47 +05:30
..
agent_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
audio_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
base_sdk_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
basic_proxy_startup_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
batches_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
benchmarks fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
code_coverage_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
documentation_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
e2e fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
enterprise fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
guardrails_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
image_gen_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
integration fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
litellm fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
litellm-proxy-extras fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
litellm_core_utils fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
litellm_utils_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
llm_responses_api_testing fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
llm_translation fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
load_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
local_testing fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
logging_callback_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
mcp_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
multi_instance_e2e_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
ocr_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
old_proxy_tests/tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
openai_endpoints_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
otel_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
pass_through_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
pass_through_unit_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
proxy_admin_ui_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
proxy_behavior fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
proxy_e2e_anthropic_messages_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
proxy_migration_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
proxy_security_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
proxy_unit_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
router_unit_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
scim_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
search_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
spend_tracking_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
store_model_in_db_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_litellm fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
unified_google_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
vector_store_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
windows_tests fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
__init__.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
_fake_openai_endpoint_server.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
_flush_vcr_cache.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
_live_test_helpers.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
_openai_record_replay_proxy.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
_vcr_conftest_common.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
_vcr_redis_persister.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
_ws_vcr.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
eval_swe_bench.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
fake_openai_endpoint.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
gettysburg.wav fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
large_text.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
openai_batch_completions.jsonl fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
pyrightconfig.json fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
README.MD fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_anthropic_compaction_usage.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_budget_management.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_callbacks_on_proxy.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_debug_warning.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_default_encoding_non_root.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_end_users.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_fallbacks.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_gpt5_azure_temperature_support.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_health.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_keys.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_litellm_proxy_responses_config.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_logging.conf fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_models.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_new_vector_store_endpoints.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_openai_endpoints.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_organizations.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_otel_thread_leak.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_presidio_latency.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_proxy_server_non_root.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_ratelimit.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_resource_cleanup.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_service_logger_otel.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_spend_logs.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_team.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_team_logging.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_team_members.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30
test_users.py fix(cohere): handle multimodal inputs in token counting and guardrails 2026-08-13 09:01:47 +05:30

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.