scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-17 23:51:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 66
litellm/tests/test_litellm/llms
History
Yucheng Zhu 0e6c4279b6 fix(guardrails): record not_run evaluation when scoping leaves nothing to scan 
Replaces the metadata transfer approach: that block read top-level
litellm_metadata which guardrail info never populates, and the SLP
builder already reads the nested bucket it lands in, so it was dead
code and is reverted.

Real cause of missing evaluations: process_input_messages skips
apply_guardrail entirely when message scoping (skip_system_message,
skip_tool, scan_only_tool_results) leaves no scannable content, so
the guardrail shows up in applied_guardrails with no
guardrail_information entry. Now records a not_run entry unless the
guardrail records its own information.
2026-08-31 21:35:32 -07:00
..
a2a/chat fix(a2a): populate response usage in a2a chat transformation (#31980) 2026-07-03 09:28:36 +05:30
aiml/image_generation test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
amazon_nova/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
anthropic Merge pull request #36008 from nuernber/litellm_bedrock_messages_disconnect_billing 2026-08-31 16:54:41 -07:00
apiserpent test: unwind environment writes in tests/test_litellm with monkeypatch (#37806) 2026-08-21 20:28:37 -07:00
azure Merge pull request #38837 from BerriAI/litellm_fix_azure_responses_anyof_tool_schema 2026-08-29 16:46:22 -07:00
azure_ai Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_model_router_spend_log_model 2026-08-25 10:55:53 -07:00
base_llm Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_keyless_key_managed_resource_owner 2026-08-29 11:33:08 -07:00
baseten/chat style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
bedrock Merge pull request #36008 from nuernber/litellm_bedrock_messages_disconnect_billing 2026-08-31 16:54:41 -07:00
bedrock_mantle fix(model_prices): add bedrock_mantle gpt-5.5/5.4 272K tiers, align sol with AWS invoice 2026-08-27 19:31:45 -07:00
black_forest_labs test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
bytez/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
cerebras fix(cerebras): add max_retries and extra_headers to get_supported_openai_params (#36601) 2026-08-25 14:10:56 -07:00
chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
chatgpt test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
cloudflare fix(cloudflare): route native Workers AI provider through OpenAI-compatible endpoint (#31053) 2026-06-23 12:09:11 -07:00
cohere test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
cometapi/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
compactifai fix(exceptions): map upstream status codes for providers with no exception_type branch 2026-08-26 00:47:28 -07:00
crusoe test: unwind environment writes in tests/test_litellm with monkeypatch (#37806) 2026-08-21 20:28:37 -07:00
custom_httpx fix(proxy): emit timing headers and overhead for /v1/messages and /v1/responses (#38840) 2026-08-29 18:11:58 -07:00
dashscope test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
databricks Merge branch 'litellm_internal_staging' into fix_databricks_oauth_url 2026-08-29 07:00:58 -07:00
datarobot test: unwind environment writes in tests/test_litellm with monkeypatch (#37806) 2026-08-21 20:28:37 -07:00
deepgram test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
deepinfra test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
deepseek fix: collapse image_url blocks whose payload lacks a url instead of forwarding them 2026-08-26 13:19:06 -07:00
docker_model_runner test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
elevenlabs test: avoid provider test module name collision 2026-04-29 22:36:57 -07:00
fal_ai test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
fastcrw/search chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
featherless_ai/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
fireworks_ai fix(streaming): preserve provider model for cost calculation 2026-08-27 23:26:00 -05:00
gdc/chat Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_opus5 2026-08-29 06:03:33 -07:00
gemini fix(registry): correct xai retired slug pricing, bedrock grok caching, and unsourced entries 2026-08-29 13:24:09 -07:00
gigachat fix(gigachat): fold cached tokens back into prompt and total token counts 2026-08-31 15:16:44 -07:00
github_copilot test(realtime): pin mode-only transcription detection and correct a stale docstring 2026-08-26 12:33:46 -07:00
gradient_ai test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
groq fix: omit thinking.type=disabled for always-on thinking Claude models (#37510) 2026-08-21 10:27:26 -07:00
heroku style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
hosted_vllm fix(hosted_vllm): forward Omni media URLs instead of downloading them 2026-08-24 17:59:52 -04:00
huggingface fix: bound Hugging Face config fetch and keep embedding tests off the network 2026-08-29 09:45:52 +00:00
inception test: unwind environment writes in tests/test_litellm with monkeypatch (#37806) 2026-08-21 20:28:37 -07:00
jina_ai/embedding test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
langflow fix(exceptions): map upstream status codes for providers with no exception_type branch 2026-08-26 00:47:28 -07:00
lemonade test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
litellm_proxy test: close mutation-testing gaps in container, skills and openai-like config factories 2026-08-28 09:28:14 -07:00
llamafile/chat style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
lm_studio style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
manus test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
meta_llama test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
minimax fix(minimax): attach MINIMAX_API_KEY on anthropic messages requests 2026-08-26 12:19:37 -07:00
mistral test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
modelscope test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
moonshot fix(moonshot, together_ai): send the reasoning effort Kimi K3 accepts (#38611) 2026-08-27 20:46:46 -07:00
nebius test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
neosantara Litellm oss staging 050626 (#29774) 2026-06-05 13:51:51 -07:00
nimble/search feat(search): add Nimble as a search provider (#36347) 2026-08-14 17:09:58 -07:00
novita/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
nscale/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
nvidia_nim/rerank test: drop rerank package marker colliding with voyage test package 2026-08-11 23:37:27 -07:00
nvidia_riva test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
oci test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
ocr test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
ollama Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_ruff_dead_test_code 2026-08-24 09:46:56 -07:00
oobabooga/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
openai fix(guardrails): record not_run evaluation when scoping leaves nothing to scan 2026-08-31 21:35:32 -07:00
openai_like fix(anthropic): cap reasoning_effort thinking budget below max_tokens on /v1/messages 2026-08-29 15:24:05 -07:00
openrouter test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
ovhcloud test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
parallel_ai test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
parasail Litellm oss staging 080626 (#29932) 2026-06-08 13:49:52 -07:00
pass_through test: remove four mirror test files that exercise none of their module 2026-07-25 10:57:19 -07:00
perplexity test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
pg_vector/vector_stores test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
publicai test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
ragflow/chat test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
recraft test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
reducto Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
runwayml fix(runwayml): skip progress scaling when Runway returns a null progress 2026-08-24 12:20:00 -07:00
s3_vectors Fix extra body error 2026-04-29 08:34:31 +05:30
sagemaker Merge pull request #38747 from BerriAI/litellm_aws_partition_helper 2026-08-29 04:10:59 -07:00
sambanova Feat/sambanova embeddings (#13308) 2025-08-12 17:15:26 -07:00
sap style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
scaleway Feature/add audio support for scaleway (#26110) 2026-04-20 14:49:41 -07:00
snowflake chore(oss): litellm oss staging 120626 (#30292) 2026-06-12 09:49:25 -07:00
soniox Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_soniox_srt_cue_grouping 2026-08-29 12:02:21 -07:00
stability test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
tencent test(tencent): drive thinking tests off the real cost map instead of patched internals 2026-08-26 11:34:06 -07:00
tinyfish test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
together_ai/chat fix(moonshot, together_ai): send the reasoning effort Kimi K3 accepts (#38611) 2026-08-27 20:46:46 -07:00
valkey/vector_stores feat(vector_stores): add Valkey as a managed vector store provider (#37002) 2026-08-18 21:45:22 +00:00
vercel_ai_gateway test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
vertex_ai fix(vertex_ai): graft default vertex path when api_base has a version-only path (#38986) 2026-08-31 12:56:34 -07:00
volcengine test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
voyage test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769) 2026-08-20 20:24:49 -07:00
wandb test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
watsonx Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_ruff_dead_test_code 2026-08-24 09:46:56 -07:00
xai fix(registry): correct xai retired slug pricing, bedrock grok caching, and unsourced entries 2026-08-29 13:24:09 -07:00
you_com test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
zai test: stop the zai tests from leaking env and litellm globals into the session (#37834) 2026-08-21 22:10:34 -07:00
test_cache_control_and_reasoning.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_file_content_block.py Litellm oss staging (#28161) 2026-05-18 16:27:44 -07:00
test_file_search_responses.py fix(tests): use canonical litellm_enterprise import path (#27699) 2026-05-12 12:32:57 -07:00
test_lifecycle_fix.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_polling_url_origin_match.py chore(security): close two unaddressed SSRF cases 2026-05-01 18:43:47 +00:00
test_predibase_transformation.py refactor(predibase): migrate transform_request and transform_response… (#25249) 2026-04-27 08:58:22 +05:30