scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-12 23:01:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 182
litellm/tests/llm_translation
History
Exact
Sameer Kankute 6e6f5be3e4
feat(triton): add embedding usage estimation for self-hosted responses (#25345) 
* feat(triton): add embedding usage estimation for self-hosted responses

Populate Triton embedding usage from request input using token counting with a safe fallback so cost/observability flows work even when provider usage is missing.

Made-with: Cursor

* fix(triton): sum per-input embedding token counts for batches

Joining batch strings with newlines before token_counter added spurious
tokens. Count each input separately and sum, matching OpenAI-style usage.

Made-with: Cursor
2026-04-08 21:14:27 -07:00
..
fixtures test fix 2025-10-16 18:00:46 -07:00
realtime [Fix] Update realtime websocket tests for websockets v15+ API 2026-03-12 14:40:48 -07:00
test-skill [Feat] New API - Claude Skills API (Anthropic) (#17042) 2025-11-24 15:01:40 -08:00
test_llm_response_utils fix(types): normalize null top_logprobs to [] in ChatCompletionTokenLogprob (#22245) 2026-02-28 15:59:29 -08:00
test_skills_data Remove Apache 2 license from SKILL.md (#22322) 2026-02-27 19:33:55 -08:00
base_audio_transcription_unit_tests.py fix: use fastuuid helper (#14903) 2025-09-25 15:47:01 -07:00
base_embedding_unit_tests.py Litellm dev 12 25 2025 p2 (#7420) 2024-12-25 18:35:34 -08:00
base_llm_unit_tests.py Fix mistral ocr failing test 2026-03-09 11:29:33 +05:30
base_rerank_unit_tests.py [Feat] Add Nvidia NIM Rerank Support (#15152) 2025-10-02 18:58:52 -07:00
conftest.py [Fix] Restore unconditional importlib.reload for llm_translation conftest 2026-03-16 00:35:02 -07:00
dog.wav (feat) Support audio param in responses streaming (#6312) 2024-10-18 19:16:14 +05:30
duck.png fix vertex ai multimodal embedding translation (#9471) 2025-03-24 23:23:28 -07:00
gettysburg.wav Litellm dev 12 25 2025 p2 (#7420) 2024-12-25 18:35:34 -08:00
guinea.png fix vertex ai multimodal embedding translation (#9471) 2025-03-24 23:23:28 -07:00
log.xt Litellm dev 04 05 2025 p2 (#9774) 2025-04-07 21:02:52 -07:00
Readme.md LiteLLM Minor Fixes & Improvements (11/29/2024) (#6965) 2024-12-01 05:24:11 -08:00
test_a2a.py [Feat] Allow calling A2A agents through LiteLLM /chat/completions API (#20358) 2026-02-03 12:52:33 -08:00
test_anthropic_completion.py Merge pull request #24445 from quora/fix/missing-content-part-added 2026-03-30 17:52:40 -07:00
test_aws_base_llm.py Add support for AWS assume_role with a session token 2025-08-23 22:37:21 -07:00
test_azure_agents.py fix(azure-ai-agents): accumulate annotations from multiple text items in streaming 2026-03-18 09:04:00 +05:30
test_azure_ai.py test_completion_azure 2026-03-30 21:54:27 -07:00
test_azure_o_series.py test: test 2026-03-28 19:17:38 -07:00
test_azure_openai.py Announce April Townhall (#25021) 2026-04-02 14:10:49 -07:00
test_bedrock_agentcore.py fix(agentcore): simplify agentcore streaming (#17141) 2026-01-19 05:20:24 -08:00
test_bedrock_agents.py test: skip test with invalid arn 2025-09-09 20:35:44 -07:00
test_bedrock_anthropic_regression.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_bedrock_common_utils.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_bedrock_completion.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_bedrock_dynamic_auth_params_unit_tests.py Add comprehensive tests for AWS external ID support 2025-09-15 19:56:31 +02:00
test_bedrock_embedding.py [Fix] Bedrock Embeddings - Ensure correct aws_region is used when provided dynamically (#16547) 2025-11-12 12:38:58 -08:00
test_bedrock_govcloud.py [Test Fix] fix gov pricing tests (#25022) 2026-04-02 15:55:55 -07:00
test_bedrock_gpt_oss.py test fix 2025-10-17 11:18:29 -07:00
test_bedrock_invoke_tests.py Litellm fix update bedrock models (#24947) 2026-04-01 19:22:54 -07:00
test_bedrock_llama.py Add bedrock llama4 pricing + handle llama4 templating on bedrock invoke route (#10582) 2025-05-06 16:06:24 -07:00
test_bedrock_moonshot.py Add tests for kimi 2 bedrock model 2026-01-08 10:07:33 +05:30
test_bedrock_nova_embedding.py fix(cost-calc): use per-image pricing for Bedrock multimodal embeddings (#21646) 2026-02-20 08:51:21 -08:00
test_bedrock_nova_json.py Extract 'thinking' from nova response + Add 'drop_params' support for gpt-image-1 (#10680) 2025-05-08 23:01:38 -07:00
test_cloudflare.py test_completion_cloudflare 2026-03-30 16:40:37 -07:00
test_cohere.py Revert "Fix xdist test isolation: capture true defaults and poll instead of sleep" 2026-03-15 22:57:39 -07:00
test_containers_api.py [Feat] Containers API - add new container API file management + UI Interface (#17745) 2025-12-09 17:33:26 -08:00
test_convert_dict_to_image.py Fix None values in usage field for gpt-image-1 model responses (#11448) 2025-06-05 13:19:18 -07:00
test_databricks.py fix: test_databricks_embeddings[True] 2026-01-06 14:20:30 +05:30
test_deepgram.py Litellm dev 12 28 2024 p3 (#7464) 2024-12-28 19:18:58 -08:00
test_deepseek_completion.py test_deepseek_mock_completion 2026-01-17 16:36:42 -08:00
test_elevenlabs.py Integrate eleven labs text-to-speech (#16573) 2025-11-24 18:49:30 -08:00
test_evals_api.py fix(tests): add flaky retries to flaky CI tests (#21795) 2026-02-21 11:56:29 -08:00
test_fireworks_ai_translation.py fix: fix setup wizard 2026-03-30 09:06:31 -07:00
test_gemini.py [Fix] Update gemini-2.0-flash to gemini-2.5-flash in test_gemini integration tests 2026-03-30 13:53:38 -07:00
test_gemini_image_usage.py fix(gemini): resolve image token undercounting in usage metadata (#22608) 2026-03-04 18:52:08 -08:00
test_gigachat.py bugfix: Remove user messages merging 2026-02-03 12:58:28 +00:00
test_gpt4o_audio.py Add gemini audio input support + handle special tokens in sagemaker response (#9640) 2025-03-29 19:23:09 -07:00
test_groq.py Add grok reasoning content 2026-01-27 16:34:57 +05:30
test_hosted_vllm_embedding_e2e.py move e2e to llm translation 2026-02-17 22:33:13 +05:30
test_huggingface_chat_completion.py Revert "Revert "fix tests (#12286)"" 2025-07-03 12:08:27 -07:00
test_hyperbolic.py Revert "Litellm dev 07 21 2025 p1 (#12848)" 2025-07-22 18:28:36 -07:00
test_infinity.py [Feat] Use aiohttp transport by default - 97% lower median latency (#11097) 2025-05-23 22:55:35 -07:00
test_jina_ai.py Litellm 12 02 2024 (#6994) 2024-12-02 22:00:01 -08:00
test_lambda_ai.py [Performance] Use O(1) Set lookups for model routing (#13879) 2025-08-21 22:56:46 -07:00
test_langgraph.py [Feat] New Provider - add langgraph (#17783) 2025-12-10 12:30:35 -08:00
test_litellm_proxy_provider.py [Bug Fix] image_edit() function returns APIConnectionError with litellm_proxy - Support for both image edits and image generations (#13735) 2025-08-18 18:26:32 -07:00
test_minimax_tts.py Add support for minmax tts 2025-12-23 21:53:53 +05:30
test_mistral_api.py test: reduce mistral direct tests b/c of rate limit errors 2025-08-23 11:15:03 -07:00
test_model_cost_map_resilience.py [Stability] Investigate + fix issue where model cost map became poorly formatted (#20895) 2026-02-10 15:17:01 -08:00
test_morph.py fix morph api tests 2025-07-22 18:44:44 -07:00
test_nvidia_nim.py TestNvidiaNim 2025-12-13 16:38:11 -08:00
test_openai.py Fix _supports_reasoning_effort_level for responses bridge 2026-03-13 13:29:39 +05:30
test_openai_o1.py fix(tests): update outdated model names in o1 and gemini tests 2026-03-12 22:50:52 -07:00
test_openrouter.py fix(test): update openrouter image generation assertion for gemini-2.5-flash-image (#23070) 2026-03-07 16:52:04 -08:00
test_optional_params.py test fix us.anthropic.claude-haiku-4-5-20251001-v1:0 (#24931) 2026-04-01 11:01:03 -07:00
test_perplexity_reasoning.py Revert "Update perplexity cost tracking (#15743)" (#16345) 2025-11-06 19:00:45 -08:00
test_prompt_caching.py (feat) openai prompt caching (non streaming) - add prompt_tokens_details in usage response (#6039) 2024-10-03 23:31:10 +05:30
test_prompt_factory.py [Infra] Merge internal dev branch with main (#25036) 2026-04-02 16:38:01 -07:00
test_replicate.py Add all replicate models in model cost map 2026-01-12 15:19:49 +05:30
test_rerank.py refactor: refactor testing 2026-03-28 18:39:32 -07:00
test_router_llm_translation_tests.py test: test 2026-03-28 19:17:38 -07:00
test_sambanova_chat_transformation.py [Bug fix] Fixes SambaNova API rejecting requests when message content is passed as a list format (#16612) 2025-11-13 17:03:14 -08:00
test_skills_api.py Reapply "feat: add model_cost aliases expansion support" 2026-03-12 13:36:57 -03:00
test_skills_e2e.py test_slack_gif_skill_creates_gif 2025-12-20 21:35:25 +05:30
test_snowflake.py Merge main and resolve Snowflake test conflict 2026-03-30 18:06:37 -07:00
test_text_completion.py Add inference providers support for Hugging Face (#8258) (#9738) (#9773) 2025-04-05 10:50:15 -07:00
test_text_completion_unit_tests.py fix(openai): use optimized async http client for text completions (#17831) 2025-12-11 14:54:26 -08:00
test_together_ai.py [Feat] Use aiohttp transport by default - 97% lower median latency (#11097) 2025-05-23 22:55:35 -07:00
test_triton.py feat(triton): add embedding usage estimation for self-hosted responses (#25345) 2026-04-08 21:14:27 -07:00
test_unit_test_bedrock_invoke.py [Oct Staging Branch] (#15460) 2025-10-17 17:52:25 -07:00
test_v0.py feat: add v0 provider support (#12751) 2025-07-18 18:26:44 -07:00
test_voyage_ai.py Add support for voyage-context-3 embedding model 2025-08-22 00:15:12 +05:30
test_watsonx.py [Fix] WatsonX tests failing on CI due to missing env vars 2026-03-30 13:27:03 -07:00
test_xai.py Fix usage in xai 2026-02-19 18:48:30 +05:30

Readme.md

Unit tests for individual LLM providers.

Name of the test file is the name of the LLM provider - e.g. test_openai.py is for OpenAI.