litellm/litellm/litellm_core_utils
2026-07-14 14:04:51 -07:00
..
audio_utils feat(vertex_ai): add Google Cloud Speech-to-Text Chirp 3 transcription support (#32274) 2026-07-06 18:25:22 -07:00
llm_cost_calc feat(pricing): add gemini-omni-flash-preview with video output token pricing 2026-07-14 14:04:51 -07:00
llm_response_utils chore(lint): zero out crash-class pyright rules and ban new type: ignore comments (#32152) 2026-07-04 16:56:12 -07:00
prompt_templates feat(router): add Router(plugins=[...]) routing-plugin pipeline (#32972) 2026-07-11 21:38:18 -07:00
specialty_caches style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
tokenizers Code Quality Improvement - remove tokenizers/ from /llms (#7163) 2024-12-10 23:50:15 -08:00
api_route_to_call_types.py Agent Guardrails - on streaming output (#21206) 2026-02-14 11:36:52 -08:00
app_crypto.py style: run black formatter on entire codebase 2026-03-11 17:07:57 -03:00
asyncify.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
cached_imports.py style: run black formatter on entire codebase 2026-03-11 17:07:57 -03:00
chat_completion_agentic_loop.py fix(websearch): websearch_interception agentic loop fixes for chat completions and anthropic messages (#31669) 2026-07-01 09:36:40 +08:00
cli_token_utils.py feat: add lite auth print-token for Claude Code apiKeyHelper support (#32846) 2026-07-11 13:31:41 -07:00
cloud_storage_security.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
completion_timeout.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
core_helpers.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
coroutine_checker.py style: run black formatter on entire codebase 2026-03-11 17:07:57 -03:00
credential_accessor.py fix(router.py): support reusable credentials via passthrough router (#9758) 2025-04-04 18:40:14 -07:00
custom_logger_registry.py feat: litellm oss 110626 (#30202) 2026-06-11 22:30:26 -07:00
dd_tracing.py _add_dd_apm_tags_for_litellm_call_id (#22219) 2026-02-26 16:42:23 -08:00
default_encoding.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
dot_notation_indexing.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
duration_parser.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
env_utils.py fix(callbacks): allow MAX_CALLBACKS override via env var (#20781) 2026-02-09 12:11:32 -08:00
exception_mapping_utils.py fix(mcp): fail closed and surface semantic filter context window errors 2026-07-10 00:41:54 -07:00
fallback_generalizations.py fix(fallback-generalizations): tolerate legacy remote rule schema and keep register_model cache-pricing inheritance 2026-07-11 00:27:45 -07:00
fallback_utils.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
get_blog_posts.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
get_litellm_params.py fix(completion): forward aws credential kwargs into litellm_params so the responses bridge keeps WIF auth 2026-07-11 16:10:39 -07:00
get_llm_provider_logic.py refactor(fallback-generalizations): split rules into routing and provider-neutral capability kinds 2026-07-11 00:27:45 -07:00
get_model_cost_map.py feat: declarative fallback generalizations for unknown models (#29718) 2026-06-27 21:01:19 -07:00
get_provider_specific_headers.py [Fix] Forward anthropic-beta headers to Bedrock, VertexAI (#15700) 2025-10-18 16:26:32 -07:00
get_supported_openai_params.py feat(tencent): add Tencent TokenHub as a provider (#31903) 2026-07-02 18:31:59 -07:00
health_check_helpers.py fix(vertex): forward realtime health check params (#32550) 2026-07-08 17:30:46 -07:00
health_check_utils.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
initialize_dynamic_callback_params.py fix(proxy): restore admin key/team callback_vars.turn_off_message_logging override (LIT-3587) (#31905) 2026-07-02 15:01:58 -07:00
json_validation_rule.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
litellm_logging.py feat(prometheus): expose video duration and image count consumption metrics (#33138) 2026-07-13 18:51:13 -07:00
llm_request_utils.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
logging_callback_manager.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
logging_utils.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
logging_worker.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
mock_functions.py Support returning virtual key in custom auth + Handle provider-specific optional params for embedding calls (#11346) 2025-06-03 07:24:13 -07:00
model_param_helper.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
model_response_utils.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
README.md Guardrails API - add streaming support (#17400) 2025-12-02 22:52:09 -08:00
realtime_streaming.py fix(responses): stop scheduling sync success_handler concurrently with async_success_handler (#32239) 2026-07-07 09:13:50 -07:00
reasoning_effort_utils.py fix(bedrock): normalize Messages system role and adaptive-thinking for Claude Invoke (#31364) 2026-06-27 11:35:36 -07:00
redact_messages.py fix: redact async complete streaming response for custom callbacks (#33106) 2026-07-13 13:35:14 -07:00
request_timeout_resolver.py fix(router): honor litellm_settings.request_timeout as an independent per-attempt timeout (#31119) 2026-06-23 14:22:54 -07:00
response_header_helpers.py fix(utils.py): guarantee openai-compatible headers always exist in response 2024-09-28 21:08:15 -07:00
rules.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00
safe_json_dumps.py perf(spend-logs): only strip NUL bytes in safe_dumps when present (#31424) 2026-06-26 09:37:19 -07:00
safe_json_loads.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
secret_redaction.py Litellm oss staging (#29492) 2026-06-02 08:48:10 -07:00
sensitive_data_masker.py fix(guardrails): mask credentials embedded in guardrail_response before persist (LIT-4314) (#32687) 2026-07-09 20:26:22 -07:00
streaming_chunk_builder_utils.py fix(anthropic): bill streaming 1h prompt-cache writes at the 1h rate (#32073) 2026-07-03 17:34:16 -07:00
streaming_handler.py chore(lint): zero out crash-class pyright rules and ban new type: ignore comments (#32152) 2026-07-04 16:56:12 -07:00
thread_pool_executor.py (Fixes) OpenAI Streaming Token Counting + Fixes usage track when litellm.turn_off_message_logging=True (#8156) 2025-01-31 15:06:37 -08:00
token_counter.py fix(token_counter): count legacy function_call.arguments (VERIA-492) (#31741) 2026-06-30 14:23:09 -07:00
url_utils.py style: unify ruff format width on 120 (#31518) 2026-06-27 12:39:29 -07:00

Folder Contents

This folder contains general-purpose utilities that are used in multiple places in the codebase.

Core files:

  • streaming_handler.py: The core streaming logic + streaming related helper utils
  • core_helpers.py: code used in types/ - e.g. map_finish_reason.
  • exception_mapping_utils.py: utils for mapping exceptions to openai-compatible error types.
  • default_encoding.py: code for loading the default encoding (tiktoken)
  • get_llm_provider_logic.py: code for inferring the LLM provider from a given model name.
  • duration_parser.py: code for parsing durations - e.g. "1d", "1mo", "10s"
  • api_route_to_call_types.py: mapping of API routes to their corresponding CallTypes (e.g., /chat/completions -> [acompletion, completion])