litellm/litellm/litellm_core_utils
yucheng-berri 2ec5c2c7cd
Merge pull request #41740 from BerriAI/litellm_otel_v2_langfuse_llm_spans_only
feat(otel v2): opt-in llm_only span scope for Langfuse destinations and the operator Langfuse exporter
2026-09-19 17:24:50 -07:00
..
audio_utils fix(audio_utils): validate MPEG frame headers before labeling sniffed audio 2026-08-29 14:20:40 -07:00
llm_cost_calc fix(cost_calc): coerce string fireworks rates and drop the match fall-through 2026-09-19 01:33:41 +00:00
llm_response_utils Merge pull request #41685 from BerriAI/litellm_prompt_injection_llm_api_check_dispatch 2026-09-18 23:53:14 -07:00
prompt_templates Merge pull request #41275 from BerriAI/litellm_azure_strip_file_format 2026-09-18 11:46:23 -07:00
specialty_caches feat(newrelic): per-team cost and usage metrics via team callbacks (#37610) 2026-08-26 23:42:02 -07:00
tokenizers
agentic_loop_settings.py fix: keep accepting a loop ceiling that spells a whole number 2026-08-22 11:24:22 -07:00
api_route_to_call_types.py fix(guardrails): resolve generateContent routes and async-first passthrough call types 2026-08-29 21:35:19 -07:00
app_crypto.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
asyncify.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
aws_partition.py fix(aws): build every AWS endpoint and ARN from the region partition 2026-08-29 01:21:59 -07:00
cached_imports.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
chat_completion_agentic_loop.py refactor(agentic-loop): narrow logging_obj before building the fake stream wrapper 2026-08-27 23:19:17 +00:00
classifier_logging.py fix(router): merge staging and retain native classifier audits 2026-09-10 14:24:56 -07:00
cli_keyring.py fix(cli): stop asking a keychain that already stopped answering 2026-08-20 04:12:23 -07:00
cli_token_utils.py fix(cli): keep the refresh token in the OS keychain, not in token.json 2026-08-20 11:43:53 -07:00
cloud_storage_security.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
completion_timeout.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
core_helpers.py fix(gemini): map every documented finishReason and reset per-candidate state 2026-09-18 14:54:18 -07:00
coroutine_checker.py fix lint review feedback 2026-09-14 15:09:53 +08:00
credential_accessor.py fix(proxy): load db credentials in the model reconcile so a worker never serves a model before its credential (#39876) 2026-09-08 10:08:24 -07:00
custom_logger_registry.py feat(pointfive): register the pointfive callback 2026-09-10 14:02:35 +03:00
dd_tracing.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
default_encoding.py feat(rust): count tiktoken cl100k_base admission tokens in Rust (#40777) 2026-09-11 23:47:05 +00:00
dot_notation_indexing.py refactor(typing): replace Any with proven types in 89 more backend files 2026-09-02 23:08:42 +00:00
duration_parser.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
env_utils.py fix(token_counter): stop large token counts from blocking the proxy event loop (#37697) 2026-08-20 16:09:07 -07:00
exception_mapping_utils.py fix(responses): keep upstream error details in response.failed 2026-09-18 13:58:53 -07:00
fallback_generalizations.py refactor(model_info): drop types import from fallback_generalizations 2026-09-14 18:58:53 +00:00
fallback_utils.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
get_blog_posts.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
get_litellm_params.py Merge pull request #41138 from BerriAI/litellm_bedrock_files_s3_endpoint_url 2026-09-18 15:04:01 -07:00
get_llm_provider_logic.py fix(spend_tracking): honour the global litellm_proxy override when inferring a model group provider 2026-09-17 00:03:01 +00:00
get_model_cost_map.py Merge pull request #40444 from BerriAI/litellm_annotate_strict_budget_helpers 2026-09-09 13:11:05 -07:00
get_provider_specific_headers.py fix(proxy): stop forwarding a client Anthropic OAuth token to Bedrock and Vertex 2026-08-21 18:02:40 -07:00
get_supported_openai_params.py fix(vertex_ai): keep reasoning_effort unsupported on Vertex AI Mistral partner models 2026-09-18 13:16:56 -07:00
health_check_helpers.py fix(ocr): send each provider a health-check document it accepts 2026-09-04 22:52:12 -07:00
health_check_utils.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
initialize_dynamic_callback_params.py feat(otel v2): opt-in llm_only span scope for Langfuse destinations and the operator Langfuse exporter 2026-09-18 02:47:29 +00:00
internal_call_metadata.py fix(auto_router): bill the routing embedding to the caller's key and team (#39532) 2026-09-03 13:53:30 -07:00
json_fragment_accumulator.py perf(streaming): add shared JSONFragmentAccumulator for Vertex and Anthropic (#36610) 2026-08-25 16:15:47 -07:00
json_validation_rule.py refactor(typing): replace Any with proven types in 89 more backend files 2026-09-02 23:08:42 +00:00
litellm_logging.py Merge pull request #41177 from BerriAI/litellm_autorouter_baseline_cache 2026-09-19 16:16:29 -07:00
llm_judge.py fix(shadow_eval): refuse a judge model that also serves one of the arms it grades (#38589) 2026-08-27 18:44:44 -07:00
llm_request_utils.py fix(hosted_vllm): forward Omni media URLs instead of downloading them 2026-08-24 17:59:52 -04:00
logging_callback_manager.py fix(proxy): surface runtime-registered callbacks in UI Logging page (#38974) 2026-09-09 22:03:16 -07:00
logging_utils.py fix(timing): union provider timing windows and anchor detailed pre-processing at receive time 2026-09-19 00:48:17 +00:00
logging_worker.py style(logging): drop explanatory comments per repo no-comments policy 2026-09-12 17:03:40 -07:00
mock_functions.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
model_param_helper.py merge litellm_internal_staging into litellm_anthropic_messages_response_cache 2026-08-11 20:48:41 +00:00
model_response_utils.py chore(techdebt): clear fresh debt from the 2026-08-29 window 2026-08-30 07:57:16 +00:00
private_json.py feat(auto-router): show routed model and savings in Claude Code and Codex (#40330) 2026-09-11 12:52:42 -07:00
ptu_pricing.py feat(spend-logs): record Azure spillover source deployment in spend log metadata 2026-09-17 05:48:51 +00:00
README.md Guardrails API - add streaming support (#17400) 2025-12-02 22:52:09 -08:00
realtime_errors.py fix(realtime): relay the upstream websocket close to the client instead of hanging 2026-09-04 19:25:12 -07:00
realtime_streaming.py feat(vertex_ai): stream Chirp speech-to-text over /v1/realtime 2026-09-17 17:57:40 -07:00
reasoning_effort_utils.py fix(bedrock): normalize Messages system role and adaptive-thinking for Claude Invoke (#31364) 2026-06-27 11:35:36 -07:00
redact_messages.py fix(redaction): redact Responses refusal parts under turn_off_message_logging 2026-09-19 19:35:08 +00:00
request_timeout_resolver.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
response_header_helpers.py
rules.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
safe_json_dumps.py fix(logging): contain every extra serializer failure and skip rescanning a stamped record 2026-09-16 17:22:12 -07:00
safe_json_loads.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
secret_redaction.py fix(utils): redact credential kwargs from the set_verbose request line 2026-09-03 01:55:56 -07:00
sensitive_data_masker.py fix(masker): cut cycles at the first back-edge and walk pydantic dumps without self-recursion 2026-09-19 03:58:47 -07:00
service_tier_utils.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
streaming_chunk_builder_utils.py refactor(streaming): compute the reasoning token cap once 2026-09-16 23:28:28 +00:00
streaming_handler.py chore(streaming): remove retired ai21/maritalk/baseten/azure raw-bytes handlers and dead palm completion code 2026-09-17 20:07:51 +00:00
thread_pool_executor.py fix(logging): bound the shared logging executor backlog (#37694) 2026-08-20 16:04:43 -07:00
token_counter.py perf: defer fastapi and tiktoken BPE imports out of import litellm 2026-09-17 08:59:48 +00:00
url_utils.py fix(image_handling): answer every SSRF rejection with one message so error text cannot probe internal hostnames 2026-09-05 00:16:14 -07:00

Folder Contents

This folder contains general-purpose utilities that are used in multiple places in the codebase.

Core files:

  • streaming_handler.py: The core streaming logic + streaming related helper utils
  • core_helpers.py: code used in types/ - e.g. map_finish_reason.
  • exception_mapping_utils.py: utils for mapping exceptions to openai-compatible error types.
  • default_encoding.py: code for loading the default encoding (tiktoken)
  • get_llm_provider_logic.py: code for inferring the LLM provider from a given model name.
  • duration_parser.py: code for parsing durations - e.g. "1d", "1mo", "10s"
  • api_route_to_call_types.py: mapping of API routes to their corresponding CallTypes (e.g., /chat/completions -> [acompletion, completion])