litellm/litellm/litellm_core_utils
Mateo Wang 4ebedf901a
Merge pull request #38696 from BerriAI/litellm_lit6376_dspy_streaming
fix(streaming): report response_cost and Anthropic citations from stream_chunk_builder
2026-08-28 15:31:23 -07:00
..
audio_utils perf(subtitle_utils): make cue grouping linear in cue count 2026-08-27 12:58:20 -07:00
llm_cost_calc fix(cost): only let a WxH request size drive image pricing 2026-08-28 09:52:31 -07:00
llm_response_utils fix(cost): make cost-breakdown headers respect service tier 2026-08-26 17:09:30 -07:00
prompt_templates fix(openai): drop tool_reference parts from tool messages at the chat boundary 2026-08-26 23:43:41 -07:00
specialty_caches feat(newrelic): per-team cost and usage metrics via team callbacks (#37610) 2026-08-26 23:42:02 -07:00
tokenizers Code Quality Improvement - remove tokenizers/ from /llms (#7163) 2024-12-10 23:50:15 -08:00
agentic_loop_settings.py fix: keep accepting a loop ceiling that spells a whole number 2026-08-22 11:24:22 -07:00
api_route_to_call_types.py fix(guardrails): scan model output on the /openai/v1/responses alias (#35818) 2026-08-04 16:46:45 -07:00
app_crypto.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
asyncify.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
cached_imports.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
chat_completion_agentic_loop.py fix: validate max_agentic_loops wherever it is set 2026-08-22 10:53:13 -07:00
cli_keyring.py fix(cli): stop asking a keychain that already stopped answering 2026-08-20 04:12:23 -07:00
cli_token_utils.py fix(cli): keep the refresh token in the OS keychain, not in token.json 2026-08-20 11:43:53 -07:00
cloud_storage_security.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
completion_timeout.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
core_helpers.py feat(guardrails): add Lakera v2 skip-message honoring and advisory (inject_system_message) mode (#34940) 2026-08-28 14:13:49 -07:00
coroutine_checker.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
credential_accessor.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
custom_logger_registry.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
dd_tracing.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
default_encoding.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
dot_notation_indexing.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
duration_parser.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
env_utils.py fix(token_counter): stop large token counts from blocking the proxy event loop (#37697) 2026-08-20 16:09:07 -07:00
exception_mapping_utils.py fix(exceptions): keep a refused connection an APIConnectionError 2026-08-27 21:00:24 -07:00
fallback_generalizations.py chore(lint): clear grandfathered over-limit lint drift and ratchet budgets down 2026-08-05 12:18:13 -07:00
fallback_utils.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
get_blog_posts.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
get_litellm_params.py feat(rust): route /chat/completions through the Rust core for anthropic and bedrock (#37241) 2026-08-20 16:15:24 -07:00
get_llm_provider_logic.py fix(router): drop a tier param the routed target cannot take (#38622) 2026-08-28 14:17:49 -07:00
get_model_cost_map.py fix(proxy): persist periodic reload schedule state so status survives restarts and fires without store_model_in_db (#35165) 2026-08-04 15:42:57 -07:00
get_provider_specific_headers.py fix(proxy): stop forwarding a client Anthropic OAuth token to Bedrock and Vertex 2026-08-21 18:02:40 -07:00
get_supported_openai_params.py fix(router): drop a tier param the routed target cannot take (#38622) 2026-08-28 14:17:49 -07:00
health_check_helpers.py fix(health): make the image_edit health probe moderation-safe 2026-08-26 15:23:08 -07:00
health_check_utils.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
initialize_dynamic_callback_params.py feat(langfuse): support langfuse_environment as a per-key dynamic callback param (#38264) 2026-08-26 16:56:55 -07:00
internal_call_metadata.py fix(logging): stop billing and logging response reads as LLM calls (#36890) 2026-08-26 18:34:17 -07:00
json_fragment_accumulator.py perf(streaming): add shared JSONFragmentAccumulator for Vertex and Anthropic (#36610) 2026-08-25 16:15:47 -07:00
json_validation_rule.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
litellm_logging.py fix(logging): preserve null end user in callbacks (#38642) 2026-08-28 10:54:53 -07:00
llm_judge.py fix(shadow_eval): refuse a judge model that also serves one of the arms it grades (#38589) 2026-08-27 18:44:44 -07:00
llm_request_utils.py fix(images): forward scalar-array edit params as repeated multipart fields 2026-08-24 12:48:44 -07:00
logging_callback_manager.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
logging_utils.py fix(otel): emit LLM Call spans for speech, image, moderation, ocr and transcription (#37752) 2026-08-22 11:11:21 -07:00
logging_worker.py fix(logging_worker): swallow cancellation in exit flush and revive dequeued tasks on loop change 2026-08-26 13:30:40 -07:00
mock_functions.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
model_param_helper.py merge litellm_internal_staging into litellm_anthropic_messages_response_cache 2026-08-11 20:48:41 +00:00
model_response_utils.py chore(lint): clear grandfathered over-limit lint drift and ratchet budgets down 2026-08-05 12:18:13 -07:00
private_json.py fix(cli): finish a refused token file rewrite in place 2026-08-20 02:58:53 -07:00
ptu_pricing.py fix(ptu): zero the Maps grounding rate on PTU deployments 2026-08-26 15:50:36 -07:00
README.md Guardrails API - add streaming support (#17400) 2025-12-02 22:52:09 -08:00
realtime_errors.py fix(realtime): bound Vertex credential resolution and make realtime failures loud 2026-08-20 02:02:00 -07:00
realtime_streaming.py fix(realtime): bill trailing audio when a Gemini transcribe Live session closes 2026-08-27 12:43:28 -07:00
reasoning_effort_utils.py fix(bedrock): normalize Messages system role and adaptive-thinking for Claude Invoke (#31364) 2026-06-27 11:35:36 -07:00
redact_messages.py chore(typing): clear fresh tech debt from the Aug 25 window 2026-08-26 07:55:16 +00:00
request_timeout_resolver.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
response_header_helpers.py fix(utils.py): guarantee openai-compatible headers always exist in response 2024-09-28 21:08:15 -07:00
rules.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
safe_json_dumps.py fix(logging): close three secret-leak paths in verbose logging (#37391) 2026-08-19 01:03:34 +00:00
safe_json_loads.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
secret_redaction.py fix(logging): close three secret-leak paths in verbose logging (#37391) 2026-08-19 01:03:34 +00:00
sensitive_data_masker.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
service_tier_utils.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
streaming_chunk_builder_utils.py fix(streaming): align assembled provider model 2026-08-28 08:13:14 -05:00
streaming_handler.py Merge pull request #38696 from BerriAI/litellm_lit6376_dspy_streaming 2026-08-28 15:31:23 -07:00
thread_pool_executor.py fix(logging): bound the shared logging executor backlog (#37694) 2026-08-20 16:04:43 -07:00
token_counter.py chore(token_counter): drop docstrings and test prose that restated the count_tokens branches 2026-08-28 06:28:21 -07:00
url_utils.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00

Folder Contents

This folder contains general-purpose utilities that are used in multiple places in the codebase.

Core files:

  • streaming_handler.py: The core streaming logic + streaming related helper utils
  • core_helpers.py: code used in types/ - e.g. map_finish_reason.
  • exception_mapping_utils.py: utils for mapping exceptions to openai-compatible error types.
  • default_encoding.py: code for loading the default encoding (tiktoken)
  • get_llm_provider_logic.py: code for inferring the LLM provider from a given model name.
  • duration_parser.py: code for parsing durations - e.g. "1d", "1mo", "10s"
  • api_route_to_call_types.py: mapping of API routes to their corresponding CallTypes (e.g., /chat/completions -> [acompletion, completion])