scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-05 02:41:56 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 49
litellm/litellm/litellm_core_utils
History
devin-ai-integration[bot] e73f949fbb
fix(params): stop stream_chunk_size reaching provider request bodies (#42664) 
* fix(params): carry stream_chunk_size through litellm_params instead of provider params

* test(integration): fence stream_chunk_size out of every provider request body

* test(bedrock): type parametrized stream chunk test params

* test(integration): drop the contracts manifest resurrected by the main merge

* test(bedrock): type the stream_chunk_size test helpers

* test(params): finish AGENTS.md typing pass on stream_chunk_size tests

* test(integration): drop the covers marker from the stream_chunk_size wire test

---------

Co-authored-by: shrey kharbanda <shreshth@berri.ai>
2026-09-23 10:27:47 -07:00
..
audio_utils fix(audio_utils): validate MPEG frame headers before labeling sniffed audio 2026-08-29 14:20:40 -07:00
llm_cost_calc fix(cost): honor deployment pricing for image generation (#39311) 2026-09-22 11:40:39 -07:00
llm_response_utils fix: answer get_api_base for github_copilot and chatgpt without running the login flow (#42602) 2026-09-22 17:23:44 -07:00
prompt_templates fix(anthropic): return 400 instead of 500 when a content list holds a bare string (#42420) 2026-09-22 15:44:30 -07:00
specialty_caches feat(newrelic): per-team cost and usage metrics via team callbacks (#37610) 2026-08-26 23:42:02 -07:00
tokenizers
agentic_followup_kwargs.py refactor(agentic-loop): build follow-up kwargs in one place so no executor can repeat a request param (#42307) 2026-09-21 20:06:01 -07:00
agentic_loop_settings.py fix: keep accepting a loop ceiling that spells a whole number 2026-08-22 11:24:22 -07:00
api_route_to_call_types.py fix(guardrails): resolve generateContent routes and async-first passthrough call types 2026-08-29 21:35:19 -07:00
app_crypto.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
asyncify.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
aws_partition.py fix(aws): build every AWS endpoint and ARN from the region partition 2026-08-29 01:21:59 -07:00
bug_report.py feat(proxy): admin-only /debug/report sharing the bug report environment (#42440) 2026-09-22 12:05:23 -07:00
cached_imports.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
chat_completion_agentic_loop.py refactor(agentic-loop): build follow-up kwargs in one place so no executor can repeat a request param (#42307) 2026-09-21 20:06:01 -07:00
classifier_logging.py fix(router): merge staging and retain native classifier audits 2026-09-10 14:24:56 -07:00
cli_keyring.py fix(cli): stop asking a keychain that already stopped answering 2026-08-20 04:12:23 -07:00
cli_token_utils.py fix(cli): keep the refresh token in the OS keychain, not in token.json 2026-08-20 11:43:53 -07:00
cloud_storage_security.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
completion_timeout.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
core_helpers.py feat(proxy): opt-in include_guardrail_response returns guardrail_information in the response (#42327) 2026-09-22 12:43:30 -07:00
coroutine_checker.py Merge remote-tracking branch 'origin/main' into litellm_decrease_anys_opus5_r5 2026-09-21 12:57:38 -07:00
credential_accessor.py fix(proxy): load db credentials in the model reconcile so a worker never serves a model before its credential (#39876) 2026-09-08 10:08:24 -07:00
custom_logger_registry.py feat(pointfive): register the pointfive callback 2026-09-10 14:02:35 +03:00
dd_tracing.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
default_encoding.py feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
dot_notation_indexing.py refactor(typing): replace Any with proven types in 89 more backend files 2026-09-02 23:08:42 +00:00
duration_parser.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
env_utils.py fix(token_counter): stop large token counts from blocking the proxy event loop (#37697) 2026-08-20 16:09:07 -07:00
error_normalization.py feat(logging): add normalized_error cluster key to error_information (#41715) 2026-09-22 15:56:50 -07:00
exception_mapping_utils.py feat(errors): prefilled GitHub issue link on unmapped internal errors (#42065) 2026-09-21 22:30:41 -07:00
fallback_generalizations.py refactor(model_info): drop types import from fallback_generalizations 2026-09-14 18:58:53 +00:00
fallback_utils.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
get_blog_posts.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
get_litellm_params.py fix(params): stop stream_chunk_size reaching provider request bodies (#42664) 2026-09-23 10:27:47 -07:00
get_llm_provider_logic.py fix(fal_ai): honour global api_base for image generation and reject non-string reasoning_effort with 400 (#42512) 2026-09-22 13:13:54 -07:00
get_model_cost_map.py fix(proxy): drop cost-map metadata echoed back on model save (#41944) 2026-09-21 19:33:16 -07:00
get_provider_specific_headers.py fix(proxy): stop forwarding a client Anthropic OAuth token to Bedrock and Vertex 2026-08-21 18:02:40 -07:00
get_supported_openai_params.py fix(vertex_ai): keep reasoning_effort unsupported on Vertex AI Mistral partner models 2026-09-18 13:16:56 -07:00
health_check_helpers.py fix(ocr): send each provider a health-check document it accepts 2026-09-04 22:52:12 -07:00
health_check_utils.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
initialize_dynamic_callback_params.py feat(arize): per-team success and error sampling rates for the Arize AX callback (#42383) 2026-09-22 11:21:40 -05:00
internal_call_metadata.py fix(auto_router): bill the routing embedding to the caller's key and team (#39532) 2026-09-03 13:53:30 -07:00
json_fragment_accumulator.py perf(streaming): add shared JSONFragmentAccumulator for Vertex and Anthropic (#36610) 2026-08-25 16:15:47 -07:00
json_validation_rule.py refactor(typing): replace Any with proven types in 89 more backend files 2026-09-02 23:08:42 +00:00
litellm_logging.py feat(logging): add normalized_error cluster key to error_information (#41715) 2026-09-22 15:56:50 -07:00
llm_judge.py fix(shadow_eval): refuse a judge model that also serves one of the arms it grades (#38589) 2026-08-27 18:44:44 -07:00
llm_request_utils.py fix(hosted_vllm): forward Omni media URLs instead of downloading them 2026-08-24 17:59:52 -04:00
logging_callback_manager.py fix(proxy): surface runtime-registered callbacks in UI Logging page (#38974) 2026-09-09 22:03:16 -07:00
logging_utils.py refactor(types): replace Any with proven types in 34 files 2026-09-20 10:16:48 +00:00
logging_worker.py test(logging_worker): cover a same-loop flush and a repeated flush after a loop change 2026-09-21 15:56:09 -07:00
mock_functions.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
model_param_helper.py merge litellm_internal_staging into litellm_anthropic_messages_response_cache 2026-08-11 20:48:41 +00:00
model_response_utils.py chore(techdebt): clear fresh debt from the 2026-08-29 window 2026-08-30 07:57:16 +00:00
private_json.py feat(auto-router): show routed model and savings in Claude Code and Codex (#40330) 2026-09-11 12:52:42 -07:00
provider_affinity.py feat: add configurable provider affinity header mapping (#41033) 2026-09-22 13:31:33 -07:00
ptu_pricing.py feat(spend-logs): record Azure spillover source deployment in spend log metadata 2026-09-17 05:48:51 +00:00
README.md feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
realtime_errors.py fix(realtime): surface an upstream handshake refusal as an error event and policy close (#42388) 2026-09-22 11:31:15 -07:00
realtime_streaming.py feat(vertex_ai): stream Chirp speech-to-text over /v1/realtime 2026-09-17 17:57:40 -07:00
reasoning_effort_utils.py fix(bedrock): normalize Messages system role and adaptive-thinking for Claude Invoke (#31364) 2026-06-27 11:35:36 -07:00
redact_messages.py fix(guardrails): store the masked output in spend logs when Presidio masks the response (#42441) 2026-09-22 01:19:22 -07:00
request_timeout_resolver.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
response_header_helpers.py
rules.py chore(lint): strip inert type: ignore comments and zero LIT009, LIT010, LIT011 headroom 2026-08-05 02:37:24 -07:00
safe_json_dumps.py refactor(types): replace Any with proven types in 32 files 2026-09-21 10:51:46 +00:00
safe_json_loads.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
secret_redaction.py feat(logger): dispatch Python logging through the Rust diagnostics processor (#42616) 2026-09-22 18:44:15 -07:00
sensitive_data_masker.py Merge remote-tracking branch 'origin/main' into litellm_decrease_anys_opus5_r5 2026-09-21 12:57:38 -07:00
served_output_texts.py fix(guardrails): store the masked output in spend logs when Presidio masks the response (#42441) 2026-09-22 01:19:22 -07:00
service_tier_utils.py feat(lint): enforce Final on locals and freeze function parameters (LIT010, LIT011) 2026-08-04 12:54:39 -07:00
streaming_chunk_builder_utils.py fix(streaming): let a later usage event zero out stale cache counts (#40736) (#42330) 2026-09-22 11:50:19 -07:00
streaming_handler.py chore(streaming): remove retired ai21/maritalk/baseten/azure raw-bytes handlers and dead palm completion code 2026-09-17 20:07:51 +00:00
thread_pool_executor.py fix(logging): bound the shared logging executor backlog (#37694) 2026-08-20 16:04:43 -07:00
token_counter.py feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
tokenizer.py feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174) 2026-09-22 04:41:11 +00:00
url_utils.py fix(ssrf): point the blocked-address remediation at litellm_settings (#42508) 2026-09-22 14:04:16 -07:00

README.md

Folder Contents

This folder contains general-purpose utilities that are used in multiple places in the codebase.

Core files:

  • streaming_handler.py: The core streaming logic + streaming related helper utils
  • core_helpers.py: code used in types/ - e.g. map_finish_reason.
  • exception_mapping_utils.py: utils for mapping exceptions to openai-compatible error types.
  • default_encoding.py: code for loading the default Python tokenizer and bundled cache
  • get_llm_provider_logic.py: code for inferring the LLM provider from a given model name.
  • duration_parser.py: code for parsing durations - e.g. "1d", "1mo", "10s"
  • api_route_to_call_types.py: mapping of API routes to their corresponding CallTypes (e.g., /chat/completions -> [acompletion, completion])

Tokenizer factories return Python tokenizer objects by default. Set LITELLM_RUST=1 or call litellm.rust(True) before constructing tokenizers to select the Rust backend through Route.TOKENIZER in the Rust catalog. Missing native bindings or unsupported native features fall back to Python. Existing tokenizer objects keep their selected backend. Rust-backed tokenizer objects carry the read-only tiktoken.Encoding / tokenizers.Tokenizer surface and are immutable: enable_padding, enable_truncation and add_tokens stay on the Python tokenizer.