mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-14 09:03:16 +00:00
cd2e4b6b60
* feat(custom-logger-v2): add types, dispatcher and route callback call sites (step 0) * feat(custom-logger-v2): add the rust contract for native callbacks and bind the proxy request scope * fix(custom-logger-v2): add field projections and keep the request scope across copies * fix(custom-logger-v2): pass the request scope to the async failure attempt_ended line * fix(custom-logger-v2): add v2 callbacks from litellm.callbacks to the success and failure lists _safe_add_callback_to_list dropped CustomLoggerV2 instances, so a V2 callback registered through litellm.callbacks got provider events but never log_prepare or attempt_ended. The manager now adds each V2 instance once by identity, and proxy startup fans V2 callbacks out like CustomLogger ones. * fix(custom-logger-v2): contain a failing v2 callback on logging hooks A V2 log_prepare writer that raised, or returned Abort or Respond, ended the line and escaped async_success_handler, so attempt_ended delivery, sync-for-async scheduling and the Redis flush were skipped. On hooks off the request path a V2 entry's exception or outcome is now counted and logged like a legacy logger's exception, and the line goes on. ContractViolation and BaseException still propagate, and legacy entries are unchanged. * fix(custom-logger-v2): run blocking and awaited v2 callbacks on synchronous logging lines run_plan_sync raised ContractViolation for a V2 log_prepare writer or any awaited V2 subscriber, the logging plane swallowed it, and every legacy logger at that hook was skipped. Legacy entries now always run in order as before. A V2 writer or awaited subscriber runs inline in list order through asyncio.run when the thread has no running event loop, so sync SDK calls apply its patch and finish it before the handler returns. On a thread with a running loop it cannot block, so it is delivered off-path and its patch is not applied. Lines without such entries take the unchanged legacy path. * fix(custom-logger-v2): map a v2 abort to its http error at every proxy hook point Only pre_call turned a V2 Abort into an HTTP error. during_call, post_call_success, post_mcp_call, list_models and the deferred stream hook raised V2Outcome, which reached the client as a 500, and post_call_failure, which never raised before, raised it too. Abort now raises the exception it carries or an HTTPException with its status, message and headers at those points, and at post_call_failure it becomes the transformed error through keep_first. Respond is still not wired and keeps raising V2Outcome. The pre_call routing test is updated to the HTTPException raise_v2_abort already makes. * fix(custom-logger-v2): apply no-log to v2 logging subscribers without a request scope The proxy request scope already carries no-log and caller opt-outs, so a V2 callback that honors no-log is skipped on proxy requests. SDK calls have no scope, and the logging plane ran V2 subscribers with an empty selection, so a no-log call was still delivered to them. Without a bound scope the logging lines now open one with no_log set when the call's litellm_params carry no-log, unless global_disable_no_log_param is on. Callbacks that do not honor no-log, and so every writer or abortable, are unaffected. * fix(custom-logger-v2): deliver a failed attempt to v2 subscribers once per logging object Callers such as MCP tool errors and Responses streaming run both failure handlers without for_async_request, so a V2 attempt_ended subscriber, now present in both failure lists, got the failure twice. The first failure handler to include V2 subscribers claims the delivery on the Logging object and the other skips them. Legacy callbacks are unchanged. * refactor(custom-logger-v2): settle a synchronous v2 run through the shared gather and use aware times in the new tests Drops the blind except the sync line used to capture a V2 callback's exception, reusing _gather_started, and gives the new logging tests timezone-aware times, so the strict ruff counts do not grow. * fix(custom_logger_v2): await a single-entry v2 stage in the caller's context A native handler in its own stage ran inside an asyncio.gather child task, so a ContextVar it set was lost when the stage ended, while a legacy callback's stays visible to later callbacks and the rest of the request. A stage with one v2 entry is now awaited directly, with gather's exception handling kept. Multi-entry parallel stages still gather. * fix(custom-logger-v2): raise a native callback panic as a runtime error * fix(custom-logger-v2): run a synchronous v2 line on a private event loop * fix(custom-logger-v2): deliver an mcp tool failure to v2 subscribers from the async failure handler * fix(custom-logger-v2): raise an abort's carried exception at the deployment hooks and keep the original failure at post_deployment_failure * fix(custom-logger-v2): apply caller opt-outs to v2 logging lines without a bound request scope * fix(custom-logger-v2): dedupe v2 callbacks in the event lists by logger key so a native built twice is delivered once * fix(custom-logger-v2): log an unregistered abort error kind and keep the carried traceback at post_call_failure * fix(callbacks): reject a selection name or alias that is not lowercase * fix(custom-logger-v2): run v2 deployment callbacks on rust bridge deployment hooks * fix(router): type the fallback event kwargs handed to the v2 dispatch * feat(custom_logger_v2): type the contract fields as records and mark derived-only fields Fields that the host derives from a hook's arguments now carry pydantic records on the Python side and matching serde records in litellm-callbacks, with records.json as the shared sample both sides check. Twelve new fields take bits 30 to 41, a write to a derived-only field is a manifest error, and pre_call can write budget_reservation and rate_limit_statuses in place. The payload reference field and the PayloadHost trait give Rust a handle to the logging payload instead of a copy * feat(custom_logger_v2): derive typed fields from hook arguments and always build call facts A slot table maps each typed field to a function over the hook's arguments. A derived slot runs only when a subscriber reads it, once per hook invocation, and off-path readers settle their reads before the hook returns. Auth fields come from the auth object only, budget_plan is claimed once, and budget_reservation and rate_limit_statuses write through to their carriers so the next pre_call subscriber sees them. Call facts are now built on unscoped logging lines and at post_call_success, post_call_failure and response_headers * feat(callbacks-python): hand the logging payload to native callbacks by reference NativeCallback stashes the shallow payload copy it is given and passes Rust a PayloadRef. The Python host renders a handle with a safe_dumps compatible encoder, on the caller's event loop while the hook runs and on the blocking pool after it, and dropping the last reference releases the stash entry. The encoder and its JSON helpers move here from the S3 logging bridge so every port shares one copy * feat(custom_logger_v2): accept every value python accepts for typed members and serve logged key identity Typed members keep the bool, int, float or str python holds through a Scalar type on both sides, so ids, limits and priorities no longer lose values that lax pydantic refused. A member the contract cannot carry is named in rejected so a port can reproduce what python does with it. Logging hooks get logged_key_identity, built from the payload metadata the python loggers read on every route, while key_identity stays on request hooks and keeps reading the auth object. * fix(custom_logger_v2): read the live budget reservation dict and carry non-finite floats both ways budget_reservation read the dict budget_reservation.py builds with a KeyError on callback_bound. Undeclared keys are now ignored, callback_bound is a member, entries keep a non-mapping item as null so capacity's model access group check stays exact, and costs read as the float budgets computes with float(x or 0.0). Float members cross as the raw json marker when they are not finite, and a native verdict returns through json_to_py, so an infinite or NaN cost survives a read and a write instead of turning into null. * feat(callbacks): carry a python value as its str() text and truthiness in scalar members Scalar gains a Stringified variant with the str() text and bool() of a value that is not a bool, int within i128, float or str, so a port can reproduce the label, path or log text python formats and the falsiness of an empty container without confusing it with a real string. Budget reservations say whether python held the entries as a list. * feat(custom_logger_v2): carry non-scalar values and any iterable tags or entries across the boundary A Scalar member now carries a list, dict, set, bytes, object or an int past i128 as its str() text and truthiness instead of naming it in rejected, which stays for values whose str() or bool() raises. request_tags carries any iterable as the items prometheus iterates, and budget reservation entries carry what budgets iterates with entries_is_list for capacity's list check, never written back to the shared dict. * feat(custom_logger_v2): say whether a stringified value hashes and whether it is an int A Stringified scalar now carries hashable, from trying hash() at the boundary, so a port can raise the TypeError capacity's priority lookup and the limiter's descriptor identities raise for a list, dict or set and not for a tuple or object. It also carries integer, so an int past i128 is told apart from any other value whose text is digits. * fix(custom_logger_v2): wrap a non-list guardrail mode as one mode like prometheus does Prometheus treats any guardrail_mode that is not a list as a single mode, so a tuple mode holds no str mode and a missing mode is one None mode. The derivation unpacked tuples and rejected other non-str modes instead. * feat(callbacks): let a native handler await typed host calls the python caller serves * test(callbacks-python): drive a native handler's host calls through the python dispatcher * test(custom_logger_v2): route a native logger with a host through the execution driver * fix(custom_logger_v2): type the wrapped guardrail modes for pyright * feat(custom_logger_v2): carry the python type name and hash error text of a stringified value * feat(custom_logger_v2): serve the model group the logging handlers compute and key remaining limits by it * feat(custom_logger_v2): tell a budget reservation that is not a mapping apart from none * feat(custom_logger_v2): name malformed team model preferences in rejected with the text capacity logs * feat(custom_logger_v2): serve a logging payload that is not a mapping apart from a missing one * fix(callbacks-python): render null keys like a python dict merge, adding a missing key last * feat(custom_logger_v2): carry an explicit output cap past i64 as a scalar * feat(custom_logger_v2): serve the custom metadata labels prometheus derives from the logged payload * feat(custom_logger_v2): serve the input sequence length bucket prometheus picks at attempt end * feat(custom_logger_v2): carry the start and end times prometheus reads from the call details * feat(custom_logger_v2): serve the limiter's body tags, session id, resolved agent and agent policy limits at pre_call * fix(callbacks): export the limiter request record * test(custom_logger_v2): give the type name cases stable ids and sort the record exports * fix(custom_logger_v2): type the new derivations and tests for pyright * fix(custom_logger_v2): drop team model preferences and serve the fallback_on_saturation handoff main's capacity callback reads * style(callbacks): rustfmt the record exports * refactor(custom_logger_v2): rename native callbacks to rust callbacks native.py becomes rust.py, with native_logger, NativeLogger, NativeHandle, NativeState and HostedNative renamed to their rust forms. The pyo3 classes NativeCallback and NativeRequestState become RustCallback and RustRequestState, and the tests, fixtures and messages follow. * refactor(custom_logger_v2): rename slots to field sources slots.py becomes field_sources.py: Slot is FieldSource, SlotFields is SourcedFields, and with_slots, NO_SLOTS and slot_table_problem follow. A table that only derives is named DERIVED_*_FIELDS, and a table that can write back keeps SOURCES (PRE_CALL_AUTH_SOURCES, PRE_CALL_SOURCES). The include_v2 comment moves down to the functions that take it. * refactor(litellm_logging): place the v2 members of Logging beside what they serve v2_scope, v2_auth and v2_failure_delivered join the other per-request class defaults, v2_logging_scope and its opt-out helper follow should_run_callback, v2_call_facts sits by the standard logging payload builder, and claim_v2_failure sits above failure_handler. bind_v2_request_scope checks for a Logging object directly instead of two runtime protocols. * refactor(proxy): rename v2_fields to v2_proxy_fields and say why it is separate * refactor(custom_logger_v2): drop the logged_payload_not_a_mapping field A logged payload that is present but not a mapping now looks the same to a rust callback as a missing one, a parity gap accepted for a case that does not occur. metadata_labels, input_sequence_length and limiter_request move down to bits 44-46, on both sides of the contract. * feat(callbacks): type a rust callback's request state Callback gains an associated State type: request_state returns it and handle and handle_hosted receive Option<&State> beside the input, which no longer carries the state. ErasedCallback is the object-safe form the python wrapper stores; it carries the state type-erased and downcasts it once, returning CallbackError::StateType rather than panicking on a mismatch. handle_hosted's default names its unused host _host. The record sample count follows the dropped logged_payload_not_a_mapping field. * docs(callbacks): say what PayloadRef, PayloadHost and RenderOptions.null_keys are * refactor(callbacks-python): render payloads with python-compat's json writers pyjson.rs reimplemented py_encode_basestring_ascii and json's floatstr. python-compat's json module now exposes write_string, generic over a String or byte sink, and float_text, and the payload renderer uses them. * docs(callbacks): note why Respond, Reroute, call_ended and flush are not wired yet * docs(custom_logger_v2): add a README for the package * fix(custom_logger_v2): import the field sources tests from field_sources * test(callbacks-python): stub the litellm package in each payload test * fix(custom_logger_v2): type the disabled callbacks lookup and the v2 dedupe check |
||
|---|---|---|
| .. | ||
| _experimental | ||
| a2a | ||
| agent_endpoints | ||
| alerting_endpoints | ||
| analytics_endpoints | ||
| anthropic_endpoints | ||
| auth | ||
| batches_endpoints | ||
| client | ||
| common_utils | ||
| config_resolvers | ||
| container_endpoints | ||
| credential_endpoints | ||
| db | ||
| decisions_endpoints | ||
| deps | ||
| discovery_endpoints | ||
| enterprise_billing | ||
| example_config_yaml | ||
| fine_tuning_endpoints | ||
| google_endpoints | ||
| guardrails | ||
| health_endpoints | ||
| hooks | ||
| image_endpoints | ||
| lens | ||
| list_api | ||
| logging_endpoints | ||
| management | ||
| management_endpoints | ||
| management_helpers | ||
| mcp_server | ||
| memory | ||
| middleware | ||
| ocr_endpoints | ||
| openai_files_endpoint | ||
| pass_through_endpoints | ||
| policy_engine | ||
| prompts | ||
| proxy_server | ||
| public_endpoints | ||
| rag_endpoints | ||
| realtime_endpoints | ||
| rerank_endpoints | ||
| response_api_endpoints | ||
| response_polling | ||
| roi_calculator | ||
| rust_forward | ||
| search_endpoints | ||
| shutdown | ||
| spend_tracking | ||
| telemetry | ||
| test_configs | ||
| types_utils | ||
| ui_crud_endpoints | ||
| utils | ||
| vector_store_endpoints | ||
| vector_store_files_endpoints | ||
| video_endpoints | ||
| __init__.py | ||
| conftest.py | ||
| proxy_server_config.yaml | ||
| router_entry_point_stubs.py | ||
| skill_submission_rows.py | ||
| test__lazy_features.py | ||
| test__types.py | ||
| test_admin_mcp.py | ||
| test_aiohttp_cleanup_closed.py | ||
| test_aiohttp_session_recovery.py | ||
| test_api_key_masking_in_errors.py | ||
| test_aproxy_startup.py | ||
| test_audio_speech_prometheus_hooks.py | ||
| test_batch_expiry.py | ||
| test_batch_metadata_none_fix.py | ||
| test_batch_retrieve_bedrock.py | ||
| test_batch_x_litellm_model_encoding.py | ||
| test_blocked_response_usage.py | ||
| test_body_snapshot_callback_params.py | ||
| test_budget_reservation.py | ||
| test_bug_report_config.py | ||
| test_caching_routes.py | ||
| test_chat_completion_metadata.py | ||
| test_claude_code_marketplace.py | ||
| test_collector.py | ||
| test_common_request_processing.py | ||
| test_component_allowlists.py | ||
| test_conftest.py | ||
| test_cors_config.py | ||
| test_credential_slot_registry.py | ||
| test_custom_callback_input.py | ||
| test_custom_logger_s3_gcs.py | ||
| test_custom_proxy.py | ||
| test_custom_tokenizer_bug.py | ||
| test_db_schema_changes.py | ||
| test_deprecated_key_grace_period.py | ||
| test_dev_reload.py | ||
| test_dynamic_mcp_route.py | ||
| test_empty_model_list.py | ||
| test_enforce_user_param.py | ||
| test_fallback_management_endpoints.py | ||
| test_fastapi_offline_routes.py | ||
| test_filter_models_by_team_access_group.py | ||
| test_get_favicon.py | ||
| test_get_image.py | ||
| test_health_check_functions.py | ||
| test_health_check_max_tokens.py | ||
| test_init_litellm_callbacks.py | ||
| test_langfuse_passthrough_security.py | ||
| test_lazy_openapi_snapshot.py | ||
| test_litellm_pre_call_utils.py | ||
| test_max_budget_env_var.py | ||
| test_mcp_asgi_response.py | ||
| test_model_based_routing_files_batches.py | ||
| test_model_deprecations_endpoint.py | ||
| test_model_dump_with_preserved_fields.py | ||
| test_model_id_header_propagation.py | ||
| test_model_info_default_limits.py | ||
| test_model_level_guardrails.py | ||
| test_model_list_aliases.py | ||
| test_model_list_callback_filter.py | ||
| test_model_list_discoverable.py | ||
| test_model_list_healthy_only.py | ||
| test_modify_response_streaming_passthrough.py | ||
| test_moyai_endpoints.py | ||
| test_native_compaction.py | ||
| test_openai_ws_passthrough_routes.py | ||
| test_openapi_schema_validation.py | ||
| test_plugin_routes.py | ||
| test_pointfive_dashboard_config.py | ||
| test_pointfive_ui_callback.py | ||
| test_pricing_field_strip.py | ||
| test_prisma_client_backoff_retry.py | ||
| test_prisma_engine_watchdog.py | ||
| test_prisma_migration.py | ||
| test_prometheus_cleanup.py | ||
| test_prometheus_metrics_server.py | ||
| test_prompt_test_endpoint.py | ||
| test_provider_url_destination_guard.py | ||
| test_proxy_cli.py | ||
| test_proxy_config_unit_test.py | ||
| test_proxy_custom_auth.py | ||
| test_proxy_logging_hook_detection.py | ||
| test_proxy_reject_logging.py | ||
| test_proxy_server.py | ||
| test_proxy_server_custom_logger.py | ||
| test_proxy_server_endpoints_and_startup.py | ||
| test_proxy_server_exception_mapping.py | ||
| test_proxy_setting_guardrails.py | ||
| test_proxy_token_counter.py | ||
| test_proxy_types.py | ||
| test_proxy_utils.py | ||
| test_proxy_utils_model_creation_and_error_logging.py | ||
| test_pyroscope.py | ||
| test_read_model_list.py | ||
| test_redis_auth_cache_flag.py | ||
| test_reducto_ocr_route.py | ||
| test_response_model_sanitization.py | ||
| test_response_polling_pre_call_checks.py | ||
| test_route_a2a_models.py | ||
| test_route_llm_request.py | ||
| test_route_priority.py | ||
| test_sensitive_route_auth.py | ||
| test_server_listener.py | ||
| test_server_root_path.py | ||
| test_shared_health_check.py | ||
| test_spend_log_cleanup.py | ||
| test_swagger_chat_completions.py | ||
| test_team_member_update.py | ||
| test_team_org_move.py | ||
| test_tools_allowlist_enforcement.py | ||
| test_tracing_endpoints.py | ||
| test_ui_path_detection.py | ||
| test_unit_test_proxy_hooks.py | ||
| test_update_llm_router_resilience.py | ||
| test_update_spend.py | ||
| test_v2_hook_routing.py | ||
| test_v2_proxy_fields.py | ||
| test_zero_cost_model_budget_bypass.py | ||
| test_zerobus_dashboard_config.py | ||
| vector_store_access_group_state.py | ||
| vertex_key.json | ||