mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-12 23:01:41 +00:00
4efce809d0
* feat(proxy): add logging_endpoints package init
* feat(proxy): add POST /v1/callbacks/logs to replay logging payloads through the success/failure callback fan-out
* feat(proxy): register callback_logs_router
* test(proxy): add logging_endpoints test package init
* test(proxy): cover /v1/callbacks/logs replay, admin guard, and partial-failure handling
* refactor(proxy): move callback-logs request/response models to litellm/types/proxy
* refactor(proxy): wrap callback-logs replay in CallbackLogsReplayer class with payload logging
* test(proxy): update callback-logs tests for class-based replayer and separated types
* fix(proxy): cover /v1/callbacks/ in backend component allowlist
The new /v1/callbacks/logs route was dropped by both component
allowlists, failing test_gateway_plus_backend_covers_full_app. It's an
admin-only spend-logging route, so it belongs on the backend (control
plane) alongside the existing /callbacks family.
* refactor(proxy): use builtin dict/list generics in callback-logs endpoint
Switch Dict/List from typing to builtin dict/list to satisfy the ruff
strict-rule budget (UP006).
* refactor(proxy): use builtin dict/list generics in callback-logs types
UP006: builtin generics over typing.Dict/List.
* chore(ui): regenerate schema.d.ts for /v1/callbacks/logs
Run npm run gen:api to add the CallbackLogRecord/CallbackLogsRequest/
CallbackLogsResponse types and the /v1/callbacks/logs path, keeping the
dashboard types in sync with the proxy OpenAPI spec.
* fix(proxy): force stream=False when replaying callback logs
A replayed StandardLoggingPayload is a terminal, fully-aggregated event —
the producer (e.g. the rust realtime gateway) already collected the whole
session before POSTing. Marking the rebuilt Logging object as streaming made
async_success_handler wait for a complete_streaming_response that never
arrives, so the spend log was never written. Realtime sessions now land in
LiteLLM_SpendLogs.
* feat(litellm-rust): CustomLogger callback layer posting to /v1/callbacks/logs
integrations/ mirrors litellm/integrations/: a sync, typed CustomLogger trait
(base contract), a typed StandardLoggingPayload, and LiteLLMPythonProxyAPILogger
— the first concrete logger, owning a bounded channel + background worker that
batches and POSTs to the Python proxy's /v1/callbacks/logs.
* feat(litellm-rust): RealTimeStreaming per-session log collector
1:1 with Python's RealTimeStreaming: observe() accumulates O(1) usage/model/id
per event (never buffers frames); log_messages() builds one StandardLoggingPayload
on session close and fans out to the CustomLogger callbacks. request_id == the
OpenAI realtime session id (sess_…), with the gateway id as fallback.
* feat(litellm-rust): wire realtime logging into the splice (lock-free observe)
The collector is owned on the splice task and observed via a synchronous &mut
callback threaded through providers::realtime::realtime() — no Arc/Mutex/atomic
on the per-frame hot path. On session close the bridge flushes one payload.
AppState carries the registered loggers; main spawns the proxy logger.
* docs(litellm-rust): ai-gateway realtime logging architecture
* docs(litellm-rust): document request-log egress to the LiteLLM control plane
Add a 'Request logging' guide to the ai-gateway README: how to point the gateway
at a LiteLLM proxy via LITELLM_PROXY_BASE_URL (+ LITELLM_MASTER_KEY for the
admin-only /v1/callbacks/logs POST), and the non-blocking / one-payload-per-session
behavior.
* feat(litellm-rust): make log-egress tunables env-overridable
Channel capacity, batch size, and flush interval now read from
LITELLM_LOG_CHANNEL_CAPACITY / LITELLM_LOG_BATCH_SIZE / LITELLM_LOG_FLUSH_INTERVAL_MS,
falling back to the DEFAULT_* consts on missing/invalid/non-positive values.
Grouped behind an EgressTunables::from_env() read once at logger construction.
* docs(litellm-rust): document log-egress tuning env vars
* docs(litellm-rust): require constants in a crate-level constants.rs
Mirror of Python's litellm/constants.py rule — magic numbers and fixed strings
go in src/constants.rs, not inline in feature modules; env-overridable tunables
keep their DEFAULT_* value there.
* refactor(litellm-rust): move ai-gateway constants into constants.rs
Per the new rule: the log-egress defaults (proxy base, ingest path, channel
capacity, batch size, flush interval) and the realtime provider default move to
crates/ai-gateway/src/constants.rs; modules import from it.
* ci: run logging_endpoints tests in the proxy-infra coverage shard
tests/test_litellm/proxy/logging_endpoints wasn't in any coverage-uploading
job, so callback_logs_endpoints.py showed only import-level coverage (~35%) on
codecov/patch despite being ~98% covered locally. Add it to proxy-infra's
test-path so the test is exercised under --cov.
* fix(litellm-rust): hash the master key before logging — never send the raw credential
Greptile/Veria P1: user_api_key_hash was the plaintext LITELLM_MASTER_KEY, which
fans out to spend logs and every callback (Langfuse/Datadog) and could be
recovered from logs. SHA-256 it (auth::hash_token, matching the proxy's
hash_token); the field is named *_hash and the proxy stores it verbatim when it
isn't sk-prefixed, so the DB value is identical with zero plaintext exposure.
* fix(litellm-rust): observe realtime logging on upstream events only
Greptile P1: observe ran on the client->upstream arm too, so an authenticated
client could send a fabricated response.done and inflate its own spend log.
session.created/response.done are server->client events; observe the upstream
arm only.
* feat(proxy): bound callback-logs batch + return per-record failures
Greptile P2: cap /v1/callbacks/logs at MAX_CALLBACK_LOG_RECORDS (default 1000,
env-overridable) so one POST can't trigger an unbounded callback/DB fan-out; and
return per-record {index, error} failures so a caller (the rust gateway) can
distinguish a transient callback error from a structurally bad payload.
* chore(ui): regenerate schema.d.ts for CallbackLogFailure / failures field
* fix(constants): make MAX_CALLBACK_LOG_RECORDS a plain constant
It doesn't need to be env-configurable (only the rust egress tunables are). As an
os.getenv var it tripped tests/documentation_tests/test_env_keys.py, which requires
every env key to be documented in the (separate-repo) config_settings.md. Plain
constant → not scanned → code-quality + documentation checks pass.
* docs(litellm-rust): trim ai-gateway ARCHITECTURE.md to one diagram + notes
* docs(litellm-rust): tighten the README request-logging section
* docs(litellm-rust): ARCHITECTURE.md is just the diagram (gateway = inference, spend = callback)
* docs(litellm-rust): drop em-dashes from the request-logging section
---------
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
128 lines
3.2 KiB
Text
128 lines
3.2 KiB
Text
.python-version
|
|
.venv
|
|
.venv_policy_test
|
|
.env
|
|
.claude
|
|
.newenv
|
|
newenv/*
|
|
litellm/proxy/myenv/*
|
|
litellm_uuid.txt
|
|
__pycache__/
|
|
*.pyc
|
|
bun.lockb
|
|
**/.DS_Store
|
|
.aider*
|
|
litellm_results.jsonl
|
|
secrets.toml
|
|
.gitignore
|
|
litellm/proxy/litellm_secrets.toml
|
|
litellm/proxy/api_log.json
|
|
.idea/
|
|
router_config.yaml
|
|
litellm_server/config.yaml
|
|
litellm/proxy/_secret_config.yaml
|
|
.aws-sam/
|
|
litellm/tests/aiologs.log
|
|
litellm/tests/exception_data.txt
|
|
litellm/tests/config_*.yaml
|
|
litellm/tests/langfuse.log
|
|
langfuse.log
|
|
.langfuse.log
|
|
.pin_list.txt
|
|
.cov_new.xml
|
|
litellm/tests/test_custom_logger.py
|
|
litellm/tests/langfuse.log
|
|
litellm/tests/dynamo*.log
|
|
.vscode/settings.json
|
|
litellm/proxy/log.txt
|
|
proxy_server_config_@.yaml
|
|
.gitignore
|
|
proxy_server_config_2.yaml
|
|
litellm/proxy/secret_managers/credentials.json
|
|
hosted_config.yaml
|
|
litellm/proxy/tests/node_modules
|
|
litellm/proxy/tests/package.json
|
|
litellm/proxy/tests/package-lock.json
|
|
ui/litellm-dashboard/.next
|
|
ui/litellm-dashboard/node_modules
|
|
ui/litellm-dashboard/next-env.d.ts
|
|
ui/litellm-dashboard/package.json
|
|
ui/litellm-dashboard/package-lock.json
|
|
deploy/charts/litellm/*.tgz
|
|
deploy/charts/litellm/charts/*
|
|
deploy/charts/*.tgz
|
|
litellm/proxy/vertex_key.json
|
|
**/.vim/
|
|
**/node_modules
|
|
kub.yaml
|
|
loadtest_kub.yaml
|
|
litellm/proxy/_new_secret_config.yaml
|
|
litellm/proxy/_new_secret_config.yaml
|
|
litellm/proxy/_super_secret_config.yaml
|
|
litellm/proxy/_super_secret_config.yaml
|
|
litellm/proxy/myenv/bin/activate
|
|
litellm/proxy/myenv/bin/Activate.ps1
|
|
myenv/*
|
|
litellm/tests/log.txt
|
|
litellm/tests/langfuse.log
|
|
litellm/tests/langfuse.log
|
|
litellm/proxy/google-cloud-sdk/*
|
|
tests/llm_translation/log.txt
|
|
venv/
|
|
tests/local_testing/log.txt
|
|
|
|
.codegpt
|
|
litellm/proxy/_new_new_secret_config.yaml
|
|
litellm/proxy/custom_guardrail.py
|
|
litellm/proxy/application.log
|
|
tests/llm_translation/vertex_test_account.json
|
|
tests/llm_translation/test_vertex_key.json
|
|
litellm/proxy/migrations/0_init/migration.sql
|
|
litellm/proxy/db/migrations/0_init/migration.sql
|
|
litellm/proxy/db/migrations/*
|
|
litellm/proxy/migrations/*config.yaml
|
|
litellm/proxy/migrations/*
|
|
litellm/proxy/to_delete_loadtest_work/*
|
|
config.yaml
|
|
tests/litellm/litellm_core_utils/llm_cost_calc/log.txt
|
|
tests/test_custom_dir/*
|
|
test.py
|
|
|
|
litellm_config.yaml
|
|
!.github/observatory/litellm_config.yaml
|
|
.cursor
|
|
litellm/proxy/to_delete_loadtest_work/*
|
|
update_model_cost_map.py
|
|
tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server_manager.py
|
|
scripts/test_vertex_ai_search.py
|
|
LAZY_LOADING_IMPROVEMENTS.md
|
|
STABILIZATION_TODO.md
|
|
**/test-results
|
|
**/playwright-report
|
|
**/*.storageState.json
|
|
**/coverage
|
|
test-config
|
|
|
|
# ---------- Terraform ----------
|
|
# Provider binaries + module cache — regenerated by `terraform init`.
|
|
**/.terraform/
|
|
# State files often contain secrets (DB passwords, API keys snapshotted from
|
|
# data sources). Keep state in a remote backend, never in git.
|
|
*.tfstate
|
|
*.tfstate.*
|
|
*.tfstate.backup
|
|
# Plan files can also contain sensitive values (variables in plaintext).
|
|
*.tfplan
|
|
# User-specific variable inputs — example files (terraform.tfvars.example) are
|
|
# tracked because they end in .example, which doesn't match the glob below.
|
|
*.tfvars
|
|
*.auto.tfvars
|
|
crash.log
|
|
crash.*.log
|
|
# .terraform.lock.hcl is intentionally NOT ignored — it pins provider versions
|
|
# and should be committed.
|
|
.vscode
|
|
.pin_list.txt
|
|
|
|
# pytest coverage data
|
|
.coverage
|