scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-17 23:51:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 67
litellm/docs/my-website/docs/proxy
History
Krrish Dholakia 386f334fee
Prompt Compression - add it to the proxy (#25729) 
* refactor: new agentic loop event hook

simplifies how to create logic for tool based multi llm calls

* fix: compress - make it work on anthropic input as well

* fix(compress.py): working prompt compression for claude code

ensures claude code messages can run through proxy easily

* docs: add agentic loop hook guide

* docs: add agentic_loop_hook to sidebar

* fix: fix multiple arguments error

* fix: fix tool call loop for compression on streaming /v1/messages

* fix: fix linting errors

* fix: fix ci/cd errors

* feat(litellm_pre_call_utils.py): use claude code session for litellm session id

allows claude code logs to be stitched together, making it easy to know they were all part of the same conversation

* fix: suppress incorrect mypy warning rE: module

* revert: drop PR's changes to litellm/proxy/_experimental/out/

Restores the 34 HTML files under _experimental/out/ to their pre-PR
paths (X/index.html -> X.html). All renames are R100 (content
unchanged); no other files are touched.

* fix: address greptile review comments on PR #25729

- Skip ``kwargs["tools"] = []`` injection when compression is a no-op —
  Anthropic Messages rejects empty tool arrays on requests that did not
  originally declare tools.
- Move agentic-loop safety guards (fingerprint cycle / max depth) out of
  the per-callback try/except so they propagate instead of being swallowed
  by the generic exception handler. Extracted _check_agentic_loop_safety.
- Gate generic ``x-<vendor>-session-id`` capture behind the
  LITELLM_CAPTURE_VENDOR_SESSION_HEADERS env var (off by default) to
  preserve backwards compatibility; explicit x-litellm-* headers are
  unaffected.
- Fix monkeypatch target in pre-call-hook test to patch the actual
  module-level binding
  (litellm.integrations.compression_interception.handler.compress).
- Add regression tests for empty-tools skip and opt-in session capture.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* revert: drop LITELLM_CAPTURE_VENDOR_SESSION_HEADERS flag

Generic x-<vendor>-session-id header capture is a new feature and only
runs *after* the explicit x-litellm-trace-id / x-litellm-session-id
checks, so it does not change behavior for any existing caller that was
already using the LiteLLM headers — no backwards-incompatibility to gate.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(compress): replace input_type with CallTypes call_type

Drop the bespoke ``CompressionInputType`` literal and use the existing
``litellm.types.utils.CallTypes`` enum instead.  ``litellm.compress()``
now takes ``call_type: Union[CallTypes, str]`` (default
``CallTypes.completion``) — no new concept to learn, and the enum is
already the way the rest of the codebase talks about request shapes.

Supported values: ``completion`` / ``acompletion`` (OpenAI chat-completions
shape) and ``anthropic_messages`` (Anthropic structured content blocks).

Updated: compress(), the compression_interception handler, tests, docs,
and the two eval scripts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-20 15:08:00 -07:00
..
guardrails docs update 2026-04-14 18:33:42 -07:00
ui ui logo (#23556) 2026-03-13 08:44:08 -07:00
access_control.md fix: prompt registry 2026-02-18 00:34:54 +05:30
access_groups.md fix: prompt registry 2026-02-18 00:34:54 +05:30
admin_ui_sso.md fix: update Okta SSO docs and custom SSO handler example 2026-03-04 17:59:55 +02:00
agentic_loop_hook.md Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
ai_hub.md docs(ai_hub.md): document mcp servers on ai hub 2025-11-22 16:07:38 -08:00
alerting.md feat: add optional digest mode for Slack alert types (#21683) 2026-02-21 15:19:17 -08:00
architecture.md (Feat) - Show Error Logs on LiteLLM UI (#8904) 2025-02-28 20:10:09 -08:00
arize_phoenix_prompts.md Arize Phoenix OSS - Prompt Management Integration (#17750) 2025-12-09 22:53:42 -08:00
auto_routing.md docs: v1.81.14-stable release notes (#21839) 2026-02-21 14:50:50 -08:00
billing.md Fix #11856: Update billing.md docs to call the new GPT-4o model (#11858) 2025-06-18 08:19:36 -07:00
budget_reset_and_tz.md docs: Clean up budget reset and timezones documentation (#22428) 2026-02-28 17:37:25 -08:00
caching.md build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
call_hooks.md Enhance documentation and sidebar structure 2026-03-17 02:10:06 +05:30
cli.md fix: prompt registry 2026-02-18 00:34:54 +05:30
cli_sso.md fix: clarify EXPERIMENTAL_UI_LOGIN ignores LITELLM_UI_SESSION_DURATION, add regression test 2026-02-26 04:41:07 -08:00
clientside_auth.md [Security] Fixes for docs (#11776) 2025-06-16 16:20:55 -07:00
config_management.md Revert "Revert "(feat) Allow using include to include external YAML files in a config.yaml (#6922)"" 2024-11-27 16:08:59 -08:00
config_settings.md fix(proxy): prioritize reasoning health check token defaults 2026-04-17 12:36:58 +05:30
configs.md docs: add LiteLLM license key environment variable instructions 2026-03-20 00:28:21 +00:00
control_plane_and_data_plane.md [Feature] UI - Allow setting base_url in API reference docs (#16674) 2025-11-18 11:27:28 -08:00
cost_tracking.md docs(troubleshoot): add cost discrepancy debugging guide 2026-04-13 11:27:16 +05:30
credential_routing.md feat(proxy): add credential overrides per team/project via model_config metadata (#24438) 2026-04-09 07:22:27 -07:00
credential_usage_tracking.md [Docs] Add Credential Usage Tracking documentation 2026-02-25 10:52:24 -08:00
custom_auth.md fix: prompt registry 2026-02-18 00:34:54 +05:30
custom_pricing.md docs: add PayGo/priority cost tracking for Gemini Vertex AI 2026-03-06 08:36:31 +05:30
custom_prompt_management.md Add sdk focused examples (#16441) 2025-11-10 18:39:34 -08:00
custom_root_ui.md Fix rendering ui on non-root images (#12226) 2025-07-01 22:08:35 -07:00
custom_sso.md fix: update Okta SSO docs and custom SSO handler example 2026-03-04 17:59:55 +02:00
customer_routing.md Controll fallback prompts client-side (#7334) 2024-12-20 19:09:53 -08:00
customer_usage.md fix: prompt registry 2026-02-18 00:34:54 +05:30
customers.md Mcp user permissions (#21462) 2026-02-18 18:53:59 -08:00
db_deadlocks.md fix: prompt registry 2026-02-18 00:34:54 +05:30
db_info.md docs: fix docs 2025-11-17 17:54:26 -08:00
debugging.md docs(debugging.md): document new feature 2025-09-11 20:17:39 -07:00
deleted_keys_teams.md fix: prompt registry 2026-02-18 00:34:54 +05:30
deploy.md docs: add Docker Image Security Guide for cosign verification and deployment best practices (#25439) 2026-04-09 23:58:35 -07:00
docker_image_security.md docs: add Docker Image Security Guide for cosign verification and deployment best practices (#25439) 2026-04-09 23:58:35 -07:00
docker_quick_start.md build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
dynamic_logging.md [Feat] AI Gateway - Allow admins to disable, dynamic callback controls (#16750) 2025-11-17 18:29:07 -08:00
dynamic_rate_limit.md docs: link dynamic TPM/RPM limiting to request prioritization doc (#22988) 2026-03-07 19:27:41 -08:00
email.md docs: document new github + gitlab ci scripts 2026-03-25 20:17:10 -07:00
embedding.md fix: prompt registry 2026-02-18 00:34:54 +05:30
endpoint_activity.md Enhance documentation and sidebar structure 2026-03-17 02:10:06 +05:30
enterprise.md docs: document new github + gitlab ci scripts 2026-03-25 20:17:10 -07:00
error_diagnosis.md [Docs] Add guide on how to debug gateway error vs provider error (#17387) 2025-12-02 14:10:00 -08:00
fallback_management.md fix: prompt registry 2026-02-18 00:34:54 +05:30
forward_client_headers.md feat(proxy): client-side provider API key precedence for Anthropic /v1/messages 2026-03-06 18:20:46 +05:30
health.md fix(proxy): prioritize reasoning health check token defaults 2026-04-17 12:36:58 +05:30
health_check_routing.md Litellm ishaan april4 2 (#25150) 2026-04-04 23:09:42 +00:00
high_availability_control_plane.md [Fix] Hide connector label on mobile, remove stale master key mention 2026-03-28 15:51:44 -07:00
image_handling.md docs(image_handling.md): architecture doc on image handling on the proxy 2025-03-21 14:19:05 -07:00
ip_address.md docs: document new github + gitlab ci scripts 2026-03-25 20:17:10 -07:00
jwt_auth_arch.md Litellm release notes 07 12 2025 (#12563) 2025-07-12 19:31:13 -07:00
jwt_key_mapping.md docs: add mermaid sequence diagram for JWT → key resolution flow 2026-03-31 16:33:27 -07:00
keys_teams_router_settings.md Enhance documentation and sidebar structure 2026-03-17 02:10:06 +05:30
litellm_managed_files.md fix: prompt registry 2026-02-18 00:34:54 +05:30
litellm_prompt_management.md docs - ai gateway prompt management (#16990) 2025-11-22 18:49:29 -08:00
load_balancing.md Merge pull request #24611 from Sameerlite/Sameerlite/order-fallback2 2026-03-27 20:15:30 +05:30
logging.md build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
logging_spec.md feat: change guardrail_information to list type (#16127) 2025-11-02 09:47:49 -08:00
managed_batches.md docs(managed_batches.md): update docs 2025-11-17 18:00:08 -08:00
managed_finetuning.md docs(managed_finetuning.md): cleanup 2025-05-24 18:21:06 -07:00
management_cli.md [Feat] AI Gateway Auth - Allow using JWTs for signing in with Proxy CLI (#16756) 2025-11-17 19:47:29 -08:00
master_key_rotations.md docs: add doc for rotating master key 2025-03-06 23:22:01 -08:00
metrics.md (docs) show /daily_metrics 2024-02-27 19:34:47 -08:00
model_access.md [Docs] Add docs on APIs for model access management (#16673) 2025-11-14 17:07:54 -08:00
model_access_groups.md [Docs] Add docs on APIs for model access management (#16673) 2025-11-14 17:07:54 -08:00
model_access_guide.md [Docs] Add docs on APIs for model access management (#16673) 2025-11-14 17:07:54 -08:00
model_compare_ui.md Enhance documentation and sidebar structure 2026-03-17 02:10:06 +05:30
model_discovery.md Litellm release notes 04 19 2025 (#10169) 2025-04-19 17:26:30 -07:00
model_management.md fix: broken link on model_management.md (#16217) 2025-11-03 17:00:03 -08:00
multi_tenant_architecture.md docs(multi_tenant_architecture.md): add new architecture doc 2025-12-04 11:13:50 -08:00
multiple_admins.md docs: document new github + gitlab ci scripts 2026-03-25 20:17:10 -07:00
native_litellm_prompt.md Gitlab based Prompt manager (#14988) 2025-10-01 18:13:11 -07:00
oauth2.md fix(auth): support selective jwt override oauth2 routing (#25252) 2026-04-07 13:52:47 -07:00
pagerduty.md Litellm release notes 07 12 2025 (#12563) 2025-07-12 19:31:13 -07:00
pass_through.md docs: remove phone numbers from readme and docs 2026-03-25 12:40:40 -07:00
pass_through_guardrails.md [Feat] UI - allow adding pass through guardrails through UI (#17226) 2025-11-27 12:27:16 -08:00
perf.md docs - fix build time issues 2024-06-15 16:12:37 -07:00
pricing_calculator.md fix: prompt registry 2026-02-18 00:34:54 +05:30
prod.md docs: document new github + gitlab ci scripts 2026-03-25 20:17:10 -07:00
project_management.md docs project management 2025-11-12 16:24:23 -08:00
prometheus.md build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
prompt_management.md Add support for prompt management for responses 2026-03-18 15:48:31 +05:30
provider_budget_routing.md tag budgets fixes 2024-12-18 10:28:37 -08:00
provider_discounts.md fix: prompt registry 2026-02-18 00:34:54 +05:30
provider_margins.md fix: prompt registry 2026-02-18 00:34:54 +05:30
public_routes.md docs: document new github + gitlab ci scripts 2026-03-25 20:17:10 -07:00
public_teams.md Doc updates + management endpoint fixes (#8138) 2025-01-30 22:56:41 -08:00
pyroscope_profiling.md build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
quick_start.md build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
rate_limit_tiers.md docs(index.md): new release notes 2024-12-26 22:01:29 -08:00
realtime_webrtc.md Fix the live tester 2026-03-12 22:14:08 +05:30
reject_clientside_metadata_tags.md Add enforce user param functionality (#17088) 2025-11-25 09:36:24 -08:00
release_cycle.md fix: prompt registry 2026-02-18 00:34:54 +05:30
reliability.md doc improvement 2026-03-09 15:39:27 -07:00
request_headers.md Mcp user permissions (#21462) 2026-02-18 18:53:59 -08:00
request_tags.md Proxy request tags docs (#22129) 2026-02-25 14:29:00 -08:00
response_headers.md docs latency headers 2025-06-26 07:58:23 -07:00
rules.md (docs) use port 4000 2024-03-08 21:59:00 -08:00
security_encryption_faq.md docs(security_encryption_faq.md): answers common questions about data at rest vs. in transit 2025-10-06 17:34:10 -07:00
self_serve.md docs: clarify that models sub-field only applies to SSO auto-created teams 2026-04-02 16:00:20 -07:00
service_accounts.md Litellm dev 07 05 2025 p4 (#12350) 2025-07-05 19:38:42 -07:00
shared_health_check.md fix: prompt registry 2026-02-18 00:34:54 +05:30
spend_logs_deletion.md fix: prompt registry 2026-02-18 00:34:54 +05:30
sync_anthropic_beta_headers.md Enhance documentation and sidebar structure 2026-03-17 02:10:06 +05:30
sync_models_github.md docs fix spend tracking (#16675) 2025-11-14 17:22:21 -08:00
tag_budgets.md [Feat] Tag Management - Add support for setting tag based budgets (#15433) 2025-10-10 19:24:50 -07:00
tag_routing.md docs: document new github + gitlab ci scripts 2026-03-25 20:17:10 -07:00
team_based_routing.md Controll fallback prompts client-side (#7334) 2024-12-20 19:09:53 -08:00
team_budgets.md added doc 2026-03-13 17:24:15 -07:00
team_logging.md docs: document new github + gitlab ci scripts 2026-03-25 20:17:10 -07:00
team_model_add.md Announce April Townhall (#25021) 2026-04-02 14:10:49 -07:00
temporary_budget_increase.md Litellm release notes 07 12 2025 (#12563) 2025-07-12 19:31:13 -07:00
timeout.md Update docs for stream_timeout and timeout (#14073) 2025-08-29 09:59:07 -07:00
token_auth.md fix(auth): support selective jwt override oauth2 routing (#25252) 2026-04-07 13:52:47 -07:00
ui.md Adding a small section to the UI docs 2025-11-20 20:31:38 -08:00
ui_credentials.md [Docs] Add Credential Usage Tracking documentation 2026-02-25 10:52:24 -08:00
ui_logs.md fix: prompt registry 2026-02-18 00:34:54 +05:30
ui_logs_sessions.md fix(session): correctly place litellm_session_id at root level instead of metadata (#11088) 2025-05-24 09:12:11 -07:00
ui_project_management.md Update docs/my-website/docs/proxy/ui_project_management.md 2026-02-28 18:15:09 -08:00
ui_spend_log_settings.md fix: prompt registry 2026-02-18 00:34:54 +05:30
ui_store_model_db_setting.md Enhance documentation and sidebar structure 2026-03-17 02:10:06 +05:30
ui_team_soft_budget_alerts.md fix(proxy): use flat soft_budget field for virtual key projected-spend alerts 2026-04-15 21:38:18 -07:00
user_keys.md build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
user_management_heirarchy.md (feat) Organizations: allow org admins to create teams on UI + (feat) IBM Guardrails (#15924) 2025-10-25 11:13:39 -07:00
user_onboarding.md Enhance documentation and sidebar structure 2026-03-17 02:10:06 +05:30
users.md feat: multiple concurrent budget windows per API key and team (#24883) (#25109) 2026-04-06 14:02:04 -07:00
veo_video_generation.md [Feat] Allow using Veo Video Generation through LiteLLM Pass through routes (#14228) 2025-09-03 18:25:43 -07:00
virtual_keys.md fix: prompt registry 2026-02-18 00:34:54 +05:30
worker_startup_hooks.md Gflags worker parameters (#22931) 2026-03-06 18:09:57 -08:00