Commit graph

23545 commits

Author SHA1 Message Date
Alex Strick van Linschoten
ab1dbe355a
feat(langfuse-otel): Add comprehensive metadata support to Langfuse OpenTelemetry integration
This commit brings the langfuse_otel integration to feature parity with the vanilla Langfuse integration by adding support for all metadata fields.

Changes:
- Extended LangfuseSpanAttributes enum with all supported metadata fields:
  - Generation-level: generation_name, generation_id, parent_observation_id, version, mask_input/output
  - Trace-level: trace_user_id, session_id, tags, trace_name, trace_id, trace_metadata, trace_version, trace_release, existing_trace_id, update_trace_keys
  - Debug: debug_langfuse

- Implemented metadata extraction and mapping in langfuse_otel.py:
  - Added _extract_langfuse_metadata() helper to extract metadata from kwargs
  - Support for header-based metadata (langfuse_* headers) via proxy
  - Enhanced _set_langfuse_specific_attributes() to map all metadata to OTEL attributes
  - JSON serialization for complex types (lists, dicts) for OTEL compatibility

- Updated documentation:
  - Added 'Metadata Support' section explaining all fields are now supported
  - Provided usage example showing how to pass metadata
  - Clarified that traces are viewed in Langfuse UI (not generic OTEL backends)
  - Added opentelemetry-exporter-otlp to required dependencies

This allows users to pass metadata like:
metadata={
    'generation_name': 'my-generation',
    'trace_id': 'trace-123',
    'session_id': 'session-456',
    'tags': ['prod', 'v1'],
    'trace_metadata': {'user_type': 'premium'}
}

All metadata is exported as OpenTelemetry span attributes with 'langfuse.*' prefix for easy filtering and analysis in the Langfuse UI.
2025-07-24 23:51:03 +02:00
Cole McIntosh
8c99c71f38
Merge pull request #12939 from superpoussin22/correct_doc
Update control_plane_and_data_plane.md
2025-07-24 09:17:01 -06:00
superpoussin22
ca9712c587
Update control_plane_and_data_plane.md 2025-07-24 16:15:55 +02:00
Ishaan Jaff
3e529fae7f
[UI] Allow setting up CloudZero Usage through LiteLLM UI (#12923)
* refactor cz settings

* allow viewing/setting CZ settings

* working button

* fixes for export button

* nicer button

* fix className
2025-07-23 21:17:12 -07:00
Apoorv Sharma
29475ab4b0
Add GA version of gemini 2.5 flash lite for both vertex and gemini (#12920)
This PR adds the config for gemini 2.5 flash lite which is out of preview now
2025-07-23 18:47:46 -07:00
Ishaan Jaff
e3b752d3dc
[Feat] LiteLLM CloudZero Integration updates - using LiteLLM_SpendLogs Table (#12922)
* use spend logs in transform.py

* fixes for using spend logs

* get_usage_data

* update get_usage_data_for_hour

* fix CloudZeroLogger

* fixes for cloudzero_dry_run_export

* working DB read

* fix _aggregate_to_hourly

* update cloudzero endpoints

* fix cloudzero_export

* fixes for CBF transformer

* use init_background_job on saving cz settings

* add is_cloudzero_setup_in_db

* init cz on startup

* fixes for CZ transform

* fix ruff check

* fix LOC violation
2025-07-23 18:46:50 -07:00
Ishaan Jaff
99031bf8b6 ci/cd new release 2025-07-23 13:50:36 -07:00
Ishaan Jaff
461cd0c30a test_completion_cost_deepseek 2025-07-23 13:16:12 -07:00
Ishaan Jaff
2d0187824c test_proxy_model_resolution_with_custom_names_documentation 2025-07-23 13:13:46 -07:00
Ishaan Jaff
79a0841719 test_router_content_policy_fallbacks 2025-07-23 13:04:28 -07:00
Ishaan Jaff
99ad24c683
[Feat] LiteLLM x Cloudzero integration - Allow exporting spend to cloudzero (#12908)
* add cz init

* add DataAnalyzer

* add CZRNGenerator -> LiteLLM resource generator

* add CloudZeroStreamer

* add CBFTransformer

* add cloudzero DB connection

* add config

* add cli.py

* cleanup

* add CloudZeroLogger

* add CloudZeroLogger

* update CloudZeroLogger

* add cloudzero_router to litellm proxy

* add CloudZeroInitRequest

* add cloudzero spend endpoints

* simple dry run endpoint

* refactor dir structure

* add well types CBFRecord

* TestCBFTransformer

* TestCloudZeroStreamer

* ruff fix

* add polars pip + docker requirements.txt

* _group_by_date

* update code qa check

* docs for CZ params
2025-07-23 13:00:58 -07:00
Krish Dholakia
c5c2a68510
Swagger - bump DOMPurify version + add back Support local swagger bundle (good for airgapped environments) + support local swagger on custom root paths (#12911)
* fix(proxy_server.py): update swagger-ui-bundle.js + bring back swagger in airgapped environments

* fix(proxy_server.py): support local swagger on custom root path

enables on prem usage of litellm
2025-07-23 11:22:56 -07:00
Ishaan Jaff
0d2e280e78 ci/cd new release 2025-07-23 09:04:26 -07:00
Ishaan Jaff
00fd020291 fix test 2025-07-23 09:04:09 -07:00
Krrish Dholakia
4277becdc2 docs(credentials.tsx): cleanup reference to dead link 2025-07-23 09:02:39 -07:00
Ishaan Jaff
60ed272ae0 ci/cd litellm 2025-07-23 07:15:33 -07:00
Ishaan Jaff
5616aaee6d docs pillar guard 2025-07-23 07:15:02 -07:00
Ishaan Jaff
477ca5c0f6 test_proxy_all_models 2025-07-23 07:11:35 -07:00
Krrish Dholakia
663dea37a8 docs: fix docs 2025-07-22 23:30:50 -07:00
Igal Boxerman
d118cc7098
feat: Add Pillar Security guardrail integration (#12791)
* feat: Add Pillar Security guardrail integration

Implements comprehensive LLM security guardrails using Pillar Security API with support for prompt injection detection, PII/secret detection, content moderation, and multi-mode execution (pre_call, during_call, post_call). Includes complete documentation, testing, and configurable actions on flagged content.

* fix: Resolve MyPy type error in Pillar guardrail config

Restructure PillarGuardrailConfigModel to properly inherit from GuardrailConfigModel[T]
and resolve return type compatibility issue in get_config_model method.

* fix: Resolve MyPy type error in Pillar guardrail config

Restructure PillarGuardrailConfigModel to properly inherit from GuardrailConfigModel[T]
and resolve return type compatibility issue in get_config_model method.

* fix docs

* fix docs

* improved docs

* fix examples, READY
2025-07-22 23:17:33 -07:00
Murad Khafizov
a6ddf5c744
feat: extended /v1/models endpoint, now it returns with fallbacks on demand (#12811)
* Extended `/v1/model` endpoint to support fallbacks

* unit tests reworked

* linting fixes

* fix lining error

* fix linting
2025-07-22 23:16:46 -07:00
Stuart Geiger
37c626a9a0
rm retired anthropic models from model_prices_and_context_window.json (#12864)
rm claude-2, claude-2.1, claude-3-sonnet-20240229
2025-07-22 23:14:37 -07:00
Jugal D. Bhatt
03d5102044
[Add health check] add architecture diagram (#12879)
* add architecture diagram

* add image path

* add new image
2025-07-22 23:12:19 -07:00
Jugal D. Bhatt
405eb7f243
[Docs ]Litellm mcp access group doc (#12883)
* add example for config

* fix typo
2025-07-22 23:11:53 -07:00
Krish Dholakia
d26f74ade5
Request Headers - support x-litellm-num-retries + Usage - support usage by model group (#12890)
* feat(litellm_pre_call_utils.py): add num_retries to litellm data for backend call

allow user to pass in num retries via request headers

* test(test_litellm_pre_call_utils.py): add unit test

* docs(request_headers.md): document new request header

* fix(common_daily_activity.py): show spend breakdown by model group

Partial fix for https://github.com/BerriAI/litellm/issues/12887

* feat(new_usage.tsx): new tab switcher for viewing usage by model group vs. received model

Closes https://github.com/BerriAI/litellm/issues/12887
2025-07-22 23:09:36 -07:00
Ishaan Jaff
a0846c3f31 ci/cd new release 2025-07-22 21:08:02 -07:00
Ishaan Jaff
642cfa26b0 remove deprecated 2025-07-22 20:59:34 -07:00
Ishaan Jaff
3a76b2762f ci/cd new release 2025-07-22 20:41:49 -07:00
Ishaan Jaff
5741cdae0f fix openrouter/qwen/qwen-vl-plus 2025-07-22 20:37:57 -07:00
Krish Dholakia
8cd6c25e1e
Fix async retryer on .acompletion() + forward clientside headers - filter out content-type from clientside request (causes llm api call to hang) (#12886)
* fix(main.py): fix async retryer

Fixes https://github.com/BerriAI/litellm/issues/12830

* fix(forward_clientside_headers_by_model_group.py): filter out 'content-type' from forwardable headers

clientside content-type != proxy content type, can cause requests to hang
2025-07-22 19:50:05 -07:00
Ishaan Jaff
b41ce5c92f
[Feat] - Track cost + add tags for health checks done by LiteLLM Proxy (#12880)
* refactor to use add_user_api_key_auth_to_request_metadata

* add get_litellm_internal_health_check_user_api_key_auth

* add get_litellm_internal_health_check_user_api_key_auth

* add _update_model_params_with_health_check_tracking_information

* add HealthCheckHelpers

* refactor to use clean helpers

* test_update_model_params_with_health_check_tracking_information

* test_get_litellm_internal_health_check_user_api_key_auth

* test_add_user_api_key_auth_to_request_metadata

* fix _update_model_params_with_health_check_tracking_information
2025-07-22 18:45:57 -07:00
Ishaan Jaff
c21dc46a33 fix morph api tests 2025-07-22 18:44:44 -07:00
Ishaan Jaff
bd8b21e700 fix mypy linting 2025-07-22 18:38:54 -07:00
Ishaan Jaff
d93e85282e test_bad_database_url 2025-07-22 18:37:51 -07:00
Ishaan Jaff
bf300f8ca7 Revert "Litellm dev 07 21 2025 p1 (#12848)"
This reverts commit e4e10aa4ed.
2025-07-22 18:28:36 -07:00
Ishaan Jaff
f0a8abb911 fix cost_calculator 2025-07-22 18:16:53 -07:00
Ishaan Jaff
4b741adadb fix recraft cost calc 2025-07-22 18:12:12 -07:00
Ishaan Jaff
e5debfc84c fix model cost map for recraft 2025-07-22 18:11:10 -07:00
Ishaan Jaff
1910cf8496 test fix vertex ai 2025-07-22 18:06:38 -07:00
Ishaan Jaff
b026f6b280
[Feat] Add cost tracking for new vertex_ai/llama-3 API models (#12878)
* add vertex_ai/meta/llama-3.1-405b-instruct-maas

* notes - vertex_ai/meta/llama-3.2-90b-vision-instruct-maas
2025-07-22 16:22:09 -07:00
Jugal D. Bhatt
c9899b5c06
[LLM Translation] Litellm gemini 2.0 live support (#12839)
* add gemini 2.0 live to model context and priceS

* added files to dump

* Update litellm/model_prices_and_context_window_backup.json

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update model_prices_and_context_window.json

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* added vertex ai live preview

* added vertex ai change

* add input video and image cosT

* add input video and image cosT

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-22 15:14:52 -07:00
Ishaan Jaff
d5ee93aa0c
[Feat] Add Recraft API - Image Edits Support (#12874)
* test_recraft_image_edit_api

* add RecraftImageEditConfig

* complete RecraftImageEditConfig

* add RecraftImageEditRequestParams in types

* update RecraftImageEditRequestParams

* working

* transform_image_edit_request

* Image Edit docs recraft

* working transform_image_edit_request

* TestRecraftImageEditTransformation
2025-07-22 15:03:08 -07:00
Ishaan Jaff
31e9303232 remove old test 2025-07-22 14:11:43 -07:00
Jugal D. Bhatt
aa1ebf37b2
pass in cmd args (#12871) 2025-07-22 14:10:23 -07:00
Ishaan Jaff
934af0e9a0
add azure-keyvault==4.2.0 (#12873) 2025-07-22 14:08:33 -07:00
dependabot[bot]
d4900d7dc0
build(deps): bump form-data from 4.0.3 to 4.0.4 in /docs/my-website (#12867)
---
updated-dependencies:
- dependency-name: form-data
  dependency-version: 4.0.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-22 11:23:41 -07:00
Ishaan Jaff
6f0c211788 [Feat] MCP Gateway - allow using MCPs with all LLM APIs when using /responses with LiteLLM (#12546)
* add MCPResponsesAPIHelper

* rename LiteLLM_Proxy_MCP_Handler

* aresponses_api_with_mcp

* mock_responses_api_response

* test response with litellm proxy MCP

* add _should_use_litellm_mcp_gateway

* fix transform_mcp_tool_to_openai_responses_api_tool

* use correct _transform_mcp_tools_to_openai

* fix config.yaml

* fixes for native MCP handling

* docs MCP with litellm proxy

* aresponses_api_with_mcp

* fix linting

* fix mypy

* fix linting

* test_aresponses_api_with_mcp_mock_integration

* docs How it works when server_url="litellm_proxy"
2025-07-22 10:22:45 -07:00
Krrish Dholakia
e683a41993 bump: version 1.74.7 → 1.74.8 2025-07-22 08:54:03 -07:00
Mateo Di Loreto
c65392cf81
Replace non-root Dockerfile base with Alpine multi-stage build; (#12707)
* Change Dockerfile.noon_root with alpine base image

* Improve non_root docker image

* Re add the build_admin_ui.sh script step

* Re add the build_admin_ui.sh script step

* Remove unnecessary workdir set

* Remove unnecessary workdir set

* Configure chainguard image

* A bit of optimization and improve comments

* delete extra build_ui script run

* Optimizie Dockerfile copy statements
2025-07-22 08:53:10 -07:00
Krrish Dholakia
964c3b7a3d docs(index.md): cleanup 2025-07-22 08:47:52 -07:00