Cast Polars Decimal columns to Float64 before calling .to_dicts() in
vantage_dry_run_export so the response contains JSON-serializable float
values instead of decimal.Decimal objects that FastAPI cannot encode.
Also cast summary totals to float for the same reason.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- DELETE /vantage/delete now removes the in-memory VantageLogger from
litellm.callbacks via remove_callbacks_by_type, preventing the
scheduler from continuing to fire exports with stale credentials
- Add test_should_cast_decimal_columns_to_float covering the
Decimal→Float64 cast in FocusCsvSerializer
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Matches the existing validator on VantageInitRequest so that empty or
whitespace-only api_key/integration_token values are rejected at update
time rather than silently persisted and failing at the next export.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Override initialize_focus_export_job in VantageLogger to use
VANTAGE_USAGE_DATA_JOB_NAME as the Redis pod lock key, preventing
silent export skips when both Focus and Vantage loggers are configured
- Add field_validator to VantageInitRequest rejecting empty-string
api_key and integration_token at init time instead of at export time
- Guard FocusLogger FOCUS_INTERVAL_SECONDS with try/except matching
the pattern already used in VantageLogger
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use `type(cb) is FocusLogger` instead of `isinstance(cb, FocusLogger)`
in both _init_custom_logger_compatible_class and
get_custom_logger_compatible_class so that a VantageLogger already in
_in_memory_loggers is not incorrectly returned for a "focus" lookup.
This ensures users with both "vantage" and "focus" in success_callbacks
get separate loggers for each export destination.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Guard FocusLogger.init_focus_export_background_job with exact type
check (type(cb) is FocusLogger) to exclude VantageLogger subclass,
preventing duplicate hourly exports when VantageLogger is registered
programmatically before startup
- Cast pl.Decimal columns to Float64 in FocusCsvSerializer so CSV
output uses standard floating-point notation instead of fixed-point
strings that Vantage's parser may reject
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Align dry-run summary to use pre-transform columns (spend, total_tokens,
team_id, model) matching FocusExportEngine internals
- Reduce token exposure in debug logs to first 4 chars
- Wrap raise_for_status in try/except to log httpx errors before re-raising
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Split VantageExportRequest (limit=None) and VantageDryRunRequest
(limit=500) so actual exports don't silently truncate large datasets
- Add try/except around each sub-batch upload in _upload_size_limited,
consistent with _upload_batched's continue-on-failure guarantee
- Guard VANTAGE_EXPORT_INTERVAL_SECONDS against non-numeric values
with try/except instead of bare int() cast
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Vantage destination:
- Skip individual CSV rows exceeding 2MB limit with a warning instead of
uploading an oversized batch that Vantage would reject
- Wrap each batch upload in try/except so remaining batches continue on
failure; re-raise the first error after all batches are attempted
Callback registration:
- Use litellm.logging_callback_manager.add_litellm_callback() instead of
raw litellm.callbacks.append() for DB-bootstrapped VantageLogger, ensuring
proper dedup and manager visibility
- Add "vantage" init handler in litellm_logging.py (both creation and
lookup branches) so config.yaml string callbacks are properly resolved
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Polars write_csv already handles empty DataFrames correctly (outputs
header-only CSV), so the conditional branch was a no-op.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add missing except HTTPException: raise in vantage_dry_run_export
(consistent with all other endpoints in the file)
- Fix is_vantage_setup_in_config() to detect both the string "vantage"
and VantageLogger instances in litellm.callbacks, preventing duplicate
logger registration on startup
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add missing `except HTTPException: raise` guard so intentional 404
responses (e.g. when settings are not configured) are not caught by the
generic Exception handler and re-raised as 500 errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Revert FocusExportEngine.dry_run_export_usage_data to use original raw
column names (spend, total_tokens, team_id, model) preserving backward
compatibility for existing callers
- Vantage dry-run endpoint computes its own summary from FOCUS columns
independently, avoiding coupling to the engine method
- Set VantageExportRequest.limit default to 500 (was None) matching docstring
- Add empty-settings guard in /vantage/export returning 404 instead of
deferring ValueError to runtime
- Fix misleading docstring in _build_tags_expr about Rust-level execution
- Clarify Tags schema comment: parquet is self-describing so existing
exports are unaffected
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Mask integration_token in GET /vantage/settings response (renamed field to integration_token_masked)
- Reuse single httpx.AsyncClient across all batch uploads in deliver()
- Align FocusExportEngine.dry_run_export_usage_data to use post-transform FOCUS columns (BilledCost, SubAccountId, ResourceType) matching the Vantage dry-run endpoint
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Enforce 10K row limit in single-shot upload path (not just 2MB size)
- Fix KeyError crash in update_vantage_settings when no settings exist
- Remove unreachable status_code==400 dead code branch
- Make dry-run endpoint work without Vantage credentials by using
FOCUS database + transformer directly instead of VantageLogger
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a pluggable Vantage destination to the existing FOCUS export pipeline,
enabling LiteLLM to export spend data in FOCUS format directly to Vantage's
cost-import API. Supports automatic hourly exports via scheduled background job,
with admin API endpoints for manual control and configuration. Includes CSV
serializer, batching for 10K row / 2MB API limits, and enriched Tags JSON with
team/user/key metadata for Vantage Token Allocation feature.
- Add CSV serializer (FocusCsvSerializer) for FOCUS data
- Add Vantage API destination with automatic batching
- Add VantageLogger that wraps FocusLogger with Vantage defaults
- Add proxy endpoints: /vantage/{init,settings,export,dry-run,delete}
- Register "vantage" callback in logger registry and literal type
- Wire up background job in proxy_server.py startup
- Populate Tags column with JSON metadata (team_id, user_id, user_email, etc.)
- Add 14 unit tests covering serializer, destination, and factory
All tests pass (23 focus tests total, no regressions).
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Fix case-insensitive tool name matching in _tool_name_matches() so that
OpenAPI operationIds (camelCase) match lowercase registered tool names
when filtering by allowed_tools
- Fix get_base_url() to resolve relative server URLs (e.g. /api/v3) by
deriving full base URL from spec_path when OpenAPI spec has relative URLs
- Add tests for case-insensitive matching and filter_tools_by_allowed_tools
Made-with: Cursor
`all_models = user_api_key_dict.models` was creating an alias, so
`_get_models_from_access_groups` (which uses `.pop()`/`.extend()`) would
mutate the cached object in-place. Now both `.models` and `.team_models`
assignments create copies via `list()`.
Added test to verify the input is not mutated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds dedup to get_key_models and get_team_models to prevent duplicate
entries when access group member models overlap with proxy_model_list.
Removes dead assignment of all_models in get_team_models.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a team has "all-proxy-models", the model list expansion now includes
model access group names so they appear in the UI key creation form.
Also fixes get_key_models not forwarding include_model_access_groups to
_get_models_from_access_groups, and removes unused _unfurl_all_proxy_models.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix perform_redaction to handle dict representation of ModelResponse (from model_dump())
- Preserve full choices structure when redacting, redact content/audio in place
- Add _redact_standard_logging_object helper for standard_logging_object field
- Update test_logging_redaction_e2e_test assertions to expect choices format
- Add charity_engine to provider_endpoints_support.json
Fixes: test_standard_logging_payload, test_standard_logging_payload_audio
Made-with: Cursor
Any param in DEFAULT_CHAT_COMPLETION_PARAM_VALUES that arrives via
completion(**kwargs) is now automatically forwarded to
get_optional_params(), even if it's not a named parameter of
completion().
Previously, get_non_default_completion_params() excluded params in
OPENAI_CHAT_COMPLETION_PARAMS (assuming they'd be forwarded via the
named-param path), while optional_param_args only contained explicitly
named params. Params like 'store' that were in the known-params list
but not named params fell through both paths and were silently dropped.
The fix adds a 7-line loop after building optional_param_args that
forwards any kwargs present in DEFAULT_CHAT_COMPLETION_PARAM_VALUES.
This means new OpenAI params only need to be added to the constants
dict — no boilerplate changes to 3+ function signatures required.
Fixes#23087
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
The SageMaker embedding handler was not using _load_credentials(),
which meant aws_role_name and aws_session_name parameters were
ignored. This prevented cross-account role assumption for embeddings
while it worked for completions.
Changes:
- Replace direct boto3 client creation with _load_credentials() call
- Create boto3.Session with assumed credentials
- Add comprehensive unit tests for role assumption
This aligns the embedding handler behavior with the completion handler,
which already supports role assumption via the BaseAWSLLM.get_credentials()
method.
Fixes cross-account SageMaker embedding access where users need to
assume a role in another account to invoke endpoints.
* feat(charity_engine): add Charity Engine provider
Charity Engine is a crowdsourced distributed computing platform that
donates processing power to charitable causes. Its inference API
provides OpenAI-compatible chat, completions, and embeddings endpoints.
* test(charity_engine): add provider config and resolution tests
Verify JSONProviderRegistry config, provider list membership,
model routing for charity_engine/<model>, and Router compatibility.
* feat(charity_engine): add Charity Engine to LlmProviders enum
Enables provider_list membership and LlmProviders.CHARITY_ENGINE
resolution required by the provider and test suite.
* fix(charity_engine): remove api_base_env to fix non-deterministic test
The CHARITY_ENGINE_API_BASE env var could override the base_url in CI,
causing test_charity_engine_provider_resolution to fail intermittently.
* fix(charity_engine): remove trailing slash from base_url
Claude's API returns assistant messages with empty text blocks
({"type": "text", "text": ""}) alongside tool_use blocks during
multi-turn tool-use conversations. These blocks are rejected when
sent back to the API with "text content blocks must be non-empty".
Sanitization already exists for other code paths (/v1/chat/completions
for both Anthropic and Bedrock), but NOT for the /v1/messages native
path. This adds the same treatment by stripping empty text blocks
from messages in async_anthropic_messages_handler before they are
forwarded to the provider.
Fixes#22930
_get_openai_compatible_provider_info already returns an api_base
ending in /v1, but get_models prepended another /v1, producing
.../inference/v1/v1/accounts/... which 404s.
Strip the trailing /v1 from api_base before re-adding it so that
both the default and any user-supplied base work correctly.
Add parametrized tests covering the default URL, trailing-slash,
custom base with /v1, and custom base without /v1.
Fixes#23106
In _get_available_deployments, when streaming mode is active the code
sums time-to-first-token values from item_ttft_latency but divides by
len(item_latency) instead of len(item_ttft_latency). These lists can
have different lengths, producing an incorrect average that skews
lowest-latency routing decisions for streaming requests.
Signed-off-by: JiangNan <1394485448@qq.com>
When value > 1 (e.g., "2mo") and current_month + value > 12,
target_month exceeds valid range (1-12), causing ValueError in
datetime constructor. For example, calling duration_in_seconds("2mo")
in November produces target_month=13.
Use modular arithmetic to correctly wrap months and increment year.
Signed-off-by: JiangNan <1394485448@qq.com>
* add new azure gpt models
* add versionless azure/gpt-5.4 models
* Undated azure/gpt-5.4 alias missing supports_service_tier
* indicate service tier support for azure/gpt-5.3-chat
* fix priority tier pricing for new azure/gpt models
The bug occurred when user data inadvertently contained reserved Python
keywords like 'self', 'params', or '__class__' as keys. When such a dict
was unpacked via **kwargs to LiteLLM_Params() or GenericLiteLLMParams(),
Python raised TypeError because 'self' was passed both implicitly and
as a keyword argument.
The fix:
- Add a Pydantic model_validator(mode='before') to GenericLiteLLMParams
that filters out reserved keys ('self', 'params', '__class__') before
validation
- Move the max_retries str-to-int conversion into the same validator
- Remove the custom __init__ methods from both GenericLiteLLMParams and
LiteLLM_Params, since the validator now handles the preprocessing
- Clean up unused VERTEX_CREDENTIALS_TYPES import
This fix applies to all classes that inherit from GenericLiteLLMParams,
including LiteLLM_Params and updateLiteLLMParams.
Added comprehensive tests in tests/test_litellm/test_litellm_params_reserved_keys.py
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
