mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-16 04:29:31 +00:00
e344dbd8db
* feat(proxy): split Azure PTU deployments across teams by share
A PTU deployment's model_info takes ptu_shares (team_id -> whole PTUs adding up to
ptu_count) in place of a single team_id. With LITELLM_ENABLE_PTU_COST_ATTRIBUTION on,
routing hides the deployment from teams holding no share, the v3 limiter enforces each
share as a per-team TPM ceiling from Azure's per-model TPM-per-PTU sizing, the daily
flat-cost rollup splits the hourly cost by share, and /team/daily/activity plus the Team
Usage page report PTU-hours next to tokens.
* fix(proxy): weight cached input in PTU ceilings and size team-scoped deployments by public name
The PTU flag is read from the process environment on every check instead of
through the secret-manager path, so a proxy with a hosted secret manager does
not pay a round trip per request. A team's ceiling now carries the model's
cached-input ratio and settlement charges cache reads at that ratio, matching
how Azure sizes a PTU. A deployment registered for one team is found by its
team_public_model_name for PTU-hours and the ceiling, since its model_name is
rewritten to a unique internal name. ptu_shares are StrictInt on ModelInfo and
parsed_ptu_shares refuses booleans, strings, and non-string team ids, a
fractional ptu_count reserves nothing, and models_processed counts distinct
deployments rather than holdings.
* fix(proxy): reserve PTU ceilings in normalized tokens, keep share routing flag-independent, and warn only shared or Azure reservations
* fix(proxy): size the uncapped output floor against the PTU share in raw output tokens and escape the exported team id
* fix(ptu): refuse ptu_shares naming a team that does not exist
* refactor(ptu): keep Azure detection under llms/azure and check share teams with one query
* fix(proxy): drop the reverted TeamIdSearchFilter import after merging main
Main reverted the team key search change that added TeamIdSearchFilter, so the PTU share team check types its where clause locally
* fix(proxy): chunk the ptu_shares team lookup under the IN-list bound
* fix(proxy): count a request naming a shared PTU deployment directly against its group's ceiling
* fix(proxy): resolve every name a shared PTU deployment is served under to its group's ceiling
* fix(ptu): refuse ptu_shares declared without ptu_count and cost_per_ptu_per_hour
* fix(ui): seed total_ptu_hours in the empty daily activity metadata
* fix(ptu): size usage rows by the name the request used
* fix(ptu): size a team's usage rows by the deployment it is served from
* fix(ptu): never size a team's usage row by another team's reservation
* fix(ptu): reserve an uncapped request against the cap the proxy writes, not its prompt length
* fix(ptu): read a share map only through the terms registration requires
A ptu_shares map stored without ptu_count, cost_per_ptu_per_hour and
ptu_effective_from, or one that does not add up to ptu_count, is refused at
registration but was enforced once loaded: every other team got the 400 and
the share holder a ceiling while no flat cost accrued. _deployment_shares now
reads the shares through ptu_terms, so such a row reserves nothing, the same
as before the map was read.
* fix(ptu): keep a declared split as the access rule and check a config split at boot with the flag off
|
||
|---|---|---|
| .. | ||
| fixtures | ||
| golden | ||
| __init__.py | ||
| _background_responses.py | ||
| _daily_activity_fixtures.py | ||
| test_agent_spend.py | ||
| test_autorouter_session_rollup.py | ||
| test_background_interaction_settlement.py | ||
| test_background_response_billed_once.py | ||
| test_background_response_billing_proxies.py | ||
| test_background_response_poll_retirement.py | ||
| test_baseline_accounting.py | ||
| test_batch_completion_accounting.py | ||
| test_batch_enqueued_tokens_redis_lua.py | ||
| test_batch_observability.py | ||
| test_batch_poll_starvation.py | ||
| test_budget_limit_envelopes.py | ||
| test_cache_activity.py | ||
| test_cache_and_quota.py | ||
| test_chaos_burst_spend_once.py | ||
| test_container_spend.py | ||
| test_daily_activity_aggregated_breakdowns.py | ||
| test_daily_activity_key_alias_probes.py | ||
| test_daily_activity_key_metadata_query_timeout.py | ||
| test_daily_activity_key_owner.py | ||
| test_daily_activity_key_owner_faults.py | ||
| test_daily_activity_key_owner_traffic.py | ||
| test_daily_activity_repository.py | ||
| test_daily_activity_routes.py | ||
| test_daily_rollup_retry.py | ||
| test_daily_tag_spend_retention.py | ||
| test_disconnected_bedrock_messages_stream_billing.py | ||
| test_end_user_budget_enforcement.py | ||
| test_end_user_spend_without_proxy_user.py | ||
| test_expired_ui_session_failure_rows.py | ||
| test_failed_dispatch_tokens.py | ||
| test_filtered_ledger.py | ||
| test_global_spend_report.py | ||
| test_global_spend_reset.py | ||
| test_google_genai_native_spend.py | ||
| test_image_generation_key_spend.py | ||
| test_key_budget_lockout.py | ||
| test_key_metadata_recovery_dropped_connection.py | ||
| test_key_metadata_recovery_probe_bounds.py | ||
| test_legacy_spend_logs_row_cap.py | ||
| test_messages_stream_usage_cost.py | ||
| test_model_router_selected_model.py | ||
| test_normalized_error_long_message.py | ||
| test_org_budget_cli_session_token.py | ||
| test_passthrough_budget_reservation.py | ||
| test_passthrough_request_tags.py | ||
| test_prompt_caching_requests_pagination.py | ||
| test_ptu_share_attribution.py | ||
| test_redis_ttl_preserving_token_increment.py | ||
| test_requester_ip_address_logging.py | ||
| test_reset_budget_leader_election.py | ||
| test_responses_bridge_spend_once.py | ||
| test_responses_cache_write_itemization.py | ||
| test_roi_branch_spend.py | ||
| test_scoped_spend_reports.py | ||
| test_service_tier_stream_billing.py | ||
| test_session_total_spend.py | ||
| test_shutdown_flush.py | ||
| test_spend_attribution_contracts.py | ||
| test_spend_calculate.py | ||
| test_spend_capture_rate_captured_spend.py | ||
| test_spend_log_read_scope.py | ||
| test_spend_log_tool_payload_content.py | ||
| test_spend_log_write_batching.py | ||
| test_spend_logs_metadata_fields.py | ||
| test_spend_rollup_accuracy.py | ||
| test_stream_alias_billing.py | ||
| test_tag_budget_enforcement.py | ||
| test_team_budget_enforcement.py | ||
| test_team_daily_activity_aggregated.py | ||
| test_team_member_budget_alerts.py | ||
| test_team_member_spend.py | ||
| test_team_member_spend_flush.py | ||
| test_user_budget_on_team_keys.py | ||
| test_vertex_image_regional_uplift.py | ||
| test_vertex_regional_uplift_spend.py | ||