scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-06 08:16:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 78
33232 commits 4106 branches 1402 tags 4.2 GiB
Commit graph

33232 commits

This branch
This branch
All branches
Author SHA1 Message Date
Yuneng Jiang
e33e5e0747
chore: fixes 2026-04-04 23:49:24 -07:00
Ishaan Jaffer
11b4d76b19 fix(tests): use monkeypatch.setenv for Redis pool max_connections tests 
Replace patch('litellm._redis._get_redis_client_logic') with monkeypatch.setenv
in test_max_connections_url_config and test_max_connections_url_config_string_value.

The mock was unreliable in CI (REDIS_URL is set to the real Redis Cloud server),
causing the pool to silently use the real config instead of the test config.
Using monkeypatch.setenv tests the full env-var→pool chain more robustly and
matches the actual production code path.
 2026-02-21 14:37:33 -08:00
Ishaan Jaff
235a47c576
fix(tests): mock test_claude_tool_use_with_gemini to fix flaky CI (#21832) 
* ui fixes

* fix(tests): mock test_claude_tool_use_with_gemini to avoid MALFORMED_FUNCTION_CALL flakiness
 2026-02-21 14:34:54 -08:00
ryan-crabbe
3da77f310b
Merge pull request #21828 from BerriAI/fix-duplicate-issues-workflow 
fix: pass prompt as env var in duplicate detection workflows
 2026-02-21 14:30:03 -08:00
Ryan Crabbe
c7d3198d9a fix: pass prompt as env var in duplicate detection workflows 
Fixes "Input must be provided either through stdin or as a prompt
argument" error by moving the prompt to a PROMPT env variable
instead of inline multiline shell string.
 2026-02-21 14:29:08 -08:00
Ishaan Jaff
fb4249005e
fix(tests): add atexit.register mock to prevent Click isolation stream closure in test_use_prisma_db_push_flag_behavior (#21829) 2026-02-21 14:28:02 -08:00
yuneng-jiang
f61dc66a02
Merge pull request #21827 from BerriAI/revert-21707-add-watchdog-prisma 
Revert "fix(proxy): recover from prisma-query-engine zombie process"
 2026-02-21 14:21:19 -08:00
yuneng-jiang
8c5be4cb62
Revert "fix(proxy): recover from prisma-query-engine zombie process (#21707)" 
This reverts commit 977ad015ca.
 2026-02-21 14:20:06 -08:00
Ishaan Jaff
56a095a079
fix(tests): update deprecated Anthropic model in test_user_model_access (#21826) 
* fix(tests): clear _async_success_callback in vertex fine-tune mocked tests to prevent Datadog interference

* fix(tests): update deprecated claude-3-5-haiku-20241022 to claude-haiku-4-5-20251001
 2026-02-21 14:18:24 -08:00
Ishaan Jaff
a1ead765ed
fix(tests): clear _async_success_callback in vertex fine-tune mocked tests to prevent Datadog interference (#21825) 2026-02-21 14:16:18 -08:00
Ishaan Jaff
494aad4a68
fix(tests): isolate auth in vertex passthrough and spend logs date range tests (#21824) 
test_vertex_passthrough_with_default_credentials and
test_view_spend_logs_with_date_range_summarized fail intermittently when a
prior xdist worker sets master_key — auth then rejects the unauthenticated
test requests before the code under test is reached.

- mock user_api_key_auth in test_vertex_passthrough_with_default_credentials
  (same pattern used for test_vertex_passthrough_with_no_default_credentials
  in #21810)
- wrap test_view_spend_logs_with_date_range_summarized in
  app.dependency_overrides[ps.user_api_key_auth] with try/finally cleanup
  (same pattern used for the other spend log tests in #21810)
 2026-02-21 14:14:50 -08:00
Ishaan Jaff
dd6a74da63
fix(tests): isolate litellm.cache and CLI env vars in flaky tests (#21821) 
- TestSpendLogsPayload: save/restore litellm.cache in setup_method/teardown_method
  so tests that run after a cache-setting test don't see a non-None cache and get
  a hash instead of "Cache OFF" in the cache_key field
- test_use_prisma_db_push_flag_behavior: apply clean_env pattern (strip DATABASE_URL/DIRECT_URL,
  then set DATABASE_URL to test value) inside the with block instead of using @patch.dict
  decorator, matching the pattern from test_skip_server_startup to avoid Click 8.3.x
  StreamMixer stream lifecycle issues in CI
 2026-02-21 14:11:48 -08:00
Ishaan Jaff
6acfa1c71d
fix(tests): clear ANTHROPIC_BASE_URL/ANTHROPIC_API_BASE in spend log api_base tests (#21820) 
Tests hardcode expected api_base as https://api.anthropic.com/v1/messages but
if ANTHROPIC_BASE_URL is set in the environment the recorded api_base changes,
causing a mismatch. Clear both env vars via monkeypatch at the start of each test.
 2026-02-21 14:11:18 -08:00
Ishaan Jaff
6ec16d583d
fix(test): add timeout to flush() call to prevent 300s hang in CI (#21819) 
GLOBAL_LOGGING_WORKER.flush() calls queue.join() which blocks until all
items are task_done(). In CI with pytest-asyncio, each test gets a fresh
event loop so the worker reinitializes its queue - items from a previous
test never get task_done(), causing an infinite hang.

Fix: wrap flush() with asyncio.wait_for(..., timeout=10.0).
 2026-02-21 14:10:57 -08:00
ryan-crabbe
b17d37eceb
Merge pull request #21815 from BerriAI/litellm_fix_openai_init_params_immutable 
fix: make cached OpenAI init params immutable and fix import ordering
 2026-02-21 13:33:03 -08:00
Ryan Crabbe
dcbac4a4af style: add missing PEP 8 blank line before top-level function 2026-02-21 13:31:19 -08:00
Ishaan Jaff
c810f5cd63
fix(tests): replace fake France Azure endpoint in test_router_azure_acompletion (#21818) 2026-02-21 13:26:59 -08:00
Ryan Crabbe
9e1d83e3de fix: make LITELLM_CLIENT_SPECIFIC_PARAMS a tuple to prevent TypeError 
tuple + list raises TypeError in get_openai_client_cache_key. Also add
test coverage for get_openai_client_cache_key to catch type mismatches.
 2026-02-21 13:25:23 -08:00
shin-bot-litellm
1be30f5129
feat(router): Add complexity-based auto routing strategy (#21789) 
* feat(router): Add complexity-based auto routing strategy

Adds a rule-based routing strategy that classifies requests by complexity
and routes them to appropriate models - with zero API calls and sub-millisecond
latency.

## Features

- **Zero external API calls** - all scoring is local
- **Sub-millisecond latency** - typically <1ms per classification
- **Weighted multi-dimensional scoring** across 7 dimensions:
  - Token count (short=simple, long=complex)
  - Code presence (code keywords → complex)
  - Reasoning markers ("step by step" → reasoning tier)
  - Technical terms (domain complexity)
  - Simple indicators ("what is" → simple, negative weight)
  - Multi-step patterns (numbered steps)
  - Question complexity (multiple questions)
- **Configurable tier boundaries** and model mappings
- **Reasoning override** - 2+ reasoning markers force REASONING tier

## Usage

```yaml
model_list:
  - model_name: smart-router
    litellm_params:
      model: auto_router/complexity_router
      complexity_router_config:
        tiers:
          SIMPLE: gpt-4o-mini
          MEDIUM: gpt-4o
          COMPLEX: claude-sonnet-4
          REASONING: o1-preview
```

Inspired by ClawRouter: https://github.com/BlockRunAI/ClawRouter

## Files Added

- litellm/router_strategy/complexity_router/complexity_router.py - Main router class
- litellm/router_strategy/complexity_router/config.py - Configuration and defaults
- litellm/router_strategy/complexity_router/__init__.py - Package exports
- litellm/router_strategy/complexity_router/README.md - Documentation
- tests/test_litellm/router_strategy/test_complexity_router.py - Test suite (37 tests)

## Files Modified

- litellm/router.py - Integration with pre_routing_hook
- litellm/types/router.py - New config params

* feat(router): Add complexity-based auto routing strategy

Adds a new rule-based routing strategy that classifies requests by complexity
and routes them to appropriate models - without any external API calls.

## Features
- Weighted scoring across 7 dimensions: token count, code presence, reasoning
  markers, technical terms, simple indicators, multi-step patterns, questions
- Maps to 4 tiers: SIMPLE, MEDIUM, COMPLEX, REASONING
- Each tier configurable to a different model
- Zero API calls, <1ms latency
- Inspired by ClawRouter

## Configuration
```yaml
model_list:
  - model_name: smart_router
    litellm_params:
      model: auto_router/complexity_router
      complexity_router_config:
        tiers:
          SIMPLE: gemini-2.0-flash
          MEDIUM: gpt-4o-mini
          COMPLEX: claude-sonnet-4
          REASONING: claude-opus-4
```

## Use Cases
- Cost optimization: route simple queries to cheaper models
- Quality optimization: route complex queries to capable models
- Zero configuration: works out of the box with sensible defaults

* feat(router): Add complexity-based auto routing strategy

Adds a new rule-based routing strategy that classifies requests by complexity
and routes them to appropriate models - without any external API calls.

- Weighted scoring across 7 dimensions: token count, code presence, reasoning
  markers, technical terms, simple indicators, multi-step patterns, questions
- Maps to 4 tiers: SIMPLE, MEDIUM, COMPLEX, REASONING
- Each tier configurable to a different model
- Zero API calls, <1ms latency
- Inspired by ClawRouter

```yaml
model_list:
  - model_name: smart_router
    litellm_params:
      model: auto_router/complexity_router
      complexity_router_config:
        tiers:
          SIMPLE: gemini-2.0-flash
          MEDIUM: gpt-4o-mini
          COMPLEX: claude-sonnet-4
          REASONING: claude-opus-4
```

- Cost optimization: route simple queries to cheaper models
- Quality optimization: route complex queries to capable models
- Zero configuration: works out of the box with sensible defaults

* feat: add enterprise presets for complexity router

Adds preset configurations for different cloud providers:
- bedrock: AWS Bedrock (Claude models)
- vertex: Google Vertex AI (Gemini models)
- azure: Azure OpenAI (GPT + o1)
- standard: Direct API (OpenAI + Anthropic)
- cost_optimized: Maximum savings (Gemini Flash + cheaper models)

Usage:
```yaml
complexity_router_config:
  preset: bedrock  # or vertex, azure, standard, cost_optimized
```

* feat(ui): update auto router submit handler for complexity router

- Handle complexity_router model type in submit handler
- Generate correct litellm_params for complexity router:
  - model: auto_router/complexity_router
  - complexity_router_config: { tiers: { SIMPLE, MEDIUM, COMPLEX, REASONING } }
- Keep existing semantic router handling intact
- Add success notification with router type name

* docs: update PR description with UI changes

* chore: remove preset feature, keep simple tier config

* fix: exclude complexity_router from auto_router check

The _is_auto_router_deployment() was matching all auto_router/* models,
causing complexity_router to fail initialization. Now it explicitly
excludes auto_router/complexity_router which has its own handler.

* fix(complexity_router): Address Greptile review feedback

Fixes 5 issues flagged in code review:

1. **Mutable singleton mutation bug** - Now always creates a new
   ComplexityRouterConfig instance instead of reusing DEFAULT_COMPLEXITY_CONFIG
   singleton, preventing cross-instance config pollution.

2. **Substring matching false positives** - Added word boundaries (spaces)
   to short keywords like 'ok', 'try', 'api', 'git', 'node', 'java', 'vue'
   to prevent matching within longer words (e.g., 'capital' matching 'api').

3. **Redundant message extraction** - Simplified to single reverse loop that
   extracts both last user message and last system prompt efficiently.

4. **Unused imports** - Removed unused DEFAULT_CREATIVE_KEYWORDS and
   DEFAULT_MULTI_STEP_PATTERNS imports.

5. **Missing async_pre_routing_hook tests** - Added comprehensive tests for:
   - Multi-turn conversations
   - List-type content handling
   - No user message case
   - Empty string content
   - Message preservation
   - Singleton mutation prevention

* fix(complexity_router): Address Greptile review feedback

- Use word boundary matching for short keywords (<5 chars) to avoid
  false positives (e.g., 'api' matching 'capital', 'git' matching 'digital')
- Remove 'ok' from simple keywords (too many false positives)
- Add tests for keyword false positive prevention
- Fix test expectations for edge cases (empty string content, list content)

Addresses: 2/5 Greptile score feedback on PR #21789

* docs(auto_routing): Add complexity router documentation

- Add Complexity Router section to auto_routing.md
- Include comparison table with semantic auto router
- Add Python SDK and Proxy Server configuration examples
- Document all configuration options (tier boundaries, token thresholds, dimension weights)
- Explain how complexity scoring works

* feat(complexity_router): Add eval suite + tune scoring parameters

Added comprehensive evaluation suite with 29 test cases covering:
- SIMPLE tier: greetings, definitions, factual questions
- MEDIUM tier: technical explanations, comparisons, debugging
- COMPLEX tier: architecture design, complex coding
- REASONING tier: explicit reasoning requests
- Regression tests: substring false positive prevention

Tuned scoring parameters based on eval results:
- Lowered tier boundaries (0.15/0.35/0.60) for better tier distribution
- Increased code/technical weights (0.30/0.25) for complex prompts
- Reduced simple indicator weight (0.05) to avoid over-penalizing
- Fixed 'hey'/'hi' keywords to require leading space

Eval results: 29/29 passed (100%)

* fix(complexity_router): Address Greptile review round 2

1. **Empty user message handling** - Changed from falsy check to None check
   to properly distinguish 'no user message' from 'empty string message'

2. **ReDoS prevention** - Changed 'first.*then' to 'first.*?then' (non-greedy)
   to prevent regex backtracking on pathological inputs

3. **Documentation sync** - Updated README.md to match actual config values:
   - Tier boundaries: 0.15/0.35/0.60 (not 0.25/0.50/0.75)
   - Dimension weights: tokenCount=0.10, codePresence=0.30, technicalTerms=0.25,
     simpleIndicators=0.05, multiStepPatterns=0.03, questionComplexity=0.02

4. **Missing UI component** - Added ComplexityRouterConfig.tsx with:
   - Tier-to-model dropdown selectors
   - Descriptions and examples for each tier
   - How classification works explanation

5. **Inline import comment** - Added explanation for why ComplexityRouter
   import is inline (matches AutoRouter pattern, avoids circular imports)

* docs(auto_routing): fix dimension weights and tier boundaries to match config.py defaults

* fix(complexity_router): skip empty string content in async_pre_routing_hook

* fix(router): remove or {} masking None complexity_router_config

* fix(config): remove unused DEFAULT_MULTI_STEP_PATTERNS and DEFAULT_CREATIVE_KEYWORDS exports

* fix(complexity_router): use word boundary matching for all single-word keywords, avoid double-scanning reasoning keywords

* fix(router): clarify circular import comment for ComplexityRouter

* docs(README): fix token thresholds to match config.py defaults

* test(complexity_router): add false positive tests for error/class/merge keyword matching

* fix(complexity_router): align .get() fallbacks with config.py defaults, document system prompt scoring

* fix(config): deduplicate keywords across code and technical lists

---------

Co-authored-by: OpenClaw Assistant <assistant@openclaw.ai>
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
 2026-02-21 13:23:37 -08:00
Ryan Crabbe
bbbec23c8b fix: update tests to match tuple return type for cached init params 2026-02-21 13:18:03 -08:00
Ishaan Jaff
886d168154
fix(logging): resolve cache_hit before hidden_params short-circuit in _response_cost_calculator (#21816) 
Cached responses carry response_cost in _hidden_params from the original call.
_response_cost_calculator was returning that pre-computed cost before checking
cache_hit, so cached responses were billed instead of returning 0.0.

Fix: move cache_hit resolution and early-return to top of the function.

Regression introduced in bdf01fa283 (fix mypy error).
 2026-02-21 13:17:02 -08:00
Ryan Crabbe
72e75c1122 Merge origin/main into litellm_fix_openai_init_params_immutable 
Resolve conflict: keep Tuple types, incorporate type: ignore comment
and alphabetical typing import order from main.
 2026-02-21 13:15:30 -08:00
Ryan Crabbe
20a685fe7f fix: make cached OpenAI init params immutable and fix import ordering 
- Move `import inspect` to stdlib import group
- Change _OPENAI_INIT_PARAMS and _AZURE_OPENAI_INIT_PARAMS from
  mutable lists to immutable tuples to prevent accidental mutation
- Update return type and helper to use Tuple[str, ...]
 2026-02-21 13:10:49 -08:00
Ishaan Jaff
8483477512
fix(test): add asyncio.sleep(0) before flush() to prevent hang in test_async_no_duplicate_spend_logs (#21813) 2026-02-21 13:09:36 -08:00
Ishaan Jaff
0a0768b3df
fix(ci): resolve mypy and check_code_and_doc_quality CI failures (#21812) 
- fix(mypy): suppress [misc] type error in common_utils.py for cls.__init__ access
- fix(mypy): move type: ignore comment to correct line in test_eval.py (line 232 not 231)
- fix(mypy): suppress [misc] and pre-existing pyright errors in vertex_ai_non_gemini.py
- fix(check_licenses): strip inline comments before parsing requirements.txt lines so CVE comments don't break packaging.requirements.Requirement()
- fix(router_coverage): add _merge_tools_from_deployment and _invalidate_access_groups_cache to ignored list (private helpers tested indirectly)
 2026-02-21 13:08:47 -08:00
github-actions[bot]
22704b0176
chore: regenerate poetry.lock to match pyproject.toml (#21811) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2026-02-21 21:07:29 +00:00
ryan-crabbe
336ecd6267
Merge pull request #21003 from BerriAI/litellm_perf_skip_usage_roundtrip 
perf: skip Usage Pydantic round-trip in logging payload
 2026-02-21 12:56:33 -08:00
Ryan Crabbe
75bc8329e2 Merge origin/main into litellm_perf_skip_usage_roundtrip 
Resolve conflict in litellm_logging.py: take main's version and
re-apply get_usage_as_dict optimization on top.
 2026-02-21 12:55:55 -08:00
Ishaan Jaff
b281181448
fix(tests): isolate auth in spend logs and vertex passthrough tests (#21810) 
* fix(tests): add app.dependency_overrides for auth in spend logs tests

test_ui_view_spend_logs_with_status, test_ui_view_spend_logs_with_model,
test_ui_view_spend_logs_with_model_id, and test_view_spend_logs_summarize_parameter
all send Bearer sk-test without mocking user_api_key_auth. When a prior test
in the same xdist worker sets master_key, the auth check fails for sk-test
and the test fails intermittently.

Fix: use app.dependency_overrides[ps.user_api_key_auth] to bypass auth,
same pattern as other tests in the same file.

* fix(tests): mock user_api_key_auth in test_vertex_passthrough_with_no_default_credentials

vertex_proxy_route calls user_api_key_auth internally. When a prior test in the
same xdist worker sets master_key, the auth check fails for the test request
and create_pass_through_route is never called, causing assert_called_once_with to fail.

Fix: patch user_api_key_auth as an AsyncMock in the with mock.patch() block.
 2026-02-21 12:51:38 -08:00
Ishaan Jaff
1ed529092f
fix(test): replace flaky test_vertex_ai_gemini_audio_ogg with mocked version (#21807) 
Previously made a real Vertex AI call with a Wikimedia URL that intermittently
failed with URL_REJECTED-REJECTED_FC_TIMEOUT.

Now mocks HTTPHandler.post and VertexBase._ensure_access_token so the test
verifies the translation (OGG -> file_data with audio/ogg mime_type) without
any real network calls. Runs in ~0.36s instead of ~60s.
 2026-02-21 12:49:06 -08:00
ryan-crabbe
da8f038178
Merge pull request #20789 from ryan-crabbe/perf/cache-openai-init-params 
perf: pre-compute OpenAI client __init__ params at module load
 2026-02-21 12:47:54 -08:00
Ishaan Jaff
d928588de0
docs: mark v1.81.12 as stable (#21809) 
* docs: mark v1.81.12 as stable, point to stable docker image and pip

* docs: fix v1.81.12 docker image to point to stable
 2026-02-21 12:45:41 -08:00
ryan-crabbe
4c393de216
Merge pull request #21213 from BerriAI/litellm_fix_streaming_connection_pool_leak 
fix: close streaming connections to prevent connection pool exhaustion
 2026-02-21 12:45:18 -08:00
Ryan Crabbe
5bcaeabfd8 Merge origin/main into litellm_fix_streaming_connection_pool_leak 
Resolve conflict in test_proxy_server.py: keep both async_data_generator
cleanup tests and store_model_in_db DB config override tests.
 2026-02-21 12:44:50 -08:00
ryan-crabbe
53f3dfa989
Merge pull request #20354 from ryan-crabbe/perf/callback-registration-routing 
perf: move async/sync callback separation from per-request to registration
 2026-02-21 12:41:28 -08:00
Ishaan Jaff
086ced1f6a
add missing sql_injection.yaml policy template (#21806) 2026-02-21 12:40:44 -08:00
Ryan Crabbe
ea32ad72c6 Merge origin/main into perf/callback-registration-routing 
Resolve conflicts:
- logging_callback_manager.py: keep PR's MAX_CALLBACKS, _is_async_callable, Callable type
- test_utils.py: keep both TestCallbackAsyncSyncSeparation and TestMetadataNoneHandling
 2026-02-21 12:40:23 -08:00
Ishaan Jaff
02670582c5
fix(tests): replace asyncio.sleep(1) with event-based wait in metadata callback tests (#21805) 2026-02-21 12:40:06 -08:00
Miguel Armenta
0fe3145e83
Fix/presidio controls (#21798) 
* check should_run_guardrail in sync logging hook path

* Add tests for CustomGuardrail logging behavior

Added tests to ensure CustomGuardrail logging behavior based on the guardrail execution state.

---------

Co-authored-by: Miguel Armenta <ma826r@att.com>
 2026-02-21 12:39:17 -08:00
ryan-crabbe
f5139716a1
Merge pull request #20882 from ryan-crabbe/perf/optimize-model-dump-preserved-fields 
perf: optimize model_dump_with_preserved_fields
 2026-02-21 12:36:02 -08:00
Ishaan Jaff
daa682e125
fix(tests): add missing start_db_health_watchdog_task mock (#21804) 
* fix(tests): add missing start_db_health_watchdog_task mock in test_proxy_server_prisma_setup

* fix(tests): add missing start_db_health_watchdog_task mock in test_health_check_not_called_when_disabled
 2026-02-21 12:31:52 -08:00
Ishaan Jaffer
0b69c21eca Revert "fix(http_handler): bypass cache when shared_session is provided for aiohttp tracing (#20630)" 
This reverts commit 7ee36c2a3a.
 2026-02-21 12:27:46 -08:00
ryan-crabbe
d7b3b2eb9e
Merge pull request #20374 from ryan-crabbe/perf/cache-access-groups 
perf: cache get_model_access_groups() no-args result on Router
 2026-02-21 12:25:45 -08:00
ryan-crabbe
469951466e
Merge pull request #20541 from ryan-crabbe/perf/cost-calculator-optimizations 
Perf/cost calculator optimizations
 2026-02-21 12:18:38 -08:00
ryan-crabbe
a8dbcb1a30
Merge pull request #20526 from ryan-crabbe/perf/add-litellm-data-to-request-optimizations 
Perf: add_litellm_data_to_request optimizations
 2026-02-21 12:16:05 -08:00
ryan-crabbe
d04e5c6a3e
Merge pull request #20448 from ryan-crabbe/perf/optimize-completion-cost 
perf: optimize completion_cost()
 2026-02-21 12:15:48 -08:00
Ishaan Jaff
08ae43ace1
fix(migrations): add ensure_project_id migration + bump litellm-proxy-extras to 0.4.46 (#21800) 
* fix(migrations): add ensure_project_id_verification_token migration

Ensures project_id column exists on LiteLLM_VerificationToken. The original
migration (20251113000000_add_project_table) adds this column, but may have
been skipped if LiteLLM_ProjectTable already existed and the migration was
resolved as idempotent. Uses IF NOT EXISTS for safety.

* bump: litellm-proxy-extras 0.4.45 → 0.4.46
 2026-02-21 12:15:21 -08:00
Ryan Crabbe
dc2c835e7b perf: optimize completion_cost() — eliminate enum overhead, reduce function call indirection 
Pre-resolve CallTypes enum values into module-level frozensets to avoid
repeated .value attribute access in the elif chain. Inline the hot-path
_store_cost_breakdown_in_logging_obj as a direct dict literal. Remove
unnecessary cast(CallTypesLiteral, call_type) call. Guard
_get_additional_costs() with azure_ai-only check since no other provider
implements additional costs.

Line profile shows 20.5% reduction in completion_cost() total time
(7.14s → 5.68s across 6,006 calls). The four targeted bottlenecks
dropped from 2.82s to 0.28s combined.
 2026-02-21 12:14:55 -08:00
ryan-crabbe
d8dda93bba
Merge pull request #20593 from ryan-crabbe/perf/reuse-litellm-params 
perf: reuse LiteLLM_Params
 2026-02-21 12:11:02 -08:00
ryan-crabbe
be016b683b
Merge pull request #20440 from ryan-crabbe/perf/skip-duplicate-logging-payload 
perf: skip duplicate get_standard_logging_object_payload for non-streaming req's
 2026-02-21 12:07:31 -08:00