When custom pricing is passed as per-request kwargs (input_cost_per_token/output_cost_per_token),
completion() registers pricing under the model name, but _select_model_name_for_cost_calc was
selecting the router deployment hash (which has no pricing data), causing response_cost to be 0.0.
Now checks whether the router_model_id entry actually has pricing before preferring it.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The --enforce_prisma_migration_check flag is now required to trigger
sys.exit(1) on DB migration failure, after #23675 flipped the default
behavior to warn-and-continue.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When callbacks are configured as a plain string (e.g., `callbacks: "my_callback"`)
instead of a list, the proxy crashes on startup with:
TypeError: can only concatenate str (not "list") to str
Normalize each callback setting to a list before concatenating.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The security_scans.sh script uses `column` to format vulnerability
output, but the package wasn't installed in the CI environment.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Part1 had 4 test files combined (was originally 2), causing cross-file
state pollution under xdist. Reverted to original grouping.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The xdist-conditional reload (manual reset in xdist mode) was missing
attributes that importlib.reload resets, causing Azure connection errors.
The original conftest used importlib.reload unconditionally (even under
xdist) and that worked on main. Restore that behavior.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GCS cache tests (test_gcs_cache_unit_tests.py) rely on module-level state
(vertex_chat_completion singleton, credential caches) that importlib.reload
resets but the xdist-safe function-scoped fixture does not. Removing -n 4
from this job restores single-process execution where module reload properly
resets all state before each test, while CI-level parallelism (parallelism: 2)
still splits test files across nodes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The old conftest never flushed HTTP client cache. Adding flush_cache() before
every test forces new TCP connections to external APIs, causing transient
connection failures under xdist parallelism. Global state isolation is already
handled by _SCALAR_DEFAULTS reset.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
test_rerank.py sets litellm.api_base = "http://localhost:4000" which leaked
to all subsequent tests on the same xdist worker, causing connection failures
across every provider (Cohere, Azure, OpenAI, etc.).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The revert of 9711e3adfe left xdist tests without proper state isolation.
Module-level assignments like `litellm.num_retries = 3` in 12+ test files
pollute shared globals, and the fixture was saving/restoring contaminated
values instead of resetting to true defaults.
- Capture true litellm defaults at conftest import time and reset before
each test (local_testing + llm_translation)
- Make llm_translation/conftest.py xdist-safe (skip reload under xdist,
add state isolation)
- Replace asyncio.sleep(2) with polling in cooldown handler tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: improve db migration failure messaging and fix pyright errors in proxy_cli
- Clarify --skip_db_migration_check messaging so users know how to opt
into warn-and-continue behavior when database setup fails
- Fix pyright reportArgumentType error by casting get_secret result to str
- Fix pyright reportPossiblyUnboundVariable by initializing litellm_settings
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: replace --skip_db_migration_check with --enforce_prisma_migration_check
Flip the default behavior: database migration failures now warn and
continue by default. Only when --enforce_prisma_migration_check (or
ENFORCE_PRISMA_MIGRATION_CHECK=true) is explicitly set will the proxy
exit on migration failure.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Fixes https://github.com/BerriAI/litellm/issues/23185
* fix(responses/main.py): ensure litellm metadata custom cost works
* refactor: move all logging updates to a common function, to have just 1 place to update logging kwarg updates
aurelio_sdk imports requests_toolbelt at load time, so it needs its deps.
Unlike semantic_router, aurelio_sdk has no conflict with openai>=2, so
--no-deps is unnecessary. Verified via uv dry-run locally.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch from expensive Linux machine (medium) to docker xlarge executor.
Drop miniconda, manual Docker CLI install, and manual PostgreSQL container
in favor of cimg/python:3.13, setup_remote_docker, and service container.
Use uv + cache for dependency installation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
semantic_router imports aurelio_sdk at module load time, so it must be
installed even when using --no-deps.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Downgrade langfuse, assistants, and python 3.13 install jobs to medium (were defaulting to large at ~25% CPU).
Bump enterprise and image_gen xdist workers to -n 4 on explicit large instances.
Drop coverage collection and persist_to_workspace for 4 jobs that no longer need it.
Downgrade verbosity from -vv to -v across all 5 jobs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The conftest fixtures were saving/restoring the current (potentially
contaminated) values of litellm globals like num_retries instead of
resetting to true defaults. Under xdist, module-level assignments
(e.g. `litellm.num_retries = 3` in 12+ test files) pollute the
shared module state and leak across tests in the same worker.
- Capture true litellm defaults at conftest import time and reset
before each test (local_testing + llm_translation)
- Make llm_translation/conftest.py xdist-safe (skip reload, add
state isolation)
- Replace asyncio.sleep(2) with polling in cooldown handler tests
- Add @pytest.mark.flaky to tests making real API calls under xdist
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch pip install -r requirements.txt to uv pip install --system -r requirements.txt
for all docker-based jobs that use the main requirements.txt. This applies the same
optimization already proven in the mapped test jobs to the rest of the CI pipeline.
Also adds --no-deps to semantic_router installs in guardrails_testing and
litellm_mapped_enterprise_tests to avoid uv's strict resolution conflict with openai>=2.
Skipped: machine executor + conda jobs (security, proxy_spend_accuracy,
proxy_multi_instance, proxy_store_model_in_db) and Group B jobs using
.circleci/requirements.txt.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
uv's strict resolver rejects transitive dep conflicts (semantic-router
wants openai<2, llm-sandbox wants pydantic>=2.11.5). Use uv for the
heavy requirements.txt install and pip for the small test dep batch.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Switch setup_litellm_test_deps from pip to uv with batched installs
- Cache installed site-packages (~/.local/lib, ~/.local/bin) instead of
pip download cache for near-instant installs on cache hit
- Remove unused coverage collection from 6 mapped test jobs (only mcps
coverage is consumed by the coverage combine step)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Router tests: expand conftest save/restore to cover all globals mutated
by router tests (default_fallbacks, tag_budget_config, request_timeout,
enable_azure_ad_token_refresh, num_retries_per_request, model_cost,
token_counter). These were leaking across xdist workers.
Proxy tests: move test_proxy_utils.py (169 parametrized) and
test_proxy_server.py (72 parametrized) from part2 to part1, balancing
~370 vs ~360 tests (was ~129 vs ~600).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Increase pytest-xdist parallelism to match available CPU on I/O-bound and
CPU-bound test jobs. Drop coverage collection from 8 jobs (still collected
by ~15 other jobs). Add dependency caching to 4 uncached jobs. Reduce
verbose output (-vv to -v) and remove -s/--log-cli-level overhead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Batching pip installs surfaced more hidden conflicts:
- respx==0.22.0 requires httpx>=0.25.0, conflicting with httpx==0.24.1
- traceloop-sdk==0.21.1 requires otel-semantic-conventions<0.46,
conflicting with opentelemetry-sdk==1.25.0 (needs ==0.46b0)
These were masked before because separate pip install calls let later
installs silently override earlier pins. Dropping the pins lets pip
resolve compatible versions. Verified with pip --dry-run locally.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Batching pip installs exposed a dependency conflict: langfuse==2.59.7
requires anyio>=4.4.0, which conflicts with the anyio==4.2.0 pin.
Dropping the pin lets pip resolve a compatible version.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Several test jobs were underutilizing their CPU allocation (~25%) because
they were either missing pytest-xdist -n or using -n 2 on 4-vCPU machines.
Batching individual pip install calls into single commands reduces resolver
overhead and saves ~30-60s per job.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test intermittently fails in CI due to Redis cache write propagation
delays, causing the second call to miss the cache and hit OpenAI directly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
LLM responses are non-deterministic and ClaudeAgentOptions doesn't expose
temperature control. The model occasionally returns unexpected short responses
(e.g. just "!") instead of the expected greeting. Add up to 3 retries with a
more explicit prompt to make the test deterministic.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test waited a fixed 3s for async callbacks to populate log_queue.
Under xdist -n 4, CPU contention can delay the GLOBAL_LOGGING_WORKER
background task beyond 3s. Replace fixed sleeps with polling loops
(up to 10s) that break as soon as the expected condition is met.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Module-level mutations (litellm.num_retries=3 in test_langfuse_e2e_test.py
and test_amazing_s3_logs.py, litellm.success_callback=['langfuse']) run
at import time, BEFORE any function fixture. The save/restore pattern
captured these polluted values as 'originals' and kept restoring them.
Fix: capture litellm defaults when conftest.py is first imported (before
test modules), then reset to those true defaults before each test instead
of saving/restoring the current (potentially polluted) state.
ThrottlingException is a transient AWS rate-limit error unrelated to code
correctness. Skip the test instead of failing the CI pipeline.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Clear _in_memory_loggers before/after each test to prevent cached logger
instances (LangsmithLogger, SlackAlerting, etc.) from leaking stale state
- Add pre_call_rules, post_call_rules to list attrs save/restore
- Add vector_store_registry to scalar attrs save/restore
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- test_avertex_batch_prediction: Add google.auth.default mock and env vars
so the test doesn't depend on real GCP credentials (was already a unit
test with mocked HTTP, just missing auth mock)
- test_async_create_batch[openai]: Add DNS pre-check that skips gracefully
when api.openai.com is unreachable instead of failing after 4 retries
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The logging tests mutate many more litellm globals than guardrails tests
(turn_off_message_logging, s3_callback_params, datadog_params, service_callback,
etc.). The initial save/restore list only covered callbacks and a few basics,
causing state leaks like redaction settings bleeding across tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add LITELLM_LOG=WARNING to suppress verbose DEBUG log output
- Remove -s flag to stop capturing all stdout
- Bump xdist workers from -n 2 to -n 4
- Add --timeout=120 for safety
- Rewrite conftest.py to use save/restore pattern (matching guardrails_tests)
instead of per-function importlib.reload + event loop creation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The retrieve_batch endpoint sets batch status to "complete" but never set
batch_processed=True, permanently blocking file deletion. CheckBatchCost
(the safety net) also excluded completed batches from its primary query,
so batch_processed was never set by either path.
Three fixes:
1. update_batch_in_database sets batch_processed=True when status reaches
"complete", with old-schema fallback retry
2. CheckBatchCost primary query no longer excludes complete/completed
(batch_processed=False filter prevents reprocessing)
3. retrieve_batch early-return now includes "complete" (DB-normalized
spelling) to avoid unnecessary provider re-polls
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
test_parallel_function_call_anthropic_error_msg was flaky because other
tests set litellm.modify_params=True without resetting it. When True,
the validation adds a dummy tool instead of raising UnsupportedParamsError.
Fix: save/restore modify_params around the test.
test_vertex_ai_llama_tool_calling failed on intermittent 404 from the
Llama model endpoint in us-east5. Fix: skip on NotFoundError like
the existing RateLimitError handling.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test used a fixed 40s sleep before checking spend logs, but async
spend logging in CI sometimes takes longer to flush. Replace with a
polling loop (10s interval, 120s max) that exits early on success.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The eager tiktoken tests were clearing all litellm modules from
sys.modules and re-importing, creating new module objects with different
class identities. This broke unittest.mock.patch for all subsequent
tests on the same xdist worker. Running these tests in subprocesses
provides perfect isolation.
Fixes: test_metadata_passed_to_custom_callback_codex_models,
test_oidc_github_success, test_oidc_google_cached,
test_oidc_google_failure,
test_encrypted_content_affinity_bypasses_rpm_limits, and 5 others.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
These jobs run on medium (2 CPU) but weren't using pytest-xdist,
leaving the second CPU idle. Added pytest-xdist dep and -n 2 to:
- auth_ui_unit_tests (~33 tests)
- litellm_router_unit_testing (~191 tests)
- mcp_testing (~112 tests)
- llm_responses_api_testing (~80 tests)
- search_testing (~53 tests)
- batches_testing (~45 tests)
- litellm_utils_testing (~205 tests)
- pass_through_unit_testing (~102 tests)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test asserts that a ttl=0 cached entry expires immediately, so the
second call should not return cached content. Both calls used the same
mock_response text, making the content != assertion always fail. Use
different mock_response values so a cache hit is distinguishable.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
