test(anthropic/chat): force PR-local model_cost map via autouse fixture

CI runs without LITELLM_LOCAL_MODEL_COST_MAP=True, so litellm.model_cost
is loaded from main-branch JSON (default model_cost_map_url) instead of
the PR's checked-out model_prices_and_context_window.json. Tests that
assert per-model flags added in this PR (supports_max_reasoning_effort,
supports_xhigh_reasoning_effort) therefore pass locally but fail in CI
with 'AssertionError: assert False is True' on 5 cases:

  - test_anthropic_model_supports_effort_param_recognizes_supporting_models
    [anthropic.claude-mythos-preview, bedrock/.../mythos-preview,
     claude-opus-4-5-20251101]
  - test_supports_effort_level_handles_provider_prefixes
    [bedrock/invoke/us.anthropic.claude-sonnet-4-6-max-True,
     claude-sonnet-4-6-max-True]

Add an autouse fixture at tests/test_litellm/llms/anthropic/chat/conftest.py
that monkey-patches litellm.model_cost to the PR-local JSON for every test
in this directory. The parent conftest already snapshots+restores
litellm.model_cost per-function, so the mutation is contained.

This is a scoped workaround. The proper fix is to set the env var
globally in the test workflow once the ~10 inline self-set test files
are audited; tracking that as a follow-up issue.
This commit is contained in:
mateo-berri 2026-05-04 09:24:59 -07:00
parent 2c9166c4f3
commit f4d6d5953d

View file

@ -0,0 +1,54 @@
"""
Local-conftest for ``tests/test_litellm/llms/anthropic/chat``.
Why this exists
---------------
``litellm.model_cost`` is loaded once at ``litellm.__init__`` time. By default
it fetches ``model_prices_and_context_window.json`` from the **main branch on
GitHub** (``litellm.model_cost_map_url``) — *not* from the PR-branch JSON in
the working tree. That works fine in production (operators get new models
without redeploying litellm) but is the wrong default for tests, which need
to validate the code in front of them against the data in front of them.
Several anthropic chat transformation tests (``test_supports_effort_level_*``,
``test_anthropic_model_supports_effort_param_*``) assert per-model flags
like ``supports_max_reasoning_effort`` / ``supports_xhigh_reasoning_effort``
that may exist in the PR-local JSON but not yet in main's JSON. Without this
fixture those tests pass locally (where AGENTS.md tells contributors to set
``LITELLM_LOCAL_MODEL_COST_MAP=True``) but fail in CI (which doesn't set the
env var) — the chicken-and-egg PR adds flag → CI fetches main without flag
→ test fails → flag never lands on main.
The fixture force-loads ``litellm.model_cost`` from the local JSON for every
test in this directory. ``tests/test_litellm/conftest.py`` already snapshots
and restores ``litellm.model_cost`` per-function, so this mutation is safe
and contained.
This is a scoped workaround. The proper fix is to set
``LITELLM_LOCAL_MODEL_COST_MAP=True`` globally in the test workflow once the
~10 inline-set test files have been audited and the few tests that exercise
the remote-fetch / integrity-validation path have been given carve-outs.
Tracked at https://github.com/BerriAI/litellm/issues/27122.
"""
import os
import pytest
import litellm
from litellm.litellm_core_utils.get_model_cost_map import get_model_cost_map
@pytest.fixture(autouse=True)
def _use_pr_local_model_cost_map(monkeypatch):
"""Force ``litellm.model_cost`` to the PR-branch JSON for the duration of
each test. ``monkeypatch`` reverts the env var after the test; the parent
conftest restores ``litellm.model_cost`` from its snapshot.
"""
monkeypatch.setenv("LITELLM_LOCAL_MODEL_COST_MAP", "True")
monkeypatch.setattr(
litellm,
"model_cost",
get_model_cost_map(url=litellm.model_cost_map_url),
)
yield