litellm/tests/llm_responses_api_testing
mateo-berri ff63bdb984 tests(vcr): only persist cassette on test pass to avoid poisoning cache
A test that fails (incl. all the failing retries before a passing one)
can otherwise overwrite a known-good cassette with a 'bad luck'
recording. Tests like test_prompt_caching, which assert on provider
state across two calls, can produce a 200 response that semantically
fails the assertion — the 2xx filter doesn't catch this because the
HTTP layer is fine.

- pytest_runtest_makereport hook attaches each phase report to the
  pytest item.
- _vcr_outcome_gate fixture (combining the verbose-mode reporter)
  reads the call-phase outcome at teardown and informs the persister
  via mark_test_outcome_for_cassette before vcrpy's Cassette.__exit__
  triggers save_cassette.
- save_cassette consults the per-key 'did the test pass?' flag and
  short-circuits when False, leaving any prior good recording intact.
- Defaults to passed=True when no marker is present so non-test
  usage of the persister still works.
2026-05-01 13:35:29 -07:00
..
base_responses_api.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
conftest.py tests(vcr): only persist cassette on test pass to avoid poisoning cache 2026-05-01 13:35:29 -07:00
test_anthropic_responses_api.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_anthropic_tool_result_empty_call_id.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_anthropic_tool_result_fix.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_azure_responses_api.py test: test 2026-03-28 19:17:38 -07:00
test_base_responses_api_streaming_iterator.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_google_ai_studio_responses_api.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_openai_responses_api.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_responses_hooks.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00