litellm/tests/logging_callback_tests
yuneng-jiang cc812cdfc7
test: point the live web search, groq and vertex image suites at models that still exist (#37733)
* test: point the live web search, groq and vertex image suites at models that still exist

Three CircleCI jobs on the staging-to-main promotion are red because the models
their live suites call have been retired by the providers, not because anything
in litellm changed.

openai/gpt-4o-search-preview now answers "has been deprecated" (its dated id
gpt-4o-search-preview-2025-03-11 carries deprecation_date 2026-07-23), so the
two web search conformance tests and the web search cost tracking test move to
gpt-5-search-api, the current search model. It keeps mode chat,
supports_web_search and a search_context_cost_per_query map, so the cost
assertion still resolves.

groq/llama-3.1-8b-instant reached its deprecation_date of 2026-08-16 and Groq
answers "does not exist or you do not have access to it". It follows
groq/llama-3.3-70b-versatile to groq/openai/gpt-oss-120b, the same replacement
PR #37422 already picked. The proxy config that job boots routes on a */*
wildcard, so no config change is needed.

vertex_ai/imagen-3.0-fast-generate-001 404s with "was not found or your project
does not have access to it". Google retired the whole Imagen family across
Vertex and the Gemini API, so there is no Imagen id left to point at. The class
is removed rather than repointed: Vertex image generation is already covered
live by TestVertexAIGeminiImageGeneration on vertex_ai/gemini-2.5-flash-image,
and the Imagen request and response transformations keep their offline coverage
in tests/test_litellm/llms/vertex_ai/image_generation/.

Only live call sites move. Remaining references to the old ids sit in offline
cost-map and transformation tests, where the string is a lookup key and no
request leaves the process.

* chore(lint): ratchet the TQ005 ceiling down to the count this branch reached

Removing the retired TestVertexImageGeneration class cleared one TQ005
violation, so the gate demands the limit come down with it.

make lint-budget-update only lowers a limit by the delta a branch cleared, and
this ceiling already sat 2 above the base count, so the tool landed on 2834
while the gate wants the limit at or below the 2832 this branch reached. The
remaining 2 are that stale headroom, which is exactly what the gate is asking
to reclaim.
2026-08-20 17:03:35 -07:00
..
gcs_pub_sub_body test(logging): pin routing_decision and internal_call_origin in the gcs pubsub spend log fixture 2026-08-01 14:34:04 -07:00
langfuse_expected_request_body [Test Fix] fix gov pricing tests (#25022) 2026-04-02 15:55:55 -07:00
base_test.py Litellm dev 11 11 2024 (#6693) 2024-11-12 00:16:35 +05:30
conftest.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
create_mock_standard_logging_payload.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
gettysburg.wav (Testing) - Add e2e testing for langfuse logging with tags (#7922) 2025-01-22 09:09:25 -08:00
log.txt test: update test 2025-04-10 14:04:57 -07:00
test_alerting.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_amazing_s3_logs.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
test_assemble_streaming_responses.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_bedrock_knowledgebase_hook.py Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
test_built_in_tools_cost_tracking.py test: point the live web search, groq and vertex image suites at models that still exist (#37733) 2026-08-20 17:03:35 -07:00
test_custom_callback_router.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_datadog.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_datadog_llm_obs.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_dynamic_otel_keys.py Litellm fix langfuse otel trace (#20382) 2026-02-03 22:40:19 -08:00
test_gcs_pub_sub.py test: add six ruff rules that catch tests which cannot fail (#37709) 2026-08-20 14:21:26 -07:00
test_generic_api_callback.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_humanloop_unit_tests.py HumanLoop integration for Prompt Management (#7479) 2024-12-30 22:26:03 -08:00
test_langfuse_dynamic_credentials.py fix(callbacks): preserve langfuse secret alias 2026-04-30 14:36:51 -07:00
test_langfuse_e2e_test.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_langfuse_unit_tests.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_langsmith_dynamic_credentials.py chore(callbacks): guard dynamic integration hosts 2026-04-30 14:27:19 -07:00
test_langsmith_unit_test.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_log_db_redis_services.py fix(ci): make litellm_internal_staging green (logging test + Bedrock Opus 4.7 self-heal) (#29344) 2026-05-30 13:57:18 -07:00
test_logging_redaction_e2e_test.py test: build redaction and batch limiter fixtures the way production does (#37416) 2026-08-18 19:37:50 -07:00
test_moderations_api_logging.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_opentelemetry_unit_tests.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_otel_logging.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_pagerduty_alerting.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_posthog.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_spend_logs.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_sqs_logger.py test: remove the five test functions a later definition shadows (#37591) 2026-08-20 17:30:48 +00:00
test_standard_logging_payload.py feat(logging): add opt-in session_id and trace_id correlation to JSON log records via contextvars (#34418) 2026-08-10 10:40:13 -07:00
test_standard_logging_payload_excluded_fields.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_token_counting.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_unit_test_litellm_logging.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_unit_tests_init_callbacks.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_view_request_resp_logs.py Fix CI: Revert security scan changes and add GitGuardian ignore rules (#18358) 2025-12-22 17:03:53 -08:00