Commit graph

4 commits

Author SHA1 Message Date
mubashir1osmani
ec8088f064
test(e2e): vendor API testing coverage (#34557)
* test(e2e): cover vendor strategy gaps for chat contract, image edits, auth, team activity

Resolves the first slice of LIT-4778 (vendor API testing strategy): image edits happy path, chat multi-turn + validation + sanitization, LLM-route auth header matrix, and /team/daily/activity structure

* test(e2e): expand vendor API strategy coverage across endpoints

Adds validation cases on existing endpoint suites, plus vector stores, search,
bedrock native, realtime HTTP secrets/calls, responses retrieve, files/batches
contract, and chat stream SSE. Registers coverage cells for LIT-4778

* test(e2e): finish vendor strategy open items

Audio transcription negatives, vector-store file attach/poll/search,
OpenAI moderation category matrix across chat/messages/responses, and
smoke model matrix for chat (LIT-4778)

* test(e2e): harden vendor strategy suite against live env edges

Fix stream [DONE] tracking, XSS no-crash contract, realtime model routing,
vector store list/search models, responses validation, and provider-denied
Bedrock paths so the suite is stable against a live proxy

* test(e2e): rename suites, drop vendor_contract, fix greptile gaps

Move shared status helpers into e2e_http, rename chat auth headers and
chat security suites, remove vendor_contract and dev_config files_settings,
and tighten transcription validation plus vector-store search assertions

* test(e2e): route bedrock stream disconnects through e2e_http

Catch mid-stream RequestException in the shared harness so bedrock native
tests do not import requests directly
2026-08-12 01:07:52 +00:00
Yuneng Jiang
472dd2716f
revert: "test(e2e): vendor API strategy coverage across endpoints (#34649)"
This reverts commit dcb4e5033c.

The suites landed without the proof-of-fix and QA runbook the PR body
itself flagged as outstanding, so the coverage they claim is unverified
against a live proxy
2026-08-04 19:00:34 -07:00
mubashir1osmani
dcb4e5033c
test(e2e): vendor API strategy coverage across endpoints (#34649)
* test(e2e): cover vendor strategy gaps for chat contract, image edits, auth, team activity

Resolves the first slice of LIT-4778 (vendor API testing strategy): image edits happy path, chat multi-turn + validation + sanitization, LLM-route auth header matrix, and /team/daily/activity structure

* test(e2e): expand vendor API strategy coverage across endpoints

Adds validation cases on existing endpoint suites, plus vector stores, search,
bedrock native, realtime HTTP secrets/calls, responses retrieve, files/batches
contract, and chat stream SSE. Registers coverage cells for LIT-4778

* test(e2e): finish vendor strategy open items

Audio transcription negatives, vector-store file attach/poll/search,
OpenAI moderation category matrix across chat/messages/responses, and
smoke model matrix for chat (LIT-4778)

* test(e2e): harden vendor strategy suite against live env edges

Fix stream [DONE] tracking, XSS no-crash contract, realtime model routing,
vector store list/search models, responses validation, and provider-denied
Bedrock paths so the suite is stable against a live proxy

* test(e2e): rename suites, drop vendor_contract, fix greptile gaps

Move shared status helpers into e2e_http, rename chat auth headers and
chat security suites, remove vendor_contract and dev_config files_settings,
and tighten transcription validation plus vector-store search assertions

* test(e2e): route bedrock stream disconnects through e2e_http

Catch mid-stream RequestException in the shared harness so bedrock native
tests do not import requests directly

* fix(e2e): address greptile and veria review on vendor strategy suite

Store search tool keys as os.environ refs and resolve them in SearchAPIRouter.
Tighten validation helpers and assertions so 5xx/empty/unrelated failures no longer pass coverage cells

* fix(e2e): drop search_api_router os.environ expansion from vendor suite

Keep the PR test-only. Search tools register without an api_key so the
proxy falls back to its own PERPLEXITY/TAVILY env, same pattern as a2a.

* test(e2e): drop search e2e suite from vendor strategy PR

Remove the /v1/search coverage file and its registry rows so this PR
no longer carries search endpoint testing.
2026-08-04 20:19:34 +00:00
mubashir1osmani
a11383de34
test(e2e): cover /v1/images/edits (#34476)
/images/edits is a distinct native route from /images/generations: a multipart
request with the source image sent as the 'image' part plus an edit prompt, not
a JSON body. Nothing exercised it end to end.

Adds a live test that registers an OpenAI image model, sends a small generated
PNG plus an edit prompt to /v1/images/edits, and asserts an image comes back
(b64 or url). Generalizes the multipart transport helper with a file_field
argument (default 'file') so the image part can be named 'image', adds an
image_edit client method, the images_edits endpoint to the coverage schema, and
the llm.images_edits.openai.basic.nonstream.works cell.
2026-07-25 10:38:07 -07:00