Commit graph

3 commits

Author SHA1 Message Date
Yassin Kortam
d5b91b94d3
test(e2e): replay a real tool-search assistant turn back to Bedrock Invoke (#36856)
The tool_search x bedrock_invoke cell only ever probed the first turn, so
nothing in the suite has sent a server_tool_use block back to a provider.
Every turn of a real Claude Code session after the first carries the
server_tool_use and tool_search_tool_result blocks the previous turn
produced, and that path was uncovered.

Adds probe_tool_search_multiturn, which takes the real assistant turn
back, answers any client-side tool_use with the id the model actually
emitted, and replays the whole thing as history with the tools still
declared. The assertion refuses to go green unless both server-tool
blocks made it into the replayed history, so a first turn truncated at
max_tokens reads as a failure instead of a vacuous pass.

The replay assertion's red paths never run in a green cell, so they get
markerless harness tests of their own alongside the existing
_builder_unit_tests tree.

No production code.
2026-08-17 11:59:26 -07:00
Yassin Kortam
0439bcbfed
refactor(e2e): fold claude_code HTTP probes onto shared Gateway methods (#33760)
* refactor(e2e): fold claude_code HTTP probes onto shared Gateway methods

Migrate tests/e2e/claude_code/http_probe.py off its own httpx client onto the
shared transport, and promote count_tokens and native anthropic messages to
first-class Gateway methods (Gateway.count_tokens / Gateway.messages) with typed
request/response models in the shared models.py so other suites reuse them.

The probes now take an injected Gateway and issue their request through the
shared count_tokens/messages methods, reusing the split control/data-plane
routing, timeout, and typed Result handling the rest of tests/e2e uses. The wire
shape is preserved: the pydantic bodies serialize byte-for-byte to what the old
httpx probes sent, and the anthropic-version header is carried by a small
AnthropicHeaders model. httpx is gone from the module.

* test(e2e): drop unit-level probe harness test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-18 19:03:01 +00:00
devin-ai-integration[bot]
56f4dbf60a
test(claude_code): move the Claude Code compatibility matrix under tests/e2e (#32548)
* test(claude_code): move the Claude Code compatibility matrix under tests/e2e

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci(claude_code): drop the CircleCI compat PR gate; the matrix runs in the scheduled e2e suite instead

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci: restore the upload-coverage job dropped by mistake with the compat gate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(e2e/claude_code): print rate-limit summary on failed compat runs and fix stale run_daily.sh header comments

* test(claude_code): assert fine-grained tool streaming via input_json_delta instead of an event-count floor

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-07-14 19:19:03 -07:00