The websearch interception handler was passing internal flags like
`_websearch_interception_converted_stream` to the follow-up LLM request.
This caused "Extra inputs are not permitted" errors from providers like
Bedrock that use strict Pydantic validation.
Fix: Filter out all kwargs starting with `_websearch_interception` prefix
before making the follow-up anthropic_messages.acreate() call.
* Consolidated change
* fix(prompt_security): update message processing to persist sanitized files and filter for API calls
* fix per krrishdholakia suggestion
Add support for 'file' and 'input_file' content types in
convert_to_gemini_tool_call_result(). File content in tool
results was previously silently dropped.
Supports base64 data URIs and HTTP URLs, matching the existing
image handling pattern. Enables PDF, audio, video, and other
file types as inline_data for Gemini.
The OpenAI Agents SDK (v0.6.9+) now passes reasoning_effort as a dict
when summary is specified: {"effort": "high", "summary": "auto"}
This change extracts the "effort" value from the dict for Vertex AI,
which only supports thinkingLevel (not summary).
Before: reasoning_effort={"effort": "high"} was silently ignored
After: reasoning_effort={"effort": "high"} correctly maps to thinkingLevel
Fixes#19411
* feat: add gemini video metadata and detail support
Implement support for video_metadata and enhanced detail parameter
for Gemini 3.0+ models:
- Add video_metadata field to ChatCompletionFileObjectFile type
- Supports fps, start_offset, and end_offset parameters
- Properly converts snake_case to camelCase for Gemini API
- Extend detail parameter to support medium and ultra_high levels
- Maps to MEDIA_RESOLUTION_MEDIUM and MEDIA_RESOLUTION_ULTRA_HIGH
- Update _process_gemini_image to handle video metadata transformation
- Add version gating to only apply features for Gemini 3+ models
- Add comprehensive test coverage (6 new test cases)
- Test detail parameter with file objects
- Test video_metadata fields (fps, start_offset, end_offset)
- Test combined detail + video_metadata usage
- Test new detail levels (medium, ultra_high)
- Test version gating (Gemini 1.5 vs 3.0)
Note: video_metadata is only supported for video files but error
handling is delegated to Vertex AI for other media types.
* refactor: rename _process_gemini_image to _process_gemini_media
The function handles multiple media types (images, audio, video, PDF),
not just images. Renamed to better reflect its actual purpose.
- Update function name in transformation.py
- Update all function calls and references
- Update test names and imports to match
- Improve docstring to clarify it handles all media types
* docs: add video metadata and media resolution control documentation
Add comprehensive documentation for Gemini 3+ video processing features:
- Document media resolution control (detail parameter) for images and videos
- Add video_metadata field documentation (fps, start_offset, end_offset)
- Include usage examples with tabs for basic, combined, and proxy scenarios
- Update both Gemini and Vertex AI provider documentation
- Clarify snake_case to camelCase field conversion for Gemini API
Signed-off-by: Kris Xia <xiajiayi0506@gmail.com>
* refactor(gemini): extract metadata application into helper function
Extract duplicated Gemini 3+ media_resolution and video_metadata
application logic from _process_gemini_media into a dedicated
_apply_gemini_3_metadata helper function to improve code maintainability.
---------
Signed-off-by: Kris Xia <xiajiayi0506@gmail.com>
