Commit graph

17 commits

Author SHA1 Message Date
Krish Dholakia
26fd6d5362
Guardrails API - support LLM tool call response checks on /chat/completions, /v1/responses, /v1/messages on regular + streaming calls (#17619)
* fix(unified_guardrails.py): send all chunks on completion of final stream

* feat(generic_guardrail_api.py): handle tool call response on streaming LLM responses

* fix(anthropic/chat/guardrail_translation): initial commit adding anthropic tool response streaming guardrails

enables guardrail checks on tool response from llm's to work via `/v1/messages`

* feat(anthropic/): working guardrail checks on tool response from LLMs

ensures guardrail checks on anthropic /v1/messages works as expected

* feat(responses/guardrail_translation): support tool call response guardrails on streaming for /v1/responses

ensures complete coverage of tool call responses

* refactor(openai.py): refactor to use consistent pydantic model for responses api tool response on streaming

enables non-openai model tool call response to work correctly with guardrail checks on /v1/responses

* test: update tests

* fix: fix linting error

* fix: fix failing tests

* fix: fix import errors

* fix(openai/chat/guardrail_transformation): fix final chunk returned on streaming
2025-12-15 18:19:52 +05:30
Krish Dholakia
a3f5a94784
fix(openai/responses/guardrail_translation): fix basemodel import (#17977)
* fix(openai/responses/guardrail_translation): fix basemodel import

ensures /v1/responses output is correctly processed by guardrail handler

* fix: fix linting error

* fix: fix linting errors
2025-12-15 16:13:45 +05:30
Krish Dholakia
51cc102c30
fix(unified_guardrail.py): support during_call event type for unified guardrails (#17514)
* fix(unified_guardrail.py): support during_call event type for unified guardrails

allows guardrails overriding apply_guardrails to work 'during_call'

* feat(generic_guardrail_api.py): support new 'tool_calls' field for generic guardrail api

returns the tool calls emitted by the LLM API to the user

* fix(generic_guardrail_api.py): working anthropic /v1/messages tool call response

send llm tool calls to guardrail api when called via `/v1/messages` API

* fix(responses/): run generic_guardrail_api on responses api tool call responses

* fix: fix tests

* test: fix tests

* fix: fix tests
2025-12-04 22:06:13 -08:00
Krish Dholakia
be0530a6b3
fix(unified_guardrail.py): correctly map a v1/messages call to the anthropic unified guardrail (#17424)
* fix(unified_guardrail.py): correctly map a v1/messages call to the anthropic unified guardrail

* fix: add more rigorous call type checks

* fix(anthropic_endpoints/endpoints.py): initialize logging object at the beginning of endpoint

ensures call id + trace id are emitted to guardrail api

* feat(anthropic/chat/guardrail_translation): support streaming guardrails

sample on every 5 chunks

* fix(openai/chat/guardrail_translation): support openai streaming guardrails

* fix: initial commit fixing output guardrails for responses api

* feat(openai/responses/guardrail_translation): handler.py - fix output checks on responses api

* fix(openai/responses/guardrail_translation/handler.py): ensure responses api guardrails work on streaming

* test: update tests

* test: update tests

* test: update tests

* fix(bedrock_guardrails.py): fix post call streaming iterator logic

* fix: fix return

* fix(bedrock_guardrails.py): fix
2025-12-03 20:54:56 -08:00
Krish Dholakia
8edcc4ecc3
Guardrails API - add streaming support (#17400)
* fix(initial-commit): adding a way to get the right response type based on the api route

* feat(unified_guardrail.py): support streaming guardrails

* test: update tests

* fix: fix linting errors

* test: update tests
2025-12-02 22:52:09 -08:00
Wei-Chiet Ku
d43c077653
Fix/issue 16759 streaming error validation (#17242)
* Enhance error handling in OpenAIResponsesAPIConfig to coalesce null error codes into a default string, preventing validation errors and improving stability during streaming iterations.

* Add test for coalescing null error codes in streaming responses

This test ensures that when a streaming error event has error.code set to None, the system correctly transforms it to 'unknown_error' and returns an ErrorEvent instance without raising a ValidationError.

---------

Co-authored-by: Ku Wei Chiet <chietkuw@fpt.com>
2025-11-27 21:24:04 -08:00
Krish Dholakia
2bd41dc034
Guardrails - Responses API, Image Gen, Text completions, Audio transcriptions, Audio Speech, Rerank, Anthropic Messages API support via the unified apply_guardrails function (#15706)
* fix(presidio.py): handle content as a list of texts

covers openai + anthropic messages api

* fix(presidio.py): safe get messages

* test: add unit testing for presidio guardrails

* fix(unified_guardrail.py): initial commit

* fix(enkryptai.py): implement apply_guardrail to enkrypt guardrail

* fix(unified_guardrail.py): support unified guardrail on input

* feat(unified_guardrail.py): add post call success hook implementation

allows us to just have 1 place to handle llm translation to guardrail api spec

* refactor: refactor initial unified guardrail component

* refactor: more refactoring

* feat(responses/): add guardrails to responses api

allows existing guardrails to work for new llm endpoints

* docs(adding_guardrail_support.md): document new guardrail endpoint support

* test: add unit tests

* feat(image_generation/): add guardrail support for image generation endpoint

* feat(openai/text_completion): support guardrails on `/v1/completions` API

* docs: document guardrails support on new endpoints

* docs: clarify when guardrails run

* feat(openai/speech): add guardrail support for input

* docs(rerank/): add guardrail support on input query

* fix: fix ruff check
2025-10-25 13:38:57 -07:00
Sameer Kankute
c0d0424eb8
Added streaming support for response api streaming image generation (#15269) 2025-10-07 08:15:57 -07:00
Sameer Kankute
4fefac1bf2 Move test to test_litellm/ folder 2025-09-05 10:24:55 +05:30
Sameer Kankute
fc9560573b
[BUG] Fix response api for reasoning item in input for litellm proxy (#14200)
* fix response api for litellm proxy

* Add test for checking if status is getting removed

* add test in correct file

* remove hardcoded fields

* Make the handling simpler

* fix lint error:
2025-09-04 10:36:48 -07:00
Krrish Dholakia
5b641380ab fix(openai/responses/transformation.py): update supported openai params 2025-08-16 01:49:54 -07:00
Ishaan Jaff
39955129f5
fix mapped tests (#12320)
* fix - use flush llm client cache

* faster mapped tests

* test_async_multiple_response_ids_routing

* fix tests

* test_ateam_member_update_admin_requires_premium

* regular mapped tests

* Revert "Fix: Initialize JSON logging for all loggers when JSON_LOGS=True (#12206)"

This reverts commit 2c60c316ec.

* reset num workers
2025-07-04 10:04:43 -07:00
hsuyuming
e3ba888c63
fix: make response api support Azure Authentication method (#11941)
* fix: make response api support Azure Authentication method
1. Support diverse Azure authentication methods
2. Use distinct headers for API key and Azure AD token base on this
   documentation (https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/responses?tabs=rest-api#generate-a-text-response)

* fix: fix path issue

* fix lint error

* rename test_transformation.py to test_azure_transformation.py

* change litellm_params as Optional type
2025-06-23 08:43:20 -07:00
Ishaan Jaff
99d851544a
[Feat] Add Azure Codex Models on LiteLLM + new /v1 preview Azure OpenAI API (#11934)
* fix get_complete_url

* fixes _is_azure_v1_api_version

* test_azure_responses_api_preview_api_version

* TestAzureResponsesAPIConfig

* add azure/codex-mini

* fix azure/codex-mini

* Update litellm/llms/azure/responses/transformation.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix linting

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-20 18:08:44 -07:00
Ishaan Jaff
f3f7bc1537
[Feat] Add Background mode for Responses API - OpenAI, AzureOpenAI (#11640)
* feat: add background as supported param responses create

* add background param for responses API
2025-06-11 15:36:25 -07:00
Ishaan Jaff
4dc9626fd5
[Feat] New LLM API Endpoint - Add List input items for Responses API (#11602)
* (feat) add list_input_items

* add alist_input_items to router

* add GET input_items for responses API

* test_basic_openai_list_input_items_endpoint

* TestTransformListInputItemsRequest

* test_ensure_initialize_azure_sdk_client_always_used
2025-06-10 15:47:16 -07:00
Krish Dholakia
ef42461c1e
Litellm fix GitHub action testing (#11163)
* test: add __init__.py files

* refactor: rename test folder to avoid naming conflict

* test: update workflows

* test: update tests

* test: update imports

* test: update tests

* test: remove unused import

* ci(test-litellm.yml): add pytest retry to github workflow

* test: fix test
2025-05-26 14:41:42 -07:00