From 0fbfc96af38410f3657effe37d9f2d3b9305b522 Mon Sep 17 00:00:00 2001 From: Sameer Kankute Date: Mon, 30 Mar 2026 16:41:58 +0530 Subject: [PATCH] docs(responses): add use_responses_api_bridge opt-in bridge docs --- docs/my-website/docs/response_api.md | 58 ++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/docs/my-website/docs/response_api.md b/docs/my-website/docs/response_api.md index 0c428000c72..36c9ee13515 100644 --- a/docs/my-website/docs/response_api.md +++ b/docs/my-website/docs/response_api.md @@ -1505,6 +1505,64 @@ curl http://localhost:4000/v1/responses \ +### Opt-in bridge for `openai/` models with custom `api_base` + +If you're using an **OpenAI-compatible third-party provider** (e.g. llama.cpp, vLLM, LM Studio) via `openai/` prefix with a custom `api_base`, LiteLLM will normally forward `/responses` requests directly to that endpoint. If the provider only supports `/chat/completions`, the request will fail. + +Set `use_responses_api_bridge: true` to force the `/responses` → `/chat/completions` bridge for these models. + +#### Python SDK Usage + +```python showLineNumbers title="Force bridge for custom openai/ endpoint" +import litellm + +response = litellm.responses( + model="openai/my-custom-model", + input="Hello!", + api_base="http://localhost:8080", + api_key="fake-key", + use_responses_api_bridge=True, +) + +print(response) +``` + +#### LiteLLM Proxy Usage + +**Setup Config:** + +```yaml showLineNumbers title="config.yaml — bridge for custom openai/ endpoint" +model_list: +- model_name: my-local-model + litellm_params: + model: openai/my-custom-model + api_base: http://localhost:8080/v1 + api_key: fake-key + use_responses_api_bridge: true +``` + +**Start Proxy:** + +```bash showLineNumbers title="Start LiteLLM Proxy" +litellm --config /path/to/config.yaml + +# RUNNING on http://0.0.0.0:4000 +``` + +**Make Request:** + +```bash showLineNumbers title="Request via bridge" +curl http://localhost:4000/v1/responses \ + -H "Content-Type: application/json" \ + -H "Authorization: Bearer sk-1234" \ + -d '{ + "model": "my-local-model", + "input": "Hello!" + }' +``` + +This is particularly useful when connecting clients that hardcode the `/responses` endpoint (e.g. OpenAI Codex CLI with `wire_api = "responses"`) to local or third-party OpenAI-compatible providers that only expose `/chat/completions`. + ## Server-side compaction For long-running conversations, you can enable **server-side compaction** so that when the rendered context size crosses a threshold, the server automatically runs compaction in-stream and emits a compaction item—no separate `POST /v1/responses/compact` call is required.