scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-09 03:18:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 108

Add allow defining default query params for a pass through

Browse source
This commit is contained in:
Sameer Kankute 2026-02-19 12:48:03 +05:30
parent 26c3d7debc
commit 4a50c55d84
5 changed files with 113 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

70
docs/my-website/docs/proxy/pass_through.md
View file

@ -58,6 +58,17 @@ Configure the required authentication and pricing:
- The Bria API requires an `api_token` header
- Enter your Bria API key as the value for the `api_token` header
**Default Query Parameters (Optional):**
- Add query parameters that will be automatically sent with every request
- Perfect for API versioning, format specifications, or default configurations
- Clients can override these parameters by providing their own values
- Example: `version=v1`, `format=json`, `timeout=30`
<Image
img={require('../../img/passthrough_query_default.png')}
style={{width: '60%', display: 'block', margin: '2rem auto'}}
/>
**Pricing Configuration:**
- Set a cost per request (e.g., $12.00 in this example)
- This enables cost tracking and billing for your users
@ -112,6 +123,9 @@ general_settings:
content-type: application/json
accept: application/json
forward_headers: true # Forward all incoming headers
default_query_params: # Optional: Default query parameters
version: "v1" # Always send version=v1
format: "json" # Default format (can be overridden)
```
### Start and Test
@ -167,6 +181,8 @@ general_settings:
forward_headers: boolean # Forward all incoming headers
include_subpath: boolean # If true, forwards requests to sub-paths (default: false)
methods: list[string] # Optional: HTTP methods (e.g., ["GET", "POST"]). If not specified, all methods are supported.
default_query_params: # Optional: Default query parameters sent with every request
<param-name>: string # Key-value pairs (e.g., version: "v1", format: "json")
headers: # Custom headers to add
Authorization: string # Auth header for target API
content-type: string # Request content type
@ -178,11 +194,17 @@ general_settings:
### Header Options
- **Authorization**: Authentication for the target API
- **content-type**: Request body format specification
- **content-type**: Request body format specification
- **accept**: Expected response format
- **LANGFUSE_PUBLIC_KEY/SECRET_KEY**: For Langfuse integration
- **Custom headers**: Any additional key-value pairs
### Default Query Parameters
- **Parameter precedence**: Client params > URL params > default params
- **Use cases**: API versioning, authentication tokens, format control, feature flags
- **Override capability**: Clients can override any default parameter
- **Examples**: `version: "v1"`, `format: "json"`, `timeout: "30"`
### Sub-path Routing
By default, pass-through endpoints only match the **exact path** specified. To forward requests to sub-paths, set `include_subpath: true`:
@ -202,7 +224,51 @@ general_settings:
---
### Method-Specific Routing
### Default Query Parameters
Pass-through endpoints support default query parameters that are automatically added to every request. This is useful for API versioning, format specifications, authentication tokens, or any default configuration.
#### How It Works
**Parameter Precedence (highest to lowest priority):**
1. **Client-provided parameters** (in the request URL)
2. **URL parameters** (from the target URL)
3. **Default parameters** (from configuration)
#### Example Configuration
```yaml
general_settings:
pass_through_endpoints:
- path: "/api/v1"
target: "https://external-api.com/service?timeout=60" # URL has timeout=60
default_query_params:
version: "v1" # Always add version=v1
format: "json" # Default format=json (can be overridden)
auth_level: "basic" # Always add auth_level=basic
```
#### Request Examples
**Client Request:** `GET /api/v1/users`
**Actual Backend Call:** `https://external-api.com/service?version=v1&format=json&auth_level=basic&timeout=60`
**Client Request:** `GET /api/v1/users?format=xml&custom=value`
**Actual Backend Call:** `https://external-api.com/service?version=v1&auth_level=basic&timeout=60&format=xml&custom=value`
- Client `format=xml` overrides default `format=json`
- Default `version=v1` and `auth_level=basic` are preserved
- URL `timeout=60` is preserved
- Client `custom=value` is added
#### Use Cases
- **API Versioning**: Always send `version=v2` to maintain compatibility
- **Authentication**: Add authentication tokens like `api_key=default_key`
- **Format Control**: Default to `format=json` but allow client override
- **Rate Limiting**: Set `rate_limit=standard` as default
- **Feature Flags**: Enable `experimental=false` by default
---
You can configure different target URLs for the same path using different HTTP methods. This is useful when different backends handle different operations:

BIN
docs/my-website/img/passthrough_query_default.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 84 KiB

19
litellm/passthrough/utils.py
View file

@ -9,7 +9,9 @@ from litellm.constants import PASS_THROUGH_HEADER_PREFIX
class BasePassthroughUtils:
@staticmethod
def get_merged_query_parameters(
existing_url: httpx.URL, request_query_params: Dict[str, Union[str, list]]
existing_url: httpx.URL,
request_query_params: Dict[str, Union[str, list]],
default_query_params: Optional[Dict[str, Union[str, list]]] = None
) -> Dict[str, Union[str, List[str]]]:
# Get the existing query params from the target URL
existing_query_string = existing_url.query.decode("utf-8")
@ -19,8 +21,19 @@ class BasePassthroughUtils:
updated_existing_query_params = {
k: v[0] if len(v) == 1 else v for k, v in existing_query_params.items()
}
# Merge the query params, giving priority to the existing ones
return {**request_query_params, **updated_existing_query_params}
# Start with default query params (lowest priority)
merged_params = {}
if default_query_params:
merged_params.update(default_query_params)
# Override with existing URL query params (medium priority)
merged_params.update(updated_existing_query_params)
# Override with request query params (highest priority - client can override anything)
merged_params.update(request_query_params)
return merged_params
@staticmethod
def forward_headers_from_request(

4
litellm/proxy/_types.py
View file

@ -1909,6 +1909,10 @@ class PassThroughGenericEndpoint(LiteLLMPydanticObjectBase):
default={},
description="Key-value pairs of headers to be forwarded with the request. You can set any key value pair here and it will be forwarded to your target endpoint",
)
default_query_params: dict = Field(
default={},
description="Key-value pairs of default query parameters to be sent with every request to this endpoint. These can be overridden by client-provided query parameters. For example: {'key': 'default_value', 'api_version': '2023-01'}",
)
include_subpath: bool = Field(
default=False,
description="If True, requests to subpaths of the path will be forwarded to the target endpoint. For example, if the path is /bria and include_subpath is True, requests to /bria/v1/text-to-image/base/2.3 will be forwarded to the target endpoint.",

27
litellm/proxy/pass_through_endpoints/pass_through_endpoints.py
View file

@ -600,6 +600,7 @@ async def pass_through_request( # noqa: PLR0915
forward_headers: Optional[bool] = False,
merge_query_params: Optional[bool] = False,
query_params: Optional[dict] = None,
default_query_params: Optional[dict] = None,
stream: Optional[bool] = None,
cost_per_request: Optional[float] = None,
custom_llm_provider: Optional[str] = None,
@ -617,6 +618,7 @@ async def pass_through_request( # noqa: PLR0915
forward_headers: Whether to forward headers
merge_query_params: Whether to merge query params
query_params: The query params
default_query_params: The default query params to be applied if not overridden by client
stream: Whether to stream the response
cost_per_request: Optional field - cost per request to the target endpoint
custom_llm_provider: Optional field - custom LLM provider for the endpoint
@ -649,13 +651,18 @@ async def pass_through_request( # noqa: PLR0915
forward_headers=forward_headers,
)
if merge_query_params:
# Apply default query parameters if provided, regardless of merge_query_params setting
if default_query_params or merge_query_params:
# Determine what to merge based on settings
request_params = dict(request.query_params) if merge_query_params else {}
# Create a new URL with the merged query params
url = url.copy_with(
query=urlencode(
HttpPassThroughEndpointHelpers.get_merged_query_parameters(
existing_url=url,
request_query_params=dict(request.query_params),
request_query_params=request_params,
default_query_params=default_query_params,
)
).encode("ascii")
)
@ -1080,6 +1087,7 @@ def create_pass_through_route(
custom_llm_provider: Optional[str] = None,
is_streaming_request: Optional[bool] = False,
query_params: Optional[dict] = None,
default_query_params: Optional[dict] = None,
guardrails: Optional[Dict[str, Any]] = None,
):
# check if target is an adapter.py or a url
@ -1171,6 +1179,7 @@ def create_pass_through_route(
"cost_per_request", cost_per_request
)
param_guardrails = target_params.get("guardrails", None)
param_default_query_params = target_params.get("default_query_params", None)
# Construct the full target URL with subpath if needed
full_target = (
@ -1208,6 +1217,7 @@ def create_pass_through_route(
forward_headers=cast(Optional[bool], param_forward_headers),
merge_query_params=cast(Optional[bool], param_merge_query_params),
query_params=final_query_params,
default_query_params=param_default_query_params,
stream=is_streaming_request or stream,
custom_body=final_custom_body,
cost_per_request=cast(Optional[float], param_cost_per_request),
@ -1849,6 +1859,7 @@ class InitPassThroughEndpointHelpers:
endpoint_id: str,
guardrails: Optional[dict] = None,
methods: Optional[List[str]] = None,
default_query_params: Optional[dict] = None,
):
"""Add exact path route for pass-through endpoint"""
# Default to all methods if none specified (backward compatibility)
@ -1886,6 +1897,7 @@ class InitPassThroughEndpointHelpers:
merge_query_params,
dependencies,
cost_per_request=cost_per_request,
default_query_params=default_query_params,
guardrails=guardrails,
),
methods=methods,
@ -1903,6 +1915,7 @@ class InitPassThroughEndpointHelpers:
"custom_headers": custom_headers,
"forward_headers": forward_headers,
"merge_query_params": merge_query_params,
"default_query_params": default_query_params,
"dependencies": dependencies,
"cost_per_request": cost_per_request,
"guardrails": guardrails,
@ -1922,6 +1935,7 @@ class InitPassThroughEndpointHelpers:
endpoint_id: str,
guardrails: Optional[dict] = None,
methods: Optional[List[str]] = None,
default_query_params: Optional[dict] = None,
):
"""Add wildcard route for sub-paths"""
# Default to all methods if none specified (backward compatibility)
@ -1960,6 +1974,7 @@ class InitPassThroughEndpointHelpers:
dependencies,
include_subpath=True,
cost_per_request=cost_per_request,
default_query_params=default_query_params,
guardrails=guardrails,
),
methods=methods,
@ -1977,6 +1992,7 @@ class InitPassThroughEndpointHelpers:
"custom_headers": custom_headers,
"forward_headers": forward_headers,
"merge_query_params": merge_query_params,
"default_query_params": default_query_params,
"dependencies": dependencies,
"cost_per_request": cost_per_request,
"guardrails": guardrails,
@ -2170,6 +2186,7 @@ async def initialize_pass_through_endpoints(
)
_forward_headers = endpoint.get("forward_headers", None)
_merge_query_params = endpoint.get("merge_query_params", None)
_default_query_params = endpoint.get("default_query_params", None)
_auth = endpoint.get("auth", None)
_dependencies = None
if _auth is not None and str(_auth).lower() == "true":
@ -2207,6 +2224,7 @@ async def initialize_pass_through_endpoints(
endpoint_id=endpoint_id,
guardrails=_guardrails,
methods=_methods,
default_query_params=_default_query_params,
)
# Generate route key with methods for tracking
@ -2228,6 +2246,7 @@ async def initialize_pass_through_endpoints(
endpoint_id=endpoint_id,
guardrails=_guardrails,
methods=_methods,
default_query_params=_default_query_params,
)
visited_endpoints.add(f"{endpoint_id}:subpath:{_path}:{methods_str}")
@ -2546,6 +2565,7 @@ async def update_pass_through_endpoints(
endpoint_id=updated_endpoint.id or endpoint_id or "",
guardrails=getattr(updated_endpoint, "guardrails", None),
methods=updated_endpoint.methods,
default_query_params=updated_endpoint.default_query_params,
)
else:
InitPassThroughEndpointHelpers.add_exact_path_route(
@ -2560,6 +2580,7 @@ async def update_pass_through_endpoints(
endpoint_id=updated_endpoint.id or endpoint_id or "",
guardrails=getattr(updated_endpoint, "guardrails", None),
methods=updated_endpoint.methods,
default_query_params=updated_endpoint.default_query_params,
)
return PassThroughEndpointResponse(
@ -2637,6 +2658,7 @@ async def create_pass_through_endpoints(
endpoint_id=created_endpoint.id or "",
guardrails=getattr(created_endpoint, "guardrails", None),
methods=created_endpoint.methods,
default_query_params=created_endpoint.default_query_params,
)
else:
InitPassThroughEndpointHelpers.add_exact_path_route(
@ -2651,6 +2673,7 @@ async def create_pass_through_endpoints(
endpoint_id=created_endpoint.id or "",
guardrails=getattr(created_endpoint, "guardrails", None),
methods=created_endpoint.methods,
default_query_params=created_endpoint.default_query_params,
)
return PassThroughEndpointResponse(endpoints=[created_endpoint])