mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-08 03:08:45 +00:00
fde4c7c97a
* feat(gdc): add Google Distributed Cloud Gemini provider support Introduce support for the Google Distributed Cloud (GDC) Gemini provider by adding "gdc" to the list of chat providers and enabling the gdc/ model prefix. The implementation defines a new GDCGeminiConfig class which handles authentication via Google Distributed Cloud service account credentials, manages token generation, formats GDC Gemini request URLs, and transforms request structures accordingly The PreProcessNonDefaultParams class is also updated to exclude vertex parameters from filtering when the custom LLM provider is GDC, allowing vertex parameters to be passed properly during GDC initialization * fix: resolve issues identified in PR #30702 * fix(gdc): harden credentials, fix vertex param filtering, add tests The supports_vertex_params branch regressed vertex_ai and vertex_ai_beta: the `if custom_llm_provider in [...]: pass` was a no-op, so those providers fell through to the config lookup, found no supports_vertex_params, and had their vertex_ params stripped. The check is now a single _provider_supports_vertex_params helper that keeps vertex_ params for the vertex family and for any config that opts in, and only swallows the expected ValueError from an unknown provider string instead of a blanket except GDC project and location now resolve from the deployment's litellm_params and the litellm.vertex_project / litellm.vertex_location globals before falling back to request optional_params, matching how vertex_ai resolves them, so a proxy caller can no longer route a request to a project the deployment did not expose A request api_key is no longer treated as a filesystem path, so a caller can't make the host open a local service-account file; api_key must be a literal service-account JSON string or a bearer token The opt-in token cache is hardened: the lock and cache dict are created in __init__ instead of via a racy hasattr lazy-init, the token is read inside the lock, and the audience is stripped of a trailing slash once so the cached and non-cached paths agree Also declares gdc_api_base, switches the lazy-import entry to the relative path every other entry uses, adds the missing trailing comma in the provider config map, and drops the api_base fallback that only ran when api_key was None Adds unit tests covering the vertex-param filter, deployment-over-request precedence, the api_key file-path rejection, URL construction branches, environment validation, token caching, and the gdc completion dispatch; transformation.py is fully covered * fix(gdc): prefer GDC-specific config, honor vertex_ai aliases, harden URL and bool parsing * fix(gdc): mint the GDCH token audience from the host, not the full base When api_base embedded /v1/projects/... and the deployment set project/location, get_complete_url rebuilt the request URL from the host while validate_environment still derived the token audience from the full original api_base, so the bearer token could target a different audience than the URL actually called. The audience is now the scheme://host of api_base in every case, matching the host get_complete_url builds against * fix(gdc): restrict JSON api_key to GDCH service accounts Only accept a credential whose type is gdch_service_account before calling google.auth.load_credentials_from_dict, so a caller-supplied external_account/identity_pool/pluggable credential carrying arbitrary token or credential_source endpoints is rejected before any token refresh runs. GDC only ever uses GDCH service accounts, and non-GDCH credentials could not have completed auth anyway (with_gdch_audience is GDCH-only), so this narrows the credential-refresh surface without changing valid GDC behavior. * fix(gdc): validate project and location as plain identifiers vertex_project and vertex_location can come from request params and were interpolated as raw path text into the GDC request URL and the x-goog-user-project header. A caller-supplied value containing / ? # or .. could reshape the path and make the proxy send its GDC-authorized request to a different endpoint under the configured host. Validate both against a strict identifier pattern before building the URL or header and raise an auth error otherwise; GCP project ids and locations are plain identifiers so valid deployments are unaffected. * fix(gdc): bind x-goog-user-project quota header to the deployment The quota project header was resolved with request-level vertex_project taking effect, so with a preformed deployment api_base a caller could set vertex_project to a different project and have it sent under the proxy's GDC credential, misattributing quota or billing. Resolve the header project the same way the URL is resolved: a preformed api_base without a deployment override binds to the project embedded in the URL, otherwise deployment and global config win over request params. This keeps the URL and the quota header consistent. * fix(gdc): always rebind x-goog-user-project, stripping caller-forwarded values The quota project header was only set when absent, so with client header forwarding an authenticated caller could send their own x-goog-user-project (any casing) and have it ride on the proxy's GDC credential, bypassing the deployment-derived binding. Strip every casing of the header and always set it from _effective_project before the request is signed. * fix(gdc): make a preformed api_base authoritative for project routing get_litellm_params copies caller-supplied vertex_project and vertex_location into litellm_params via OPTIONAL_KWARGS_KEYS, so litellm_params cannot be treated as a deployment-only source. The previous _deployment_overrides_path inference let an authenticated caller flip a pinned preformed api_base such as /v1/projects/pinned/... to /v1/projects/attacker/..., driving requests to a caller-chosen project with the proxy's configured GDC credentials and quota header A preformed /v1/projects/ api_base is now authoritative; get_complete_url returns it unchanged and _effective_project binds the x-goog-user-project quota header to the project embedded in that URL, so a caller can no longer redirect a pinned deployment or move the quota header off it. The two tests that asserted the override behavior are now regression tests that fail if the rewrite is reintroduced * fix(gdc): make a preformed api_base self-sufficient in get_complete_url get_complete_url resolved and required a params-derived vertex_project before returning a preformed /v1/projects/ api_base, so a deployment that pins its project in the api_base path was forced to also pass vertex_project or hit 'project is required'. validate_environment already extracts the project from a preformed URL and needs no such param, so the two paths disagreed The preformed-URL early return now runs before project/location resolution, matching validate_environment: a preformed api_base is returned as-is with no redundant param, and non-preformed bases still require vertex_project and vertex_location as before. Adds a regression test that a preformed base with no project/location params returns the URL unchanged --------- Co-authored-by: Paige O'Connor <lostpaige@google.com> Co-authored-by: Tim Laubach <tlaubach@google.com> |
||
|---|---|---|
| .. | ||
| aiml/image_generation | ||
| amazon_nova/chat | ||
| anthropic | ||
| apiserpent | ||
| azure | ||
| azure_ai | ||
| base_llm | ||
| baseten/chat | ||
| bedrock | ||
| bedrock_mantle | ||
| black_forest_labs | ||
| bytez/chat | ||
| chat | ||
| chatgpt | ||
| cloudflare | ||
| cohere | ||
| cometapi/chat | ||
| compactifai | ||
| crusoe | ||
| custom_httpx | ||
| dashscope | ||
| databricks | ||
| datarobot | ||
| deepgram | ||
| deepinfra | ||
| deepseek | ||
| docker_model_runner | ||
| elevenlabs | ||
| fal_ai/image_generation | ||
| fastcrw/search | ||
| featherless_ai/chat | ||
| fireworks_ai | ||
| gdc/chat | ||
| gemini | ||
| github_copilot | ||
| heroku | ||
| hosted_vllm | ||
| huggingface | ||
| inception | ||
| jina_ai/embedding | ||
| langflow | ||
| lemonade | ||
| litellm_proxy | ||
| llamafile/chat | ||
| lm_studio | ||
| manus | ||
| meta_llama | ||
| minimax | ||
| mistral | ||
| modelscope | ||
| moonshot | ||
| nebius | ||
| neosantara | ||
| novita/chat | ||
| nscale/chat | ||
| nvidia_riva | ||
| oci | ||
| ocr | ||
| ollama | ||
| openai | ||
| openai_like | ||
| openrouter | ||
| ovhcloud | ||
| parallel_ai | ||
| parasail | ||
| pass_through | ||
| perplexity | ||
| pg_vector/vector_stores | ||
| publicai | ||
| ragflow/chat | ||
| recraft | ||
| reducto | ||
| runwayml | ||
| s3_vectors | ||
| sagemaker | ||
| sambanova | ||
| sap | ||
| scaleway | ||
| snowflake | ||
| soniox | ||
| stability | ||
| tinyfish | ||
| vercel_ai_gateway | ||
| vertex_ai | ||
| volcengine | ||
| voyage | ||
| wandb | ||
| watsonx | ||
| xai | ||
| you_com | ||
| zai | ||
| test_cache_control_and_reasoning.py | ||
| test_file_content_block.py | ||
| test_file_search_responses.py | ||
| test_lifecycle_fix.py | ||
| test_oom_fixes.py | ||
| test_polling_url_origin_match.py | ||
| test_predibase_transformation.py | ||