feat(litellm): add Vispark provider with Vision Small/Medium/Large (#41192)
Some checks are pending
unit / rust-bridge-harness (push) Blocked by required conditions
unit / llms-providers (push) Blocked by required conditions
unit / proxy-endpoints (push) Blocked by required conditions
unit / proxy-feature-endpoints (push) Blocked by required conditions
unit / rust-bridge (push) Waiting to run
unit / assert-shard-coverage (push) Waiting to run
unit / enterprise-managed-files (push) Blocked by required conditions
unit / enterprise-repositories-secrets (push) Blocked by required conditions
unit / proxy-server (push) Blocked by required conditions
unit / lens-python-310 (push) Blocked by required conditions
unit / caching-local (push) Blocked by required conditions
unit / core-utils (push) Blocked by required conditions
unit / endpoints (push) Blocked by required conditions
unit / enterprise-package (push) Blocked by required conditions
unit / integrations (push) Blocked by required conditions
unit / llms-anthropic (push) Blocked by required conditions
unit / llms-bedrock (push) Blocked by required conditions
unit / llms-openai-meta (push) Blocked by required conditions
unit / llms-vertex-ai (push) Blocked by required conditions
unit / mcp-elicitation (push) Blocked by required conditions
unit / proxy-auth (push) Blocked by required conditions
unit / mcp-server (push) Blocked by required conditions
unit / proxy-extras (push) Blocked by required conditions
unit / proxy-hooks-client (push) Blocked by required conditions
unit / proxy-infra-db-spend (push) Blocked by required conditions
unit / proxy-infra-root-a-p (push) Blocked by required conditions
unit / proxy-infra-root-q-z (push) Blocked by required conditions
unit / docs (push) Waiting to run
unit / OpenAI 2.20.0 compatibility (push) Blocked by required conditions
unit / helm (push) Waiting to run
unit / OpenAI 3.0.0 compatibility (push) Blocked by required conditions
unit / OpenAI 3.25.0 compatibility (push) Blocked by required conditions
unit / redis-py 7.4.1 compatibility (push) Blocked by required conditions
unit / redis-py 5.3.1 compatibility (push) Blocked by required conditions
unit / redis-py 6.4.0 compatibility (push) Blocked by required conditions
unit / redis-py 8.0.1 compatibility (push) Blocked by required conditions
unit / coverage (push) Blocked by required conditions
unit / unit passed (push) Blocked by required conditions
GitHub Actions Security Analysis / zizmor (push) Waiting to run
CodSpeed Benchmarks / benchmarks (push) Waiting to run
Cost map pins recheck / rerun-cost-map-guard (push) Waiting to run
Publish lint base counts / publish (basedpyright, litellm/scripts/type_check_gate.py) (push) Waiting to run
Publish lint base counts / publish (openapi-docs, litellm/scripts/openapi_docs_gate.py) (push) Waiting to run
Publish lint base counts / publish (ruff-strict, litellm/scripts/ruff_strict_gate.py) (push) Waiting to run
Publish lint base counts / publish (test-quality, litellm/scripts/test_quality_gate.py) (push) Waiting to run
Publish lint base counts / publish (type-discipline, litellm/scripts/type_discipline_gate.py) (push) Waiting to run
Root entry limit / root-entry-limit (push) Waiting to run
Scorecard supply-chain security / Scorecard analysis (push) Waiting to run
smoke / dashboard-build (push) Waiting to run
smoke / py3.10 (push) Waiting to run
smoke / py3.11 (push) Waiting to run
smoke / py3.12 (push) Waiting to run
smoke / py3.13 (push) Waiting to run
smoke / py3.14 (push) Waiting to run
smoke / py3.15 (push) Waiting to run
smoke / all-extras-py3.12 (push) Waiting to run
smoke / all-extras-py3.13 (push) Waiting to run
smoke / windows-sdk (push) Waiting to run
smoke / proxy-build-from-pip (push) Waiting to run
smoke / smoke passed (push) Blocked by required conditions
LiteLLM Rust DB / db-queries (push) Waiting to run
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
LiteLLM Rust / rust-wheel (push) Waiting to run
unit / auth-checks (push) Blocked by required conditions
unit / budgets (push) Blocked by required conditions
unit / custom-logging (push) Blocked by required conditions
unit / endpoints-and-responses (push) Blocked by required conditions
unit / enterprise-routing (push) Blocked by required conditions
unit / jwt-and-keys (push) Blocked by required conditions
unit / key-generation (push) Blocked by required conditions
unit / logging-misc (push) Blocked by required conditions
unit / db-and-spend (push) Blocked by required conditions
unit / guardrails-hooks (push) Blocked by required conditions
unit / mcp-oauth (push) Blocked by required conditions
unit / proxy-runtime (push) Blocked by required conditions
unit / proxy-server-core (push) Blocked by required conditions
unit / root (push) Blocked by required conditions
unit / proxy-utils (push) Blocked by required conditions
unit / ui-unit (push) Waiting to run
unit / proxy-infra-routing (push) Blocked by required conditions
unit / proxy-subprocess (push) Blocked by required conditions
unit / responses-caching-types (push) Blocked by required conditions
unit / router (push) Blocked by required conditions

* feat(litellm): add Vispark provider with Vision Small/Medium/Large

Rebased onto latest main: re-applied vispark registrations (model sets,
provider enum, JSON provider registry, pricing catalogs, dashboard
fields, endpoints matrix) and moved provider test to
tests/unit/llms/openai_like/ per current layout.

* fix(litellm): keep basedpyright reportUnknownArgumentType budget flat for vispark

* fix(litellm): satisfy type-discipline budget for vispark cast

* chore: retrigger CI (prior MCP dep-resolution leg cancelled by infra)

* test(vispark): cover routing, text completion and cost; sync endpoint matrix backup

* test(vispark): tolerate cost-map entries without a provider when listing models

* chore(vispark): regenerate the provider endpoint matrix with main's generator

* fix(vispark): bill the 600 input token minimum and bridge JSON schemas and PDF file parts

Vispark charges every request for at least 600 input tokens, so the cost map
gains min_billed_input_tokens and the generic cost path bills up to it.
Vispark answers 500 to a json_schema that sets additionalProperties, which the
OpenAI SDK always sends for a Pydantic model, so the provider strips it.
Vispark reads PDFs only as image_url parts, so file parts are sent that way.

* fix(vispark): bill the input token minimum under custom pricing and strip every additionalProperties value

* refactor(router): inline the billing minimum lookup into the deployment cost payload

* fix(vispark): scope the input minimum per deployment and reject map-field schemas

Keep a deployment's min_billed_input_tokens on its own cost-map entry so it
no longer changes billing for other deployments of the same backend model.
Strip boolean additionalProperties for Vispark, answer 400 for schema-valued
ones unless drop_params is set, and do it without recursion.

* fix(budget): reserve at least a deployment's billed input minimum

* test(budget): patch the renamed deployment info helper

* fix(vispark): reject file_id parts, keep properties named additionalProperties, bill the minimum on dashscope and perplexity

* fix(vispark): send inline file_data when a file part also carries a file_id

* fix: keep provider-stated cost for minimum-only deployments and skip the minimum without a counted prompt

A deployment that sets only min_billed_input_tokens no longer drops Perplexity or Kyma stated cost, streamed cost headers no longer bill the minimum before usage arrives, and tiered deployments now carry the minimum

* fix: bill provider-stated cost for minimum-only deployments and charge a disconnect only the group's shared minimum

* style: drop the restating comment on the vispark provider entry

* fix(cost): keep the provider-reported video cost for a deployment that sets only a billing minimum

* fix(cost): read the billing minimum the way the cost map guard accepts it

A cost map row can write min_billed_input_tokens as "600" or 600.0, which the
catalog's count parser accepts. The chat cost path subtracted the raw value and
raised TypeError on a string, and the custom-rate path billed no minimum unless
the value was an int. Both now read it through one non-negative int adapter

Also loosen the Scaleway file part test, since main now adds a default filename

* fix(vispark): keep schema literals and reserve at the short prompt's tier

Removing additionalProperties now skips the literal data in const, enum,
default and examples, so a value holding that key reaches Vispark unchanged

Budget reservation picks the pricing tier from the counted prompt and only then
applies the billed input minimum, the way billing does, so a minimum that
crosses into a cheaper tier no longer reserves less than the request bills

* fix(utils): list JSON providers whose API key env var differs from their name

* fix(vispark): answer 400 for a file part with no file instead of crashing

* fix(cost): use a deployment's own input minimum with per-token rates

/spend/calculate and /cost/estimate priced custom-rate deployments with the
cost map's min_billed_input_tokens, so a deployment that sets its own
minimum (0 or 1000, say) got a quote that differs from what its live
requests bill. Both endpoints now pass the deployment's minimum with the
rates, and the per-token floor also reads it from the requested model info
before falling back to the cost map

* chore(vispark): keep catalog and endpoint formatting aligned

* fix(vispark): mark provider registry reference final

* fix(vispark): reject missing provider credentials before fallback

* fix(vispark): preserve authenticated text completion clients

* fix(vispark): preserve authenticated chat completion clients

* fix(vispark): reject unused SDK credentials on HTTP transport

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
This commit is contained in:
Vansh 2026-11-17 14:18:52 +05:30 • committed by GitHub
parent 78d0a25fcd
commit 0cc072c999
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
11 changed files with 544 additions and 0 deletions

View file

@ -703,6 +703,7 @@ streamlake_models: Final[Set[str]] = set()
wally_models: Final[Set[str]] = set()
zerogpu_models: Final[Set[str]] = set()
v0_models: Set = set()
vispark_models: Final[Set[str]] = set()
morph_models: Set = set()
opencode_go_models: Final[set[str]] = set() # mutable-ok: populated when the shared model catalog loads
lambda_ai_models: Set = set()
@ -1003,6 +1004,8 @@ def _populate_provider_model_sets(model_cost_map: Mapping[str, Mapping[str, obje
zerogpu_models.add(key)
elif value.get("litellm_provider") == "v0":
v0_models.add(key)
elif value.get("litellm_provider") == "vispark":
vispark_models.add(key)
elif value.get("litellm_provider") == "morph":
morph_models.add(key)
elif value.get("litellm_provider") == "opencode_go":
@ -1183,6 +1186,7 @@ model_list = list(
| wally_models
| zerogpu_models
| v0_models
| vispark_models
| morph_models
| opencode_go_models
| lambda_ai_models
@ -1315,6 +1319,7 @@ def _build_models_by_provider() -> dict:
"wally": wally_models,
"zerogpu": zerogpu_models,
"v0": v0_models,
"vispark": vispark_models,
"morph": morph_models,
"opencode_go": opencode_go_models,
"lambda_ai": lambda_ai_models,

View file

@ -1113,6 +1113,7 @@ openai_compatible_endpoints: Final[list[str]] = [
"https://inference.coralbricks.ai/v1",
"https://api.cortecs.ai/v1",
"https://api.scx.ai/v1",
"https://api.lab.vispark.in/v1",
"https://api.prisminference.com/v1",
"https://api.reka.ai/v1",
"https://inference.runanywhere.ai/v1",
@ -1193,6 +1194,7 @@ openai_compatible_providers: Final[list[str]] = [
"pinstripes", # Pinstripes - JSON-configured provider
"openinfer",
"darkbloom",
"vispark",
"meta", # Meta Model API (Muse Spark) - JSON-configured provider
"cognition",
"coralbricks",
@ -1229,6 +1231,7 @@ openai_text_completion_compatible_providers: Final[list] = [ # providers that s
"poe",
"chutes",
"v0",
"vispark",
"lambda_ai",
"hyperbolic",
"wandb",

View file

@ -314,5 +314,17 @@
"supports_function_calling": true,
"supports_reasoning": true
}
},
"vispark": {
"base_url": "https://api.lab.vispark.in/v1",
"api_key_env": "VISPARK_LAB_API_KEY",
"api_base_env": "VISPARK_LAB_API_BASE",
"param_mappings": {
"max_completion_tokens": "max_tokens"
},
"supported_endpoints": [
"/v1/chat/completions",
"/v1/completions"
]
}
}

View file

@ -1974,6 +1974,13 @@ def _complete_azure_ai(ctx: CompletionDispatchContext) -> _CompletionDispatchRes
return response
def _validate_vispark_api_key(provider: str | None, api_key: str | None, model: str, has_client: bool = False) -> None:
if provider == "vispark" and not api_key and not has_client:
raise litellm.AuthenticationError(
message="Set VISPARK_LAB_API_KEY or pass api_key", llm_provider="vispark", model=model
)
def _complete_text_completion_openai(
ctx: CompletionDispatchContext,
) -> _CompletionDispatchResult:
@ -2006,6 +2013,8 @@ def _complete_text_completion_openai(
openai.api_version = None
# set API KEY
_validate_vispark_api_key(custom_llm_provider, api_key, model, has_client=client is not None)
api_key = api_key or litellm.api_key or litellm.openai_key or get_secret("OPENAI_API_KEY")
headers = headers or litellm.headers
@ -2874,6 +2883,14 @@ def _complete_custom_openai(
# for deepinfra/perplexity/anyscale/groq/friendliai we check in get_llm_provider and pass in the api base from there
api_base = _resolve_openai_api_base(api_base)
_validate_vispark_api_key(
custom_llm_provider,
api_key,
model,
has_client=isinstance(client, (openai.OpenAI, openai.AsyncOpenAI))
and not get_secret_bool("EXPERIMENTAL_OPENAI_BASE_LLM_HTTP_HANDLER"),
)
if custom_llm_provider == "bigmodel" and not api_key:
raise litellm.AuthenticationError(
message="Set BIGMODEL_API_KEY or pass api_key", llm_provider="bigmodel", model=model

View file

@ -51727,6 +51727,102 @@
"supports_tool_choice": true,
"supports_vision": true
},
"vispark/vision-small": {
"cache_read_input_token_cost": 0.0,
"input_cost_per_token": 1.05e-06,
"litellm_provider": "vispark",
"max_input_tokens": 1000000,
"max_output_tokens": 65536,
"mode": "chat",
"output_cost_per_token": 3.16e-06,
"source": "https://api.lab.vispark.in/v1/models",
"supported_endpoints": [
"/v1/chat/completions",
"/v1/completions"
],
"supported_modalities": [
"text",
"image",
"audio",
"video"
],
"supported_output_modalities": [
"text"
],
"supports_function_calling": true,
"supports_native_streaming": true,
"supports_parallel_function_calling": true,
"supports_pdf_input": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_system_messages": true,
"supports_tool_choice": true,
"supports_vision": true
},
"vispark/vision-medium": {
"cache_read_input_token_cost": 0.0,
"input_cost_per_token": 4.21e-06,
"litellm_provider": "vispark",
"max_input_tokens": 1000000,
"max_output_tokens": 65536,
"mode": "chat",
"output_cost_per_token": 1.263e-05,
"source": "https://api.lab.vispark.in/v1/models",
"supported_endpoints": [
"/v1/chat/completions",
"/v1/completions"
],
"supported_modalities": [
"text",
"image",
"audio",
"video"
],
"supported_output_modalities": [
"text"
],
"supports_function_calling": true,
"supports_native_streaming": true,
"supports_parallel_function_calling": true,
"supports_pdf_input": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_system_messages": true,
"supports_tool_choice": true,
"supports_vision": true
},
"vispark/vision-large": {
"cache_read_input_token_cost": 0.0,
"input_cost_per_token": 7.37e-06,
"litellm_provider": "vispark",
"max_input_tokens": 1000000,
"max_output_tokens": 65536,
"mode": "chat",
"output_cost_per_token": 2.211e-05,
"source": "https://api.lab.vispark.in/v1/models",
"supported_endpoints": [
"/v1/chat/completions",
"/v1/completions"
],
"supported_modalities": [
"text",
"image",
"audio",
"video"
],
"supported_output_modalities": [
"text"
],
"supports_function_calling": true,
"supports_native_streaming": true,
"supports_parallel_function_calling": true,
"supports_pdf_input": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_system_messages": true,
"supports_tool_choice": true,
"supports_vision": true
},
"vercel_ai_gateway/alibaba/qwen-3-14b": {
"input_cost_per_token": 8e-08,
"litellm_provider": "vercel_ai_gateway",

View file

@ -4332,6 +4332,32 @@
"generateContent": true
}
},
"vispark": {
"display_name": "Vispark (`vispark`)",
"url": "https://lab.vispark.in/#vision",
"endpoints": {
"chat_completions": true,
"responses": true,
"text_completion": true,
"embeddings": false,
"image_generations": false,
"image_edits": false,
"audio_transcriptions": false,
"audio_speech": false,
"rerank": false,
"ocr": false,
"search": false,
"moderations": false,
"realtime": false,
"video_generations": false,
"batches": false,
"messages": true,
"interactions": true,
"systemone": false,
"a2a": true,
"generateContent": true
}
},
"vercel_ai_gateway": {
"display_name": "Vercel AI Gateway (`vercel_ai_gateway`)",
"url": "https://docs.litellm.ai/docs/providers/vercel_ai_gateway",

View file

@ -4548,5 +4548,33 @@
}
],
"default_model_placeholder": "bigmodel/glm-5.3"
},
{
"provider": "VISPARK",
"provider_display_name": "Vispark",
"litellm_provider": "vispark",
"credential_fields": [
{
"key": "api_base",
"label": "API Base",
"placeholder": "https://api.lab.vispark.in/v1",
"tooltip": null,
"required": false,
"field_type": "text",
"options": null,
"default_value": null
},
{
"key": "api_key",
"label": "API Key",
"placeholder": null,
"tooltip": null,
"required": true,
"field_type": "password",
"options": null,
"default_value": null
}
],
"default_model_placeholder": "vispark/vision-medium"
}
]

View file

@ -4747,6 +4747,7 @@ class LlmProviders(str, Enum):
MOONSHOT = "moonshot"
PUBLICAI = "publicai"
V0 = "v0"
VISPARK = "vispark"
MORPH = "morph"
OPENCODE_GO = "opencode_go"
LAMBDA_AI = "lambda_ai"

View file

@ -51727,6 +51727,102 @@
"supports_tool_choice": true,
"supports_vision": true
},
"vispark/vision-small": {
"cache_read_input_token_cost": 0.0,
"input_cost_per_token": 1.05e-06,
"litellm_provider": "vispark",
"max_input_tokens": 1000000,
"max_output_tokens": 65536,
"mode": "chat",
"output_cost_per_token": 3.16e-06,
"source": "https://api.lab.vispark.in/v1/models",
"supported_endpoints": [
"/v1/chat/completions",
"/v1/completions"
],
"supported_modalities": [
"text",
"image",
"audio",
"video"
],
"supported_output_modalities": [
"text"
],
"supports_function_calling": true,
"supports_native_streaming": true,
"supports_parallel_function_calling": true,
"supports_pdf_input": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_system_messages": true,
"supports_tool_choice": true,
"supports_vision": true
},
"vispark/vision-medium": {
"cache_read_input_token_cost": 0.0,
"input_cost_per_token": 4.21e-06,
"litellm_provider": "vispark",
"max_input_tokens": 1000000,
"max_output_tokens": 65536,
"mode": "chat",
"output_cost_per_token": 1.263e-05,
"source": "https://api.lab.vispark.in/v1/models",
"supported_endpoints": [
"/v1/chat/completions",
"/v1/completions"
],
"supported_modalities": [
"text",
"image",
"audio",
"video"
],
"supported_output_modalities": [
"text"
],
"supports_function_calling": true,
"supports_native_streaming": true,
"supports_parallel_function_calling": true,
"supports_pdf_input": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_system_messages": true,
"supports_tool_choice": true,
"supports_vision": true
},
"vispark/vision-large": {
"cache_read_input_token_cost": 0.0,
"input_cost_per_token": 7.37e-06,
"litellm_provider": "vispark",
"max_input_tokens": 1000000,
"max_output_tokens": 65536,
"mode": "chat",
"output_cost_per_token": 2.211e-05,
"source": "https://api.lab.vispark.in/v1/models",
"supported_endpoints": [
"/v1/chat/completions",
"/v1/completions"
],
"supported_modalities": [
"text",
"image",
"audio",
"video"
],
"supported_output_modalities": [
"text"
],
"supports_function_calling": true,
"supports_native_streaming": true,
"supports_parallel_function_calling": true,
"supports_pdf_input": true,
"supports_reasoning": true,
"supports_response_schema": true,
"supports_system_messages": true,
"supports_tool_choice": true,
"supports_vision": true
},
"vercel_ai_gateway/alibaba/qwen-3-14b": {
"input_cost_per_token": 8e-08,
"litellm_provider": "vercel_ai_gateway",

View file

@ -4332,6 +4332,32 @@
"generateContent": true
}
},
"vispark": {
"display_name": "Vispark (`vispark`)",
"url": "https://lab.vispark.in/#vision",
"endpoints": {
"chat_completions": true,
"responses": true,
"text_completion": true,
"embeddings": false,
"image_generations": false,
"image_edits": false,
"audio_transcriptions": false,
"audio_speech": false,
"rerank": false,
"ocr": false,
"search": false,
"moderations": false,
"realtime": false,
"video_generations": false,
"batches": false,
"messages": true,
"interactions": true,
"systemone": false,
"a2a": true,
"generateContent": true
}
},
"vercel_ai_gateway": {
"display_name": "Vercel AI Gateway (`vercel_ai_gateway`)",
"url": "https://docs.litellm.ai/docs/providers/vercel_ai_gateway",

View file

@ -0,0 +1,234 @@
import json
from pathlib import Path
from typing import Final
import pytest
import respx
import litellm
_CHAT_COMPLETION: Final = {
"id": "chatcmpl_vispark",
"object": "chat.completion",
"created": 1_790_000_000,
"model": "vision-medium",
"choices": [
{
"index": 0,
"message": {"role": "assistant", "content": "Hello from Vispark"},
"finish_reason": "stop",
}
],
"usage": {"prompt_tokens": 4, "completion_tokens": 3, "total_tokens": 7},
}
_TEXT_COMPLETION: Final = {
"id": "cmpl_vispark",
"object": "text_completion",
"created": 1_790_000_000,
"model": "vision-small",
"choices": [{"index": 0, "text": "Hello from Vispark", "finish_reason": "stop", "logprobs": None}],
"usage": {"prompt_tokens": 4, "completion_tokens": 3, "total_tokens": 7},
}
def test_vispark_provider_resolution(monkeypatch: pytest.MonkeyPatch):
from litellm.litellm_core_utils.get_llm_provider_logic import get_llm_provider
monkeypatch.setenv("VISPARK_LAB_API_KEY", "vispark-env-key")
monkeypatch.delenv("VISPARK_LAB_API_BASE", raising=False)
model, provider, api_key, api_base = get_llm_provider(
model="vispark/vision-medium",
custom_llm_provider=None,
api_base=None,
api_key=None,
)
assert model == "vision-medium"
assert provider == "vispark"
assert api_key == "vispark-env-key"
assert api_base == "https://api.lab.vispark.in/v1"
def test_vispark_provider_keeps_explicit_credentials(monkeypatch: pytest.MonkeyPatch):
from litellm.litellm_core_utils.get_llm_provider_logic import get_llm_provider
monkeypatch.setenv("VISPARK_LAB_API_KEY", "vispark-env-key")
monkeypatch.setenv("VISPARK_LAB_API_BASE", "https://vispark.env.example/v1")
_, provider, api_key, api_base = get_llm_provider(
model="vispark/vision-medium",
custom_llm_provider=None,
api_base="https://vispark.internal.example/v1",
api_key="vispark-explicit-key",
)
assert provider == "vispark"
assert api_key == "vispark-explicit-key"
assert api_base == "https://vispark.internal.example/v1"
def test_vispark_api_base_env_overrides_default(monkeypatch: pytest.MonkeyPatch):
from litellm.litellm_core_utils.get_llm_provider_logic import get_llm_provider
monkeypatch.setenv("VISPARK_LAB_API_KEY", "vispark-env-key")
monkeypatch.setenv("VISPARK_LAB_API_BASE", "https://vispark.env.example/v1")
_, provider, _, api_base = get_llm_provider(
model="vispark/vision-medium",
custom_llm_provider=None,
api_base=None,
api_key=None,
)
assert provider == "vispark"
assert api_base == "https://vispark.env.example/v1"
def test_vispark_api_base_autodetects_provider(monkeypatch: pytest.MonkeyPatch):
from litellm.litellm_core_utils.get_llm_provider_logic import get_llm_provider
monkeypatch.setenv("VISPARK_LAB_API_KEY", "vispark-env-key")
model, provider, api_key, api_base = get_llm_provider(
model="vision-medium",
custom_llm_provider=None,
api_base="https://api.lab.vispark.in/v1",
api_key=None,
)
assert model == "vision-medium"
assert provider == "vispark"
assert api_key == "vispark-env-key"
assert api_base == "https://api.lab.vispark.in/v1"
def test_vispark_chat_completion_request_maps_max_completion_tokens():
with respx.mock() as upstream:
route: Final = upstream.post("https://api.lab.vispark.in/v1/chat/completions").respond(
200, json=_CHAT_COMPLETION
)
response: Final = litellm.completion(
model="vispark/vision-medium",
messages=[{"role": "user", "content": "Say hello"}],
max_completion_tokens=64,
api_key="vispark-test-key",
)
request: Final = route.calls.last.request
body: Final = json.loads(request.content)
assert route.call_count == 1
assert request.headers["authorization"] == "Bearer vispark-test-key"
assert body["model"] == "vision-medium"
assert body["max_tokens"] == 64
assert "max_completion_tokens" not in body
assert response.choices[0].message.content == "Hello from Vispark"
def test_vispark_text_completion_request():
with respx.mock() as upstream:
route: Final = upstream.post("https://api.lab.vispark.in/v1/completions").respond(200, json=_TEXT_COMPLETION)
response: Final = litellm.text_completion(
model="vispark/vision-small",
prompt="Say hello",
api_key="vispark-test-key",
)
body: Final = json.loads(route.calls.last.request.content)
assert route.call_count == 1
assert body["model"] == "vision-small"
assert body["prompt"] == "Say hello"
assert response.choices[0].text == "Hello from Vispark"
def test_vispark_cost_follows_the_cost_map():
cost_entry: Final = litellm.model_cost["vispark/vision-large"]
prompt_cost, completion_cost = litellm.cost_per_token(
model="vision-large",
custom_llm_provider="vispark",
prompt_tokens=1_000,
completion_tokens=500,
)
assert prompt_cost == pytest.approx(1_000 * cost_entry["input_cost_per_token"])
assert completion_cost == pytest.approx(500 * cost_entry["output_cost_per_token"])
def test_vispark_models_are_listed_for_the_provider():
from litellm.utils import get_valid_models
expected: Final = {name for name, entry in litellm.model_cost.items() if entry.get("litellm_provider") == "vispark"}
assert expected
assert set(litellm.models_by_provider["vispark"]) == expected
assert expected <= set(get_valid_models(custom_llm_provider="vispark"))
def test_vispark_endpoint_matrix_matches_between_root_and_backup():
backup_path: Final = Path(litellm.__file__).parent / "provider_endpoints_support_backup.json"
root_path: Final = Path(litellm.__file__).parent.parent / "provider_endpoints_support.json"
backup_entry: Final = json.loads(backup_path.read_text())["providers"]["vispark"]
root_entry: Final = json.loads(root_path.read_text())["providers"]["vispark"]
assert backup_entry == root_entry
assert root_entry["endpoints"]["chat_completions"] is True
@pytest.mark.parametrize("text_completion", (False, True))
def test_vispark_missing_key_rejects_before_openai_fallback(monkeypatch: pytest.MonkeyPatch, text_completion: bool):
monkeypatch.delenv("VISPARK_LAB_API_KEY", raising=False)
monkeypatch.setenv("OPENAI_API_KEY", "unrelated-provider-key")
with respx.mock(assert_all_called=False) as upstream:
route: Final = upstream.post(url__regex=r"https://api\.lab\.vispark\.in/v1/.*").respond(200, json=_CHAT_COMPLETION)
if text_completion:
with pytest.raises(litellm.AuthenticationError, match="VISPARK_LAB_API_KEY"):
litellm.text_completion(model="vispark/vision-small", prompt="Hello")
else:
with pytest.raises(litellm.AuthenticationError, match="VISPARK_LAB_API_KEY"):
litellm.completion(model="vispark/vision-medium", messages=[{"role": "user", "content": "Hello"}])
assert route.call_count == 0
def test_vispark_text_completion_uses_authenticated_client(monkeypatch: pytest.MonkeyPatch):
from openai import OpenAI
monkeypatch.delenv("VISPARK_LAB_API_KEY", raising=False)
monkeypatch.setenv("OPENAI_API_KEY", "unrelated-provider-key")
with OpenAI(api_key="vispark-client-key", base_url="https://api.lab.vispark.in/v1") as client:
with respx.mock() as upstream:
route: Final = upstream.post("https://api.lab.vispark.in/v1/completions").respond(200, json=_TEXT_COMPLETION)
response: Final = litellm.text_completion(model="vispark/vision-small", prompt="Hello", client=client)
assert route.calls.last.request.headers["authorization"] == "Bearer vispark-client-key"
assert response.choices[0].text == "Hello from Vispark"
def test_vispark_chat_completion_uses_authenticated_client(monkeypatch: pytest.MonkeyPatch):
from openai import OpenAI
monkeypatch.delenv("VISPARK_LAB_API_KEY", raising=False)
monkeypatch.setenv("OPENAI_API_KEY", "unrelated-provider-key")
with OpenAI(api_key="vispark-client-key", base_url="https://api.lab.vispark.in/v1") as client:
with respx.mock() as upstream:
route: Final = upstream.post("https://api.lab.vispark.in/v1/chat/completions").respond(200, json=_CHAT_COMPLETION)
response: Final = litellm.completion(
model="vispark/vision-medium", messages=[{"role": "user", "content": "Hello"}], client=client
)
assert route.calls.last.request.headers["authorization"] == "Bearer vispark-client-key"
assert response.choices[0].message.content == "Hello from Vispark"
def test_vispark_http_transport_rejects_unused_sdk_credentials(monkeypatch: pytest.MonkeyPatch):
from openai import OpenAI
monkeypatch.delenv("VISPARK_LAB_API_KEY", raising=False)
monkeypatch.setenv("OPENAI_API_KEY", "unrelated-provider-key")
monkeypatch.setenv("EXPERIMENTAL_OPENAI_BASE_LLM_HTTP_HANDLER", "true")
with OpenAI(api_key="vispark-client-key", base_url="https://api.lab.vispark.in/v1") as client:
with respx.mock(assert_all_called=False) as upstream:
route: Final = upstream.post("https://api.lab.vispark.in/v1/chat/completions").respond(200, json=_CHAT_COMPLETION)
with pytest.raises(litellm.AuthenticationError, match="VISPARK_LAB_API_KEY"):
litellm.completion(model="vispark/vision-medium", messages=[{"role": "user", "content": "Hello"}], client=client)
assert route.call_count == 0