litellm/litellm/harness/context.py
Mateo Wang d80f8c28ca
refactor(types): replace Any with proven types in 137 files (#44478)
* refactor(types): prove runtime types at harness, search, rag and client boundaries

Replace Any with adapter-validated types in the litellm.agent() harness, the
search provider transformations, RAG ingestion and query, the vector store
pre-call hook and registry, the galileo and opik logging integrations and the
proxy client CLI. Each boundary gets unit tests for well-formed and malformed
payloads.

* chore(typing): prove types at more provider boundaries and restore search transformations

Second pass over a2a, embedding, rerank, image, audio and small provider
modules. Search transformations go back to their previous form because
validating their response bodies would change the proxy status for
malformed upstream bodies from 400 to 500.

* refactor(types): prove types at logging, files, rerank, image, audio and management boundaries

Replace Any with validated or annotation-only types in 37 more files: logging
integrations, token counters, provider files/rerank/image generation/audio
transcription transformations, pass-through logging handlers and management
endpoints. No proxy HTTP status or error type changes.

* refactor(types): prove types at repository, spend, files and router boundaries

Replace Any with repository table accessors, validated mappings and
annotation-only types in 33 more files: Prisma repositories, the enterprise
batch and responses cost checkers, budget reservation, files endpoints,
management endpoints, the policy registry, the adaptive and complexity routers
and the secret managers. No proxy HTTP status or error type changes.

* test(types): run the aiohttp transformation test in-process and cover repository row conversion

The aiohttp chat transformation test no longer starts a server. It feeds the
transformation a response whose json() returns the body under test.

The proxy unit shards now exercise stored model rows whose params are JSON
strings and the object permission create and update paths.
2026-10-05 11:01:45 -07:00

62 lines
2 KiB
Python

"""Per-session state shared by the runtime, handlers and harness configs."""
from __future__ import annotations
from collections.abc import Awaitable, Callable, Mapping, Sequence
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, TypeAlias
from pydantic import BaseModel
from litellm.harness.options import HarnessOptions
from litellm.harness.sandbox.base import Sandbox
from litellm.harness.types import Approval, Harness, PermissionMode
if TYPE_CHECKING:
from litellm.harness.endpoint import ModelEndpoint
ApprovalHandler: TypeAlias = Callable[
[Approval], bool | Awaitable[bool] # mutable-ok: Callable parameter list in a type alias, not a runtime collection
]
@dataclass(frozen=True)
class GatewayTarget:
"""Resolved LiteLLM AI Gateway for `litellm_proxy/` models. Internal, not exported."""
api_base: str
api_key: str
@dataclass
class SessionContext:
"""Everything a handler and config need for a session. Owned by the runtime."""
harness: Harness
sandbox: Sandbox
session_id: str
# Model name as sent to the runtime (litellm_proxy/ prefix already stripped).
model: str | None = None
gateway: GatewayTarget | None = None
api_key: str | None = None
api_base: str | None = None
endpoint: ModelEndpoint | None = None
instructions: str | None = None
tools: Sequence[Callable[..., object]] = ()
skills: Sequence[str] = ()
disable_tools: Sequence[str] = ()
permissions: PermissionMode = "full"
on_approval: ApprovalHandler | None = None
output: type[BaseModel] | None = None
max_turns: int | None = None
timeout: float | None = None
metadata: Mapping[str, object] = field(default_factory=dict)
options: HarnessOptions | None = None
# Set by the handler after each turn.
final_text: str = ""
output_json: str | None = None
# Usage for in-process harnesses that call LiteLLM directly (no model endpoint).
input_tokens: int = 0
output_tokens: int = 0
cost: float = 0.0
calls: int = 0