This document identifies all external dependencies that could impact LLM
calling logic at runtime and provides mitigation strategies.
Key findings:
- Only ONE critical external dependency: GitHub model cost map
- This is mitigable with LITELLM_LOCAL_MODEL_COST_MAP=True
- Other external calls (HuggingFace, Together AI, etc.) fail gracefully
- No dynamic code loading from external sources
Co-authored-by: ishaan <ishaan@berri.ai>
* Add async_post_call_response_headers_hook to CustomLogger (#20070)
Allow CustomLogger callbacks to inject custom HTTP response headers
into streaming, non-streaming, and failure responses via a new
async_post_call_response_headers_hook method.
* async_post_call_response_headers_hook
---------
Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
* Add /realtime API benchmarks to Benchmarks documentation
- Added new section showing performance improvements for /realtime endpoint
- Included before/after metrics showing 182× faster p99 latency
- Added test setup specifications and key optimizations
- Referenced from v1.80.5-stable release notes
Co-authored-by: ishaan <ishaan@berri.ai>
* Update /realtime benchmarks to show current performance only
- Removed before/after comparison, showing only current metrics
- Clarified that benchmarks are e2e latency against fake realtime endpoint
- Simplified table format for better readability
Co-authored-by: ishaan <ishaan@berri.ai>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
* Add LangSmith mock client support
- Create langsmith_mock_client.py following GCS and Langfuse patterns
- Add mock mode detection via LANGSMITH_MOCK environment variable
- Intercept LangSmith API calls via AsyncHTTPHandler.post patching
- Add verbose logging throughout mock implementation
- Update LangsmithLogger to initialize mock client when mock mode enabled
- Supports configurable mock latency via LANGSMITH_MOCK_LATENCY_MS
* Add Datadog mock client support
- Create datadog_mock_client.py following GCS, Langfuse, and LangSmith patterns
- Add mock mode detection via DATADOG_MOCK environment variable
- Intercept Datadog API calls via AsyncHTTPHandler.post and httpx.Client.post patching
- Add verbose logging throughout mock implementation
- Update DataDogLogger and DataDogLLMObsLogger to initialize mock client when mock mode enabled
- Supports both async and sync logging paths
- Supports configurable mock latency via DATADOG_MOCK_LATENCY_MS
* refactor: consolidate mock client logic into factory pattern
- Create mock_client_factory.py to centralize common mock HTTP client logic
- Refactor GCS, Langfuse, LangSmith, and Datadog mock clients to use factory
- Improve GET/DELETE mock accuracy for GCS (return valid StandardLoggingPayload)
- Fix DELETE mock to return empty body (204 No Content) instead of JSON
- Reduce code duplication across integration mock clients
* feat: add PostHog mock client support
- Create posthog_mock_client.py using factory pattern
- Integrate mock client into PostHogLogger with mock mode detection
- Add verbose logging for mock mode initialization and batch operations
- Enable mock mode via POSTHOG_MOCK environment variable
* Add Helicone mock client support
- Created helicone_mock_client.py using factory pattern (similar to GCS)
- Integrated mock mode detection and initialization in HeliconeLogger
- Mock client patches HTTPHandler.post to intercept Helicone API calls
- Uses factory pattern for should_use_mock and MockResponse utilities
- Custom HTTPHandler.post patching required since HTTPHandler uses self.client.send()
* Add mock support for Braintrust integration and extend mock client factory
- Add braintrust_mock_client.py with mock HTTP client for Braintrust integration testing
- Integrate mock client into BraintrustLogger with mock mode detection
- Refactor Helicone mock client to fully utilize factory's HTTPHandler.post patching
- Extend mock_client_factory to support patching HTTPHandler.post for sync calls
- Enable endpoint-specific mock responses for Braintrust (/project vs /project_logs)
- All mock clients now properly handle both async (AsyncHTTPHandler) and sync (HTTPHandler) calls
* Fix linter errors: remove unused imports and suppress complexity warning
- Remove unused imports from gcs_bucket_mock_client.py (httpx, json, timedelta, Dict, Optional)
- Remove unused Callable import from mock_client_factory.py
- Add noqa comment to suppress PLR0915 complexity warning for create_mock_client_factory function
* Document mock environment variables for PostHog, Helicone, Braintrust, Datadog, and Langsmith integrations
- Add POSTHOG_MOCK and POSTHOG_MOCK_LATENCY_MS documentation
- Add HELICONE_MOCK and HELICONE_MOCK_LATENCY_MS documentation
- Add BRAINTRUST_MOCK and BRAINTRUST_MOCK_LATENCY_MS documentation
- Add DATADOG_MOCK and DATADOG_MOCK_LATENCY_MS documentation
- Add LANGSMITH_MOCK and LANGSMITH_MOCK_LATENCY_MS documentation
All mock env vars follow the same pattern: enable mock mode for integration testing by intercepting API calls and returning mock responses without making actual network calls.
* Fix security issue
* feat: add feature to make silent calls
* add test or silent feat
* add docs for silent feat
* fix lint issues and UI logs
* add docs of ab testing and deep copy
Add support for /embeddings endpoint via Vercel AI Gateway.
Closes#19658
Changes:
- Add VercelAIGatewayEmbeddingConfig in litellm/llms/vercel_ai_gateway/embedding/
- Register provider in utils.py and main.py
- Add unit tests for embedding transformation
- Update documentation with embeddings examples
Usage:
```python
from litellm import embedding
response = embedding(
model="vercel_ai_gateway/openai/text-embedding-3-small",
input="Hello world",
api_key="your-api-key"
)
```
