Fix metric name inconsistency for litellm_remaining_requests_metric
and litellm_remaining_tokens_metric. The factory received names
without the _metric suffix, causing _is_metric_enabled to fail when
users configured these metrics in prometheus_metrics_config.
Fixes#18221
Signed-off-by: majiayu000 <1835304752@qq.com>
Fix Ollama_chatException "illegal base64 data at input byte 4" error
when using images with ollama_chat provider. Ollama expects pure base64
data, not the full data URL format (data:image/png;base64,...).
Fixes#18338
Signed-off-by: majiayu000 <1835304752@qq.com>
Instead of limiting grpcio < 1.68.0, specifically exclude the versions
affected by the reconnect bug, and allow installation with either
older or newer versions.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Add support for Z.AI GLM-4.7, latest flagship model with enhanced reasoning capabilities.
Changes:
- Add zai/glm-4.7 to model pricing with /bin/bash.60/M input, .20/M output
- Add cached input pricing (/bin/bash.11/M) for GLM-4.7
- Add supports_reasoning flag to enable thinking parameter
- Update ZAIChatConfig to support thinking parameter for models with reasoning
- Update documentation with GLM-4.7 as latest flagship model
- Add cached input column to pricing table (GLM-4.7 only)
- Add tests for GLM-4.7 reasoning support and cost calculation
- Update all examples to use GLM-4.7
Model specifications:
- Context: 200K input, 128K output
- Supports: reasoning, function calling, tool choice, prompt caching
- Pricing: Same as GLM-4.6 with cache support
See: https://docs.z.ai/guides/llm/glm-4.7
Add the output_text convenience property to ResponsesAPIResponse that
aggregates all output_text items from the output list, matching the
OpenAI SDK's Response.output_text behavior.
The property iterates through output items, collects text content from
message-type outputs, and returns them concatenated into a single
string. Returns empty string if no output_text content exists.
Handles both dict and Pydantic model access patterns for compatibility
with different output formats.
Fixes#18470🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: yurekami <yurekami@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Some OpenAI-compatible providers (e.g., Apertis) return empty error
objects even on successful responses. The previous check only verified
that error was not None, causing spurious APIErrors.
Now the code checks if the error object contains meaningful data:
- For dict errors: non-empty message OR non-null code
- For string errors: non-empty string
- Other truthy values are still treated as errors
Fixes#18407🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: yurekami <yurekami@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Fixes#18599
When OpenAI models (gpt-5-nano, o1-*, o3-*) and other providers return
reasoning_tokens in completion_tokens_details but don't provide text_tokens,
LiteLLM was incorrectly calculating costs using only reasoning_tokens,
ignoring the remaining completion tokens.
Changes:
- Modified generic_cost_per_token() in llm_cost_calc/utils.py to calculate
text_tokens as: completion_tokens - reasoning_tokens - audio_tokens - image_tokens
when text_tokens is not explicitly provided
- Added comprehensive test case test_reasoning_tokens_without_text_tokens_gpt5_nano()
to verify all completion_tokens are billed correctly
Example:
- completion_tokens: 977
- reasoning_tokens: 768
- Before: only 768 tokens billed (21% less)
- After: all 977 tokens billed correctly
Affected models:
- OpenAI: gpt-5-nano, o1-*, o3-*
- Perplexity: sonar-reasoning*
- Any model returning reasoning_tokens without text_tokens
- Lazy load remove_index_from_tool_calls via __getattr__
- Lazy load _service_logger module via __getattr__ in __init__.py
- Lazy load audio_utils.utils with module-level caching in utils.py
- Remove direct imports that trigger heavy module loading
These changes reduce import-time memory usage by deferring imports
until they are actually needed, while maintaining performance through
caching for subsequent accesses.
- /v2/model/info now returns {"data": []} when llm_router is None or model_list is empty
- /model_group/info now returns {"data": []} when llm_model_list is None or empty
- Fixes UI crash on fresh installs with STORE_MODEL_IN_DB=True
- Added 4 unit tests for empty model list scenarios
Adds log_format parameter supporting json_array (default), ndjson, and single formats. NDJSON format enables webhook integrations like Sumo Logic to parse individual log records at ingest time. Defaults to json_array for backward compatibility.
* fix(cost_calculator): correct gpt-image-1 cost calculation using token-based pricing (#13847)
gpt-image-1 uses token-based pricing (like chat models), not pixel-based pricing
like DALL-E. The old code was calculating incorrect costs by treating it as DALL-E.
Changes:
- Update model pricing JSON with correct token-based costs for gpt-image-1
- Add dedicated cost calculator for OpenAI gpt-image models
- Route gpt-image-1 to token-based calculator in cost router
- Add comprehensive tests for the new calculator
* refactor: simplify gpt-image-1 cost calculator using responses API helper
Reuse _transform_response_api_usage_to_chat_usage and generic_cost_per_token
for gpt-image-1 cost calculation since ImageUsage has the same spec as
ResponseAPIUsage.
