mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-18 09:45:23 +00:00
* fix(langfuse_otel): send the completion start time langfuse_otel never set langfuse.observation.completion_start_time, so Langfuse showed no time to first token for any call, streamed or not. The legacy langfuse callback already sends kwargs["completion_start_time"]. Send the same value as an ISO timestamp with an explicit offset. * fix(langfuse_otel): type kwargs and test against a real span Type the kwargs of _set_langfuse_specific_attributes as dict[str, Any], the declared type of model_call_details, so the new completion start time read adds no unknown-type errors. The tests now check a real SDK span instead of patching safe_set_attribute, and the parameter is typed. * fix(otel): send the completion start time from the v2 Langfuse mapper With LITELLM_OTEL_V2 the langfuse_otel spans come from LangfuseMapper, which had no completion start time either. Carry the payload's completionStartTime on LLMCallSpanData for streamed calls, the same calls that get time_to_first_chunk_seconds, and map it to langfuse.observation.completion_start_time. * fix(langfuse_otel): take the logging kwargs as a read-only Mapping The new kwargs annotation tripped LIT001. Both helpers only read the logging kwargs, so type them Mapping[str, Any] instead of dict. * fix(langfuse_otel): time /v1/messages streams from the call start and send the V2 completion start time like the default emitter * fix(langfuse_otel): time Gemini, Interactions and A2A streams from their first chunk and send the start time as JSON The native Gemini, Interactions and A2A streams never recorded a first chunk, so with completion_start_time now sent both emitters showed TTFT equal to latency on them. They now record it, and Gemini and Interactions take the call's start time like /v1/messages. Both emitters send the completion start time as a JSON string, the form the Langfuse SDK and the langfuse callback send, through one serializer. * fix(langfuse_otel): time A2A streams from the call start and record a cached replay's first event A2A streams took their start time when the iterator was built, so their latency left out the work before it. They now use the call's start time like every other route. Cached /v1/messages stream replays recorded no first event, so their time to first token fell back to the end time. The replay now records it on the first event it yields. The tests use the real Interactions config and typed logger hooks. * fix(a2a): time streamed A2A calls from before the agent card fetch and make the cached replay test clock-free * test(a2a): type the card fetch test and stop rebinding the seed helper's recorder * test(otel): expect completion_start_time and cache_hit in the B3 span keys --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| adapters | ||
| __init__.py | ||
| main.py | ||
| Readme.md | ||
| streaming_iterator.py | ||
LiteLLM Google GenAI Interface
Interface to interact with Google GenAI Functions in the native Google interface format.
Overview
This module provides a native interface to Google's Generative AI API, allowing you to use Google's content generation capabilities with both streaming and non-streaming modes, in both synchronous and asynchronous contexts.
Available Functions
Non-Streaming Functions
generate_content()- Synchronous content generationagenerate_content()- Asynchronous content generation
Streaming Functions
generate_content_stream()- Synchronous streaming content generationagenerate_content_stream()- Asynchronous streaming content generation
Usage Examples
Basic Non-Streaming Usage
from litellm.google_genai import generate_content, agenerate_content
from google.genai.types import ContentDict, PartDict
# Synchronous usage
contents = ContentDict(
parts=[
PartDict(text="Hello, can you tell me a short joke?")
],
)
response = generate_content(
contents=contents,
model="gemini-pro", # or your preferred model
# Add other model-specific parameters as needed
)
print(response)
Async Non-Streaming Usage
import asyncio
from litellm.google_genai import agenerate_content
from google.genai.types import ContentDict, PartDict
async def main():
contents = ContentDict(
parts=[
PartDict(text="Hello, can you tell me a short joke?")
],
)
response = await agenerate_content(
contents=contents,
model="gemini-pro",
# Add other model-specific parameters as needed
)
print(response)
# Run the async function
asyncio.run(main())
Streaming Usage
from litellm.google_genai import generate_content_stream
from google.genai.types import ContentDict, PartDict
# Synchronous streaming
contents = ContentDict(
parts=[
PartDict(text="Tell me a story about space exploration")
],
)
for chunk in generate_content_stream(
contents=contents,
model="gemini-pro",
):
print(f"Chunk: {chunk}")
Async Streaming Usage
import asyncio
from litellm.google_genai import agenerate_content_stream
from google.genai.types import ContentDict, PartDict
async def main():
contents = ContentDict(
parts=[
PartDict(text="Tell me a story about space exploration")
],
)
async for chunk in agenerate_content_stream(
contents=contents,
model="gemini-pro",
):
print(f"Async chunk: {chunk}")
asyncio.run(main())
Testing
This module includes comprehensive tests covering:
- Sync and async non-streaming requests
- Sync and async streaming requests
- Response validation
- Error handling scenarios
See tests/unified_google_tests/base_google_test.py for test implementation examples.