mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-16 09:22:22 +00:00
* fix(count_tokens): normalize Google system parts and null tools * fix(count_tokens): preserve typed validator return * fix(count_tokens): validate single system part without typing.cast * fix(count_tokens): accept every Google spelling of a repeated field Google countTokens now reads a repeated field sent as one object, null, or a list with null items the same way as its list form, for system parts, tools, and function declarations, matching what Vertex accepts. * fix(count_tokens): read every Google spelling of contents and skip null system text contents and each content's parts now read one object, null, and null items like their list form, the same as system parts and tools. A one-object parts used to translate to no messages, so non-Google models counted it as empty. A system part whose text is null is skipped, as Vertex AI does. * fix(google_genai): read null-text media system parts as media and one-object content parts on generateContent * fix(google_proxy): count null-text system parts that carry only part metadata, like Vertex AI A system part with null or missing text and only metadata such as thought or thoughtSignature, or an empty part, now counts as empty instead of answering 400. Parts that carry an image or file with null text still answer 400. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| adapters | ||
| __init__.py | ||
| main.py | ||
| Readme.md | ||
| streaming_iterator.py | ||
LiteLLM Google GenAI Interface
Interface to interact with Google GenAI Functions in the native Google interface format.
Overview
This module provides a native interface to Google's Generative AI API, allowing you to use Google's content generation capabilities with both streaming and non-streaming modes, in both synchronous and asynchronous contexts.
Available Functions
Non-Streaming Functions
generate_content()- Synchronous content generationagenerate_content()- Asynchronous content generation
Streaming Functions
generate_content_stream()- Synchronous streaming content generationagenerate_content_stream()- Asynchronous streaming content generation
Usage Examples
Basic Non-Streaming Usage
from litellm.google_genai import generate_content, agenerate_content
from google.genai.types import ContentDict, PartDict
# Synchronous usage
contents = ContentDict(
parts=[
PartDict(text="Hello, can you tell me a short joke?")
],
)
response = generate_content(
contents=contents,
model="gemini-pro", # or your preferred model
# Add other model-specific parameters as needed
)
print(response)
Async Non-Streaming Usage
import asyncio
from litellm.google_genai import agenerate_content
from google.genai.types import ContentDict, PartDict
async def main():
contents = ContentDict(
parts=[
PartDict(text="Hello, can you tell me a short joke?")
],
)
response = await agenerate_content(
contents=contents,
model="gemini-pro",
# Add other model-specific parameters as needed
)
print(response)
# Run the async function
asyncio.run(main())
Streaming Usage
from litellm.google_genai import generate_content_stream
from google.genai.types import ContentDict, PartDict
# Synchronous streaming
contents = ContentDict(
parts=[
PartDict(text="Tell me a story about space exploration")
],
)
for chunk in generate_content_stream(
contents=contents,
model="gemini-pro",
):
print(f"Chunk: {chunk}")
Async Streaming Usage
import asyncio
from litellm.google_genai import agenerate_content_stream
from google.genai.types import ContentDict, PartDict
async def main():
contents = ContentDict(
parts=[
PartDict(text="Tell me a story about space exploration")
],
)
async for chunk in agenerate_content_stream(
contents=contents,
model="gemini-pro",
):
print(f"Async chunk: {chunk}")
asyncio.run(main())
Testing
This module includes comprehensive tests covering:
- Sync and async non-streaming requests
- Sync and async streaming requests
- Response validation
- Error handling scenarios
See tests/unified_google_tests/base_google_test.py for test implementation examples.