litellm/litellm/google_genai
zoveki a11e76fe8f
fix(google_genai): read one-object and null spellings of repeated fields in countTokens and generateContent (#48771)
* fix(count_tokens): normalize Google system parts and null tools

* fix(count_tokens): preserve typed validator return

* fix(count_tokens): validate single system part without typing.cast

* fix(count_tokens): accept every Google spelling of a repeated field

Google countTokens now reads a repeated field sent as one object, null, or a
list with null items the same way as its list form, for system parts, tools,
and function declarations, matching what Vertex accepts.

* fix(count_tokens): read every Google spelling of contents and skip null system text

contents and each content's parts now read one object, null, and null items like their list form, the same as system parts and tools. A one-object parts used to translate to no messages, so non-Google models counted it as empty. A system part whose text is null is skipped, as Vertex AI does.

* fix(google_genai): read null-text media system parts as media and one-object content parts on generateContent

* fix(google_proxy): count null-text system parts that carry only part metadata, like Vertex AI

A system part with null or missing text and only metadata such as thought or thoughtSignature, or an empty part, now counts as empty instead of answering 400. Parts that carry an image or file with null text still answer 400.

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-28 21:46:06 -07:00
..
adapters fix(google_genai): read one-object and null spellings of repeated fields in countTokens and generateContent (#48771) 2026-10-28 21:46:06 -07:00
__init__.py refactor(lint): apply every safe ruff autofix and zero 28 strict-rule budgets 2026-08-01 15:43:29 -07:00
main.py fix(vertex_ai): price streamed pass-through calls with the regional uplift (#44795) 2026-10-19 08:10:52 +00:00
Readme.md [Feat] Add Support for calling Gemini/Vertex models in their native format (#12046) 2025-06-25 18:37:03 -07:00
streaming_iterator.py fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00

LiteLLM Google GenAI Interface

Interface to interact with Google GenAI Functions in the native Google interface format.

Overview

This module provides a native interface to Google's Generative AI API, allowing you to use Google's content generation capabilities with both streaming and non-streaming modes, in both synchronous and asynchronous contexts.

Available Functions

Non-Streaming Functions

  • generate_content() - Synchronous content generation
  • agenerate_content() - Asynchronous content generation

Streaming Functions

  • generate_content_stream() - Synchronous streaming content generation
  • agenerate_content_stream() - Asynchronous streaming content generation

Usage Examples

Basic Non-Streaming Usage

from litellm.google_genai import generate_content, agenerate_content
from google.genai.types import ContentDict, PartDict

# Synchronous usage
contents = ContentDict(
    parts=[
        PartDict(text="Hello, can you tell me a short joke?")
    ],
)

response = generate_content(
    contents=contents,
    model="gemini-pro",  # or your preferred model
    # Add other model-specific parameters as needed
)

print(response)

Async Non-Streaming Usage

import asyncio
from litellm.google_genai import agenerate_content
from google.genai.types import ContentDict, PartDict

async def main():
    contents = ContentDict(
        parts=[
            PartDict(text="Hello, can you tell me a short joke?")
        ],
    )
    
    response = await agenerate_content(
        contents=contents,
        model="gemini-pro",
        # Add other model-specific parameters as needed
    )
    
    print(response)

# Run the async function
asyncio.run(main())

Streaming Usage

from litellm.google_genai import generate_content_stream
from google.genai.types import ContentDict, PartDict

# Synchronous streaming
contents = ContentDict(
    parts=[
        PartDict(text="Tell me a story about space exploration")
    ],
)

for chunk in generate_content_stream(
    contents=contents,
    model="gemini-pro",
):
    print(f"Chunk: {chunk}")

Async Streaming Usage

import asyncio
from litellm.google_genai import agenerate_content_stream
from google.genai.types import ContentDict, PartDict

async def main():
    contents = ContentDict(
        parts=[
            PartDict(text="Tell me a story about space exploration")
        ],
    )
    
    async for chunk in agenerate_content_stream(
        contents=contents,
        model="gemini-pro",
    ):
        print(f"Async chunk: {chunk}")

asyncio.run(main())

Testing

This module includes comprehensive tests covering:

  • Sync and async non-streaming requests
  • Sync and async streaming requests
  • Response validation
  • Error handling scenarios

See tests/unified_google_tests/base_google_test.py for test implementation examples.