scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-08-28 05:25:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 36

[Feat] New provider TTS - Add AWS polly API for TTS (#18326)

Browse source 
* add aws_polly as new provider

* init AWSPollyTextToSpeechConfig

* test_aws_polly_tts_with_native_voice

* init aws_polly + AWS polly dispatch

* init AWSPollyTextToSpeechConfig

* fix transform

* add aws_polly as a new provider for TTS API

* add to sidebar

* docs aws polly

* code qa fix

* add AWS Polly Text-to-Speech

* add cost tracking for AWS polly

* docs fix
This commit is contained in:
Ishaan Jaff 2025-12-22 18:19:34 +05:30 • committed by GitHub
parent 9f9d7fe7a0
commit 2677d9d30d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
15 changed files with 1065 additions and 15 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

364
docs/my-website/docs/providers/aws_polly.md Normal file
View file

@ -0,0 +1,364 @@
# AWS Polly Text to Speech (tts)
## Overview
| Property | Details |
|-------|-------|
| Description | Convert text to natural-sounding speech using AWS Polly's neural and standard TTS engines |
| Provider Route on LiteLLM | `aws_polly/` |
| Supported Operations | `/audio/speech` |
| Link to Provider Doc | [AWS Polly SynthesizeSpeech ↗](https://docs.aws.amazon.com/polly/latest/dg/API_SynthesizeSpeech.html) |
## Quick Start
### **LiteLLM SDK**
```python showLineNumbers title="SDK Usage"
import litellm
from pathlib import Path
import os
# Set environment variables
os.environ["AWS_ACCESS_KEY_ID"] = ""
os.environ["AWS_SECRET_ACCESS_KEY"] = ""
os.environ["AWS_REGION_NAME"] = "us-east-1"
# AWS Polly call
speech_file_path = Path(__file__).parent / "speech.mp3"
response = litellm.speech(
model="aws_polly/neural",
voice="Joanna",
input="the quick brown fox jumped over the lazy dogs",
)
response.stream_to_file(speech_file_path)
```
### **LiteLLM PROXY**
```yaml showLineNumbers title="proxy_config.yaml"
model_list:
- model_name: polly-neural
litellm_params:
model: aws_polly/neural
aws_access_key_id: "os.environ/AWS_ACCESS_KEY_ID"
aws_secret_access_key: "os.environ/AWS_SECRET_ACCESS_KEY"
aws_region_name: "us-east-1"
```
## Polly Engines
AWS Polly supports different speech synthesis engines. Specify the engine in the model name:
| Model | Engine | Cost (per 1M chars) | Description |
|-------|--------|---------------------|-------------|
| `aws_polly/standard` | Standard | $4.00 | Original Polly voices, faster and lowest cost |
| `aws_polly/neural` | Neural | $16.00 | More natural, human-like speech (recommended) |
| `aws_polly/generative` | Generative | $30.00 | Most expressive, highest quality (limited voices) |
| `aws_polly/long-form` | Long-form | $100.00 | Optimized for long content like articles |
### **LiteLLM SDK**
```python showLineNumbers title="Using Different Engines"
import litellm
# Neural engine (recommended)
response = litellm.speech(
model="aws_polly/neural",
voice="Joanna",
input="Hello world",
)
# Standard engine (lower cost)
response = litellm.speech(
model="aws_polly/standard",
voice="Joanna",
input="Hello world",
)
# Generative engine (highest quality)
response = litellm.speech(
model="aws_polly/generative",
voice="Matthew",
input="Hello world",
)
```
### **LiteLLM PROXY**
```yaml showLineNumbers title="proxy_config.yaml"
model_list:
- model_name: polly-neural
litellm_params:
model: aws_polly/neural
aws_region_name: "us-east-1"
- model_name: polly-standard
litellm_params:
model: aws_polly/standard
aws_region_name: "us-east-1"
- model_name: polly-generative
litellm_params:
model: aws_polly/generative
aws_region_name: "us-east-1"
```
## Available Voices
### Native Polly Voices
AWS Polly has many voices across different languages. Here are popular US English voices:
| Voice | Gender | Engine Support |
|-------|--------|----------------|
| `Joanna` | Female | Neural, Standard |
| `Matthew` | Male | Neural, Standard, Generative |
| `Ivy` | Female (child) | Neural, Standard |
| `Kendra` | Female | Neural, Standard |
| `Amy` | Female (British) | Neural, Standard |
| `Brian` | Male (British) | Neural, Standard |
### **LiteLLM SDK**
```python showLineNumbers title="Using Native Polly Voices"
import litellm
# US English female
response = litellm.speech(
model="aws_polly/neural",
voice="Joanna",
input="Hello from Joanna",
)
# US English male
response = litellm.speech(
model="aws_polly/neural",
voice="Matthew",
input="Hello from Matthew",
)
# British English female
response = litellm.speech(
model="aws_polly/neural",
voice="Amy",
input="Hello from Amy",
)
```
### **LiteLLM PROXY**
```yaml showLineNumbers title="proxy_config.yaml"
model_list:
- model_name: polly-joanna
litellm_params:
model: aws_polly/neural
voice: "Joanna"
aws_region_name: "us-east-1"
- model_name: polly-matthew
litellm_params:
model: aws_polly/neural
voice: "Matthew"
aws_region_name: "us-east-1"
```
### OpenAI Voice Mappings
LiteLLM also supports OpenAI voice names, which are automatically mapped to Polly voices:
| OpenAI Voice | Maps to Polly Voice |
|--------------|---------------------|
| `alloy` | Joanna |
| `echo` | Matthew |
| `fable` | Amy |
| `onyx` | Brian |
| `nova` | Ivy |
| `shimmer` | Kendra |
### **LiteLLM SDK**
```python showLineNumbers title="Using OpenAI Voice Names"
import litellm
# These are equivalent
response = litellm.speech(
model="aws_polly/neural",
voice="alloy", # Maps to Joanna
input="Hello world",
)
response = litellm.speech(
model="aws_polly/neural",
voice="Joanna", # Native Polly voice
input="Hello world",
)
```
## SSML Support
AWS Polly supports SSML (Speech Synthesis Markup Language) for advanced control over speech output. LiteLLM automatically detects SSML input.
### **LiteLLM SDK**
```python showLineNumbers title="SSML Example"
import litellm
ssml_input = """
<speak>
Hello, <break time="500ms"/>
this is a test with <emphasis level="strong">emphasis</emphasis>
and <prosody rate="slow">slower speech</prosody>.
</speak>
"""
response = litellm.speech(
model="aws_polly/neural",
voice="Joanna",
input=ssml_input,
)
```
### **LiteLLM PROXY**
```bash showLineNumbers title="cURL Request with SSML"
curl -X POST http://localhost:4000/v1/audio/speech \
-H "Authorization: Bearer sk-1234" \
-H "Content-Type: application/json" \
-d '{
"model": "polly-neural",
"voice": "Joanna",
"input": "<speak>Hello <break time=\"500ms\"/> world</speak>"
}' \
--output speech.mp3
```
## Supported Parameters
```python showLineNumbers title="All Parameters"
response = litellm.speech(
model="aws_polly/neural",
voice="Joanna", # Required: Voice selection
input="text to convert", # Required: Input text (or SSML)
response_format="mp3", # Optional: mp3, ogg_vorbis, pcm
# AWS-specific parameters
language_code="en-US", # Optional: Language code
sample_rate="22050", # Optional: Sample rate in Hz
)
```
## Response Formats
| Format | Description |
|--------|-------------|
| `mp3` | MP3 audio (default) |
| `ogg_vorbis` | Ogg Vorbis audio |
| `pcm` | Raw PCM audio |
### **LiteLLM SDK**
```python showLineNumbers title="Different Response Formats"
import litellm
# MP3 (default)
response = litellm.speech(
model="aws_polly/neural",
voice="Joanna",
input="Hello",
response_format="mp3",
)
# Ogg Vorbis
response = litellm.speech(
model="aws_polly/neural",
voice="Joanna",
input="Hello",
response_format="ogg_vorbis",
)
```
## AWS Authentication
LiteLLM supports multiple AWS authentication methods.
### **LiteLLM SDK**
```python showLineNumbers title="Authentication Options"
import litellm
import os
# Option 1: Environment variables (recommended)
os.environ["AWS_ACCESS_KEY_ID"] = "your-access-key"
os.environ["AWS_SECRET_ACCESS_KEY"] = "your-secret-key"
os.environ["AWS_REGION_NAME"] = "us-east-1"
response = litellm.speech(model="aws_polly/neural", voice="Joanna", input="Hello")
# Option 2: Pass credentials directly
response = litellm.speech(
model="aws_polly/neural",
voice="Joanna",
input="Hello",
aws_access_key_id="your-access-key",
aws_secret_access_key="your-secret-key",
aws_region_name="us-east-1",
)
# Option 3: IAM Role (when running on AWS)
response = litellm.speech(
model="aws_polly/neural",
voice="Joanna",
input="Hello",
aws_region_name="us-east-1",
)
# Option 4: AWS Profile
response = litellm.speech(
model="aws_polly/neural",
voice="Joanna",
input="Hello",
aws_profile_name="my-profile",
)
```
### **LiteLLM PROXY**
```yaml showLineNumbers title="proxy_config.yaml"
model_list:
# Using environment variables
- model_name: polly-neural
litellm_params:
model: aws_polly/neural
aws_access_key_id: "os.environ/AWS_ACCESS_KEY_ID"
aws_secret_access_key: "os.environ/AWS_SECRET_ACCESS_KEY"
aws_region_name: "us-east-1"
# Using IAM Role (when proxy runs on AWS)
- model_name: polly-neural-iam
litellm_params:
model: aws_polly/neural
aws_region_name: "us-east-1"
# Using AWS Profile
- model_name: polly-neural-profile
litellm_params:
model: aws_polly/neural
aws_profile_name: "my-profile"
```
## Async Support
```python showLineNumbers title="Async Usage"
import litellm
import asyncio
async def main():
response = await litellm.aspeech(
model="aws_polly/neural",
voice="Joanna",
input="Hello from async AWS Polly",
aws_region_name="us-east-1",
)
with open("output.mp3", "wb") as f:
f.write(response.content)
asyncio.run(main())
```

9
docs/my-website/docs/text_to_speech.md
View file

@ -14,7 +14,7 @@ import TabItem from '@theme/TabItem';
| Fallbacks | ✅ | Works between supported models |
| Loadbalancing | ✅ | Works between supported models |
| Guardrails | ✅ | Applies to input text (non-streaming only) |
| Supported Providers | OpenAI, Azure OpenAI, Vertex AI | |
| Supported Providers | OpenAI, Azure OpenAI, Vertex AI, AWS Polly, ElevenLabs | |
## **LiteLLM Python SDK Usage**
### Quick Start
@ -101,6 +101,7 @@ litellm --config /path/to/config.yaml
| OpenAI | [Usage](#quick-start) |
| Azure OpenAI| [Usage](../docs/providers/azure#azure-text-to-speech-tts) |
| Azure AI Speech Service (AVA)| [Usage](../docs/providers/azure_ai_speech) |
| AWS Polly | [Usage](#aws-polly-text-to-speech) |
| Vertex AI | [Usage](../docs/providers/vertex#text-to-speech-apis) |
| Gemini | [Usage](#gemini-text-to-speech) |
| ElevenLabs | [Usage](../docs/providers/elevenlabs#text-to-speech-tts) |
@ -246,6 +247,12 @@ curl http://0.0.0.0:4000/v1/audio/speech \
--output vertex_speech.mp3
```
### AWS Polly Text-to-Speech
AWS Polly provides neural and standard text-to-speech engines with support for multiple voices and languages.
See the [AWS Polly provider documentation](../docs/providers/aws_polly) for detailed usage examples.
## ✨ Enterprise LiteLLM Proxy - Set Max Request File Size
Use this when you want to limit the file size for requests sent to `audio/transcriptions`

1
docs/my-website/sidebars.js
View file

@ -664,6 +664,7 @@ const sidebars = {
"providers/bedrock_agents",
"providers/bedrock_writer",
"providers/bedrock_batches",
"providers/aws_polly",
"providers/bedrock_vector_store",
]
},

0
litellm/llms/aws_polly/__init__.py Normal file
View file

0
litellm/llms/aws_polly/text_to_speech/__init__.py Normal file
View file

391
litellm/llms/aws_polly/text_to_speech/transformation.py Normal file
View file

@ -0,0 +1,391 @@
"""
AWS Polly Text-to-Speech transformation
Maps OpenAI TTS spec to AWS Polly SynthesizeSpeech API
Reference: https://docs.aws.amazon.com/polly/latest/dg/API_SynthesizeSpeech.html
"""
import json
from typing import TYPE_CHECKING, Any, Coroutine, Dict, Optional, Tuple, Union
import httpx
from litellm.llms.base_llm.text_to_speech.transformation import (
BaseTextToSpeechConfig,
TextToSpeechRequestData,
)
from litellm.llms.bedrock.base_aws_llm import BaseAWSLLM
if TYPE_CHECKING:
from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj
from litellm.types.llms.openai import HttpxBinaryResponseContent
else:
LiteLLMLoggingObj = Any
HttpxBinaryResponseContent = Any
class AWSPollyTextToSpeechConfig(BaseTextToSpeechConfig, BaseAWSLLM):
"""
Configuration for AWS Polly Text-to-Speech
Reference: https://docs.aws.amazon.com/polly/latest/dg/API_SynthesizeSpeech.html
"""
def __init__(self):
BaseTextToSpeechConfig.__init__(self)
BaseAWSLLM.__init__(self)
# Default settings
DEFAULT_VOICE = "Joanna"
DEFAULT_ENGINE = "neural"
DEFAULT_OUTPUT_FORMAT = "mp3"
DEFAULT_REGION = "us-east-1"
# Voice name mappings from OpenAI voices to Polly voices
VOICE_MAPPINGS = {
"alloy": "Joanna", # US English female
"echo": "Matthew", # US English male
"fable": "Amy", # British English female
"onyx": "Brian", # British English male
"nova": "Ivy", # US English female (child)
"shimmer": "Kendra", # US English female
}
# Response format mappings from OpenAI to Polly
FORMAT_MAPPINGS = {
"mp3": "mp3",
"opus": "ogg_vorbis",
"aac": "mp3", # Polly doesn't support AAC, use MP3
"flac": "mp3", # Polly doesn't support FLAC, use MP3
"wav": "pcm",
"pcm": "pcm",
}
# Valid Polly engines
VALID_ENGINES = {"standard", "neural", "long-form", "generative"}
def dispatch_text_to_speech(
self,
model: str,
input: str,
voice: Optional[Union[str, Dict]],
optional_params: Dict,
litellm_params_dict: Dict,
logging_obj: "LiteLLMLoggingObj",
timeout: Union[float, httpx.Timeout],
extra_headers: Optional[Dict[str, Any]],
base_llm_http_handler: Any,
aspeech: bool,
api_base: Optional[str],
api_key: Optional[str],
**kwargs: Any,
) -> Union[
"HttpxBinaryResponseContent",
Coroutine[Any, Any, "HttpxBinaryResponseContent"],
]:
"""
Dispatch method to handle AWS Polly TTS requests
This method encapsulates AWS-specific credential resolution and parameter handling
Args:
base_llm_http_handler: The BaseLLMHTTPHandler instance from main.py
"""
# Get AWS region from kwargs or environment
aws_region_name = kwargs.get("aws_region_name") or self._get_aws_region_name_for_polly(
optional_params=optional_params
)
# Convert voice to string if it's a dict
voice_str: Optional[str] = None
if isinstance(voice, str):
voice_str = voice
elif isinstance(voice, dict):
voice_str = voice.get("name") if voice else None
# Update litellm_params with resolved values
# Note: AWS credentials (aws_access_key_id, aws_secret_access_key, etc.)
# are already in litellm_params_dict via get_litellm_params() in main.py
litellm_params_dict["aws_region_name"] = aws_region_name
litellm_params_dict["api_base"] = api_base
litellm_params_dict["api_key"] = api_key
# Call the text_to_speech_handler
response = base_llm_http_handler.text_to_speech_handler(
model=model,
input=input,
voice=voice_str,
text_to_speech_provider_config=self,
text_to_speech_optional_params=optional_params,
custom_llm_provider="aws_polly",
litellm_params=litellm_params_dict,
logging_obj=logging_obj,
timeout=timeout,
extra_headers=extra_headers,
client=None,
_is_async=aspeech,
)
return response
def _get_aws_region_name_for_polly(self, optional_params: Dict) -> str:
"""Get AWS region name for Polly API calls."""
aws_region_name = optional_params.get("aws_region_name")
if aws_region_name is None:
aws_region_name = self.get_aws_region_name_for_non_llm_api_calls()
return aws_region_name
def get_supported_openai_params(self, model: str) -> list:
"""
AWS Polly TTS supports these OpenAI parameters
"""
return ["voice", "response_format", "speed"]
def map_openai_params(
self,
model: str,
optional_params: Dict,
voice: Optional[Union[str, Dict]] = None,
drop_params: bool = False,
kwargs: Dict = {},
) -> Tuple[Optional[str], Dict]:
"""
Map OpenAI parameters to AWS Polly parameters
"""
mapped_params = {}
# Map voice - support both native Polly voices and OpenAI voice mappings
mapped_voice: Optional[str] = None
if isinstance(voice, str):
if voice in self.VOICE_MAPPINGS:
# OpenAI voice -> Polly voice
mapped_voice = self.VOICE_MAPPINGS[voice]
else:
# Assume it's already a Polly voice name
mapped_voice = voice
# Map response format
if "response_format" in optional_params:
format_name = optional_params["response_format"]
if format_name in self.FORMAT_MAPPINGS:
mapped_params["output_format"] = self.FORMAT_MAPPINGS[format_name]
else:
mapped_params["output_format"] = format_name
else:
mapped_params["output_format"] = self.DEFAULT_OUTPUT_FORMAT
# Extract engine from model name (e.g., "aws_polly/neural" -> "neural")
engine = self._extract_engine_from_model(model)
mapped_params["engine"] = engine
# Pass through Polly-specific parameters (use AWS API casing)
if "language_code" in kwargs:
mapped_params["LanguageCode"] = kwargs["language_code"]
if "lexicon_names" in kwargs:
mapped_params["LexiconNames"] = kwargs["lexicon_names"]
if "sample_rate" in kwargs:
mapped_params["SampleRate"] = kwargs["sample_rate"]
return mapped_voice, mapped_params
def _extract_engine_from_model(self, model: str) -> str:
"""
Extract engine from model name.
Examples:
- aws_polly/neural -> neural
- aws_polly/standard -> standard
- aws_polly/long-form -> long-form
- aws_polly -> neural (default)
"""
if "/" in model:
parts = model.split("/")
if len(parts) >= 2:
engine = parts[1].lower()
if engine in self.VALID_ENGINES:
return engine
return self.DEFAULT_ENGINE
def validate_environment(
self,
headers: dict,
model: str,
api_key: Optional[str] = None,
api_base: Optional[str] = None,
) -> dict:
"""
Validate AWS environment and set up headers.
AWS SigV4 signing will be done in transform_text_to_speech_request.
"""
validated_headers = headers.copy()
validated_headers["Content-Type"] = "application/json"
return validated_headers
def get_complete_url(
self,
model: str,
api_base: Optional[str],
litellm_params: dict,
) -> str:
"""
Get the complete URL for AWS Polly SynthesizeSpeech request
Polly endpoint format:
https://polly.{region}.amazonaws.com/v1/speech
"""
if api_base is not None:
return api_base.rstrip("/") + "/v1/speech"
aws_region_name = litellm_params.get("aws_region_name", self.DEFAULT_REGION)
return f"https://polly.{aws_region_name}.amazonaws.com/v1/speech"
def is_ssml_input(self, input: str) -> bool:
"""
Returns True if input is SSML, False otherwise.
Based on AWS Polly SSML requirements - must contain <speak> tag.
"""
return "<speak>" in input or "<speak " in input
def _sign_polly_request(
self,
request_body: Dict[str, Any],
endpoint_url: str,
litellm_params: Dict,
) -> Tuple[Dict[str, str], str]:
"""
Sign the AWS Polly request using SigV4.
Returns:
Tuple of (signed_headers, json_body_string)
"""
try:
from botocore.auth import SigV4Auth
from botocore.awsrequest import AWSRequest
except ImportError:
raise ImportError("Missing boto3 to call AWS Polly. Run 'pip install boto3'.")
# Get AWS region
aws_region_name = litellm_params.get("aws_region_name", self.DEFAULT_REGION)
# Get AWS credentials
credentials = self.get_credentials(
aws_access_key_id=litellm_params.get("aws_access_key_id"),
aws_secret_access_key=litellm_params.get("aws_secret_access_key"),
aws_session_token=litellm_params.get("aws_session_token"),
aws_region_name=aws_region_name,
aws_session_name=litellm_params.get("aws_session_name"),
aws_profile_name=litellm_params.get("aws_profile_name"),
aws_role_name=litellm_params.get("aws_role_name"),
aws_web_identity_token=litellm_params.get("aws_web_identity_token"),
aws_sts_endpoint=litellm_params.get("aws_sts_endpoint"),
aws_external_id=litellm_params.get("aws_external_id"),
)
# Serialize request body to JSON
json_body = json.dumps(request_body)
# Create headers for signing
headers = {
"Content-Type": "application/json",
}
# Create AWS request for signing
aws_request = AWSRequest(
method="POST",
url=endpoint_url,
data=json_body,
headers=headers,
)
# Sign the request
SigV4Auth(credentials, "polly", aws_region_name).add_auth(aws_request)
# Return signed headers and body
return dict(aws_request.headers), json_body
def transform_text_to_speech_request(
self,
model: str,
input: str,
voice: Optional[str],
optional_params: Dict,
litellm_params: Dict,
headers: dict,
) -> TextToSpeechRequestData:
"""
Transform OpenAI TTS request to AWS Polly SynthesizeSpeech format.
Supports:
- Native Polly voices (Joanna, Matthew, etc.)
- OpenAI voice mapping (alloy, echo, etc.)
- SSML input (auto-detected via <speak> tag)
- Multiple engines (neural, standard, long-form, generative)
Returns:
TextToSpeechRequestData: Contains signed request for Polly API
"""
# Get voice (already mapped in main.py, or use default)
polly_voice = voice or self.DEFAULT_VOICE
# Get output format
output_format = optional_params.get("output_format", self.DEFAULT_OUTPUT_FORMAT)
# Get engine
engine = optional_params.get("engine", self.DEFAULT_ENGINE)
# Build request body
request_body: Dict[str, Any] = {
"Engine": engine,
"OutputFormat": output_format,
"Text": input,
"VoiceId": polly_voice,
}
# Auto-detect SSML
if self.is_ssml_input(input):
request_body["TextType"] = "ssml"
else:
request_body["TextType"] = "text"
# Add optional Polly parameters (already in AWS casing from map_openai_params)
for key in ["LanguageCode", "LexiconNames", "SampleRate"]:
if key in optional_params:
request_body[key] = optional_params[key]
# Get endpoint URL
endpoint_url = self.get_complete_url(
model=model,
api_base=litellm_params.get("api_base"),
litellm_params=litellm_params,
)
# Sign the request with AWS SigV4
signed_headers, json_body = self._sign_polly_request(
request_body=request_body,
endpoint_url=endpoint_url,
litellm_params=litellm_params,
)
# Return as ssml_body so the handler uses data= instead of json=
# This preserves the exact JSON string that was signed
return TextToSpeechRequestData(
ssml_body=json_body,
headers=signed_headers,
)
def transform_text_to_speech_response(
self,
model: str,
raw_response: httpx.Response,
logging_obj: "LiteLLMLoggingObj",
) -> "HttpxBinaryResponseContent":
"""
Transform AWS Polly response to standard format.
Polly returns the audio data directly in the response body.
"""
from litellm.types.llms.openai import HttpxBinaryResponseContent
return HttpxBinaryResponseContent(raw_response)

58
litellm/main.py
View file

@ -105,10 +105,22 @@ from litellm.llms.vertex_ai.common_utils import (
from litellm.realtime_api.main import _realtime_health_check
from litellm.secret_managers.main import get_secret_bool, get_secret_str
from litellm.types.router import GenericLiteLLMParams
from litellm.types.utils import RawRequestTypedDict, StreamingChoices
from litellm.types.utils import (
ModelResponseStream,
RawRequestTypedDict,
StreamingChoices,
)
from litellm.utils import (
Choices,
CustomStreamWrapper,
EmbeddingResponse,
Message,
ModelResponse,
ProviderConfigManager,
TextChoices,
TextCompletionResponse,
TextCompletionStreamWrapper,
TranscriptionResponse,
Usage,
_get_model_info_helper,
add_provider_specific_params_to_optional_params,
@ -166,8 +178,8 @@ from .llms.azure_ai.anthropic.handler import AzureAnthropicChatCompletion
from .llms.azure_ai.embed import AzureAIEmbedding
from .llms.bedrock.chat import BedrockConverseLLM, BedrockLLM
from .llms.bedrock.embed.embedding import BedrockEmbedding
from .llms.bedrock.image_generation.image_handler import BedrockImageGeneration
from .llms.bedrock.image_edit.handler import BedrockImageEdit
from .llms.bedrock.image_generation.image_handler import BedrockImageGeneration
from .llms.bytez.chat.transformation import BytezChatConfig
from .llms.clarifai.chat.transformation import ClarifaiConfig
from .llms.codestral.completion.handler import CodestralTextCompletion
@ -240,18 +252,6 @@ from .types.utils import (
all_litellm_params,
)
from litellm.types.utils import ModelResponseStream
from litellm.utils import (
Choices,
EmbeddingResponse,
Message,
ModelResponse,
TextChoices,
TextCompletionResponse,
TextCompletionStreamWrapper,
TranscriptionResponse,
)
####### ENVIRONMENT VARIABLES ###################
openai_chat_completions = OpenAIChatCompletion()
openai_text_completions = OpenAITextCompletion()
@ -6471,6 +6471,35 @@ def speech( # noqa: PLR0915
api_key=api_key,
**kwargs,
)
elif custom_llm_provider == "aws_polly":
from litellm.llms.aws_polly.text_to_speech.transformation import (
AWSPollyTextToSpeechConfig,
)
# AWS Polly Text-to-Speech
if text_to_speech_provider_config is None:
text_to_speech_provider_config = AWSPollyTextToSpeechConfig()
# Cast to specific AWS Polly config type to access dispatch method
aws_polly_config = cast(
AWSPollyTextToSpeechConfig, text_to_speech_provider_config
)
response = aws_polly_config.dispatch_text_to_speech(
model=model,
input=input,
voice=voice,
optional_params=optional_params,
litellm_params_dict=litellm_params_dict,
logging_obj=logging_obj,
timeout=timeout,
extra_headers=extra_headers,
base_llm_http_handler=base_llm_http_handler,
aspeech=aspeech or False,
api_base=api_base,
api_key=api_key,
**kwargs,
)
if response is None:
raise Exception(
@ -6905,6 +6934,7 @@ def _get_encoding():
global _encoding_cache
if _encoding_cache is None:
import sys
# Access via module to trigger __getattr__ if not cached
_encoding_cache = sys.modules[__name__].encoding
return _encoding_cache

36
litellm/model_prices_and_context_window_backup.json
View file

@ -25423,6 +25423,42 @@
"/v1/audio/speech"
]
},
"aws_polly/standard": {
"input_cost_per_character": 4e-06,
"litellm_provider": "aws_polly",
"mode": "audio_speech",
"supported_endpoints": [
"/v1/audio/speech"
],
"source": "https://aws.amazon.com/polly/pricing/"
},
"aws_polly/neural": {
"input_cost_per_character": 1.6e-05,
"litellm_provider": "aws_polly",
"mode": "audio_speech",
"supported_endpoints": [
"/v1/audio/speech"
],
"source": "https://aws.amazon.com/polly/pricing/"
},
"aws_polly/long-form": {
"input_cost_per_character": 1e-04,
"litellm_provider": "aws_polly",
"mode": "audio_speech",
"supported_endpoints": [
"/v1/audio/speech"
],
"source": "https://aws.amazon.com/polly/pricing/"
},
"aws_polly/generative": {
"input_cost_per_character": 3e-05,
"litellm_provider": "aws_polly",
"mode": "audio_speech",
"supported_endpoints": [
"/v1/audio/speech"
],
"source": "https://aws.amazon.com/polly/pricing/"
},
"us.amazon.nova-lite-v1:0": {
"input_cost_per_token": 6e-08,
"litellm_provider": "bedrock_converse",

1
litellm/types/utils.py
View file

@ -2915,6 +2915,7 @@ class LlmProviders(str, Enum):
BYTEZ = "bytez"
REPLICATE = "replicate"
RUNWAYML = "runwayml"
AWS_POLLY = "aws_polly"
HUGGINGFACE = "huggingface"
TOGETHER_AI = "together_ai"
OPENROUTER = "openrouter"

6
litellm/utils.py
View file

@ -8096,6 +8096,12 @@ class ProviderConfigManager:
)
return VertexAITextToSpeechConfig()
elif litellm.LlmProviders.AWS_POLLY == provider:
from litellm.llms.aws_polly.text_to_speech.transformation import (
AWSPollyTextToSpeechConfig,
)
return AWSPollyTextToSpeechConfig()
return None
@staticmethod

36
model_prices_and_context_window.json
View file

@ -25423,6 +25423,42 @@
"/v1/audio/speech"
]
},
"aws_polly/standard": {
"input_cost_per_character": 4e-06,
"litellm_provider": "aws_polly",
"mode": "audio_speech",
"supported_endpoints": [
"/v1/audio/speech"
],
"source": "https://aws.amazon.com/polly/pricing/"
},
"aws_polly/neural": {
"input_cost_per_character": 1.6e-05,
"litellm_provider": "aws_polly",
"mode": "audio_speech",
"supported_endpoints": [
"/v1/audio/speech"
],
"source": "https://aws.amazon.com/polly/pricing/"
},
"aws_polly/long-form": {
"input_cost_per_character": 1e-04,
"litellm_provider": "aws_polly",
"mode": "audio_speech",
"supported_endpoints": [
"/v1/audio/speech"
],
"source": "https://aws.amazon.com/polly/pricing/"
},
"aws_polly/generative": {
"input_cost_per_character": 3e-05,
"litellm_provider": "aws_polly",
"mode": "audio_speech",
"supported_endpoints": [
"/v1/audio/speech"
],
"source": "https://aws.amazon.com/polly/pricing/"
},
"us.amazon.nova-lite-v1:0": {
"input_cost_per_token": 6e-08,
"litellm_provider": "bedrock_converse",

16
provider_endpoints_support.json
View file

@ -205,6 +205,22 @@
"a2a": true
}
},
"aws_polly": {
"display_name": "AWS - Polly (`aws_polly`)",
"url": "https://docs.litellm.ai/docs/providers/aws_polly",
"endpoints": {
"chat_completions": false,
"messages": false,
"responses": false,
"embeddings": false,
"image_generations": false,
"audio_transcriptions": false,
"audio_speech": true,
"moderations": false,
"batches": false,
"rerank": false
}
},
"azure": {
"display_name": "Azure (`azure`)",
"url": "https://docs.litellm.ai/docs/providers/azure",

BIN
tests/audio_tests/aws_polly_speech.mp3 Normal file
View file

Binary file not shown.

BIN
tests/audio_tests/aws_polly_speech_generative.mp3 Normal file
View file

Binary file not shown.

162
tests/audio_tests/test_audio_speech.py
View file

@ -522,3 +522,165 @@ async def test_azure_ava_tts_fable_voice_mapping():
assert "Testing voice mapping" in ssml_body
assert "<speak" in ssml_body
assert "<voice" in ssml_body
@pytest.mark.asyncio
async def test_aws_polly_tts_with_native_voice():
"""
Test AWS Polly TTS with a native Polly voice (Joanna).
Verifies the request is formatted correctly for the Polly API.
"""
import json
from unittest.mock import MagicMock, patch
import httpx
# Mock response - Polly returns audio bytes directly
mock_response_content = b"fake_audio_data"
mock_httpx_response = MagicMock(spec=httpx.Response)
mock_httpx_response.content = mock_response_content
mock_httpx_response.status_code = 200
mock_httpx_response.headers = {"content-type": "audio/mpeg"}
with patch("litellm.llms.custom_httpx.http_handler.AsyncHTTPHandler.post") as mock_post:
mock_post.return_value = mock_httpx_response
response = await litellm.aspeech(
model="aws_polly/neural",
voice="Joanna",
input="Hello, this is a test of AWS Polly",
aws_region_name="us-east-1",
)
# Verify the mock was called
assert mock_post.called
# Get the call arguments - AWS Polly uses data= with JSON string (for SigV4 signing)
call_args = mock_post.call_args
request_data = call_args.kwargs.get("data")
# Parse the JSON body
assert request_data is not None
request_body = json.loads(request_data)
# Verify the request body is formatted correctly for Polly
assert request_body["VoiceId"] == "Joanna"
assert request_body["Text"] == "Hello, this is a test of AWS Polly"
assert request_body["OutputFormat"] == "mp3"
assert request_body["Engine"] == "neural"
assert request_body.get("TextType", "text") == "text"
@pytest.mark.asyncio
async def test_aws_polly_tts_with_openai_voice_mapping():
"""
Test AWS Polly TTS with OpenAI voice mapping (alloy -> Joanna).
Verifies that OpenAI voices are correctly mapped to Polly voices.
"""
import json
from unittest.mock import MagicMock, patch
import httpx
mock_response_content = b"fake_audio_data"
mock_httpx_response = MagicMock(spec=httpx.Response)
mock_httpx_response.content = mock_response_content
mock_httpx_response.status_code = 200
mock_httpx_response.headers = {"content-type": "audio/mpeg"}
with patch("litellm.llms.custom_httpx.http_handler.AsyncHTTPHandler.post") as mock_post:
mock_post.return_value = mock_httpx_response
response = await litellm.aspeech(
model="aws_polly/neural",
voice="alloy",
input="Testing OpenAI voice mapping",
aws_region_name="us-east-1",
)
assert mock_post.called
call_args = mock_post.call_args
request_data = call_args.kwargs.get("data")
# Parse the JSON body
assert request_data is not None
request_body = json.loads(request_data)
# Verify alloy was mapped to Joanna
assert request_body["VoiceId"] == "Joanna"
assert request_body["Text"] == "Testing OpenAI voice mapping"
@pytest.mark.asyncio
async def test_aws_polly_tts_with_ssml():
"""
Test AWS Polly TTS with SSML input.
Verifies that SSML is detected and TextType is set correctly.
"""
import json
from unittest.mock import MagicMock, patch
import httpx
mock_response_content = b"fake_audio_data"
mock_httpx_response = MagicMock(spec=httpx.Response)
mock_httpx_response.content = mock_response_content
mock_httpx_response.status_code = 200
mock_httpx_response.headers = {"content-type": "audio/mpeg"}
ssml_input = '<speak>Hello, <break time="500ms"/> this is SSML.</speak>'
with patch("litellm.llms.custom_httpx.http_handler.AsyncHTTPHandler.post") as mock_post:
mock_post.return_value = mock_httpx_response
response = await litellm.aspeech(
model="aws_polly/neural",
voice="Joanna",
input=ssml_input,
aws_region_name="us-east-1",
)
assert mock_post.called
call_args = mock_post.call_args
request_data = call_args.kwargs.get("data")
# Parse the JSON body
assert request_data is not None
request_body = json.loads(request_data)
# Verify SSML is detected and TextType is set to ssml
assert request_body["Text"] == ssml_input
assert request_body["TextType"] == "ssml"
assert request_body["VoiceId"] == "Joanna"
@pytest.mark.asyncio
async def test_aws_polly_tts_real_api():
"""
Test AWS Polly TTS with real API request.
Requires AWS credentials to be configured.
"""
speech_file_path = Path(__file__).parent / "aws_polly_speech_generative.mp3"
response = await litellm.aspeech(
model="aws_polly/generative",
voice="Joanna",
input="Hello, this is a test of AWS Polly text to speech integration with LiteLLM.",
aws_region_name="us-east-1",
)
from litellm.types.llms.openai import HttpxBinaryResponseContent
assert isinstance(response, HttpxBinaryResponseContent)
binary_content = response.content
assert len(binary_content) > 0
# MP3 files start with ID3 tag or MPEG sync word
assert binary_content[:3] == b"ID3" or binary_content[:2] == b"\xff\xfb" or binary_content[:2] == b"\xff\xf3"
response.stream_to_file(speech_file_path)
assert speech_file_path.exists()
assert speech_file_path.stat().st_size > 0
print(f"AWS Polly TTS audio saved to: {speech_file_path}")