add example using tts on vertex ai

This commit is contained in:
Ishaan Jaff 2024-08-23 16:04:51 -07:00
parent 5c3a1484c3
commit 82c3357454
2 changed files with 99 additions and 0 deletions

View file

@ -1768,6 +1768,98 @@ response = await litellm.aimage_generation(
)
```
## **Text to Speech APIs**
:::info
LiteLLM supports calling [Vertex AI Text to Speech API](https://console.cloud.google.com/vertex-ai/generative/speech/text-to-speech) in the OpenAI text to speech API format
:::
Usage
<Tabs>
<TabItem value="sdk" label="SDK">
Vertex AI does not support passing a `model` param - so passing `model=vertex_ai/` is the only required param
**Sync Usage**
```python
speech_file_path = Path(__file__).parent / "speech_vertex.mp3"
response = litellm.speech(
model="vertex_ai/",
input="hello what llm guardrail do you have",
)
response.stream_to_file(speech_file_path)
```
**Async Usage**
```python
speech_file_path = Path(__file__).parent / "speech_vertex.mp3"
response = litellm.aspeech(
model="vertex_ai/",
input="hello what llm guardrail do you have",
)
response.stream_to_file(speech_file_path)
```
</TabItem>
<TabItem value="proxy" label="LiteLLM PROXY (Unified Endpoint)">
1. Add model to config.yaml
```yaml
model_list:
- model_name: multimodalembedding@001
litellm_params:
model: vertex_ai/multimodalembedding@001
vertex_project: "adroit-crow-413218"
vertex_location: "us-central1"
vertex_credentials: adroit-crow-413218-a956eef1a2a8.json
litellm_settings:
drop_params: True
```
2. Start Proxy
```
$ litellm --config /path/to/config.yaml
```
3. Make Request use OpenAI Python SDK
```python
import openai
client = openai.OpenAI(api_key="sk-1234", base_url="http://0.0.0.0:4000")
# # request sent to model set on litellm proxy, `litellm --model`
response = client.embeddings.create(
model="multimodalembedding@001",
input = None,
extra_body = {
"instances": [
{
"image": {
"bytesBase64Encoded": "base64"
},
"text": "this is a unicorn",
},
],
}
)
print(response)
```
</TabItem>
</Tabs>
## Extra
### Using `GOOGLE_APPLICATION_CREDENTIALS`

View file

@ -78,6 +78,13 @@ litellm --config /path/to/config.yaml
# RUNNING on http://0.0.0.0:4000
```
## **Supported Providers**
| Provider | Link to Usage |
|-------------|--------------------|
| OpenAI | |
| Azure OpenAI| |
| Vertex AI | [Usage](../docs/providers/vertex#text-to-speech-apis) |
## Azure Usage