scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-06 02:48:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103
litellm/docs/my-website/docs/providers
History
Exact
ryanh-ai 374c3458d5
feat: add sagemaker_nova provider for Amazon Nova models on SageMaker (#21542) 
* feat: add sagemaker_nova provider for Nova models on SageMaker

Add support for custom/fine-tuned Amazon Nova models (Nova Micro, Nova Lite,
Nova 2 Lite) deployed on SageMaker Inference real-time endpoints.

Nova uses OpenAI-compatible request/response format with additional
Nova-specific parameters (top_k, reasoning_effort, allowed_token_ids,
truncate_prompt_tokens) and requires stream:true in the request body.
Nova endpoints also reject 'model' in the request body.

Changes:
- New provider: sagemaker_nova/<endpoint-name>
- SagemakerNovaConfig inherits from SagemakerChatConfig
- Override transform_request to strip 'model' from request body
- Override supports_stream_param_in_request_body (True for Nova)
- Extend get_supported_openai_params with Nova-specific params
- Refactored SagemakerChatConfig to use custom_llm_provider param
  instead of hardcoded strings (backwards-compatible)
- Consolidated main.py routing for sagemaker_chat and sagemaker_nova
- 22 unit tests + 9 integration tests (skip-gated)
- Documentation with SDK, streaming, multimodal, and proxy examples
- All tests verified against live SageMaker Nova endpoint

* fix: move integration tests to tests/local_testing/ per test directory policy

* fix: remove unused module-level SagemakerNovaConfig instance

The sagemaker_nova_config singleton was never imported or used — the
ProviderConfigManager creates its own instance via the lambda registered
in utils.py. Removing this leftover boilerplate.

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2026-03-14 15:10:01 -07:00
..
azure added note for using Azure Active Directory Tokens with all the other endpoints (#17733) 2025-12-09 11:51:28 -08:00
azure_ai feat(azure_ai): add router flat cost when response contains actual model 2026-03-06 18:18:06 +05:30
gemini Add Vertex and Gemini Videos API with Cost Tracking + UI support (#16323) 2025-11-08 16:03:51 -08:00
google_ai_studio fix: prompt registry 2026-02-18 00:34:54 +05:30
openai docs(responses): remove unused json import from tool search example 2026-03-10 18:41:54 -03:00
runwayml [Feat] RunwayML - Add support for /audio/speech eleven_multilingual_v2 endpoint (#16604) 2025-11-13 14:32:09 -08:00
vertex_ai Add Vertex and Gemini Videos API with Cost Tracking + UI support (#16323) 2025-11-08 16:03:51 -08:00
watsonx feat: Add IBM watsonx.ai rerank support (#21303) 2026-02-16 20:12:16 -08:00
abliteration.md fix: prompt registry 2026-02-18 00:34:54 +05:30
ai21.md docs: update ai21 docs 2024-09-11 13:35:40 +03:00
aiml.md [Feat] New LLM API - AI/ML API for Image Gen (#13893) 2025-08-23 13:12:44 -07:00
aleph_alpha.md adding vllm to docs 2023-09-06 20:58:47 -07:00
amazon_nova.md docs fix 2025-12-06 16:09:27 -08:00
anthropic.md feat(anthropic): add Files API support for SDK 2026-03-11 12:45:19 -03:00
anthropic_effort.md fix(anthropic): map reasoning_effort to output_config for Claude 4.6 models 2026-02-26 17:18:32 -03:00
anthropic_programmatic_tool_calling.md Update documentation for azure 4 feats 2025-11-26 21:18:47 +05:30
anthropic_tool_input_examples.md Update documentation for azure 4 feats 2025-11-26 21:18:47 +05:30
anthropic_tool_search.md fix: prompt registry 2026-02-18 00:34:54 +05:30
anyscale.md updating anyscale link to latest json present in main branch 2023-12-27 01:41:17 +05:30
apertis.md fix: prompt registry 2026-02-18 00:34:54 +05:30
aws_polly.md fix: prompt registry 2026-02-18 00:34:54 +05:30
aws_sagemaker.md feat: add sagemaker_nova provider for Amazon Nova models on SageMaker (#21542) 2026-03-14 15:10:01 -07:00
azure_ai.md docs(azure_ai.md): document anthropic model usage on azure ai 2025-12-01 14:06:31 -08:00
azure_ai_agents.md fix: prompt registry 2026-02-18 00:34:54 +05:30
azure_ai_img.md fix: prompt registry 2026-02-18 00:34:54 +05:30
azure_ai_img_edit.md add docs 2025-09-23 02:51:19 +00:00
azure_ai_speech.md [Feat] Add SSML Support for Azure Text-to-Speech (AVA) (#16747) 2025-11-17 15:41:22 -08:00
azure_ai_vector_stores.md (feat) Azure AI Vector Stores - support "virtual" indexes + create vector store on passthrough API (#16160) 2025-11-01 12:01:32 -07:00
azure_document_intelligence.md [Feat] Add Azure AI Doc Intelligence OCR (#16219) 2025-11-03 17:22:19 -08:00
azure_ocr.md [Feat] Add Azure AI Doc Intelligence OCR (#16219) 2025-11-03 17:22:19 -08:00
baseten.md Update Baseten LiteLLM integration 2025-08-19 12:21:05 -07:00
bedrock.md fix: prompt registry 2026-02-18 00:34:54 +05:30
bedrock_agentcore.md fix(mcp): add AWS SigV4 auth for Bedrock AgentCore MCP servers (#22782) 2026-03-10 11:11:20 -07:00
bedrock_agents.md docs(bedrock_agents.md): show example of passing provider specific params to bedrock agents 2025-07-02 14:23:07 -07:00
bedrock_batches.md Add support for file content download for bedrock batches 2025-12-04 13:27:53 +05:30
bedrock_embedding.md fix: prompt registry 2026-02-18 00:34:54 +05:30
bedrock_image_gen.md fix(opentelemetry.py): fix issue where headers were not being split correctly + feat(bedrock): add titan image generations w/ cost tracking (#15916) 2025-10-25 13:45:13 -07:00
bedrock_imported.md fix: prompt registry 2026-02-18 00:34:54 +05:30
bedrock_mantle.md docs: add Amazon Bedrock Mantle provider page 2026-03-05 00:10:00 -05:00
bedrock_realtime_with_audio.md fix: prompt registry 2026-02-18 00:34:54 +05:30
bedrock_rerank.md fix(opentelemetry.py): fix issue where headers were not being split correctly + feat(bedrock): add titan image generations w/ cost tracking (#15916) 2025-10-25 13:45:13 -07:00
bedrock_vector_store.md (feat) Milvus - search vector store support + (fix) Passthrough Endpoints - support multi-part form data on passthrough (#16035) 2025-11-01 12:00:29 -07:00
bedrock_writer.md [Feat] New model - add bedrock writer models (#17685) 2025-12-08 17:49:06 -08:00
black_forest_labs.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
black_forest_labs_img_edit.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
bytez.md Integration: Bytez as a model provider (#12121) 2025-07-12 10:50:39 -07:00
cerebras.md Update model path and documentation for cerebras API call (#8862) 2025-02-27 20:23:03 -08:00
chatgpt.md docs(chatgpt): add gpt-5.3-chat-latest proxy example 2026-03-05 16:50:38 -05:00
chutes.md fix: prompt registry 2026-02-18 00:34:54 +05:30
clarifai.md docs fix 2025-10-18 13:36:48 -07:00
cloudflare_workers.md (docs) cloudflare 2023-12-29 12:10:32 +05:30
codestral.md docs - codestral 2024-06-17 16:46:46 -07:00
cohere.md Litellm sameer oct staging (#15806) 2025-10-24 12:17:22 -07:00
cometapi.md feat(cometapi): Add CometAPI provider support (embeddings, image generation, docs) 2025-10-16 13:08:14 +08:00
compactifai.md Fix: MDX compilation error in CompactifAI documentation 2025-09-17 07:50:38 +02:00
custom_llm_server.md fix: prompt registry 2026-02-18 00:34:54 +05:30
dashscope.md fix: prompt registry 2026-02-18 00:34:54 +05:30
databricks.md fix: prompt registry 2026-02-18 00:34:54 +05:30
datarobot.md Apply suggestions from code review 2025-08-28 23:00:09 -06:00
deepgram.md docs(deepgram.md): add table clarifying supported openai endpoint 2024-12-28 20:21:31 -08:00
deepinfra.md [Documentation] Litellm rerank deepinfra endpoint (#13845) 2025-08-22 09:51:42 -07:00
deepseek.md feat(deepseek): add native support for thinking and reasoning_effort params (#17712) 2025-12-11 15:28:43 -08:00
docker_model_runner.md [Feat] New LLM Provider - Docker Model Runner (#16948) 2025-11-21 16:09:32 -08:00
elevenlabs.md fix: prompt registry 2026-02-18 00:34:54 +05:30
empower.md add document 2024-05-31 18:55:22 -07:00
fal_ai.md Vector store files Stable Release (#16643) 2025-11-15 13:00:33 -08:00
featherless_ai.md Add provider Featherless AI (access to 4200+ models) (#10596) 2025-05-17 19:38:46 -07:00
fireworks_ai.md fix: prompt registry 2026-02-18 00:34:54 +05:30
friendliai.md FriendliAI: Documentation Updates (#7517) 2025-01-04 22:44:24 -08:00
galadriel.md fix(key_management_endpoints.py): override metadata field value on up… (#7008) 2024-12-03 23:03:50 -08:00
gemini.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
gemini_file_search.md Add gemini file search support 2025-11-26 11:17:30 +05:30
gigachat.md fix: prompt registry 2026-02-18 00:34:54 +05:30
github.md [Feat] Add FAL AI Image Generations on LiteLLM (#16067) 2025-10-29 13:10:51 -07:00
github_copilot.md fix: prompt registry 2026-02-18 00:34:54 +05:30
gmi.md fix: prompt registry 2026-02-18 00:34:54 +05:30
gradient_ai.md Add digitalocean provider (#12169) 2025-08-09 16:26:33 -07:00
groq.md feat: add groq/openai/gpt-oss-safeguard-20b model pricing (#21951) 2026-02-23 21:03:18 -08:00
helicone.md Add Helicone as a provider and update observability documentation (#17663) 2025-12-08 12:34:11 -08:00
heroku.md Update docs/my-website/docs/providers/heroku.md 2025-08-13 13:18:37 -06:00
huggingface.md fix img URL for tests 2025-11-22 09:41:15 -08:00
huggingface_rerank.md [Docs] v1.72.2.rc (#11519) 2025-06-07 17:20:15 -07:00
hyperbolic.md Revert "Litellm dev 07 21 2025 p1 (#12848)" 2025-07-22 18:28:36 -07:00
infinity.md [Feat] Add infinity embedding support (contributor pr) (#10196) 2025-04-21 20:01:29 -07:00
jina_ai.md docs: add docs on jina ai rerank support 2024-11-15 01:45:57 +05:30
lambda_ai.md Revert "Litellm dev 07 21 2025 p1 (#12848)" 2025-07-22 18:28:36 -07:00
langgraph.md fix: prompt registry 2026-02-18 00:34:54 +05:30
lemonade.md [Fix] Proxy Auth - Ensure LLM_API_KEYs can access pass through routes (#15115) 2025-10-01 14:09:01 -07:00
litellm_proxy.md fix: prompt registry 2026-02-18 00:34:54 +05:30
llamafile.md Add llamafile as a provider (#10203) (#10482) 2025-05-01 18:36:55 -07:00
llamagate.md fix: prompt registry 2026-02-18 00:34:54 +05:30
lm_studio.md Enable structured JSON schema outputs in LM Studio (for validated responses) (#10929) 2025-05-18 20:43:49 -07:00
manus.md fix: prompt registry 2026-02-18 00:34:54 +05:30
meta_llama.md [Feat] Enable Tool Calling for meta_llama (#11895) 2025-06-19 13:44:22 -07:00
milvus_vector_stores.md fix: prompt registry 2026-02-18 00:34:54 +05:30
minimax.md fix: prompt registry 2026-02-18 00:34:54 +05:30
mistral.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
moonshot.md fix(moonshot): preserve image_url blocks in multimodal messages 2026-02-19 16:44:38 -03:00
morph.md Revert "Litellm dev 07 21 2025 p1 (#12848)" 2025-07-22 18:28:36 -07:00
nano-gpt.md fix: prompt registry 2026-02-18 00:34:54 +05:30
nebius.md [Security] Fixes for docs (#11776) 2025-06-16 16:20:55 -07:00
nlp_cloud.md updates 2023-09-29 11:33:58 -07:00
novita.md fix: fix novita ai docs (#10882) 2025-05-16 10:04:55 -07:00
nscale.md Update Nscale model providers to point to website (#10764) 2025-05-12 17:10:39 -07:00
nvidia_nim.md [Feat] Add Nvidia NIM Rerank Support (#15152) 2025-10-02 18:58:52 -07:00
nvidia_nim_rerank.md [Feat] New model - add nvidia nim llama-3.2-nv-rerankqa-1b-v2 (#17670) 2025-12-08 15:25:23 -08:00
oci.md OCI Provider - Add oci_endpoint_id Parameter for OCI Dedicated Endpoints #16723 2025-11-17 19:59:34 -08:00
ollama.md docs: document streaming iterator guardrail 2025-03-22 20:20:36 -07:00
openai.md fix(openai): drop all reasoning_effort for gpt-5.4 + tools, including 'none' 2026-03-12 16:22:40 -03:00
openai_compatible.md [Feat] Allow adding OpenAI compatible chat providers using .json + add public ai provider (#17448) 2025-12-04 17:59:25 -08:00
openrouter.md docs(openrouter): add image edit documentation 2026-03-03 14:50:41 -03:00
ovhcloud.md Add support of audio transcription for OVHcloud (#17305) 2025-12-01 18:26:39 -08:00
perplexity.md feat(perplexity): update Responses API integration to match Agent API 2026-02-23 18:05:03 +00:00
perplexity_embedding.md feat(perplexity): add embedding support for pplx-embed-v1 models (#22610) 2026-03-02 17:37:50 -08:00
petals.md add petals streaming to docs 2023-10-02 09:04:34 -07:00
poe.md fix: prompt registry 2026-02-18 00:34:54 +05:30
predibase.md fix typo in predibase.md 2025-03-22 08:47:33 -04:00
publicai.md [Feature]: Add Provider publicai.co (#17230) 2025-11-27 13:08:03 -08:00
pydantic_ai_agent.md fix: prompt registry 2026-02-18 00:34:54 +05:30
ragflow.md Add ragflow support 2025-12-02 18:18:08 +05:30
ragflow_vector_store.md Add vector store support for ragflow 2025-12-03 15:29:47 +05:30
recraft.md [Feat] Add Recraft API - Image Edits Support (#12874) 2025-07-22 15:03:08 -07:00
replicate.md docs(replicate.md): add proxy examples to docs 2024-04-25 08:46:21 -07:00
sambanova.md Feat/sambanova embeddings (#13308) 2025-08-12 17:15:26 -07:00
sap.md fix: prompt registry 2026-02-18 00:34:54 +05:30
sarvam.md fix: prompt registry 2026-02-18 00:34:54 +05:30
scaleway.md fix: prompt registry 2026-02-18 00:34:54 +05:30
snowflake.md Snowflake provider support: added embeddings, PAT, account_id (#15727) 2025-11-17 20:27:46 -08:00
stability.md fix: prompt registry 2026-02-18 00:34:54 +05:30
synthetic.md fix: prompt registry 2026-02-18 00:34:54 +05:30
text_completion_openai.md docs - fix build time issues 2024-06-15 16:12:37 -07:00
togetherai.md docs add tg ai rerank docs 2024-08-27 18:14:40 -07:00
topaz.md New stable release - release notes (#8148) 2025-01-31 10:02:59 -08:00
triton-inference-server.md docs add 1.55.8 changelog 2024-12-21 20:51:39 -08:00
v0.md feat: add v0 provider support (#12751) 2025-07-18 18:26:44 -07:00
vercel_ai_gateway.md fix: prompt registry 2026-02-18 00:34:54 +05:30
vertex.md feat(vertex_ai): support explicit AWS credentials for WIF auth (#21472) 2026-03-06 09:27:20 -08:00
vertex_ai_agent_engine.md fix: prompt registry 2026-02-18 00:34:54 +05:30
vertex_batch.md docs: use custom-llm-provider header in examples (#16055) 2025-10-29 19:06:48 -07:00
vertex_embedding.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
vertex_image.md Update docs related to vertex ai image gen 2025-11-25 13:31:50 +05:30
vertex_ocr.md fix: prompt registry 2026-02-18 00:34:54 +05:30
vertex_partner.md docs: add VertexAI ZAI (GLM) documentation 2026-03-11 13:18:12 -03:00
vertex_realtime.md feat(vertex_ai): Vertex AI Gemini Live via unified /realtime endpoint (#22153) 2026-02-25 22:11:06 -08:00
vertex_self_deployed.md [Feat] VertexAI Gemma model family streaming support + Added MedGemma (#15427) 2025-10-10 14:22:27 -07:00
vertex_speech.md fix: prompt registry 2026-02-18 00:34:54 +05:30
vllm.md docs(vllm.md): document new endpoint 2025-09-13 09:46:39 -07:00
vllm_batches.md fix: prompt registry 2026-02-18 00:34:54 +05:30
volcano.md Add Volcengine embedding module with handler and transformation logic 2025-08-28 15:05:11 +08:00
voyage.md feat(voyage): add rerank API support (#17744) 2025-12-09 17:34:09 -08:00
wandb_inference.md Add information about personal entities error 2025-10-07 20:10:38 +05:30
xai.md Feat: add support for Grok 4.1 Fast models (#16936) 2025-11-21 15:57:55 -08:00
xai_realtime.md fix: prompt registry 2026-02-18 00:34:54 +05:30
xiaomi_mimo.md fix: prompt registry 2026-02-18 00:34:54 +05:30
xinference.md [Feat] Add XInference Image Generation API Provider (#12439) 2025-07-08 21:17:38 -07:00
zai.md fix: prompt registry 2026-02-18 00:34:54 +05:30