scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-21 00:21:49 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 74
litellm/docs/my-website/docs
History
ryanh-ai 374c3458d5
feat: add sagemaker_nova provider for Amazon Nova models on SageMaker (#21542) 
* feat: add sagemaker_nova provider for Nova models on SageMaker

Add support for custom/fine-tuned Amazon Nova models (Nova Micro, Nova Lite,
Nova 2 Lite) deployed on SageMaker Inference real-time endpoints.

Nova uses OpenAI-compatible request/response format with additional
Nova-specific parameters (top_k, reasoning_effort, allowed_token_ids,
truncate_prompt_tokens) and requires stream:true in the request body.
Nova endpoints also reject 'model' in the request body.

Changes:
- New provider: sagemaker_nova/<endpoint-name>
- SagemakerNovaConfig inherits from SagemakerChatConfig
- Override transform_request to strip 'model' from request body
- Override supports_stream_param_in_request_body (True for Nova)
- Extend get_supported_openai_params with Nova-specific params
- Refactored SagemakerChatConfig to use custom_llm_provider param
  instead of hardcoded strings (backwards-compatible)
- Consolidated main.py routing for sagemaker_chat and sagemaker_nova
- 22 unit tests + 9 integration tests (skip-gated)
- Documentation with SDK, streaming, multimodal, and proxy examples
- All tests verified against live SageMaker Nova endpoint

* fix: move integration tests to tests/local_testing/ per test directory policy

* fix: remove unused module-level SagemakerNovaConfig instance

The sagemaker_nova_config singleton was never imported or used — the
ProviderConfigManager creates its own instance via the lambda registered
in utils.py. Removing this leftover boilerplate.

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2026-03-14 15:10:01 -07:00
..
adding_provider feat(guardrails): team-based guardrail registration and approval workflow (#22459) 2026-03-02 22:06:49 -08:00
anthropic_unified docs(anthropic): add v1/messages → /responses parameter mapping reference 2026-03-05 15:51:19 +05:30
caching fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
completion merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
contribute_integration docs(community.md): add new integration partner doc 2025-12-09 18:17:14 -08:00
contributing merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
debugging
embedding merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
extras fix: prompt registry 2026-02-18 00:34:54 +05:30
guides fix: prompt registry 2026-02-18 00:34:54 +05:30
integrations fix: prompt registry 2026-02-18 00:34:54 +05:30
langchain added tags to langchain 2025-09-10 00:32:26 -04:00
observability fix: req changes 2026-02-25 22:27:51 +05:30
pass_through Development environment setup (#22432) 2026-02-28 14:50:06 -08:00
projects fix: prompt registry 2026-02-18 00:34:54 +05:30
provider_registration merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
providers feat: add sagemaker_nova provider for Amazon Nova models on SageMaker (#21542) 2026-03-14 15:10:01 -07:00
proxy [Feat] - Ishaan main merge branch (#23596) 2026-03-14 09:40:00 -07:00
search feat(search): add Serper (serper.dev) as search provider (#23112) 2026-03-09 08:40:37 -07:00
secret_managers fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
troubleshoot docs: pip venv upgrade workflow (#23290) 2026-03-10 13:53:54 -07:00
tutorials docs: add Claude Code skills page for litellm-skills (#23642) 2026-03-14 13:24:16 -07:00
vector_stores Add vector store support for ragflow 2025-12-03 15:29:47 +05:30
a2a.md Agents - add max budget + tpm/rpm limiting per agent AND per agent session (#22849) 2026-03-07 19:12:42 -08:00
a2a_agent_headers.md docs(agents): add A2A agent authentication headers guide 2026-03-05 15:11:18 +05:30
a2a_agent_permissions.md [Docs] A2a - Permission management (#17515) 2025-12-04 17:29:47 -08:00
a2a_cost_tracking.md fix: prompt registry 2026-02-18 00:34:54 +05:30
a2a_invoking_agents.md fix: prompt registry 2026-02-18 00:34:54 +05:30
a2a_iteration_budgets.md Agents - add max budget + tpm/rpm limiting per agent AND per agent session (#22849) 2026-03-07 19:12:42 -08:00
aiohttp_benchmarks.md
anthropic_count_tokens.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
apply_guardrail.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
assistants.md docs: improve Getting Started page and SDK documentation structure (#17614) 2025-12-08 13:05:50 -08:00
audio_transcription.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
batches.md fix: prompt registry 2026-02-18 00:34:54 +05:30
bedrock_converse.md docs boto3 instructions 2025-10-16 16:34:56 -07:00
bedrock_invoke.md docs boto3 instructions 2025-10-16 16:34:56 -07:00
benchmarks.md fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
budget_manager.md
contact.md docs: update broken Slack invite links to support page (#16546) 2025-11-12 12:41:55 -08:00
container_files.md fix: prompt registry 2026-02-18 00:34:54 +05:30
containers.md [docs] add docs for containers files api + code interpreter on LiteLLM (#17749) 2025-12-09 18:11:28 -08:00
contributing.md fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
count_tokens.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
data_retention.md fix: prompt registry 2026-02-18 00:34:54 +05:30
data_security.md docs(data_security.md): data_security.md 2025-06-09 17:53:11 -07:00
default_code_snippet.md
enterprise.md fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
evals_api.md fix: prompt registry 2026-02-18 00:34:54 +05:30
exception_mapping.md Adds support for returning Azure Content Policy error information when exceptions from Azure OpenAI occur (#16231) 2025-11-08 16:04:36 -08:00
files_endpoints.md feat(anthropic): add Files API support for SDK 2026-03-11 12:45:19 -03:00
fine_tuning.md fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
generateContent.md fix: Metadata / Trace ID Missing in S3 Streaming Callbacks 2026-02-25 14:16:42 +05:30
image_edits.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
image_generation.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
image_variations.md
index.md fix: prompt registry 2026-02-18 00:34:54 +05:30
interactions.md fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
load_test.md fix: prompt registry 2026-02-18 00:34:54 +05:30
load_test_advanced.md fix: prompt registry 2026-02-18 00:34:54 +05:30
load_test_rpm.md docs: usaged-based routing perf warnings (#14080) 2025-08-29 17:31:12 -07:00
load_test_sdk.md
mcp.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
mcp_aws_sigv4.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
mcp_control.md [Chore] update mcp documentation for header forwarding 2026-03-04 13:07:25 +05:30
mcp_cost.md [MCP Gateway] QA/Fixes - Ensure Team/Key level enforcement works for MCPs (#15305) 2025-10-07 17:34:48 -07:00
mcp_guardrail.md merge: resolve conflicts between main and litellm_oss_staging_03_11_2026 2026-03-12 09:38:31 -03:00
mcp_oauth.md fix: prompt registry 2026-02-18 00:34:54 +05:30
mcp_openapi.md feat(mcp): allow admins to override tool name and description per MCP server (#22828) 2026-03-04 17:58:05 -08:00
mcp_public_internet.md fix: prompt registry 2026-02-18 00:34:54 +05:30
mcp_semantic_filter.md fix: prompt registry 2026-02-18 00:34:54 +05:30
mcp_troubleshoot.md fix: prompt registry 2026-02-18 00:34:54 +05:30
mcp_usage.md [MCP Gateway] QA/Fixes - Ensure Team/Key level enforcement works for MCPs (#15305) 2025-10-07 17:34:48 -07:00
migration.md
migration_policy.md
moderation.md docs(moderation.md): fix moderation quick start docs 2025-11-07 16:25:08 -08:00
ocr.md Enable local file support for OCR (#22133) 2026-02-27 10:50:02 -08:00
oidc.md fix: prompt registry 2026-02-18 00:34:54 +05:30
old_guardrails.md Use the -d flag in docs instead of -D (#12179) 2025-06-30 15:25:42 -07:00
projects.md
proxy_api.md Corrected docs updates sept 2025 (#14916) 2025-09-25 15:49:19 -07:00
proxy_auth.md fix: prompt registry 2026-02-18 00:34:54 +05:30
proxy_server.md
rag_ingest.md fix: prompt registry 2026-02-18 00:34:54 +05:30
rag_query.md fix: prompt registry 2026-02-18 00:34:54 +05:30
realtime.md feat(realtime): guardrails support for /v1/realtime WebSocket endpoint (#22152) 2026-02-25 21:34:22 -08:00
reasoning_content.md fix(openai): drop all reasoning_effort for gpt-5.4 + tools, including 'none' 2026-03-12 16:22:40 -03:00
rerank.md feat: Add IBM watsonx.ai rerank support (#21303) 2026-02-16 20:12:16 -08:00
response_api.md Add documentation related to responses websocket 2026-03-04 18:26:43 +05:30
response_api_compact.md fix: prompt registry 2026-02-18 00:34:54 +05:30
router_architecture.md
routing.md fix: prompt registry 2026-02-18 00:34:54 +05:30
rules.md
scheduler.md docs: usaged-based routing perf warnings (#14080) 2025-08-29 17:31:12 -07:00
sdk_custom_pricing.md
secret.md fix(ollama): thread api_base to get_model_info + graceful fallback (#21970) 2026-02-23 21:00:37 -08:00
set_keys.md
skills.md [Docs] Claude Skills API (#17052) 2025-11-24 15:32:03 -08:00
text_completion.md Guardrails - Responses API, Image Gen, Text completions, Audio transcriptions, Audio Speech, Rerank, Anthropic Messages API support via the unified apply_guardrails function (#15706) 2025-10-25 13:38:57 -07:00
text_to_speech.md fix: prompt registry 2026-02-18 00:34:54 +05:30
traffic_mirroring.md fix: prompt registry 2026-02-18 00:34:54 +05:30
troubleshoot.md fix: prompt registry 2026-02-18 00:34:54 +05:30
vector_store_files.md fix docs 2025-11-15 16:58:47 -08:00
vertex_batch_passthrough.md Litellm sameer oct staging (#15806) 2025-10-24 12:17:22 -07:00
videos.md [Feat] New Provider - Add RunwayML Provider for video generations (#16505) 2025-11-11 18:48:23 -08:00
wildcard_routing.md