TensorNull
55a6dd3a8b
feat(cometapi): Add CometAPI provider support (embeddings, image generation, docs)
...
- Add CometAPI embedding and image generation transformations and configs
- Add image cost calculator and export/init files
- Register provider in constants, utils, main (embedding path) and sidebars
- Add CometAPI docs page and cookbook notebook (Colab) for usage examples
2025-10-16 13:08:14 +08:00
Ishaan Jaff
bc26845ec4
[Feat] Native /ocr endpoint support ( #15573 )
...
* [Feat] Add native litellm.ocr() functions (#15567 )
* fix get_supported_ocr_params
* add get_provider_ocr_config
* init OCR
* init ocr functions
* add OCRResponse Base Model
* add ocr to llm http handlers
* add main.py for OCR
* fix linting for OCR
* TestMistralOCR
* update to use DocumentType for Mistral
* fix _prepare_ocr_request
* fix transform
* add main.py for OCR
* add spec to init
* fix OCR
* TestMistralOCR
* ruff fix
* Potential fix for code scanning alert no. 3521: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
---------
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* [Feat] Add /ocr route on LiteLLM AI Gateway - Adds support for native mistral ocr calling (#15571 )
* fix get_supported_ocr_params
* add get_provider_ocr_config
* init OCR
* init ocr functions
* add OCRResponse Base Model
* add ocr to llm http handlers
* add main.py for OCR
* fix linting for OCR
* TestMistralOCR
* update to use DocumentType for Mistral
* fix _prepare_ocr_request
* fix transform
* add main.py for OCR
* add spec to init
* fix OCR
* TestMistralOCR
* ruff fix
* add router.ocr() methods
* add OCR routes
* feat add ocr routes
* add OCR routes
* feat: add OCR routes in proxy server
* working /ocr routes
* test_router_aocr_with_mistral
* docs Mistral OCR
* docs OCR
* [Feat] Add Azure AI Mistral OCR Integration (#15572 )
* fix get_supported_ocr_params
* add get_provider_ocr_config
* init OCR
* init ocr functions
* add OCRResponse Base Model
* add ocr to llm http handlers
* add main.py for OCR
* fix linting for OCR
* TestMistralOCR
* update to use DocumentType for Mistral
* fix _prepare_ocr_request
* fix transform
* add main.py for OCR
* add spec to init
* fix OCR
* TestMistralOCR
* ruff fix
* add router.ocr() methods
* add OCR routes
* feat add ocr routes
* add OCR routes
* feat: add OCR routes in proxy server
* working /ocr routes
* test_router_aocr_with_mistral
* docs Mistral OCR
* docs OCR
* add azure ai to get_provider_ocr_config
* add AzureAIOCRConfig
* TestAzureAIOCR
* TestAzureAIOCR
* test fixes for azure ai ocr
* fix async OCR transform for Azure
* fix transform_ocr_request
---------
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-10-15 17:20:01 -07:00
Krrish Dholakia
0526334d9c
docs: fix url
2025-10-15 08:33:28 -07:00
Davi S. Zucon
9d944b1404
small fix code snippet custom_prompt_management.md
...
passing parameter:
prompt_id directly to openai client, raises error:
TypeError: Completions.create() got an unexpected keyword argument 'prompt_id'
instead use:
extra_body={
"prompt_id": "1234"
}
2025-10-14 16:37:00 -03:00
Krrish Dholakia
0c86c3d791
docs(index.md): bump rc
2025-10-13 10:56:14 -07:00
Krrish Dholakia
611bda94cb
docs: fix doc
2025-10-13 09:59:07 -07:00
Krish Dholakia
d81843070a
Merge pull request #15461 from BerriAI/litellm_update_docs
...
[Docs] - Update benchmark results
2025-10-12 22:11:57 -07:00
mubashir1osmani
950ee924fc
add ecs to docs
2025-10-12 16:13:56 -04:00
Ishaan Jaffer
e761665f22
docs fix
2025-10-11 18:21:35 -07:00
AlexsanderHamir
5d1fa76d14
update benchmarks
2025-10-11 18:06:37 -07:00
AlexsanderHamir
2cb1db27cb
docs: update benchmark results with improved infrastructure
...
- Update to 2 instance baseline (1035 RPS @ 200ms median)
- Add 4 instance results (1170 RPS @ 100ms median)
- Update machine specs to 4 CPU / 8GB RAM
- Update Locust settings to 1000 users
2025-10-11 18:00:26 -07:00
Imad Saddik
133189e191
Fixed a few typos ( #15267 )
2025-10-11 17:53:42 -07:00
Alexsander Hamir
300a142926
[Add]: perf summary ( #15458 )
...
* add: perf summary
* update docs
* fix: focus on p99
2025-10-11 17:38:13 -07:00
Ishaan Jaffer
087bd2d306
docs fix
2025-10-11 16:49:11 -07:00
Ishaan Jaffer
1ccd949495
docs fix
2025-10-11 16:34:47 -07:00
Ishaan Jaffer
b9fef02f04
docs v1.78.0
2025-10-11 16:33:46 -07:00
Ishaan Jaffer
7288f4355f
docs fix
2025-10-11 16:24:21 -07:00
Ishaan Jaffer
54b8bde60c
docs fix
2025-10-11 16:23:46 -07:00
Ishaan Jaffer
8d29fc8cdd
docs
2025-10-11 16:12:00 -07:00
Krish Dholakia
8c4c4acda2
Merge pull request #15457 from BerriAI/litellm_docs_10_11_2025
...
Litellm docs 10 11 2025
2025-10-11 15:16:13 -07:00
Krrish Dholakia
3f921d8dbe
docs(index.md): document mcp oauth support
2025-10-11 15:14:15 -07:00
Krrish Dholakia
9b5943ae50
docs: cleanup docs
2025-10-11 15:09:24 -07:00
Krrish Dholakia
fdaa163758
docs(claude_responses_api.md): update docs with mcp server tutorial
2025-10-11 15:07:54 -07:00
Ishaan Jaffer
e4f3d78579
docs v1.78.0.rc.1
2025-10-11 14:57:28 -07:00
Krrish Dholakia
73257be604
docs(index.md): initial commit for github mcp oauth docs
2025-10-11 14:00:50 -07:00
Ishaan Jaff
f6e0ef4ee7
[Docs] Key Rotations ( #15455 )
...
* docs keys
* docs key r
2025-10-11 13:24:47 -07:00
Ishaan Jaffer
0cde6a2843
docs key rotations
2025-10-11 13:10:17 -07:00
Krish Dholakia
07ba5c1ea9
Merge branch 'main' into litellm_dev_10_10_2025_p3
2025-10-11 12:58:53 -07:00
Krish Dholakia
88000d09bf
Merge pull request #15454 from BerriAI/litellm_ui_qa_10_11_2025
...
docs: cleanup docs + build ui
2025-10-11 12:54:37 -07:00
Ishaan Jaffer
8292b27073
docs fix
2025-10-11 12:15:30 -07:00
Ishaan Jaff
d45dfa899b
[Docs] v1.78.0-stable ( #15452 )
...
* docs fix
* docs fix
2025-10-11 12:12:41 -07:00
Krrish Dholakia
c03c66266e
docs: cleanup docs
2025-10-11 11:05:55 -07:00
Ishaan Jaffer
698c0b219c
test code QA checks
2025-10-11 08:41:56 -07:00
Ishaan Jaffer
5377b17bdd
doc fix
2025-10-10 19:55:02 -07:00
Ishaan Jaff
527c8f59fa
[Feat] Tag Management - Add support for setting tag based budgets ( #15433 )
...
* feat: add LiteLLM_TagTable
* fix: use new table for tag management
* fix - allow setting budgets for tags
* working tag creation
* fix schema.prisma
* add tag info
* ui fixes
* ui fix tag info
* TAG_CACHE_IN_MEMORY_TTL_SECONDS
* add Litellm_EntityType
* fix get_aggregated_db_spend_update_transactions
* fix: _update_entity_spend_in_db
* fix _tag_max_budget_check
* add tag budget check
* add tag_list_transactions
* test_get_tag_objects_batch
* test_update_tag_db_without_prisma_client
* fix get_tags_from_request_body
* get_tags_from_request_body
* fix get_tags_from_request_body
* fix spend tracking utils
* get_tags_from_request_body
* test_get_tags_from_request_body_with_metadata_tags
* feat: add _update_tag_cache spend tracking
* fix _PROXY_track_cost_callback
* test_tag_cache_update_multiple_tags
* fix tag info
* docs fix
* docs tag budgets
* doc fix
* docs fix
* fix tag budget
* docs tag budgets
* docs fix
* ruff fix
2025-10-10 19:24:50 -07:00
Krrish Dholakia
144ea09549
docs(openai.md): document how to get reasoning content for gpt-5 models
2025-10-10 18:42:19 -07:00
Ishaan Jaff
bf209415da
[Feat] VertexAI Gemma model family streaming support + Added MedGemma ( #15427 )
...
* test_acompletion_filters_stream_and_stream_options
* fix: stream_options
* docs medgemma
* lint fix
* docs
2025-10-10 14:22:27 -07:00
Sameer Kankute
a2123f7596
fix lint errors ( #15406 )
...
* fix lint errors
* fix mypy error
* Update docs error
2025-10-10 14:16:23 -07:00
Sameer Kankute
513925fd12
Add OCI Cohere support with tool calling and streaming capabilities ( #15365 )
...
* Add cohere models for OCI integration
* fix mypy error
2025-10-10 11:20:18 -07:00
Krish Dholakia
d944717d4b
Merge pull request #15278 from xprilion/wandb-inference-docs
...
Add W&B Inference documentation
2025-10-09 22:35:16 -07:00
Krish Dholakia
611a876889
Merge pull request #15380 from BerriAI/litellm_shared_heatlh_status
...
Implement Shared Health Check State Across Pods
2025-10-09 22:25:45 -07:00
Krish Dholakia
e1e1ea31dc
Merge pull request #15379 from PostHog/feat/posthog-per-request-api-key
...
feat: posthog per request api key
2025-10-09 22:24:51 -07:00
Alexsander Hamir
1c7463f42f
[Perf]: optimize SSL/TLS handshake performance with prioritized cipher ( #15398 )
...
* perf: optimize SSL/TLS handshake performance with prioritized cipher suites
Add configurable SSL cipher suite list that strongly prefers fast modern
ciphers (TLS 1.3, ECDHE+GCM) while maintaining backward compatibility with
widely-supported fallback ciphers. Set minimum TLS version to 1.2 for
better performance.
Changes:
- Add DEFAULT_SSL_CIPHERS constant with prioritized cipher list
(configurable via LITELLM_SSL_CIPHERS env var)
- Update get_ssl_configuration() to apply optimized ciphers by default
- Set TLS minimum version to 1.2
- Preserve existing ssl_security_level parameter for custom configs
This balances handshake performance (~50-100ms with modern ciphers) with
broad provider compatibility.
* fix: missing documented constants
2025-10-09 19:21:37 -07:00
Ishaan Jaff
52bbabd788
[Feat] Support for Vertex AI Gemma Models on Custom Endpoints ( #15397 )
...
* TestVertexGemmaiCompletion
* test vertex Gemma
* fix file name
* fix file naming
* add VertexAIGemmaModels
* add cost_router for vertexai
* fix main.py
* fix VertexGemmaConfig
* fix Vertex AI Gemma-AI Models Handler
* docs gemma
* fix ids
* test fix
* ruff check fixes
* docs fix
* docs fix
2025-10-09 19:20:02 -07:00
Ishaan Jaffer
be587363f3
docs fix
2025-10-09 16:28:38 -07:00
Ishaan Jaff
6cf57d85c3
[Feat] Add EnkryptAI Guardrails on LiteLLM ( #15390 )
...
* feat: add EnkryptAIGuardrail
* fix types
* fix EnkryptAIGuardrails
* __all__ = ["EnkryptAIGuardrail"]
fix
* fix EnkryptAIGuardrail
* TestEnkryptAIExtraction
* fix mypy linting
* docs enkrypt ai
* add EnkryptAIGuardrailConfigModel
* fix EnkryptAIGuardrailConfigModel
* fix import struct
* fix EnkryptAIGuardrails
2025-10-09 16:27:04 -07:00
Anubhav Singh
f59f1971cc
Merge branch 'BerriAI:main' into wandb-inference-docs
2025-10-09 22:26:37 +05:30
Sameer Kankute
dce6cd1051
Add shared healthcheck
2025-10-09 22:18:05 +05:30
Carlos Marchal
dd560f792e
feat: posthog per request api key
2025-10-09 18:32:11 +02:00
Krrish Dholakia
f88e3ebdfa
docs: refactor to have function call / computer use / etc. on top level
2025-10-09 07:25:25 -07:00
