Commit graph

23371 commits

Author SHA1 Message Date
Krish Dholakia
bc2ab84342
Teams - allow setting custom key duration + show many user + service account keys have been created (#12722)
* fix(team_info.tsx): allow setting custom key duration

more flexible than previous pre-set options

* feat(team_info.tsx): show how many user + service account keys have been created within a team
2025-07-17 22:33:59 -07:00
Krish Dholakia
b77c9f5de2
fix(team_endpoints.py): ensure user id correctly added when new team … (#12719)
* fix(team_endpoints.py): ensure user id correctly added when new team created with user email as member

Fixes issue where user not correctly added to team on /team/new

* fix(internal_user_endpoints.py): make user email validation check case insensitive

Fixes issue where uppercase email was added even when lowercase email existed

* test: update test
2025-07-17 22:31:44 -07:00
Krish Dholakia
fb3a5bc1b7
feat(internal_user_endpoints.py): new /user/bulk_update endpoint (#12720)
* feat(internal_user_endpoints.py): new `/user/bulk_update` endpoint

enable bulk updating users on the UI

* refactor: cleanup unused import
2025-07-17 22:28:29 -07:00
Krrish Dholakia
15585adb8c bump: version 1.74.4 → 1.74.5 2025-07-17 22:26:32 -07:00
Krish Dholakia
b515d051ff
Litellm encrypt admin UI values (#12675)
* build: move build_and_test to use prisma migrate

* feat(proxy_setting_endpoints.py): encrypt env var before storing in db

Ensures env var can be read when loaded in from DB

Fixes issue when trying to add SSO from admin UI

* test: update tests
2025-07-17 22:24:58 -07:00
Krrish Dholakia
4d97cd66fb docs(index.md): document new release 2025-07-17 22:13:26 -07:00
Jugal D. Bhatt
f4a94ec911
Added name change (#12702) 2025-07-17 18:35:01 -07:00
Jugal D. Bhatt
842af61a05
[MCP Gateway] add fix to update object permission on update/delete key/team (#12701)
* add fix to update object permission on update/delete

* fix edit team page dropdown
2025-07-17 18:34:35 -07:00
Jugal D. Bhatt
7832e9f2f6
[MCP Gateway] added docs for mcp namespacing by URL (#12700)
* added docs

* added changes to comma separated

* change url

* highlight

* added server in angle brackets and _ namespaced the access group
2025-07-17 18:33:48 -07:00
Ishaan Jaff
33c84846e9
[Refactor] Vector Stores - Use class VectorStorePreCallHook for all Vector Store Integrations (#12715)
* add VectorStorePreCallHook

* vector_store_pre_call_hook

* add pop_vector_stores_to_run

* async_get_chat_completion_prompt

* working e2e tests

* test_e2e_bedrock_knowledgebase_retrieval_with_completion

* delete old files

* fix logging test

* VectorStorePreCallHook

* fix ruff check

* vector_store_pre_call_hook

* linting error fixes
2025-07-17 16:31:58 -07:00
Robert Gambee
52b2984792
[Bug Fix] Always include tool calls in output of trim_messages (#11517)
* Check content and order of trimmed messages

* Assert tool calls are preserved if below max_tokens

* Unreverse order of tool calls

* Return tool calls alongside other messages

* Write test for trimming untokenizable field

* Return original messages in case of exception
2025-07-17 16:01:59 -07:00
Krrish Dholakia
134eb743f4 docs(mcp.md): remove 'beta' 2025-07-17 12:16:54 -07:00
Krrish Dholakia
aeb270b759 build: move build_and_test to use prisma migrate 2025-07-17 12:16:06 -07:00
Ishaan Jaff
37b6814400
[Refactor] Use Existing config structure for bedrock vector stores (#12672)
* add transform_search_vector_store_request

* add BedrockVectorStoreConfig

* unit tests

* TestBedrockVectorStore
2025-07-17 11:55:11 -07:00
Ishaan Jaff
95f7290e44
[Feat] Proxy - New LLM API Routes /v1/vector_stores and /v1/vector_stores/vs_abc123/search (#12699)
* add Search + Create vector stores

* add new vector_stores routes

* add avector_store_search routes

* docs vector stores

* docs vector stores

* Create Vector Store

* fix - refactor

* add vector_store_router

* fix imports

* fix ruff
2025-07-17 11:19:25 -07:00
Ishaan Jaff
52a525ab9c
[Feat] Bedrock Guardrails - Allow disabling exception on 'BLOCKED' action (#12693)
* flag: disable_exception_on_block

* allow setting disable_exception_on_block

* test_bedrock_guardrail_disable_exception_on_block_non_streaming

* docs Disabling Exceptions on Bedrock BLOCK

* fix recognition_metadata test
2025-07-17 08:46:33 -07:00
Cole McIntosh
77191c7b2a
Fix incorrect environment variable names in LiteLLM gateway documentation (#12686)
Updated ANTHROPIC_API_KEY to ANTHROPIC_AUTH_TOKEN to match the official documentation requirements for LiteLLM gateway configuration.
2025-07-17 08:02:28 -07:00
Cole McIntosh
3a0ea80c75
Add Claude Code LiteLLM tutorial (#12650)
* Add concise Claude Code + LiteLLM Gateway tutorial

- Create focused tutorial matching existing tutorial style
- Step-by-step guide from installation to advanced configurations
- Multi-provider configuration examples (AWS Bedrock, Azure OpenAI, Load Balancing)
- Based on Anthropic's official LiteLLM configuration documentation
- Added to sidebar with clean title 'Use LiteLLM with Claude Code'
- Fixed sidebar reference from 'secret' to 'set_keys' for proper document resolution

* Update config_settings.md to correct documentation links for key management and Hashicorp Vault settings. Changed references from 'secret.md' to 'set_keys.md' for improved clarity and accuracy.

* Update sidebar and config_settings.md to reflect changes in key management documentation. Changed sidebar reference from 'set_keys' to 'secret' and updated links in config_settings.md for Hashicorp Vault settings to point to 'secret.md' for improved accuracy.

* Remove extra tutorial and update sidebar accordingly

* Update tutorial title from 'WebUI' to 'Open WebUI' for clarity and consistency in documentation.

* Remove Python version requirement from Claude Responses API tutorial for clarity and to align with updated prerequisites.
2025-07-16 23:07:29 -07:00
Jugal D. Bhatt
4b09d0d517
[Liveness/Liveliness probe] add separate health app for liveness probes in files (#12669)
* add separate health app

* add new docs

* refactor

* fix colons

* Update config_settings.md

* refactor

* docs

* add unit test
2025-07-16 20:35:09 -07:00
Ishaan Jaff
e5f0a8477b
[Feat] New Vector Store - PG Vector (#12667)
* add PGVectorStoreConfig

* add PGVectorStoreConfig

* test_environment_variable_support

* fix code QA check

* rename test

* add PG vector img

* allow adding vector stores

* add pg vector

* add vector store

* TestPGVectorStoreConfig

* TestPGVectorStoreConfig
2025-07-16 18:17:05 -07:00
Cole McIntosh
b2080ec9af
feat: add input_fidelity parameter for OpenAI image generation (#12662)
* feat: add input_fidelity parameter for OpenAI image generation

- Add input_fidelity to OpenAIImageGenerationOptionalParams type
- Update image_generation function signature to accept input_fidelity
- Add input_fidelity to default_params in get_optional_params_image_gen
- Include input_fidelity in openai_params list for proper handling
- Update documentation with input_fidelity parameter description
- Add test for input_fidelity parameter functionality

This enables control over how closely the model follows the input prompt
for gpt-image-1 model, improving prompt adherence and image quality.

* feat: add input_fidelity to optional parameters for image generation

- Include input_fidelity in the list of OpenAIImageGenerationOptionalParams
- This addition enhances the flexibility of image generation by allowing control over input fidelity.

* test: enhance test for gpt-image-1 with input_fidelity parameter

- Update test_gpt_image_1_with_input_fidelity to include mocking of OpenAI response
- Validate that the OpenAI client is called with correct parameters, including input_fidelity
- Improve response validation to ensure expected output structure and values
2025-07-16 16:56:05 -07:00
Ishaan Jaff
d9943f9812 fix cohere InternalServerError error mapping 2025-07-16 16:13:34 -07:00
Ishaan Jaff
0141ef924f test_create_mcp_server_invalid_alias 2025-07-16 15:58:34 -07:00
Ishaan Jaff
43e7d5288f
[Bug Fix] SCIM - add GET /ServiceProviderConfig (#12664)
* add scimv2 config endpoint

* test_get_service_provider_config
2025-07-16 15:43:25 -07:00
Ishaan Jaff
6a7aab7b84
[Bug Fix] StandardLoggingPayload on cache_hits should track custom llm provider + DD LLM Obs span type (#12652)
* bug fix - ensure custom llm provider is tracked on cache hit

* fix config.yaml

* test_cache_hit_includes_custom_llm_provider

* fix _get_datadog_span_kind

* test_datadog_span_kind_mapping

* fix ruff check

* test_datadog_span_kind_mapping
2025-07-16 15:43:15 -07:00
Ishaan Jaff
538339e1a8
[Feat] UI - Add end_user filter on UI (#12663)
* add end_user filter on Logs

* add end user filter
2025-07-16 15:20:31 -07:00
Ishaan Jaff
9b817b511e
[Feat] Allow reading custom logger python scripts from s3 (#12623)
* add _load_instance_from_remote_storage

* load from s3

* fix download_python_file_from_s3

* fix download_python_file_from_s3

* add _load_instance_from_remote_storage

* tests

* Step 2b - Loading Custom Callbacks

* docs

* fix code qa check
2025-07-16 15:07:01 -07:00
Jugal D. Bhatt
83b0c4cba7
[MCP Gateway] Allow MCP sse and http to have namespaced url for better segregation LIT-304 (#12658)
* fix tools fetch for keys

* Add namespacing in url

* add test for namespacing url

* helper method

* fix test
2025-07-16 14:47:45 -07:00
Jugal D. Bhatt
a7d0b122b9
[MCP Gateway] List tools from access list for keys (#12657)
* fix tools fetch for keys

* add unit test
2025-07-16 14:28:53 -07:00
Jugal D. Bhatt
b3a214bce7
[MCP Gateway] Allow MCP access groups to be added via the config LIT-312 (#12654)
* allow mcp access groups to be added via the config

* fix code and doc quality

* fix mypy

* create helpers:
2025-07-16 13:57:43 -07:00
Krrish Dholakia
6e426c8b7c refactor(admin_ui_sso.md): refactor to include /sso/debug/callback in this doc
more natural place
2025-07-16 11:39:02 -07:00
Krrish Dholakia
047b106197 docs(custom_auth.md): add virtual keys doc 2025-07-16 11:09:57 -07:00
Krrish Dholakia
0fb7c01382 fix(make_model_public_form.tsx): make distinction clearer 2025-07-16 10:58:23 -07:00
Krrish Dholakia
8a1a90bc7a test: update test 2025-07-16 10:28:08 -07:00
Cole McIntosh
9c4b4303d3
fix: remove unused imports in completion_extras transformation (#12655)
- Remove unused GenericResponseOutputItem import
- Remove unused OutputFunctionToolCall import
- Fixes F401 linting errors
2025-07-16 10:23:13 -07:00
Krrish Dholakia
aa12a868c5 fix(migrate_keys.py): add script for migrating keys to new db 2025-07-16 10:18:36 -07:00
Cole McIntosh
d51aee1b84
Add GitHub Copilot LiteLLM tutorial (#12649)
* Add comprehensive GitHub Copilot + LiteLLM integration tutorial

- Complete setup guide from installation to production deployment
- Multiple configuration examples including authentication, load balancing, and cost tracking
- Docker and Kubernetes deployment configurations
- Troubleshooting section with common issues and solutions
- Best practices for security, monitoring, and reliability
- Usage examples for code completion, chat interface, and direct API integration

* Add concise GitHub Copilot + LiteLLM tutorial

- Create focused tutorial matching Gemini CLI style
- Step-by-step guide from installation to production deployment
- Multi-provider configuration examples (OpenAI, Anthropic, Bedrock)
- Load balancing and fallback configuration
- Docker deployment instructions
- Troubleshooting section with common issues
- Updated sidebar with clean title 'Use LiteLLM with GitHub Copilot'

* Refactor GitHub Copilot integration tutorial

- Removed outdated production deployment and direct API usage sections
- Streamlined troubleshooting steps for clarity
- Ensured documentation aligns with current best practices and configurations

* Add proper credit to Sergio Pino for GitHub Copilot tutorial

- Reference original DEV.to article in info box
- Add credits section acknowledging foundational work
- Maintain attribution to original author's guide
2025-07-16 09:40:27 -07:00
Krrish Dholakia
f4131b023e fix: don't fail request if unmapped item in responses list
not every responses item has a 1:1 mapping with chat completions
2025-07-16 09:25:26 -07:00
Ishaan Jaff
e8a748161f
[Bug Fix] grok-4 does not support the stop param (#12646)
* bug fix - using stop reason with grok 4

* fixes for XAI stop params

* test_xai_grok_4_stop_not_supported

* test_xai_grok_4_stop_not_supported
2025-07-16 09:19:25 -07:00
Krrish Dholakia
604075a36c test: update test 2025-07-16 09:15:05 -07:00
Krrish Dholakia
9cac629ca6 test: update test 2025-07-16 09:13:16 -07:00
Krrish Dholakia
446ed6039e docs(admin_ui_sso.md): document /fallback/login flow 2025-07-16 09:07:42 -07:00
Krrish Dholakia
5a8762b6a1 test: update tests 2025-07-16 08:56:41 -07:00
Cole McIntosh
c0935b9d58
Merge pull request #12648 from colesmcintosh/add-groq-moonshotai-kimi-k2-instruct
Add groq/moonshotai-kimi-k2-instruct model configuration
2025-07-16 09:50:52 -06:00
Cole McIntosh
ca40efab31 feat: add groq/moonshotai-kimi-k2-instruct model configuration
- Add model configuration for groq/moonshotai-kimi-k2-instruct
- Set max_tokens: 131072, max_input_tokens: 131072, max_output_tokens: 16384
- Configure pricing: 1e-06 input cost, 3e-06 output cost per token
- Enable function calling, response schema, reasoning, and tool choice support
2025-07-16 09:22:42 -06:00
Ishaan Jaff
0d1a1cfcfb
add together_ai/moonshotai/Kimi-K2-Instruct (#12645) 2025-07-16 07:53:27 -07:00
Michael Nguyen
f505977b62
Update bedrock nova micro and lite info (#12619) 2025-07-16 07:53:15 -07:00
Cole McIntosh
79e4d77bcf
fix: Handle circular references in spend tracking metadata JSON serialization (#12643)
* fix: Handle circular references in spend tracking metadata JSON serialization

- Fixes issue #12634 where circular references in metadata caused
  ValueError: Circular reference detected when logging spend data
- Adds _safe_json_dumps() function that detects and handles circular
  references by replacing them with placeholder strings
- Maintains full functionality for normal objects while preventing
  crashes from circular references
- Adds comprehensive tests for circular reference handling
- Critical fix for v1.74.3 stable release

* fix: Replace bare except clauses with specific Exception handling

- Fixes E722 linting errors in _safe_json_dumps function
- Maintains same error handling behavior while following best practices
- All tests continue to pass

* refactor: Use existing safe_dumps utility instead of custom implementation

- Replace custom _safe_json_dumps() with existing safe_dumps() from litellm_core_utils
- Remove duplicate code and leverage existing circular reference handling
- Update tests to use safe_dumps function
- Maintains same functionality while reducing code duplication
- All tests continue to pass
2025-07-16 07:24:41 -07:00
Krrish Dholakia
4e9440ee85 docs: cleanup docs 2025-07-16 07:18:13 -07:00
Krrish Dholakia
e22390a39a docs(openai.md): cleanup bridge doc 2025-07-15 22:59:10 -07:00