* fix(litellm_proxy/chat/transformation.py): support 'user' and all other openai chat completion params
Fixes issue where 'user' was not being sent in request to litellm proxy via sdk
* fix(prisma_migration.py): remove 'use_prisma_migrate' flag, is now default
* docs: cleanup docs
* fix(proxy_cli.py): remove --use_prisma_migrate flag
* refactor: remove references to use_prisma_migrate env var
This is now the default flow for db migrations
* fix(create_key_button.tsx): add prompts on UI
* feat(key_management_endpoints.py): support adding prompt to key via `/key/update`
* fix(key_info_view.tsx): show existing prompts on key in key_info_view.tsx
* fix(key_edit_view.tsx): UX - disable premium feature for non-premium users
prevent accidental clicking
* fix(create_key_button.tsx): disable premium features behind flag, prevent errors
* feat(prompts.tsx): add new ui component to view created prompts
enables viewing prompts created on config
* feat(prompt_info.tsx): add component for viewing the prompt information
* feat(prompt_endpoints.py): support converting dotprompt to json structure + accept json structure in promptmanager
allows prompt manager to work with api endpoints
* test(test_prompt_manager.py): add unit tests for json data input
* feat(dotprompt/__init__.py): add prompt data to dotpromptmanager
* fix(prompt_endpoints.py): working crud endpoints for prompt management
* feat(prompts/): support `prompt_file` for dotprompt
allows to precisely point to the prompt file a prompt should use
* feat(proxy/utils.py): resolve prompt id correctly
resolves user sent prompt id with internal prompt id
* feat(schema.prisma): initial pr with db schema for prompt management table
allows post endpoints to work with backend
* feat(prompt_endpoints.py): use db in patch_prompt endpoint
* feat(prompt_endpoints.py): use db for update_prompt endpoint
* feat(prompt_endpoints.py): use db on prompt delete endpoint
* build(schema.prisma): add prompt tale to schema.prisma in litellm-proxy-extras
* build(migration.sql): add new sql migration file
* fix(init_prompts.py): fix init
* feat(prompt_info_view.tsx): show the raw prompt template on ui
allows developer to know the prompt template they'll be calling
* feat(add_prompt_form.tsx): working ui add prompt flow
allows user to add prompts to litellm via ui
* build(ui/): styling fixes
* build(ui/): prompts.tsx
styling improvements
* fix(add_prompt_form.tsx): styling improvements
* build(prompts.tsx): styling improvements
* build(ui/): styling improvements
* build(ui/): fix ui error
* fix: fix ruff check
* docs: document new api params
* test: update tests
* fix(proxy_cli.py): make use_prisma_migrate proxy default
Fixes https://github.com/BerriAI/litellm/issues/13046
Prisma migrate deploy prevents resetting db
* fix(auth_checks.py): resolve team only models while doing auth checks on model access groups
Fixes issue where key had access via an access group, but team only model could not be called
* test(test_router.py): add unit testing
* feat(provider_specific_fields.tsx): add aws sagemaker on UI
* fix(proxy_cli.py): check for module not found error on proxy import
Fixes https://github.com/BerriAI/litellm/issues/11836
* feat(proxy_server.py): utility function to get all models across all teams user is in
Allows user to see all team models on UI
* feat(proxy_server.py): return models accessible via team id's in `/v2/model/info`
Allows UI to tell user which team they can use to access model
* feat(columsn.tsx): initial PR to add 'accessible via Team ID's on model hub
allows user to know what teams they can access a model through
* Revert "feat(columsn.tsx): initial PR to add 'accessible via Team ID's on model hub"
This reverts commit f844c79383.
* fix(proxy_server.py): backend model info endpoint improvements
* UI Improvements for Default User access (#11952)
* feat(ui/): add a 'current team' and 'view' filters to the models page
allow user to see what all models they have access to within a specific team
* feat: working ui for seeing models in teams
* fix(model_dashboard.tsx): make current team filter more prominent
* style(model_dashboard.tsx): add a helpful note telling user how to create a model for the team they've selected
* style(model_dashboard.tsx): only show helpful note when current view is team, not for global
* fix(team_dropdown.tsx): allow searching by team id on create key modal
* feat(create_key_button.tsx): add helpful message when team selection is required
* fix: fix linting checks
* fix: fix ui linting error
* docs(team_endpoints.py): document new param
* Enhance proxy CLI with Rich formatting and improved user experience
- Integrated Rich library for better console output in `proxy_cli.py`, including version display, health check results, and test completion responses.
- Updated health check and test completion methods to provide progress indicators and formatted tables.
- Refactored feedback display in `proxy_server.py` to use Rich for a more visually appealing user interface.
- Adjusted tests in `test_proxy_cli.py` to mock console output instead of using print statements, ensuring compatibility with Rich formatting.
* fix linting error
* refactor(proxy_cli.py): simplify DB setup logging
- Removed progress indicators for IAM token generation and environment variable decryption to simplify the code.
- Consolidated the logic for generating the database URL and setting environment variables.
- Enhanced error handling for configuration loading and database setup, ensuring clearer feedback
* Update test-linting workflow to include proxy-dev dependencies in Poetry installation
* Enhance proxy server initialization with Rich console for improved model display. Added support for loading model parameters from environment variables and refined provider identification logic. Fallback to original print formatting if Rich is not available.
* Refactor feedback handling: Moved feedback message generation and custom warning display to utils.py. Enhanced feedback box with rich formatting and fallback to ASCII for environments without rich. Cleaned up proxy_server.py by removing obsolete code.
* fix linting error
* Refactor model initialization display: Moved model initialization logic to a new utility function `display_model_initialization` for improved readability and maintainability. Enhanced model provider extraction with a dedicated function. Fallback to basic logging if Rich console is unavailable.
* Refactor model provider extraction: Replace the `_extract_provider_from_model` function with a more robust approach using `get_llm_provider`. Implement fallback logic for provider identification and improve error handling. Ensure compatibility with Rich console for model initialization display.
* fix: handle DB_USER, DB_PASSWORD, DB_HOST problem I faced, since this could be with special character, so better be url encoded, similar problem also happen in DATABASE_URL
* test: added the test cases
* add: test case of url with sepcial character
* build(README.md): initial commit adding a separate folder for additional proxy files. Meant to reduce size of core package
* build(litellm-proxy-extras/): new pip package for storing migration files
allows litellm proxy to use migration files, without adding them to core repo
* build(litellm-proxy-extras/): cleanup pyproject.toml
* build: move prisma migration files inside new proxy extras package
* build(run_migration.py): update script to write to correct folder
* build(proxy_cli.py): load in migration files from litellm-proxy-extras
Closes https://github.com/BerriAI/litellm/issues/9558
* build: add MIT license to litellm-proxy-extras
* test: update test
* fix: fix schema
* bump: version 0.1.0 → 0.1.1
* build(publish-proxy-extras.sh): add script for publishing new proxy-extras version
* build(liccheck.ini): add litellm-proxy-extras to authorized packages
* fix(litellm-proxy-extras/utils.py): move prisma migrate logic inside extra proxy pkg
easier since migrations folder already there
* build(pre-commit-config.yaml): add litellm_proxy_extras to ci tests
* docs(config_settings.md): document new env var
* build(pyproject.toml): bump relevant files when litellm-proxy-extras version changed
* build(pre-commit-config.yaml): run poetry check on litellm-proxy-extras as well
* fix running litellm on windows
* fix importing litellm
* _init_hypercorn_server
* linting fix
* TestProxyInitializationHelpers
* ci/cd run again
* ci/cd run again
* add SecretManager to httpxSpecialProvider
* fix importing AWSSecretsManagerV2
* add unit testing for writing keys to AWS secret manager
* use KeyManagementEventHooks for key/generated events
* us event hooks for key management endpoints
* working AWSSecretsManagerV2
* fix write secret to AWS secret manager on /key/generate
* fix KeyManagementSettings
* use tasks for key management hooks
* add async_delete_secret
* add test for async_delete_secret
* use _delete_virtual_keys_from_secret_manager
* fix test secret manager
* test_key_generate_with_secret_manager_call
* fix check for key_management_settings
* sync_read_secret
* test_aws_secret_manager
* fix sync_read_secret
* use helper to check when _should_read_secret_from_secret_manager
* test_get_secret_with_access_mode
* test - handle eol model claude-2, use claude-2.1 instead
* docs AWS secret manager
* fix test_read_nonexistent_secret
* fix test_supports_response_schema
* ci/cd run again
* refactor(proxy_server.py): add debug logging around license check event (refactor position in startup_event logic)
* fix(proxy/_types.py): allow admin_allowed_routes to be any str
* fix(router.py): raise 400-status code error for no 'model_name' error on router
Fixes issue with status code when unknown model name passed with pattern matching enabled
* fix(converse_handler.py): add claude 3-5 haiku to bedrock converse models
* test: update testing to replace claude-instant-1.2
* fix(router.py): fix router.moderation calls
* test: update test to remove claude-instant-1
* fix(router.py): support model_list values in router.moderation
* test: fix test
* test: fix test
* feat(proxy_cli.py): add new 'log_config' cli param
Allows passing logging.conf to uvicorn on startup
* docs(cli.md): add logging conf to uvicorn cli docs
* fix(get_llm_provider_logic.py): fix default api base for litellm_proxy
Fixes https://github.com/BerriAI/litellm/issues/6332
* feat(openai_like/embedding): Add support for jina ai embeddings
Closes https://github.com/BerriAI/litellm/issues/6337
* docs(deploy.md): update entrypoint.sh filepath post-refactor
Fixes outdated docs
* feat(prometheus.py): emit time_to_first_token metric on prometheus
Closes https://github.com/BerriAI/litellm/issues/6334
* fix(prometheus.py): only emit time to first token metric if stream is True
enables more accurate ttft usage
* test: handle vertex api instability
* fix(get_llm_provider_logic.py): fix import
* fix(openai.py): fix deepinfra default api base
* fix(anthropic/transformation.py): remove anthropic beta header (#6361)
* fix: enable new 'disable_prisma_schema_update' flag
* build(config.yml): remove setup remote docker step
* ci(config.yml): give container time to start up
* ci(config.yml): update test
* build(config.yml): actually start docker
* build(config.yml): simplify grep check
* fix(prisma_client.py): support reading disable_schema_update via env vars
* ci(config.yml): add test to check if all general settings are documented
* build(test_General_settings.py): check available dir
* ci: check ../ repo path
* build: check ./
* build: fix test
* feat(together_ai/completion): handle together ai completion calls
* fix: handle list of int / list of list of int for text completion calls
* fix(utils.py): check if base model in bedrock converse model list
Fixes https://github.com/BerriAI/litellm/issues/6003
* test(test_optional_params.py): add unit tests for bedrock optional param mapping
Fixes https://github.com/BerriAI/litellm/issues/6003
* feat(utils.py): enable passing dummy tool call for anthropic/bedrock calls if tool_use blocks exist
Fixes https://github.com/BerriAI/litellm/issues/5388
* fixed an issue with tool use of claude models with anthropic and bedrock (#6013)
* fix(utils.py): handle empty schema for anthropic/bedrock
Fixes https://github.com/BerriAI/litellm/issues/6012
* fix: fix linting errors
* fix: fix linting errors
* fix: fix linting errors
* fix(proxy_cli.py): fix import route for app + health checks path (#6026)
* (testing): Enable testing us.anthropic.claude-3-haiku-20240307-v1:0. (#6018)
* fix(proxy_cli.py): fix import route for app + health checks gettsburg.wav
Fixes https://github.com/BerriAI/litellm/issues/5999
---------
Co-authored-by: David Manouchehri <david.manouchehri@ai.moda>
---------
Co-authored-by: Ved Patwardhan <54766411+vedpatwardhan@users.noreply.github.com>
Co-authored-by: David Manouchehri <david.manouchehri@ai.moda>
