* Add `litellm-proxy` CLI (#10478)
* First cut at a Python client module for proxy
* Add UnauthorizedError + add_model method
* Add delete_model method
* Add example model_id to delete_model docstring
* Make delete_model raise NotFoundError
* Add get_model
* Add get_all_model_info
* Rename models.list_models to models.list
* Rename models.get_all_model_info to models.info
* Move ModelsManagementClient.get_all_model_group_info to ModelGroupsManagementClient.info
* Rename get_model to get
* Rename add_model to new
* Rename delete_model to delete
* In client classes, rename base_url attribute to _base_url and api_key attribute to _api_key
* Add ModelsManagementClient.updae method
* Add client.chat.completions (ChatClient)
* ruff format litellm/proxy/client
* ruff format tests/litellm/proxy/client/*.py
* Add latest changes
* Rename KeysManagementClient.create to KeysManagementClient.generate
* Add new parameters to KeysManagementClient.generate
* Add CredentialsManagementClient
* Remove api_key parameter from KeysManagementClient.generate
* Fix lint errors
* Add litellm/proxy/client/README.md
* README.md: Remove api_key param to client.keys.generate
* Fix mypy errors
* First cut at litellm-proxy cli
* Add test for `litellm-proxy models list`
* Nicer get_models_info
* get_models_info: --columns option
* Use format_timestamp in list_models
* ruff format litellm/proxy/client
* Simpler JSON printing with rich.print_json
* Move models-related commands to separate file
From `cli.py` to `groups/models.py`
* Improve directory structure
* Cleanup cli/groups/models.py - esp. usage of rich
* Refactoring
* Refactor mocking in cli/test_main.py
* Dedup models commands tests
* Update poetry.lock
* Fix mypy errors
* ruff format litellm/proxy/client/cli
* ruff format tests/litellm/proxy/client/*.py
* Fix timezone issue in test_models_list_table_format
* Add cli/README.md
* Small README.md tweaks
* README.md enhancements
* Add credentials commands
* Add chat commands
* Add http commands
* ruff format litellm/proxy/client/cli
* Fix lint errors in credentials and http commands
* json => json_lib
* test-key => sk-test-key
* Mock HTTP responses so http command tests pass
* Fix mypy error in credentials.py
* bump: version 1.67.5 → 1.67.6
* build: update litellm version
* cli/main.py: show_envvar=True
* Increase test job timeout to 8 minutes
because it looks like maybe the job is getting canceled because it takes
too long with the additional tests?
This probably could be reverted once #10484 is merged, since that speeds
up pytest runs greatly.
* Add keys functionality to library/CLI
* Add info about keys commands to litellm/proxy/client/cli/README.md
* Move Model Information section in CLI README
* Make Model Information a level 4 heading
* Move rich to extras
as suggested by @ishaan-jaff
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
* pin rich=13.7.1
---------
Co-authored-by: Marc Abramowitz <abramowi@adobe.com>
Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
* Simple fix for #9339 - upgrade the underlying library and cache the azure storage client (#9965)
* fix - use constants for caching azure storage client
---------
Co-authored-by: Adrian Lyjak <adrian@chatmeter.com>
* feat(schema.prisma): initial commit adding aggregate table for team spend
allows team spend to be visible at 1m+ logs
* feat(db_spend_update_writer.py): support logging aggregate team spend
allows usage dashboard to work at 1m+ logs
* feat(litellm-proxy-extras/): add new migration file
* fix(db_spend_update_writer.py): fix return type
* build: bump requirements
* fix: fix ruff error
* fix(openai.py): ensure openai file object shows up on logs
* fix(managed_files.py): return unified file id as b64 str
allows retrieve file id to work as expected
* fix(managed_files.py): apply decoded file id transformation
* fix: add unit test for file id + decode logic
* fix: initial commit for litellm_proxy support with CRUD Endpoints
* fix(managed_files.py): support retrieve file operation
* fix(managed_files.py): support for DELETE endpoint for files
* fix(managed_files.py): retrieve file content support
supports retrieve file content api from openai
* fix: fix linting error
* test: update tests
* fix: fix linting error
* feat(managed_files.py): support reading / writing files in DB
* feat(managed_files.py): support deleting file from DB on delete
* test: update testing
* fix(spend_tracking_utils.py): ensure each file create request is logged correctly
* fix(managed_files.py): fix storing / returning managed file object from cache
* fix(files/main.py): pass litellm params to azure route
* test: fix test
* build: add new prisma migration
* build: bump requirements
* test: add more testing
* refactor: cleanup post merge w/ main
* fix: fix code qa errors
* feat(internal_user_endpoints.py): return 'total_tokens' in `/user/daily/analytics`
* test(test_internal_user_endpoints.py): add unit test to assert spend metrics and dailyspend metadata always report the same fields
* build(schema.prisma): record success + failure calls to daily user table
allows understanding why model requests might exceed provider requests (e.g. user hit rate limit error)
* fix(internal_user_endpoints.py): report success / failure requests in API
* fix(proxy/utils.py): default to success
status can be missing or none at times for successful requests
* feat(new_usage.tsx): show success/failure calls on UI
* style(new_usage.tsx): ui cleanup
* fix: fix linting error
* fix: fix linting error
* feat(litellm-proxy-extras/): add new migration files
* build(README.md): initial commit adding a separate folder for additional proxy files. Meant to reduce size of core package
* build(litellm-proxy-extras/): new pip package for storing migration files
allows litellm proxy to use migration files, without adding them to core repo
* build(litellm-proxy-extras/): cleanup pyproject.toml
* build: move prisma migration files inside new proxy extras package
* build(run_migration.py): update script to write to correct folder
* build(proxy_cli.py): load in migration files from litellm-proxy-extras
Closes https://github.com/BerriAI/litellm/issues/9558
* build: add MIT license to litellm-proxy-extras
* test: update test
* fix: fix schema
* bump: version 0.1.0 → 0.1.1
* build(publish-proxy-extras.sh): add script for publishing new proxy-extras version
* build(liccheck.ini): add litellm-proxy-extras to authorized packages
* fix(litellm-proxy-extras/utils.py): move prisma migrate logic inside extra proxy pkg
easier since migrations folder already there
* build(pre-commit-config.yaml): add litellm_proxy_extras to ci tests
* docs(config_settings.md): document new env var
* build(pyproject.toml): bump relevant files when litellm-proxy-extras version changed
* build(pre-commit-config.yaml): run poetry check on litellm-proxy-extras as well
* Adding VertexAI Claude 3.7 Sonnet (#8774)
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
* build(model_prices_and_context_window.json): add anthropic 3-7 models on vertex ai and bedrock
* Support video_url (#8743)
* Support video_url
Support VLMs that works with video.
Example implemenation in vllm: https://github.com/vllm-project/vllm/pull/10020
* llms openai.py: Add ChatCompletionVideoObject
Add data structures to support `video_url` in chat completion
* test test_completion.py: add test for video_url
* Arize Phoenix - ensure correct endpoint/protocol are used; and default to phoenix cloud (#8750)
* minor fixes to default to http and to ensure that the correct endpoint is used
* Update test_arize_phoenix.py
* prioritize http over grpc
* update sentry_sdk to latest version (#8588)
* Add anthropic thinking + reasoning content support (#8778)
* feat(anthropic/chat/transformation.py): add anthropic thinking param support
* feat(anthropic/chat/transformation.py): support returning thinking content for anthropic on streaming responses
* feat(anthropic/chat/transformation.py): return list of thinking blocks (include block signature)
allows usage in tool call responses
* fix(types/utils.py): extract and map reasoning_content from anthropic as content str
* test: add testing to ensure thinking_blocks are returned at the root
* fix(anthropic/chat/handler.py): return thinking blocks on streaming - include signature
* feat(factory.py): handle anthropic thinking blocks translation if in assistant response
* test: handle openai internal instability
* test: handle openai audio instability
* ci: pin anthropic dep
* test: handle openai audio instability
* fix: fix linting error
* refactor(anthropic/chat/transformation.py): refactor function to remain <50 LOC
* fix: fix linting error
* fix: fix linting error
* fix: fix linting error
* fix: fix linting error
* test: handle index error
* bump: version 1.61.15 → 1.61.16
---------
Co-authored-by: Emerson Gomes <emerson.gomes@gmail.com>
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
Co-authored-by: Pang Wu <104795337+pang-wu@users.noreply.github.com>
Co-authored-by: Nate Mar <67926244+nate-mar@users.noreply.github.com>
Co-authored-by: stephaneminisini <stephane.minisini@gmail.com>
