scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-08 03:08:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 115

fix(proxy_server.py): fix linting errors

Browse source
This commit is contained in:
Krrish Dholakia 2023-12-01 20:40:49 -08:00
parent 72381c3cc2
commit 47730da262
2 changed files with 8 additions and 8 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
docs/my-website/docs/simple_proxy.md
View file

@ -6,9 +6,9 @@ import TabItem from '@theme/TabItem';
LiteLLM Server manages:
* Calling 100+ LLMs [Huggingface/Bedrock/TogetherAI/etc.](#other-supported-models) in the OpenAI `ChatCompletions` & `Completions` format
* Load balancing - between [Multiple Models](#multiple-models---quick-start) + [Deployments of the same model](#multiple-instances-of-1-model) **LiteLLM proxy can handle 1k+ requests/second during load tests**
* Authentication & Spend Tracking [Virtual Keys](#managing-auth---virtual-keys)
* **Unified Interface**: Calling 100+ LLMs [Huggingface/Bedrock/TogetherAI/etc.](#other-supported-models) in the OpenAI `ChatCompletions` & `Completions` format
* **Load Balancing**: between [Multiple Models](#multiple-models---quick-start) + [Deployments of the same model](#multiple-instances-of-1-model) - LiteLLM proxy can handle 1.5k+ requests/second during load tests.
* **Cost tracking**: Authentication & Spend Tracking [Virtual Keys](#managing-auth---virtual-keys)
[**See LiteLLM Proxy code**](https://github.com/BerriAI/litellm/tree/main/litellm/proxy)

10
litellm/proxy/proxy_server.py
View file

@ -955,10 +955,10 @@ async def info_key_fn(key: str = fastapi.Query(..., description="Key in the requ
#### [BETA] - This is a beta endpoint, format might change based on user feedback. - https://github.com/BerriAI/litellm/issues/964
@router.post("/model/new", description="Allows adding new models to the model list in the config.yaml", tags=["model management"], dependencies=[Depends(user_api_key_auth)])
async def add_new_model(model_params: ModelParams):
global llm_router, llm_model_list, general_settings
global llm_router, llm_model_list, general_settings, user_config_file_path
try:
# Load existing config
with open(user_config_file_path, "r") as config_file:
with open(f"{user_config_file_path}", "r") as config_file:
config = yaml.safe_load(config_file)
# Add the new model to the config
@ -968,7 +968,7 @@ async def add_new_model(model_params: ModelParams):
})
# Save the updated config
with open(user_config_file_path, "w") as config_file:
with open(f"{user_config_file_path}", "w") as config_file:
yaml.dump(config, config_file, default_flow_style=False)
# update Router
@ -983,9 +983,9 @@ async def add_new_model(model_params: ModelParams):
#### [BETA] - This is a beta endpoint, format might change based on user feedback. - https://github.com/BerriAI/litellm/issues/933
@router.get("/model/info", description="Provides more info about each model in /models, including config.yaml descriptions (except api key and api base)", tags=["model management"], dependencies=[Depends(user_api_key_auth)])
async def model_info(request: Request):
global llm_model_list, general_settings
global llm_model_list, general_settings, user_config_file_path
# Load existing config
with open(user_config_file_path, "r") as config_file:
with open(f"{user_config_file_path}", "r") as config_file:
config = yaml.safe_load(config_file)
all_models = config['model_list']