Marc Abramowitz
ddf77656fb
Allow non-admins to use OpenAI routes
...
The problem we were having is non-admin users trying to use
`/engines/{model}/chat/completions` were getting an HTTP 401 error.
```shell
$ curl -sSL 'http://localhost:4000/engines/gpt-35-turbo-0125/chat/completions ' \
--header "Authorization: Bearer ${LITELLM_KEY}" \
--header 'Content-Type: application/json' \
--data ' {
"model": "gpt-35-turbo-0125",
"messages": [
{
"role": "user",
"content": "Write a poem about LiteLLM"
}
]
}' \
| jq '.'
{
"error": {
"message": "Authentication Error, Only proxy admin can be used to generate, delete, update info for new keys/users/teams. Route=/engines/gpt-35-turbo-0125/chat/completions. Your role=unknown. Your user_id=someone@company.com",
"type": "auth_error",
"param": "None",
"code": 401
}
}
```
This seems to be related to code in `user_api_key_auth` that checks that the URL
matches a list of routes that are allowed for non-admin users, where the list of
routes is in `LiteLLMRoutes.openai_routes.value`. The problem is that the route
`/engines/{model}/chat/completions` is not in that list and furthermore, that
wouldn't even work if it were, because the comparison is done with
`request.url.path` and that will have the actual model name in it (e.g.:
`gpt-35-turbo-0125`), rather than `{model}`.
I added a new list `LiteLLMRoutes.openai_route_names` and added the route
**names** to that list. Then I added a check in `user_api_key_auth` to see if
the route name is in the list of route names.
2024-05-15 17:36:11 -07:00
Ishaan Jaff
6d8ea641ec
(ci/cd) fix test_content_policy_exception_azure
2024-05-15 14:47:39 -07:00
Ishaan Jaff
371043d683
fix - test mistral/large _parallel_function_call
2024-05-15 14:31:00 -07:00
Ishaan Jaff
3e831b4e1a
fix debug logs on router test
2024-05-15 14:28:17 -07:00
Ishaan Jaff
fdf7a4d8c8
fix - test_lowest_latency_routing_first_pick
2024-05-15 14:24:13 -07:00
Ishaan Jaff
c00d232900
Merge pull request #3655 from BerriAI/litellm_only_show_debug_info_server_side
...
Fix Proxy Server - only show API base, Model server log exceptions, not on client side
2024-05-15 14:20:14 -07:00
Ishaan Jaff
1a11b151b7
fix - show litellm debug info in errors
2024-05-15 14:18:12 -07:00
Ishaan Jaff
f6e342303e
show debug info on slack alerts
2024-05-15 13:23:44 -07:00
Krrish Dholakia
d9ad7c6218
fix(router.py): fix validation error for default fallback
2024-05-15 13:23:00 -07:00
Ishaan Jaff
59e18f23e0
fix - show litellm_debug_info
2024-05-15 13:07:04 -07:00
Ishaan Jaff
74f093bb4b
Merge pull request #3653 from BerriAI/litellm_fix_text_completions
...
[Fix] - Alerting on `/completions` - don't raise hanging request alert for /completions
2024-05-15 11:28:55 -07:00
Krrish Dholakia
dba713ea43
fix(router.py): add validation for how router fallbacks are setup
...
prevent user errors
2024-05-15 10:44:16 -07:00
Ishaan Jaff
efca96baf8
Merge pull request #3647 from paul-gauthier/openrouter-gpt-4o
...
cloned gpt-4o models into openrouter/openai in costs&context.json
2024-05-15 10:33:19 -07:00
Ishaan Jaff
2e81347607
fix - don't raise hanging request alert for /completions
2024-05-15 10:27:02 -07:00
Ishaan Jaff
5177e4408e
Merge pull request #3651 from BerriAI/litellm_improve_load_balancing
...
[Feat] Proxy + router - don't cooldown on 4XX error that are not 429, 408, 401
2024-05-15 10:24:34 -07:00
Ishaan Jaff
f17f0a09d8
feat - router use _is_cooldown_required
2024-05-15 10:03:55 -07:00
Krrish Dholakia
f5d73547c7
fix(lowest_latency.py): allow ttl to be a float
2024-05-15 09:59:21 -07:00
Ishaan Jaff
ae80148c12
test - router cooldowns
2024-05-15 09:43:30 -07:00
Ishaan Jaff
52f8c39bbf
feat - don't cooldown deployment on BadRequestError
2024-05-15 09:03:27 -07:00
Krrish Dholakia
f43da3597d
test: fix test
2024-05-15 08:51:40 -07:00
Krrish Dholakia
fffb2427f3
fix(types/init.py): don't import openai assistants types by default
2024-05-15 08:50:31 -07:00
Krrish Dholakia
51a02de4cf
refactor(proxy_server.py): update doc string for /user/update
2024-05-15 08:25:14 -07:00
Krrish Dholakia
1840919ebd
fix(main.py): testing fix
2024-05-15 08:23:00 -07:00
Krrish Dholakia
1a3b001432
docs(langfuse_integration.md): cleanup docs
2024-05-15 07:37:04 -07:00
Krrish Dholakia
8117af664c
fix(huggingface_restapi.py): fix task extraction from model name
2024-05-15 07:28:19 -07:00
Krrish Dholakia
900bb9aba8
test(test_token_counter.py): fix load test
2024-05-15 07:12:43 -07:00
Paul Gauthier
e0152c0b61
cloned gpt-4o models into openrouter/openai
2024-05-15 06:20:51 -07:00
Krrish Dholakia
4ff0703a31
fix(slack_alerting.py): fix timezone utc issue
2024-05-14 22:54:33 -07:00
Krrish Dholakia
b06f989871
refactor(main.py): trigger new build
2024-05-14 22:46:44 -07:00
Krrish Dholakia
e0c1fe91f5
test(test_end_users.py): fix end user test
2024-05-14 22:34:26 -07:00
Krrish Dholakia
8f3bf584be
docs(vertex.md): add gemini 1.5 flash to vertex docs
2024-05-14 22:26:56 -07:00
Krrish Dholakia
83ba819602
bump: version 1.37.9 → 1.37.10
2024-05-14 22:17:52 -07:00
Krrish Dholakia
3b5c06747d
refactor(main.py): trigger new build
2024-05-14 22:17:40 -07:00
Krrish Dholakia
54587db402
fix(alerting.py): fix datetime comparison logic
2024-05-14 22:10:09 -07:00
Ishaan Jaff
0bac40b0f2
ci/cd run again
2024-05-14 21:53:14 -07:00
Ishaan Jaff
6290de36df
(ci/cd) run again
2024-05-14 21:39:09 -07:00
Krrish Dholakia
73b6b5e804
test(test_token_counter.py): fix token counting test
2024-05-14 21:35:28 -07:00
Ishaan Jaff
faa58c7938
(ci/cd) run again
2024-05-14 20:45:07 -07:00
Ishaan Jaff
e7af8d61cd
fix check_request_disconnected = None case
2024-05-14 20:38:32 -07:00
Ishaan Jaff
6d1ae5b9c4
(ci/cd) run again
2024-05-14 20:18:12 -07:00
Ishaan Jaff
aaea02dee8
Merge pull request #3640 from BerriAI/litellm_fix_client_side_disconnecting_reqs
...
[Feat] Proxy - cancel tasks when fast api request is cancelled
2024-05-14 20:14:42 -07:00
Ishaan Jaff
4466982507
feat - cancel tasks when fast api request is cancelled
2024-05-14 19:58:51 -07:00
Krrish Dholakia
0262c480be
refactor(main.py): trigger new build
2024-05-14 19:52:23 -07:00
Krrish Dholakia
9eee2f3889
docs(prod.md): add 'disable load_dotenv' tutorial to docs
2024-05-14 19:13:22 -07:00
Krrish Dholakia
1ab4974773
fix: disable 'load_dotenv' for prod environments
2024-05-14 19:09:36 -07:00
Krrish Dholakia
298fd9b25c
fix(main.py): ignore model_config param
2024-05-14 19:03:17 -07:00
Krrish Dholakia
6de358cf21
build(model_prices_and_context_window.json): support new gemini 1.5 preview models
2024-05-14 18:40:14 -07:00
Krrish Dholakia
a1dd341ca1
fix(utils.py): default claude-3 to tiktoken (0.8s faster than hf tokenizer)
2024-05-14 18:37:14 -07:00
Krish Dholakia
46a81524ab
Merge pull request #3637 from BerriAI/revert-3444-main
...
Revert "Logfire Integration"
2024-05-14 17:39:34 -07:00
Krish Dholakia
b04a8d878a
Revert "Logfire Integration"
2024-05-14 17:38:47 -07:00
