Krrish Dholakia
6a2ddc2791
docs(routing.md): add docs on lowest latency routing buffer
2024-04-30 22:41:50 -07:00
Krrish Dholakia
cfc1eeb3c3
test(test_router_fallbacks.py): rename test to run earlier
2024-04-30 22:04:20 -07:00
Krrish Dholakia
e506e71cb9
fix(test_router_fallbacks.py): reduce test rpm
2024-04-30 22:00:48 -07:00
Krish Dholakia
9f55a99e98
Merge pull request #3376 from BerriAI/litellm_routing_logic
...
fix(router.py): unify retry timeout logic across sync + async function_with_retries
2024-04-30 19:58:45 -07:00
Krrish Dholakia
e2eddac406
test(test_ratelimit.py): fix test to send below rpm
2024-04-30 19:35:21 -07:00
Krrish Dholakia
4761345311
fix(main.py): fix mock completion response
2024-04-30 19:30:18 -07:00
Krrish Dholakia
bc5c9d7da9
fix(test_router_fallbacks.py): fix tests
2024-04-30 18:48:39 -07:00
Ishaan Jaff
5da931f297
ui - clean up table
2024-04-30 18:48:32 -07:00
Ishaan Jaff
0c464f7f61
fix - viewing model metrics
2024-04-30 18:26:14 -07:00
Ishaan Jaff
ace3b02d97
ui - model analytics show failed requests %
2024-04-30 18:23:31 -07:00
Krish Dholakia
47017f5bc4
Merge pull request #3377 from BerriAI/revert-3374-abramowi/disambiguate-invalid-model-name-errors
...
Revert "Disambiguate invalid model name errors"
2024-04-30 17:58:00 -07:00
Krish Dholakia
82095731c1
Revert "Disambiguate invalid model name errors"
2024-04-30 17:57:52 -07:00
Krish Dholakia
fe3496961a
Merge pull request #3374 from msabramo/abramowi/disambiguate-invalid-model-name-errors
...
Disambiguate invalid model name errors
2024-04-30 17:57:29 -07:00
Krrish Dholakia
1baad80c7d
fix(router.py): cooldown deployments, for 401 errors
2024-04-30 17:54:00 -07:00
Ishaan Jaff
f2849d0641
fix - track litellm_model_name in LiteLLM_ErrorLogs
2024-04-30 17:31:40 -07:00
Ishaan Jaff
8a1a043801
backend - show model latency per token
2024-04-30 17:23:36 -07:00
Ishaan Jaff
8177ef5ec0
ui - show model latency / token
2024-04-30 17:23:27 -07:00
Ishaan Jaff
ce1817380e
feat ui - modelExceptionsCall
2024-04-30 16:56:45 -07:00
Ishaan Jaff
a2a8fef8f4
fix passing starttime and endtime to model/exceptions
2024-04-30 16:53:53 -07:00
Krrish Dholakia
8ee51a96f4
test(test_router_debug_logs.py): fix retry logic
2024-04-30 16:42:10 -07:00
Krish Dholakia
ce9ede6110
Merge pull request #3370 from BerriAI/litellm_latency_buffer
...
fix(lowest_latency.py): allow setting a buffer for getting values within a certain latency threshold
2024-04-30 16:01:47 -07:00
Ishaan Jaff
26a5d85869
fix - backend return exceptions
2024-04-30 15:41:16 -07:00
Krrish Dholakia
0267069c6a
fix(router.py): return routing args as dict
2024-04-30 15:39:14 -07:00
Krrish Dholakia
668a5353ee
fix(router.py): fix linting issue
2024-04-30 15:35:16 -07:00
Krrish Dholakia
6a2b4bcab8
fix(router.py): only check /v1 for azure ai studio models
...
Fixes https://github.com/BerriAI/litellm/issues/3346
2024-04-30 15:29:50 -07:00
Krrish Dholakia
87ff26ff27
fix(router.py): unify retry timeout logic across sync + async function_with_retries
2024-04-30 15:23:19 -07:00
Ishaan Jaff
49f83ce204
ui - show models analytics
2024-04-30 15:16:25 -07:00
Marc Abramowitz
dd166680d1
Move chat_completions before completions
...
so that the `chat_completions` route is defined before the `completions` route.
This is necessary because the `chat_completions` route is more
specific than the `completions` route, and the order of route definitions
matters in FastAPI.
Without this, doing a request to
`/openai/deployments/{model_in_url}/chat/completions` might trigger
`completions` being called (with `model` set to `{model_in_url}/chat` instead of
`chat_completions` getting called, which is the correct function.
Fixes: GH-3372
2024-04-30 15:07:10 -07:00
Marc Abramowitz
c794c09679
Disambiguate invalid model name errors
...
because that error can be thrown in several different places, so
knowing the function it's being thrown from can be very useul for debugging.
2024-04-30 14:34:54 -07:00
Ishaan Jaff
b9a0a13516
ui - show model usage
2024-04-30 14:28:19 -07:00
Ishaan Jaff
1f4f1c6f70
stash /model/metrics/exceptions endpoints
2024-04-30 14:19:23 -07:00
Krrish Dholakia
3cc82f558e
fix(utils.py): add exception mapping for gemini error
2024-04-30 14:17:10 -07:00
Krrish Dholakia
638477a023
test: fix test
2024-04-30 14:06:09 -07:00
Krrish Dholakia
df43012bdd
test(test_router_fallbacks.py): use rpm test -> more stable
2024-04-30 14:01:01 -07:00
Krrish Dholakia
470dbf9745
build(ui): allow user to modify 'lowest_latency_buffer' on UI
2024-04-30 13:54:43 -07:00
Ishaan Jaff
0b0be700fc
Merge pull request #3371 from BerriAI/litellm_log_errors_db
...
[Feat] Write LLM Exception to LiteLLM Proxy DB
2024-04-30 13:36:41 -07:00
Ishaan Jaff
4b8fda4ac4
log startTime and EndTime for exceptions
2024-04-30 13:34:14 -07:00
Ishaan Jaff
3aad034a8b
feat log request kwargs in error logs
2024-04-30 13:28:26 -07:00
Ishaan Jaff
ad5fddef15
fix log model_group
2024-04-30 13:11:09 -07:00
Ishaan Jaff
ee2a2ce559
fix - log api_base in errors
2024-04-30 13:02:42 -07:00
Ishaan Jaff
06804bc70a
fix - working exception writing
2024-04-30 12:48:17 -07:00
Ishaan Jaff
22725bd44d
fix types for errorLog
2024-04-30 12:31:33 -07:00
Ishaan Jaff
c7f979e0fe
fix schema error logs
2024-04-30 12:31:19 -07:00
Ishaan Jaff
ac1cabe963
add LiteLLM_ErrorLogs to types
2024-04-30 12:16:03 -07:00
Krrish Dholakia
fee488bd53
test(test_image_generation.py): fix test
2024-04-30 12:15:37 -07:00
Krrish Dholakia
285a3733a9
test(test_image_generation.py): fix test
2024-04-30 12:14:29 -07:00
Krrish Dholakia
90cdfef1c1
fix(lowest_latency.py): allow setting a buffer for getting values within a certain latency threshold
...
if an endpoint is slow - it's completion time might not be updated till the call is completed. This prevents us from overloading those endpoints, in a simple way.
2024-04-30 12:00:26 -07:00
Ishaan Jaff
d6f7fa7f4e
v0 prisma schema
2024-04-30 11:42:17 -07:00
Krrish Dholakia
398d503590
build(model_prices_and_context_window.json): add bedrock llama3 pricing
2024-04-30 11:36:29 -07:00
Krrish Dholakia
00d1440d0d
test(test_image_generation.py): change img model for test - bedrock EOL
2024-04-30 08:55:40 -07:00
