scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-10 22:41:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 82

(docs) add instructions on running with FLASK

Browse source
This commit is contained in:
ishaan-jaff 2023-11-08 17:48:12 -08:00
parent c57ed0a9d7
commit fc45f41818
1 changed files with 30 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

30
docs/my-website/docs/simple_proxy.md
View file

@ -213,6 +213,36 @@ python3 -m lm_eval \
</TabItem>
<TabItem value="flask evals" label="FLASK Evals">
FLASK - Fine-grained Language Model Evaluation
Use litellm to evaluate any LLM on FLASK https://github.com/kaistAI/FLASK
**Step 1: Start the local proxy**
```shell
$ litellm --model huggingface/bigcode/starcoder
```
**Step 2: Set OpenAI API Base & Key**
```shell
$ export OPENAI_API_BASE=http://0.0.0.0:8000
```
**Step 3 Run with FLASK**
```shell
git clone https://github.com/kaistAI/FLASK
```
```shell
cd FLASK/gpt_review
```
Run the eval
```shell
python gpt4_eval.py -q '../evaluation_set/flask_evaluation.jsonl'
```
</TabItem>
<TabItem value="continue-dev" label="ContinueDev">
Continue-Dev brings ChatGPT to VSCode. See how to [install it here](https://continue.dev/docs/quickstart).