From 68e668df139cc35d486175865cbd8fdc2698d075 Mon Sep 17 00:00:00 2001
From: ishaan-jaff <ishaanjaffer0324@gmail.com>
Date: Wed, 8 Nov 2023 17:21:12 -0800
Subject: [PATCH] (docs) lm harness

---
 .../docs/tutorials/lm_evaluation_harness.md   | 25 +++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/docs/my-website/docs/tutorials/lm_evaluation_harness.md b/docs/my-website/docs/tutorials/lm_evaluation_harness.md
index 3cb63d0087f..a4488076e4b 100644
--- a/docs/my-website/docs/tutorials/lm_evaluation_harness.md
+++ b/docs/my-website/docs/tutorials/lm_evaluation_harness.md
@@ -41,6 +41,31 @@ python3 -m lm_eval \
 
 ```
 
+## FLASK - Fine-grained Language Model Evaluation Based on Alignment Skill Sets
+Use litellm to evaluate any LLM on FLASK https://github.com/kaistAI/FLASK 
+
+**Step 1: Start the local proxy**
+```shell
+$ litellm --model huggingface/bigcode/starcoder
+```
+
+**Step 2: Set OpenAI API Base & Key**
+```shell
+$ export OPENAI_API_BASE=http://0.0.0.0:8000
+```
+
+**Step 3 Run with FLASK** 
+
+```shell
+cd FLASK
+cd gpt_review
+```
+
+Run the eval 
+``shell
+python gpt4_eval.py -q '../evaluation_set/flask_evaluation.jsonl'
+```
+
 ## Debugging 
 
 ### Making a test request to your proxy