From 37720d51b0fdfa91cf842fc7960ebec786d15057 Mon Sep 17 00:00:00 2001 From: Anno Yanzhe Chen <54897166+ChenAnno@users.noreply.github.com> Date: Mon, 29 Sep 2025 08:56:44 +0000 Subject: [PATCH] Update README.md --- README.md | 165 +----------------------------------------------------- 1 file changed, 2 insertions(+), 163 deletions(-) diff --git a/README.md b/README.md index 647130e..ae179de 100644 --- a/README.md +++ b/README.md @@ -41,167 +41,6 @@ --- -## ๐ŸŒŸ Overview +This repository contains the source code for the [Code2Video](https://chenanno.github.io/Code2Video/) Github.io website. -

- Overview -

- -**Code2Video** is an **agentic, code-centric framework** that generates high-quality **educational videos** from knowledge points. -Unlike pixel-based text-to-video models, our approach leverages executable **Manim code** to ensure **clarity, coherence, and reproducibility**. - -**Key Features**: -- ๐ŸŽฌ **Code-Centric Paradigm** โ€” executable code as the unified medium for both temporal sequencing and spatial organization of educational videos. -- ๐Ÿค– **Modular Tri-Agent Design** โ€” Planner (storyboard expansion), Coder (debuggable code synthesis), and Critic (layout refinement with anchors) work together for structured generation. -- ๐Ÿ“š **MMMC Benchmark** โ€” the first benchmark for code-driven video generation, covering 117 curated learning topics inspired by 3Blue1Brown, spanning diverse areas. -- ๐Ÿงช **Multi-Dimensional Evaluation** โ€” systematic assessment on efficiency, aesthetics, and end-to-end knowledge transfer. - ---- - -## ๐Ÿš€ How to Create -- Code2Video - -

- Approach -

- -### 1. Requirements - -```bash -pip install -r requirements.txt -```` - -### 2. Configure LLM API Keys - -Fill in your **API credentials** in `gpt_config.json`. - -* **LLM API**: - * Required for Planner & Coder. - * Best Manim code quality achieved with **Claude-4-Opus**. -* **VLM API**: - * Required for Planner Critic. - * For layout and aesthetics optimization, provide **Gemini API key**. - * Best quality achieved with **gemini-2.5-pro-preview-05-06**. - -* **Visual Assets API**: - - * To enrich videos with icons, set `ICONFINDER_API_KEY` from [IconFinder](https://www.iconfinder.com/account/applications). - -### 3. Run Agents - -We provide two shell scripts for different generation modes: - -#### (a) Full Benchmark Mode - -Script: `run_agent.sh` - -Runs all (or a subset of) learning topics defined in `long_video_topics_list.json`. - -```bash -sh run_agent.sh -``` - -**Important parameters inside `run_agent.sh`:** - -* `API`: specify which LLM to use. -* `FOLDER_PREFIX`: name prefix for saving output folders (e.g., `TEST-LIST`). -* `MAX_CONCEPTS`: number of concepts to include (`-1` means all). -* `PARALLEL_GROUP_NUM`: number of groups to run in parallel. - ---- - -#### (b) Single Knowledge Point Mode - -Script: `run_agent_single.sh` - -Generates a video from a single **knowledge point** specified in the script. - -```bash -sh run_agent_single.sh --knowledge_point "Linear transformations and matrices" -``` - -**Important parameters inside `run_agent_single.sh`:** - -* `API`: specify which LLM to use. -* `FOLDER_PREFIX`: output folder prefix (e.g., `TEST-single`). -* `KNOWLEDGE_POINT`: target concept, e.g. `"Linear transformations and matrices"`. - ---- - -### 4. Project Organization - -A suggested directory structure: - -``` -Code2Video/ -โ”‚โ”€โ”€ agent.py -โ”‚โ”€โ”€ run_agent.sh -โ”‚โ”€โ”€ run_agent_single.sh -โ”‚โ”€โ”€ api_config.json -โ”‚โ”€โ”€ ... -โ”‚ -โ”œโ”€โ”€ assets/ -โ”‚ โ”œโ”€โ”€ icons/ # downloaded visual assets cache via IconFinder API -โ”‚ โ””โ”€โ”€ reference/ # reference images -โ”‚ -โ”œโ”€โ”€ json_files/ # JSON-based topic lists & metadata -โ”œโ”€โ”€ prompts/ # prompt templates for LLM calls -โ”œโ”€โ”€ CASES/ # generated cases, organized by FOLDER_PREFIX -โ”‚ โ””โ”€โ”€ TEST-LIST/ # example multi-topic generation results -โ”‚ โ””โ”€โ”€ TEST-single/ # example single-topic generation results -``` - - ---- - -## ๐Ÿ“Š How to Evaluate -- MMMC - -We evaluate along **three complementary dimensions**: - -1. **Knowledge Transfer (TeachQuiz)** - - ```bash - python3 eval_TQ.py - ``` - -2. **Aesthetic & Structural Quality (AES)** - - ```bash - python3 eval_AES.py - ``` - -3. **Efficiency Metrics (During Creating)** - - * Token usage - * Execution time - - -๐Ÿ‘‰ More data and evaluation scripts are available at: -[HuggingFace: MMMC Benchmark](https://huggingface.co/datasets/YanzheChen/MMMC) - ---- - -## ๐Ÿ™ Acknowledgements - -* Video data is sourced from the **[3Blue1Brown official lessons](https://www.3blue1brown.com/#lessons)**. - These videos represent the **upper bound of clarity and aesthetics** in educational video design and inform our evaluation metrics. -* We thank all the **Show Lab @ NUS** members for support! -* This project builds upon open-source contributions from **Manim Community** and the broader AI research ecosystem. -* High-quality visual assets (icons) are provided by **[IconFinder](https://www.iconfinder.com/)** and **[Icons8](https://icons8.com/icons)**, which were used to enrich the educational videos. - - ---- - - +For more details and demos, please visit our [project page](https://github.com/showlab/Code2Video).