Merge pull request #278 from sergiopaniego/llm-grpo-trl

Added new `Post-training an LLM using GRPO with TRL` recipe 🧑‍🍳️
huggingface · Feb 5, 2025 · 063ca3e · 063ca3e
2 parents 1c90308 + 0e45d15
commit 063ca3e
Show file tree

Hide file tree

Showing 3 changed files with 895 additions and 3 deletions.
diff --git a/notebooks/en/_toctree.yml b/notebooks/en/_toctree.yml
@@ -74,7 +74,9 @@
           title: Phoenix Observability Dashboard on HF Spaces
         - local: search_and_learn
           title: Scaling Test-Time Compute for Longer Thinking in LLMs
-
+        - local: fine_tuning_llm_grpo_trl
+          title: Post training an LLM for reasoning with GRPO in TRL
+
     - title: Computer Vision Recipes
       isExpanded: false
       sections:

diff --git a/notebooks/en/fine_tuning_llm_grpo_trl.ipynb b/notebooks/en/fine_tuning_llm_grpo_trl.ipynb
diff --git a/notebooks/en/index.md b/notebooks/en/index.md
@@ -7,12 +7,11 @@ applications and solving various machine learning tasks using open-source tools
 
 Check out the recently added notebooks:
 
+- [Post training an LLM for reasoning with GRPO in TRL](fine_tuning_llm_grpo_trl)
 - [Evaluating AI Search Engines with `judges` - the open-source library for LLM-as-a-judge evaluators](llm_judge_evaluating_ai_search_engines_with_judges_library)
 - [Structured Generation from Images or Documents Using Vision Language Models](structured_generation_vision_language_models)
 - [Vector Search on Hugging Face with the Hub as Backend](vector_search_with_hub_as_backend)
 - [Multi-Agent Order Management System with MongoDB](mongodb_smolagents_multi_micro_agents)
-- [Scaling Test-Time Compute for Longer Thinking in LLMs](search_and_learn)
-- [Signature-Aware Model Serving from MLflow with Ray Serve](mlflow_ray_serve)
 
 You can also check out the notebooks in the cookbook's [GitHub repo](https://github.com/huggingface/cookbook).