From b2402e2abbba377deac93360756720aa409c3327 Mon Sep 17 00:00:00 2001 From: davidberenstein1957 Date: Fri, 24 Jan 2025 12:47:05 +0100 Subject: [PATCH] Refactor Jupyter notebook for structured generation using Vision Language Models - Updated the description to clarify the use of the SmolVLM-Instruct model and its integration with the HuggingFace Transformers and Outlines libraries. - Added a reference to an outlines tutorial for better guidance. - Modified the installation command to remove the Gradio library, streamlining the dependencies. --- .../en/structured_generation_vision_languag_models.ipynb | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/notebooks/en/structured_generation_vision_languag_models.ipynb b/notebooks/en/structured_generation_vision_languag_models.ipynb index bd4a0589..3e3b331d 100644 --- a/notebooks/en/structured_generation_vision_languag_models.ipynb +++ b/notebooks/en/structured_generation_vision_languag_models.ipynb @@ -6,7 +6,8 @@ "source": [ "# Structured Generation from Documents Using Vision Language Models\n", "\n", - "We will be using the SmolVLM-500M-Instruct model from HuggingFaceTB to extract structured information from documents. We will do so using the HuggingFace Transformers library and the Outlines library, which facilitates structured generation based on limiting token sampling probabilities. We will also use the Gradio library to create a simple UI for uploading and extracting structured information from documents.\n", + "We will be using the SmolVLM-Instruct model from HuggingFaceTB to extract structured information from documents We will run the VLM using the HuggingFace Transformers library and the Outlines library, which facilitates structured generation based on limiting token sampling probabilities. \n", + "This approach is based on a [outlines tutorial](https://dottxt-ai.github.io/outlines/latest/cookbook/atomic_caption/) library.\n", "\n", "## Dependencies and imports\n", "\n", @@ -19,7 +20,7 @@ "metadata": {}, "outputs": [], "source": [ - "%pip install accelerate outlines transformers torch flash-attn outlines datasets sentencepiece gradio" + "%pip install accelerate outlines transformers torch flash-attn outlines datasets sentencepiece" ] }, {