Skip to content

Commit

Permalink
Refactor Jupyter notebook for structured generation using Vision Lang…
Browse files Browse the repository at this point in the history
…uage Models

- Updated the description to clarify the use of the SmolVLM-Instruct model and its integration with the HuggingFace Transformers and Outlines libraries.
- Added a reference to an outlines tutorial for better guidance.
- Modified the installation command to remove the Gradio library, streamlining the dependencies.
  • Loading branch information
davidberenstein1957 committed Jan 24, 2025
1 parent 8d35edd commit b2402e2
Showing 1 changed file with 3 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@
"source": [
"# Structured Generation from Documents Using Vision Language Models\n",
"\n",
"We will be using the SmolVLM-500M-Instruct model from HuggingFaceTB to extract structured information from documents. We will do so using the HuggingFace Transformers library and the Outlines library, which facilitates structured generation based on limiting token sampling probabilities. We will also use the Gradio library to create a simple UI for uploading and extracting structured information from documents.\n",
"We will be using the SmolVLM-Instruct model from HuggingFaceTB to extract structured information from documents We will run the VLM using the HuggingFace Transformers library and the Outlines library, which facilitates structured generation based on limiting token sampling probabilities. \n",
"This approach is based on a [outlines tutorial](https://dottxt-ai.github.io/outlines/latest/cookbook/atomic_caption/) library.\n",
"\n",
"## Dependencies and imports\n",
"\n",
Expand All @@ -19,7 +20,7 @@
"metadata": {},
"outputs": [],
"source": [
"%pip install accelerate outlines transformers torch flash-attn outlines datasets sentencepiece gradio"
"%pip install accelerate outlines transformers torch flash-attn outlines datasets sentencepiece"
]
},
{
Expand Down

0 comments on commit b2402e2

Please sign in to comment.