Update a bunch of tests and docs, preparing for 0.1.0

Author: thmsmlr

.formatter.exs

@@ -6,6 +6,7 @@
     "{mix,.formatter}.exs",
     "{config,lib,test}/**/*.{ex,exs}",
     "pages/cookbook/**/*.{ex,exs}",
-    "examples/**/*.{ex,exs}"
+    "examples/**/*.{ex,exs}",
+    "scripts/**/*.{ex,exs}"
   ]
 ]

CHANGELOG.md

@@ -2,6 +2,29 @@
 
 ## [Unreleased](https://github.com/thmsmlr/instructor_ex/compare/v0.0.5..main)
 
+### Added
+- **New Adapters**: Anthropic, Gemini, Groq, Ollama, and VLLM. Each of these provides specialized support for their respective LLM APIs.
+- **`:json_schema` Mode**: The OpenAI adapter and others now support a `:json_schema` mode for more structured JSON outputs.
+- **`Instructor.Extras.ChainOfThought`**: A new module to guide multi-step reasoning processes with partial returns and final answers.
+- **Enhanced Streaming**: More robust partial/array streaming pipelines, plus improved SSE-based parsing for streamed responses.
+- **Re-ask/Follow-up Logic**: Adapters can now handle re-asking the LLM to correct invalid JSON responses when `max_retries` is set.
+
+### Changed
+- **OpenAI Adapter Refactor**: A major internal refactor for more flexible streaming modes, additional “response format” options, and better error handling.
+- **Ecto Dependency**: Updated from `3.11` to `3.12`. 
+- **Req Dependency**: Now supports `~> 0.5` or `~> 1.0`.
+
+### Deprecated
+- **Schema Documentation via `@doc`**: Schemas using `@doc` to send instructions to the LLM will now emit a warning. Please migrate to `@llm_doc` via `use Instructor`.
+
+### Breaking Changes
+- Some adapter configurations now require specifying an `:api_path` or `:auth_mode`. Verify your adapter config matches the new format.
+- The OpenAI adapter’s `:json_schema` mode strips unsupported fields (e.g., `format`, `pattern`) from schemas before sending them to the LLM.
+
+### Fixed
+- Various improvements to JSON parsing and streaming handling, including better handling of partial/invalid responses.
+
+
 ## [v0.0.5](https://github.com/thmsmlr/instructor_ex/compare/v0.0.4..v0.0.5)
 
 ### Added

README.md

@@ -13,26 +13,49 @@ _Structured, Ecto outputs with OpenAI (and OSS LLMs)_
 
 <!-- Docs -->
 
- Structured prompting for LLMs. Instructor is a spiritual port of the great [Instructor Python Library](https://github.com/jxnl/instructor) by [@jxnlco](https://twitter.com/jxnlco), check out his [talk on YouTube](https://www.youtube.com/watch?v=yj-wSRJwrrc).
- 
- The Instructor library is useful for coaxing an LLM to return JSON that maps to an Ecto schema that you provide, rather than the default unstructured text output. If you define your own validation logic, Instructor can automatically retry prompts when validation fails (returning natural language error messages to the LLM, to guide it when making corrections).
+Check out our [Quickstart Guide](https://hexdocs.pm/instructor/quickstart.html) to get up and running with Instructor in minutes.
 
-Instructor is designed to be used with the [OpenAI API](https://platform.openai.com/docs/api-reference/chat-completions/create) by default, but it also works with [llama.cpp](https://github.com/ggerganov/llama.cpp) and [Bumblebee](https://github.com/elixir-nx/bumblebee) (Coming Soon!) by using an extendable adapter behavior.
+Instructor provides structured prompting for LLMs. It is a spiritual port of the great [Instructor Python Library](https://github.com/jxnl/instructor) by [@jxnlco](https://twitter.com/jxnlco).
+
+Instructor allows you to get structured output out of an LLM using Ecto.  
+You don't have to define any JSON schemas.
+You can just use Ecto as you've always used it.  
+And since it's just ecto, you can provide change set validations that you can use to ensure that what you're getting back from the LLM is not only properly structured, but semantically correct.
+
+To learn more about the philosophy behind Instructor and its motivations, check out this Elixir Denver Meetup talk:
+
+<div style="text-align: center">
+
+[![Instructor: Structured prompting for LLMs](assets/youtube-thumbnail.png)](https://www.youtube.com/watch?v=RABXu7zqnT0)
+
+</div>
+
+While Instructor is designed to be used with OpenAI, it also supports every major AI lab and open source LLM inference server:
+
+- OpenAI
+- Anthropic
+- Groq
+- Ollama
+- Gemini
+- vLLM
+- llama.cpp
 
 At its simplest, usage is pretty straightforward: 
 
-1. Create an ecto schema, with a `@doc` string that explains the schema definition to the LLM. 
-2. Define a `validate_changeset/1` function on the schema, and use the `Instructor.Validator` macro in order for Instructor to know about it.
+1. Create an ecto schema, with a `@llm_doc` string that explains the schema definition to the LLM. 
+2. Define a `validate_changeset/1` function on the schema, and use the `use Instructor` macro in order for Instructor to know about it.
 2. Make a call to `Instructor.chat_completion/1` with an instruction for the LLM to execute.
 
 You can use the `max_retries` parameter to automatically, iteratively go back and forth with the LLM to try fixing validation errorswhen they occur.
 
 ```elixir
+Mix.install([:instructor])
+
 defmodule SpamPrediction do
   use Ecto.Schema
-  use Instructor.Validator
+  use Validator
 
-  @doc """
+  @llm_doc """
   ## Field Descriptions:
   - class: Whether or not the email is spam.
   - reason: A short, less than 10 word rationalization for the classification.
@@ -57,7 +80,7 @@ end
 
 is_spam? = fn text ->
   Instructor.chat_completion(
-    model: "gpt-3.5-turbo",
+    model: "gpt-4o-mini",
     response_model: SpamPrediction,
     max_retries: 3,
     messages: [
@@ -69,9 +92,10 @@ is_spam? = fn text ->
         They sell all types of clothing.
 
         Classify the following email: 
-        ```
-        #{text}
-        ```
+
+        <email>
+          #{text}
+        </email>
         """
       }
     ]
@@ -83,17 +107,6 @@ is_spam?.("Hello I am a Nigerian prince and I would like to send you money")
 # => {:ok, %SpamPrediction{class: :spam, reason: "Nigerian prince email scam", score: 0.98}}
 ```
 
-Check out our [Quickstart Guide](https://hexdocs.pm/instructor/quickstart.html) for more code snippets that you can run locally (in Livebook). Or, to get a better idea of the thinking behind Instructor, read more about our [Philosophy & Motivations](https://hexdocs.pm/instructor/philosophy.html).
-
-Optionally, you can also customize the your llama.cpp calls (with defaults shown):
-```elixir
-llamacpp
-config :instructor, adapter: Instructor.Adapters.Llamacpp
-config :instructor, :llamacpp,
-    chat_template: :mistral_instruct,
-    api_url: "http://localhost:8080/completion"
-````
-
 <!-- Docs -->
 
 ## Installation
@@ -103,67 +116,7 @@ In your mix.exs,
 ```elixir
 def deps do
   [
-    {:instructor, "~> 0.0.5"}
-  ]
-end
-```
-
-InstructorEx uses [Code.fetch_docs/1](https://hexdocs.pm/elixir/1.16.2/Code.html#fetch_docs/1) to fetch LLM instructions from the Ecto schema specified in `response_model`. If your project is deployed using [releases](https://hexdocs.pm/mix/Mix.Tasks.Release.html), add the following configuration to mix.exs to prevent docs from being stripped from the release:
-
-```elixir
-def project do
-  # ...
-  releases: [
-    myapp: [
-      strip_beams: [keep: ["Docs"]]
-    ]
+    {:instructor, "~> 0.1.0"}
   ]
 end
-```
-
-## TODO
-
-- [ ] Partial Schemaless doesn't work since fields are set to required in Ecto.
-- [x] Groq adapter
-- [ ] @doc gets stripped in release, find a workaround
-- [ ] ChainOfThought doesn't work with max_retries
-- [ ] Logging for Distillation / Finetuning
-- [ ] Add a Bumblebee adapter
-- [ ] Support naked ecto types by auto-wrapping, not just maps of ecto types, do not wrap if we don't need to... Current codepaths are muddled
-- [ ] Optional/Maybe types
-- [ ] Add Livebook Tutorials, include in Hexdocs
-    - [x] Text Classification
-    - [ ] Self Critique
-    - [ ] Image Extracting Tables
-    - [ ] Moderation
-    - [x] Citations
-    - [ ] Knowledge Graph
-    - [ ] Entity Resolution
-    - [ ] Search Queries
-    - [ ] Query Decomposition
-    - [ ] Recursive Schemas
-    - [x] Table Extraction
-    - [x] Action Item and Dependency Mapping
-    - [ ] Multi-File Code Generation
-    - [ ] PII Data Sanitizatiommersed
-- [x] Update hexdocs homepage to include example for tutorial
-
-## Blog Posts
-
-- [ ] Why structured prompting?
-
-    Meditations on new HCI.
-    Finally we have software that can understand text. f(text) -> text.
-    This is great, as it gives us a new domain, but the range is still text.
-    While we can use string interpolation to map Software 1.0 into f(text), the outputs are not interoperable with Software 1.0.
-    Hence why UXs available to us are things like Chatbots as our users have to interpret the output.
-
-    Instructor, structure prompting, gives use f(text) -> ecto_schema.
-    Schemas are the lingua franca of Software 1.0.
-    With Instrutor we can now seamlessly move back and forth between Software 1.0 and Software 2.0.
-
-    Now we can maximally leverage AI...
-
-- [ ] From GPT-4 to zero-cost production - Distilation, local-llms, and the cost structure of AI.
-
-    ... 😘
+```

assets/youtube-thumbnail.png

@@ -3,6 +3,8 @@ defmodule Instructor do
 
   alias Instructor.JSONSchema
 
+  @type stream :: Enumerable.t()
+
   @external_resource "README.md"
 
   [_, readme_docs, _] =
@@ -36,7 +38,7 @@ defmodule Instructor do
   ## Examples
 
       iex> Instructor.chat_completion(
-      ...>   model: "gpt-3.5-turbo",
+      ...>   model: "gpt-4o-mini",
       ...>   response_model: Instructor.Demos.SpamPrediction,
       ...>   messages: [
       ...>     %{
@@ -56,7 +58,7 @@ defmodule Instructor do
   Partial streaming will emit the record multiple times until it's complete.
 
       iex> Instructor.chat_completion(
-      ...>   model: "gpt-3.5-turbo",
+      ...>   model: "gpt-4o-mini",
       ...>   response_model: {:partial, %{name: :string, birth_date: :date}}
       ...>   messages: [
       ...>     %{
@@ -74,7 +76,7 @@ defmodule Instructor do
   and instructor will emit them one at a time as they arrive in complete form and validated.
 
       iex> Instructor.chat_completion(
-      ...>   model: "gpt-3.5-turbo",
+      ...>   model: "gpt-4o-mini",
       ...>   response_model: {:array, %{name: :string, birth_date: :date}}
       ...>   messages: [
       ...>     %{
@@ -94,7 +96,7 @@ defmodule Instructor do
   If there's a validation error, it will return an error tuple with the change set describing the errors.
 
       iex> Instructor.chat_completion(
-      ...>   model: "gpt-3.5-turbo",
+      ...>   model: "gpt-4o-mini",
       ...>   response_model: Instructor.Demos.SpamPrediction,
       ...>   messages: [
       ...>     %{
@@ -118,7 +120,7 @@ defmodule Instructor do
           {:ok, Ecto.Schema.t()}
           | {:error, Ecto.Changeset.t()}
           | {:error, String.t()}
-          | Stream.t()
+          | stream()
   def chat_completion(params, config \\ nil) do
     params =
       params

lib/instructor.ex

@@ -6,9 +6,10 @@ defmodule Instructor.Adapter do
   @type params :: [Keyword.t()]
   @type config :: any()
   @type raw_response :: any()
+  @type stream :: Enumerable.t()
 
   @callback chat_completion(params(), config()) ::
-              Stream.t() | {:ok, raw_response(), String.t()} | {:error, String.t()}
+              stream() | {:ok, raw_response(), String.t()} | {:error, String.t()}
 
   @callback reask_messages(raw_response(), params(), config()) :: [map()]
 end

lib/instructor/adapter.ex

@@ -125,6 +125,40 @@ defmodule Instructor.Adapters.Anthropic do
     args
   end
 
+  @impl true
+  def reask_messages(raw_response, params, _config) do
+    reask_messages_for_mode(params[:mode], raw_response)
+  end
+
+  defp reask_messages_for_mode(:tools, %{
+         "choices" => [
+           %{
+             "message" =>
+               %{
+                 "tool_calls" => [
+                   %{"id" => tool_call_id, "function" => %{"name" => name, "arguments" => args}} =
+                     function
+                 ]
+               } = message
+           }
+         ]
+       }) do
+    [
+      Map.put(message, "content", function |> Jason.encode!())
+      |> Map.new(fn {k, v} -> {String.to_atom(k), v} end),
+      %{
+        role: "tool",
+        tool_call_id: tool_call_id,
+        name: name,
+        content: args
+      }
+    ]
+  end
+
+  defp reask_messages_for_mode(_mode, _raw_response) do
+    []
+  end
+
   defp url(config), do: api_url(config) <> "/v1/messages"
 
   defp api_url(config), do: Keyword.fetch!(config, :api_url)