Skip to content

About

Type-safe structured output extraction from LLMs (crate: instructors)

Resources

Stars

0 stars

Watchers

0 watching

Forks

 
 

Repository files navigation

instructors

Crates.io docs.rs License Downloads MSRV

English | 简体中文 | 日本語

Type-safe structured output extraction from LLMs. The Rust instructor.

Define a Rust struct → instructors generates the JSON Schema → LLM returns valid JSON → you get a typed value. With automatic validation and retry.

Highlights

  • 6 providers — OpenAI, Anthropic, Gemini, DeepSeek, Together, any OpenAI/Anthropic/Gemini-compatible API
  • Validation + retry — invalid output is fed back to the LLM with error details for automatic correction
  • JSON auto-repair — fixes trailing commas, single quotes, markdown fences before retry, saving tokens and latency
  • Provider fallback — chain multiple providers for automatic failover
  • Streaming — partial JSON tokens as they arrive (OpenAI, Anthropic, Gemini)
  • Batch + concurrent — process hundreds of prompts with configurable concurrency via Semaphore
  • Vision — extract structured data from images (URL or base64)
  • Thinking control — dial Gemini's reasoning depth up or down per client
  • Cost tracking — per-request token count and cost estimation via tiktoken

Why instructors?

Aspect Raw API + serde instructors
Schema enforcement Manual JSON Schema writing Auto-generated from #[derive(JsonSchema)]
Parse failures Crash or silent data loss Automatic retry with error feedback to LLM
Malformed JSON Application error Auto-repaired before deserialization
Multiple providers Rewrite per provider One interface, swap with one line
Validation Manual if/else after parsing .validate() with LLM-aware retry
Cost tracking Count tokens yourself Built-in via tiktoken

Quick Start

use instructors::prelude::*;

#[derive(Debug, Deserialize, JsonSchema)]
struct Contact { name: String, email: Option<String> }

let client = Client::openai("sk-...");
let contact: Contact = client
    .extract("Contact John Doe at john@example.com")
    .await?.value;

Installation

[dependencies]
instructors = "1"

Providers

Provider Constructor Mechanism
OpenAI Client::openai(key) response_format strict JSON Schema
Anthropic Client::anthropic(key) forced tool_use, or native output_config (opt-in)
OpenAI-compatible Client::openai_compatible(key, url) Same as OpenAI (DeepSeek, Together, etc.)
Anthropic-compatible Client::anthropic_compatible(key, url) Same as Anthropic
Google Gemini Client::gemini(key) response_schema structured JSON, optional thinking control
Gemini-compatible Client::gemini_compatible(key, url) Same as Gemini

Default models

When no model is set, each provider uses its current balanced tier:

Provider Default
OpenAI gpt-5.6-terra
Anthropic claude-sonnet-5
Gemini gemini-3.6-flash

Model generations turn over roughly twice a year and retired ids stop resolving, so these move with every minor release. Pin .with_model("...") if you need one specific model to survive upgrades.

Anthropic native structured outputs

Anthropic extraction defaults to a forced tool_use call, which every Claude generation and every Anthropic-compatible proxy accepts. On the generations that support it, output_config.format constrains the response text directly with no tool round-trip:

let client = Client::anthropic("sk-ant-...")
    .with_anthropic_structured_output();

Gemini thinking

Gemini models allow configuring their thinking effort. Version 2 and 3 models differ in how they can be configured. Version 2 models accept a budgeting parameter and version 3 models accept a thinking level.

You can configure these parameters with the with_gemini_thinking setter:

use instructors::{GeminiThinking, ThinkingLevel};

// a mostly-transcription extraction: don't pay for deep reasoning
let client = Client::gemini("AIza...")
    .with_gemini_thinking(GeminiThinking::Level(ThinkingLevel::Low));

// a schema that needs inference across a long document
let client = Client::gemini("AIza...")
    .with_gemini_thinking(GeminiThinking::Level(ThinkingLevel::High));
Variant Wire Generation Value
Level(ThinkingLevel) thinkingLevel Gemini 3 Minimal, Low, Medium, High
Budget(i32) thinkingBudget Gemini 2.5 (legacy) token ceiling, or -1 for dynamic
let client = Client::gemini("AIza...")
    .with_model("gemini-2.5-flash")
    .with_gemini_thinking(GeminiThinking::Budget(1024));

Thinking tokens are billed as output tokens, so they are summed into usage.output_tokens and the cost estimate along with the answer's own tokens.

Gemini only. The method is a no-op on OpenAI and Anthropic clients.

// OpenAI
let client = Client::openai("sk-...");

// Anthropic
let client = Client::anthropic("sk-ant-...");

// DeepSeek, Together, or any OpenAI-compatible API
let client = Client::openai_compatible("sk-...", "https://api.deepseek.com/v1");

// Anthropic-compatible proxy
let client = Client::anthropic_compatible("sk-...", "https://proxy.example.com/v1");

// Google Gemini
let client = Client::gemini("AIza...");

// Gemini-compatible proxy
let client = Client::gemini_compatible("AIza...", "https://proxy.example.com/v1beta");

Streaming

Stream partial JSON tokens as they arrive:

let result = client.extract::<Contact>("...")
    .on_stream(|chunk| {
        print!("{chunk}");  // partial JSON fragments
    })
    .await?;

All three providers (OpenAI, Anthropic, Gemini) support streaming. The final result is assembled from all chunks and deserialized as usual.

Image Input

Extract structured data from images using vision-capable models:

use instructors::ImageInput;

// from URL
let result = client.extract::<Description>("Describe this image")
    .image(ImageInput::Url("https://example.com/photo.jpg".into()))
    .model("gpt-5.6-terra")
    .await?;

// from base64
let result = client.extract::<Description>("Describe this image")
    .image(ImageInput::Base64 {
        media_type: "image/png".into(),
        data: base64_string,
    })
    .await?;

// multiple images
let result = client.extract::<Comparison>("Compare these images")
    .images(vec![
        ImageInput::Url("https://example.com/a.jpg".into()),
        ImageInput::Url("https://example.com/b.jpg".into()),
    ])
    .await?;

Provider Fallback

Chain multiple providers for automatic failover:

let client = Client::openai("sk-...")
    .with_fallback(Client::anthropic("sk-ant-..."))
    .with_fallback(Client::openai_compatible("sk-...", "https://api.deepseek.com/v1"));

// tries OpenAI first → Anthropic on failure → DeepSeek as last resort
let result = client.extract::<Contact>("...").await?;

Each fallback is tried in order after the primary provider exhausts its retries.

Validation

Validate extracted data with automatic retry — invalid results are fed back to the LLM with error details.

Closure-based

let user: User = client.extract("...")
    .validate(|u: &User| {
        if u.age > 150 { Err("age must be <= 150".into()) } else { Ok(()) }
    })
    .await?.value;

Trait-based

use instructors::prelude::*;

#[derive(Debug, Deserialize, JsonSchema)]
struct Email { address: String }

impl Validate for Email {
    fn validate(&self) -> Result<(), ValidationError> {
        if self.address.contains('@') { Ok(()) }
        else { Err("invalid email".into()) }
    }
}

let email: Email = client.extract("...").validated().await?.value;

List Extraction

Extract multiple items from text with extract_many:

#[derive(Debug, Deserialize, JsonSchema)]
struct Entity {
    name: String,
    entity_type: String,
}

let entities: Vec<Entity> = client
    .extract_many("Apple CEO Tim Cook met Google CEO Sundar Pichai")
    .await?.value;

Batch Processing

Process multiple prompts concurrently with configurable concurrency:

let prompts = vec!["review 1".into(), "review 2".into(), "review 3".into()];

let results = client
    .extract_batch::<Review>(prompts)
    .concurrency(5)
    .validate(|r: &Review| { /* ... */ Ok(()) })
    .run()
    .await;

// each result is independent — partial failures don't affect others
for result in results {
    match result {
        Ok(r) => println!("{:?}", r.value),
        Err(e) => eprintln!("failed: {e}"),
    }
}

Multi-turn Conversations

Pass message history for context-aware extraction:

use instructors::Message;

let result = client.extract::<Summary>("summarize the above")
    .messages(vec![
        Message::user("Here is a long document..."),
        Message::assistant("I see the document."),
    ])
    .await?;

Classification

Enums work naturally for classification tasks:

#[derive(Debug, Deserialize, JsonSchema)]
enum Sentiment { Positive, Negative, Neutral }

let sentiment: Sentiment = client
    .extract("This product is amazing!")
    .await?.value;

Nested Types

Complex nested structures with vectors, options, and enums:

#[derive(Debug, Deserialize, JsonSchema)]
struct Paper {
    title: String,
    authors: Vec<Author>,
    keywords: Vec<String>,
}

#[derive(Debug, Deserialize, JsonSchema)]
struct Author {
    name: String,
    affiliation: Option<String>,
}

let paper: Paper = client.extract(&pdf_text).model("gpt-5.6-terra").await?.value;

Retry & Timeout

Enable exponential backoff on HTTP 429/503 errors and set an overall request timeout:

use std::time::Duration;
use instructors::BackoffConfig;

let client = Client::openai("sk-...")
    .with_retry_backoff(BackoffConfig::default())  // 500ms base, 30s cap, 3 retries
    .with_timeout(Duration::from_secs(120));        // overall timeout

// per-request override
let result = client.extract::<Contact>("...")
    .retry_backoff(BackoffConfig {
        base_delay: Duration::from_millis(200),
        max_delay: Duration::from_secs(10),
        jitter: true,
        max_http_retries: 5,
    })
    .timeout(Duration::from_secs(30))
    .await?;

Without backoff configured, HTTP 429/503 errors fail immediately (default behavior unchanged).

Configuration

let result: MyStruct = client
    .extract("input text")
    .model("gpt-5.6-luna")            // override model
    .system("You are an expert...")   // custom system prompt
    .temperature(0.0)                 // deterministic output
    .max_tokens(2048)                 // limit output tokens
    .max_retries(3)                   // retry on parse/validation failure
    .context("extra context...")      // append to prompt
    .retry_backoff(BackoffConfig::default()) // HTTP 429/503 backoff
    .timeout(Duration::from_secs(30))        // overall timeout
    .await?
    .value;

temperature is not sent to Anthropic. The current Claude generations reject a non-default temperature with a 400, so the Anthropic request never carries it and the setting has no effect there. It is sent to the OpenAI and Gemini families as usual.

Client Defaults

Set defaults once, override per-request:

let client = Client::openai("sk-...")
    .with_model("gpt-5.6-luna")
    .with_temperature(0.0)
    .with_max_retries(3)
    .with_system("Extract data precisely.");

// all extractions use the defaults above
let a: TypeA = client.extract("...").await?.value;
let b: TypeB = client.extract("...").await?.value;

// override for a specific request
let c: TypeC = client.extract("...").model("gpt-5.6-terra").await?.value;

Two defaults are provider-specific and can only be set on the client, because they change the shape of the request rather than one field of it: with_anthropic_structured_output() and with_gemini_thinking(GeminiThinking::Level(ThinkingLevel::Low)). Each is a no-op on the other providers' clients.

Cost Tracking

Built-in token counting and cost estimation via tiktoken:

let result = client.extract::<Contact>("...").await?;

println!("input:  {} tokens", result.usage.input_tokens);
println!("output: {} tokens", result.usage.output_tokens);
println!("cost:   ${:.6}", result.usage.cost.unwrap_or(0.0));
println!("retries: {}", result.usage.retries);

cost is None for a model the price table does not carry. Model ids are resolved by tiktoken, so the id you sent to the provider — dashes where the table uses a dot, a release-date suffix, Bedrock or Vertex decoration — prices as that model rather than silently costing nothing.

Disable with default-features = false:

[dependencies]
instructors = { version = "1", default-features = false }

JSON Repair

When an LLM returns malformed JSON — trailing commas, single quotes, unquoted keys, markdown code fences, etc. — instructors automatically attempts to repair the output before deserialization. If repair succeeds, the fixed JSON is parsed directly without burning a retry. This saves both tokens and latency, especially with smaller or open-source models that are more likely to produce slightly broken output.

Repair is transparent: you don't need to configure anything. It runs on every response before serde_json parsing, and falls back to the normal retry path if the output can't be fixed.

Lifecycle Hooks

Observe requests and responses:

let result = client.extract::<Contact>("...")
    .on_request(|model, prompt| {
        println!("[req] model={model}, prompt_len={}", prompt.len());
    })
    .on_response(|usage| {
        println!("[res] tokens={}, cost={:?}", usage.total_tokens, usage.cost);
    })
    .await?;

How It Works

  1. #[derive(JsonSchema)] generates a JSON Schema from your Rust type (via schemars)
  2. Schema is cached per type (thread-local, zero lock contention)
  3. The schema is transformed for the target provider:
    • OpenAI: wrapped in response_format with strict mode (additionalProperties: false, all fields required)
    • Anthropic: wrapped as a tool with input_schema, forced via tool_choice
    • Gemini: passed as response_schema with response_mime_type: "application/json"
  4. LLM is constrained to produce valid JSON matching the schema
  5. Response JSON is automatically repaired if malformed (trailing commas, single quotes, unquoted keys, markdown fences)
  6. Response is deserialized with serde_json::from_str::<T>()
  7. If Validate trait or .validate() closure is present, validation runs
  8. On parse/validation failure, error feedback is sent back and the request is retried

Ecosystem

tiktoken · @goliapkg/tiktoken-wasm · instructors · chunkedrs · embedrs

License

MIT

About

Type-safe structured output extraction from LLMs (crate: instructors)

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages