Pydantic models in LLM API calls - Part 3

Exploring the use of Pydantic models directly in LLM API calls and with the Instructor library

Passing the Pydantic model directly in the API call to LLMs

In this article I’ll try to use Pydantic in the API calls to LLM experimenting with OpenAIs beta API and the Instructor package. The way Pydantic works with Instructor is described in detail here.

|# code-summary: "Import all required libraries"
# Import packages
from pydantic import BaseModel, Field, EmailStr
from typing import List, Literal, Optional
from openai import OpenAI
# instructor is a library for extracting structured data from Large Language Models (LLMs). It is built on top of Pydantic.
# instructor extracts the JSON schema from a Pydantic model and passes it in the prompt. It handles retries and parsing of the response.
# Instructor takes a 'response_model' parameter in the API call, which is a Pydantic model that defines the structure of the expected response.
import instructor
import anthropic
from dotenv import load_dotenv
from datetime import date
import os

load_dotenv()

ANTHROPIC_API_KEY = os.getenv("ANTHROPIC_API_KEY")
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

# Define the UserInput model for customer support queries
class UserInput(BaseModel):
    name: str
    email: EmailStr
    query: str
    order_id: Optional[int] = Field(
        # Default value is None
        None,
        description="5-digit order number (cannot start with 0)",
        #Greater or equal to 10000 and less than or equal to 99999
        ge=10000,
        le=99999
    )
    purchase_date: Optional[date] = None

# Define the CustomerQuery model that inherits from UserInput. It adds fields that will be populated by the LLM for priority, category, complaint status, and tags.
class CustomerQuery(UserInput):
    priority: str = Field(
        ..., description="Priority level: low, medium, high"
    )
    category: Literal[
        'refund_request', 'information_request', 'other'
    ] = Field(..., description="Query category")
    is_complaint: bool = Field(
        ..., description="Whether this is a complaint"
    )
    tags: List[str] = Field(..., description="Relevant keyword tags")

# Create a sample user input as a JSON string
user_input_json = '''{
    "name": "Ula Dobra",
    "email": "ula.dobra@podlasem.com",
    "query": "I would like to know the status of my order.",
    "order_id": 87647,
    "purchase_date": "2025-01-01"
}'''

I’ll use the Pydantic UserInput model to validate the user input JSON data. Validated data should be returned if there are no errors.

#Validate the user input against the UserInput model using the model_validate_json method
user_input = UserInput.model_validate_json(user_input_json)
print("User Input:")
print(user_input)

User Input:
name='Ula Dobra' email='ula.dobra@podlasem.com' query='I would like to know the status of my order.' order_id=87647 purchase_date=datetime.date(2025, 1, 1)

When using this method, I’ll rely on Instructor to pass the JSON schema of the Pydantic model in the API call.

I’ll first create prompt for LLM to analyze customer query and provide structured response. It doesn’t include a mention of the Pydantic model.

#No mention of the desired data structure in the prompt. Instructor will handle that.
prompt = (
    f"Analyze the following customer query {user_input} "
    f"and provide a structured response."
)

I’ll then create an Anthropic client using Instructor and pass the Pydantic model as the ‘response_model’ parameter in the API call.

# Use Anthropic with Instructor to get structured output (Instructor will work with OpenAI, Grok, Gemini and others as well)
anthropic_client = instructor.from_anthropic(
    anthropic.Anthropic(api_key=ANTHROPIC_API_KEY)
)

response = anthropic_client.messages.create(
    model="claude-3-7-sonnet-latest",  
    max_tokens=1024,
    messages=[
        {
            "role": "user", 
            "content": prompt
        }
    ],
    response_model=CustomerQuery  
)

This should return a valid response - as an instance of the CustomerQuery Pydantic model! 👍

print(response)
# Inspect the returned structured data in JSON format
print(type(response))
print(response.model_dump_json(indent=2))

name='Ula Dobra' email='ula.dobra@podlasem.com' query='I would like to know the status of my order.' order_id=87647 purchase_date=datetime.date(2025, 1, 1) priority='medium' category='information_request' is_complaint=False tags=['order status', 'inquiry']
<class '__main__.CustomerQuery'>
{
  "name": "Ula Dobra",
  "email": "ula.dobra@podlasem.com",
  "query": "I would like to know the status of my order.",
  "order_id": 87647,
  "purchase_date": "2025-01-01",
  "priority": "medium",
  "category": "information_request",
  "is_complaint": false,
  "tags": [
    "order status",
    "inquiry"
  ]
}

Testing OpenAI beta API which accepts a ‘response_format’ argument

Turns out that OpenAI has a version of their API that also accepts a ‘response_format’ argument that is a Pydantic model. No need to use Instructor in this case. Turns out other LLM providers are starting to incorporate Pydantic models in a similar way.

# Initialize OpenAI client and call passing CustomerQuery in your API call
openai_client = OpenAI()
response = openai_client.beta.chat.completions.parse(
    model="gpt-4o",
    messages=[{"role": "user", "content": prompt}],
    #They call the below 'constrained generation'
    response_format=CustomerQuery
)
response_content = response.choices[0].message.content
print(type(response_content))
print(response_content)

<class 'str'>
{"name":"Ula Dobra","email":"ula.dobra@podlasem.com","query":"I would like to know the status of my order.","order_id":87647,"purchase_date":"2025-01-01","priority":"medium","category":"information_request","is_complaint":false,"tags":["order_status","customer_service"]}

Testing Pydantic AI framework with Google Gemini

Pydantic AI provides a nice wrapper for working with multiple LLMs. A Pydantic model can be passed as the ‘output_type’ argument.

# Try out the Pydantic AI package for defining an agent and getting a structured response
from pydantic_ai import Agent
import nest_asyncio
nest_asyncio.apply()

agent = Agent(
    # The model can be changed to "gpt-4o" or "gpt-4o-mini" for OpenAI models and others as well
    model="openai:gpt-4o",
    # model="google-gla:gemini-2.0-flash",
    output_type=CustomerQuery,
)

response = agent.run_sync(prompt)

And we get a valid response!

response

AgentRunResult(output=CustomerQuery(name='Ula Dobra', email='ula.dobra@podlasem.com', query='I would like to know the status of my order.', order_id=87647, purchase_date=datetime.date(2025, 1, 1), priority='medium', category='information_request', is_complaint=False, tags=['order_status', 'information', 'customer_query']))

Conclusion

In this article, we’ve explored how Pydantic models can be seamlessly integrated into LLM API calls for robust data validation and structured responses. By leveraging libraries like Instructor, OpenAI’s beta API, and Pydantic AI, developers can ensure consistent output formats and simplify downstream processing. As LLM providers continue to adopt Pydantic support, building reliable, type-safe AI applications becomes more accessible and efficient.