Debugging and Troubleshooting

Oct 13, 2025 by talynt.org 82 views

Bug: KeyError: 'title' in get_function_tool when using structured_predict with Pydantic models containing Forward References

Bug Description

Bug Description: A KeyError: 'title' was encountered when using the structured_predict function with Pydantic models that employ forward references. This issue appears to specifically manifest when the get_function_tool function attempts to extract the title from the JSON schema of the Pydantic model.

Expected Behavior: structured_predict should be able to process Pydantic models that use forward references without raising errors. It is expected that the get_function_tool function correctly extracts the necessary information from the model's JSON schema, even in the presence of forward references, and that structured_predict functions as intended, returning the structured output based on the provided Pydantic model.

Actual Behavior: A KeyError: 'title' occurs, and the program terminates abnormally. The error specifically occurs in the line name=schema["title"] within the get_function_tool function in the file llama_index/core/program/function_program.py.

Root Cause Analysis: When a Pydantic model uses forward references, the result of output_cls.model_json_schema() does not contain the "title" key directly in the root of the JSON schema. Instead, the definitions and titles for models referenced via forward references are moved inside the "$defs" key in the JSON schema.

The get_function_tool function incorrectly assumes that the "title" key is always present in the root of the JSON schema, causing the KeyError when it encounters models with forward references.

Possible Solution: Modify the get_function_tool function to correctly handle JSON schemas generated by Pydantic models with forward references.

A possible solution could be: Check if the "title" key exists in the root of the schema. If present, use it as usual.

If the "title" key is not present in the root, check the " $defs" key. Within "$ defs" there might be the definitions of the referenced models, including their titles. It may be necessary to navigate within "$defs" to find the correct title associated with the output_cls model.

A more robust fix would require a deeper analysis of the JSON schema structure in the presence of forward references and an adaptation of the title extraction in get_function_tool to correctly handle these structures.

Version

0.12.23

Steps to Reproduce

from typing import List
from pydantic import BaseModel, Field
from llama_index.llms.openai import OpenAI
import os
from llama_index.core.prompts import PromptTemplate
from dotenv import load_dotenv


from typing import List
from pydantic import BaseModel, Field

class Parent(BaseModel):
    children: List["Child"] = Field(...)
    name: str = Field(...)

class Child(BaseModel):
    parents: List["Parent"] = Field(...)

Parent.model_rebuild()
Child.model_rebuild()

load_dotenv()

llm = OpenAI(
    model=os.environ.get("OPENAI_MODEL_NAME"),
    api_key=os.environ.get("OPENAI_API_KEY", ""),
    temperature=0,
    max_tokens=8192,
)


prompt = PromptTemplate(
    "{name}"
)

response = llm.structured_predict(
    Child, prompt, test="test"
)

print(response)

Relevant Logs/Tracbacks

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/code/src/autoquery/react.py", line 31, in <module>
    response = llm.structured_predict(
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 322, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/llama_index/llms/openai/base.py", line 1005, in structured_predict
    return super().structured_predict(*args, llm_kwargs=llm_kwargs, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 322, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/llama_index/core/llms/llm.py", line 375, in structured_predict
    result = program(llm_kwargs=llm_kwargs, **prompt_args)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 322, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/llama_index/core/program/function_program.py", line 134, in __call__
    tool = get_function_tool(self._output_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/llama_index/core/program/function_program.py", line 49, in get_function_tool
    name=schema["title"],
         ~~~~~~^^^^^^^^^
KeyError: 'title'

Debugging and Troubleshooting

To debug this issue, we can start by examining the JSON schema generated by the Pydantic model with forward references. We can use the output_cls.model_json_schema() method to obtain the JSON schema and then inspect it to see if the "title" key is present in the root or within the "$defs" key.

We can also add print statements or use a debugger to step through the get_function_tool function and see where the KeyError is occurring. This can help us identify the specific line of code that is causing the error and make the necessary modifications to fix the issue.

Solution Implementation

To fix this issue, we can modify the get_function_tool function to correctly handle JSON schemas generated by Pydantic models with forward references. We can add a check to see if the "title" key exists in the root of the schema, and if not, we can navigate within the "$defs" key to find the correct title associated with the output_cls model.

Here is an example of how we can modify the get_function_tool function to fix this issue:

def get_function_tool(output_cls):
    schema = output_cls.model_json_schema()
    if "title" in schema:
        # If the "title" key is present in the root, use it as usual
        name = schema["title"]
    else:
        # If the "title" key is not present in the root, check the "$defs" key
        if "$defs" in schema:
            # Navigate within "$defs" to find the correct title associated with the output_cls model
            for definition in schema["$defs"]:
                if definition["$id"] == output_cls.__name__:
                    name = definition["title"]
                    break
        else:
            # If the "$defs" key is not present, raise an error
            raise ValueError("Could not find title in schema")
    return name

This modified function will correctly handle JSON schemas generated by Pydantic models with forward references and will not raise a KeyError when trying to extract the title from the schema.
Q&A: Bug: KeyError: 'title' in get_function_tool when using structured_predict with Pydantic models containing Forward References

Q: What is the bug and how does it occur?

A: The bug is a KeyError: 'title' that occurs when using the structured_predict function with Pydantic models that employ forward references. This issue specifically manifests when the get_function_tool function attempts to extract the title from the JSON schema of the Pydantic model.

Q: What is the expected behavior of the structured_predict function?

A: The expected behavior of the structured_predict function is to process Pydantic models that use forward references without raising errors. It is expected that the get_function_tool function correctly extracts the necessary information from the model's JSON schema, even in the presence of forward references, and that structured_predict functions as intended, returning the structured output based on the provided Pydantic model.

Q: What is the actual behavior of the structured_predict function?

A: The actual behavior of the structured_predict function is to raise a KeyError: 'title' when trying to extract the title from the JSON schema of the Pydantic model. This occurs because the get_function_tool function incorrectly assumes that the "title" key is always present in the root of the JSON schema.

Q: What is the root cause of the bug?

A: The root cause of the bug is that the get_function_tool function does not correctly handle JSON schemas generated by Pydantic models with forward references. When a Pydantic model uses forward references, the result of output_cls.model_json_schema() does not contain the "title" key directly in the root of the JSON schema. Instead, the definitions and titles for models referenced via forward references are moved inside the "$defs" key in the JSON schema.

Q: How can the bug be fixed?

A: The bug can be fixed by modifying the get_function_tool function to correctly handle JSON schemas generated by Pydantic models with forward references. This can be done by adding a check to see if the "title" key exists in the root of the schema, and if not, navigating within the "$defs" key to find the correct title associated with the output_cls model.

Q: What is the solution implementation?

A: The solution implementation involves modifying the get_function_tool function to correctly handle JSON schemas generated by Pydantic models with forward references. This can be done by adding a check to see if the "title" key exists in the root of the schema, and if not, navigating within the "$defs" key to find the correct title associated with the output_cls model.

Q: How can the solution be tested?

A: The solution can be tested by using the structured_predict function with Pydantic models that employ forward references and verifying that the get_function_tool function correctly extracts the title from the JSON schema.

Q: What are the benefits of the solution?

A: The benefits of the solution are that it correctly handles JSON schemas generated by Pydantic models with forward references, preventing the KeyError: 'title' from occurring. This ensures that the structured_predict function functions as intended, returning the structured output based on the provided Pydantic model.

Q: What are the potential risks of the solution?

A: The potential risks of the solution are that it may not correctly handle all possible JSON schema structures generated by Pydantic models with forward references. However, this can be mitigated by thoroughly testing the solution with a variety of different Pydantic models and JSON schema structures.

Q: How can the solution be maintained and updated?

A: The solution can be maintained and updated by regularly reviewing and updating the get_function_tool function to ensure that it correctly handles the latest JSON schema structures generated by Pydantic models with forward references. This can be done by monitoring the latest developments in the Pydantic and JSON schema communities and adapting the solution accordingly.