Support for structured output mode in AgentChat

[BenConstable9]

What feature would you like to be added?

Support for open ai structured output mode in agent chat. The ability to give an agent a Pydantic class that it must adhere to in the response.

This could be dumped to json as a message for intermediate agents?

Why is this needed?

This would make integrating AutoGen with api requests easier. e.g. consider an answer agent at the end of a complex agentic group chat. It might need to formulate the answer in a specific json format, once it takes into account all the previous messages.

OpenAI structured data mode makes the likelihood of a correct JSON structure far more likely than prompting alone.

[rysweet]

thanks @BenConstable9

[rysweet]

@ekzhu - I feel like this has been discussed recently - is there another relevant issue?

[ekzhu]

There was a PR a while back that add the response format option to model client. It is already available for OpenAIChatCompletionClient and AzureOpenAIChatCompletionClient.

https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/cookbook/structured-output-agent.html

So a fix can be to add response_format parameter to AssistantAgent, and it accepts a Pydantic type.