Python: Add support for AutoGen's 0.2 ConversableAgent (#10607)

### Motivation and Context

For those who use AutoGen's 0.2 `ConversableAgent`, we're providing
support in Semantic Kernel to run this agent type. This assumes one will
port their existing AG 0.2.X `ConversableAgent` code and run in Semantic
Kernel.

Note: as we move towards GA for our agent chat patterns, we are
analyzing how to support AutoGen agents with a shared time. This PR does
not provide support for the AG `ConversibleAgent` group chat patterns
that exist in the 0.2.X package.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Add support and samples for the AG 0.2.X ConversableAgent. 
- Add unit test coverage.
- Add samples and READMEs.
- Closes #10407

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄

---------

Co-authored-by: Chris <[email protected]>

Author: moonbox3

python/pyproject.toml

@@ -50,6 +50,9 @@ dependencies = [
 
 ### Optional dependencies
 [project.optional-dependencies]
+autogen = [
+    "autogen-agentchat >= 0.2, <0.4"
+]
 azure = [
     "azure-ai-inference >= 1.0.0b6",
     "azure-ai-projects >= 1.0.0b5",

python/samples/concepts/agents/autogen_conversable_agent/README.md

@@ -0,0 +1,20 @@
+## AutoGen Conversable Agent (v0.2.X)
+
+Semantic Kernel Python supports running AutoGen Conversable Agents provided in the 0.2.X package.
+
+### Limitations
+
+Currently, there are some limitations to note:
+
+- AutoGen Conversable Agents in Semantic Kernel run asynchronously and do not support streaming of agent inputs or responses.
+- The `AutoGenConversableAgent` in Semantic Kernel Python cannot be configured as part of a Semantic Kernel `AgentGroupChat`. As we progress towards GA for our agent group chat patterns, we will explore ways to integrate AutoGen agents into a Semantic Kernel group chat scenario.
+
+### Installation
+
+Install the `semantic-kernel` package with the `autogen` extra:
+
+```bash
+pip install semantic-kernel[autogen]
+```
+
+For an example of how to integrate an AutoGen Conversable Agent using the Semantic Kernel Agent abstraction, please refer to [`autogen_conversable_agent_simple_convo.py`](autogen_conversable_agent_simple_convo.py).

python/samples/concepts/agents/autogen_conversable_agent/autogen_conversable_agent_code_executor.py

@@ -0,0 +1,61 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+import asyncio
+
+from autogen import ConversableAgent
+from autogen.coding import LocalCommandLineCodeExecutor
+
+from semantic_kernel.agents.autogen.autogen_conversable_agent import AutoGenConversableAgent
+
+"""
+The following sample demonstrates how to use the AutoGenConversableAgent to create a reply from an agent
+to a message with a code block. The agent executes the code block and replies with the output.
+
+The sample follows the AutoGen flow outlined here:
+https://microsoft.github.io/autogen/0.2/docs/tutorial/code-executors#local-execution
+"""
+
+
+async def main():
+    # Create a temporary directory to store the code files.
+    import os
+
+    # Configure the temporary directory to be where the script is located.
+    temp_dir = os.path.dirname(os.path.realpath(__file__))
+
+    # Create a local command line code executor.
+    executor = LocalCommandLineCodeExecutor(
+        timeout=10,  # Timeout for each code execution in seconds.
+        work_dir=temp_dir,  # Use the temporary directory to store the code files.
+    )
+
+    # Create an agent with code executor configuration.
+    code_executor_agent = ConversableAgent(
+        "code_executor_agent",
+        llm_config=False,  # Turn off LLM for this agent.
+        code_execution_config={"executor": executor},  # Use the local command line code executor.
+        human_input_mode="ALWAYS",  # Always take human input for this agent for safety.
+    )
+
+    autogen_agent = AutoGenConversableAgent(conversable_agent=code_executor_agent)
+
+    message_with_code_block = """This is a message with code block.
+The code block is below:
+```python
+import numpy as np
+import matplotlib.pyplot as plt
+x = np.random.randint(0, 100, 100)
+y = np.random.randint(0, 100, 100)
+plt.scatter(x, y)
+plt.savefig('scatter.png')
+print('Scatter plot saved to scatter.png')
+```
+This is the end of the message.
+"""
+
+    async for content in autogen_agent.invoke(message=message_with_code_block):
+        print(f"# {content.role} - {content.name or '*'}: '{content.content}'")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

python/samples/concepts/agents/autogen_conversable_agent/autogen_conversable_agent_convo_with_tools.py

@@ -0,0 +1,95 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+import asyncio
+import os
+from typing import Annotated, Literal
+
+from autogen import ConversableAgent, register_function
+
+from semantic_kernel.agents.autogen.autogen_conversable_agent import AutoGenConversableAgent
+from semantic_kernel.contents.function_call_content import FunctionCallContent
+from semantic_kernel.contents.function_result_content import FunctionResultContent
+
+"""
+The following sample demonstrates how to use the AutoGenConversableAgent to create a conversation between two agents
+where one agent suggests a tool function call and the other agent executes the tool function call.
+
+In this example, the assistant agent suggests a calculator tool function call to the user proxy agent. The user proxy
+agent executes the calculator tool function call. The assistant agent and the user proxy agent are created using the
+ConversableAgent class. The calculator tool function is registered with the assistant agent and the user proxy agent.
+
+This sample follows the AutoGen flow outlined here:
+https://microsoft.github.io/autogen/0.2/docs/tutorial/tool-use
+"""
+
+
+Operator = Literal["+", "-", "*", "/"]
+
+
+async def main():
+    def calculator(a: int, b: int, operator: Annotated[Operator, "operator"]) -> int:
+        if operator == "+":
+            return a + b
+        if operator == "-":
+            return a - b
+        if operator == "*":
+            return a * b
+        if operator == "/":
+            return int(a / b)
+        raise ValueError("Invalid operator")
+
+    assistant = ConversableAgent(
+        name="Assistant",
+        system_message="You are a helpful AI assistant. "
+        "You can help with simple calculations. "
+        "Return 'TERMINATE' when the task is done.",
+        # Note: the model "gpt-4o" leads to a "division by zero" error that doesn't occur with "gpt-4o-mini"
+        # or even "gpt-4".
+        llm_config={
+            "config_list": [{"model": os.environ["OPENAI_CHAT_MODEL_ID"], "api_key": os.environ["OPENAI_API_KEY"]}]
+        },
+    )
+
+    # Create a Semantic Kernel AutoGenConversableAgent based on the AutoGen ConversableAgent.
+    assistant_agent = AutoGenConversableAgent(conversable_agent=assistant)
+
+    user_proxy = ConversableAgent(
+        name="User",
+        llm_config=False,
+        is_termination_msg=lambda msg: msg.get("content") is not None and "TERMINATE" in msg["content"],
+        human_input_mode="NEVER",
+    )
+
+    assistant.register_for_llm(name="calculator", description="A simple calculator")(calculator)
+
+    # Register the tool function with the user proxy agent.
+    user_proxy.register_for_execution(name="calculator")(calculator)
+
+    register_function(
+        calculator,
+        caller=assistant,  # The assistant agent can suggest calls to the calculator.
+        executor=user_proxy,  # The user proxy agent can execute the calculator calls.
+        name="calculator",  # By default, the function name is used as the tool name.
+        description="A simple calculator",  # A description of the tool.
+    )
+
+    # Create a Semantic Kernel AutoGenConversableAgent based on the AutoGen ConversableAgent.
+    user_proxy_agent = AutoGenConversableAgent(conversable_agent=user_proxy)
+
+    async for content in user_proxy_agent.invoke(
+        recipient=assistant_agent,
+        message="What is (44232 + 13312 / (232 - 32)) * 5?",
+        max_turns=10,
+    ):
+        for item in content.items:
+            match item:
+                case FunctionResultContent(result=r):
+                    print(f"# {content.role} - {content.name or '*'}: '{r}'")
+                case FunctionCallContent(function_name=fn, arguments=arguments):
+                    print(f"# {content.role} - {content.name or '*'}: Function Name: '{fn}', Arguments: '{arguments}'")
+                case _:
+                    print(f"# {content.role} - {content.name or '*'}: '{content.content}'")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

python/samples/concepts/agents/autogen_conversable_agent/autogen_conversable_agent_simple_convo.py

@@ -0,0 +1,61 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+import asyncio
+import os
+
+from autogen import ConversableAgent
+
+from semantic_kernel.agents.autogen.autogen_conversable_agent import AutoGenConversableAgent
+
+"""
+The following sample demonstrates how to use the AutoGenConversableAgent to create a conversation between two agents
+where one agent suggests a joke and the other agent generates a joke.
+
+The sample follows the AutoGen flow outlined here:
+https://microsoft.github.io/autogen/0.2/docs/tutorial/introduction#roles-and-conversations
+"""
+
+
+async def main():
+    cathy = ConversableAgent(
+        "cathy",
+        system_message="Your name is Cathy and you are a part of a duo of comedians.",
+        llm_config={
+            "config_list": [
+                {
+                    "model": os.environ["OPENAI_CHAT_MODEL_ID"],
+                    "temperature": 0.9,
+                    "api_key": os.environ.get("OPENAI_API_KEY"),
+                }
+            ]
+        },
+        human_input_mode="NEVER",  # Never ask for human input.
+    )
+
+    cathy_autogen_agent = AutoGenConversableAgent(conversable_agent=cathy)
+
+    joe = ConversableAgent(
+        "joe",
+        system_message="Your name is Joe and you are a part of a duo of comedians.",
+        llm_config={
+            "config_list": [
+                {
+                    "model": os.environ["OPENAI_CHAT_MODEL_ID"],
+                    "temperature": 0.7,
+                    "api_key": os.environ.get("OPENAI_API_KEY"),
+                }
+            ]
+        },
+        human_input_mode="NEVER",  # Never ask for human input.
+    )
+
+    joe_autogen_agent = AutoGenConversableAgent(conversable_agent=joe)
+
+    async for content in cathy_autogen_agent.invoke(
+        recipient=joe_autogen_agent, message="Tell me a joke about the stock market.", max_turns=3
+    ):
+        print(f"# {content.role} - {content.name or '*'}: '{content.content}'")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

python/semantic_kernel/agents/autogen/README.md

@@ -0,0 +1,20 @@
+## AutoGen Conversable Agent (v0.2.X)
+
+Semantic Kernel Python supports running AutoGen Conversable Agents provided in the 0.2.X package.
+
+### Limitations
+
+Currently, there are some limitations to note:
+
+- AutoGen Conversable Agents in Semantic Kernel run asynchronously and do not support streaming of agent inputs or responses.
+- The `AutoGenConversableAgent` in Semantic Kernel Python cannot be configured as part of a Semantic Kernel `AgentGroupChat`. As we progress towards GA for our agent group chat patterns, we will explore ways to integrate AutoGen agents into a Semantic Kernel group chat scenario.
+
+### Installation
+
+Install the `semantic-kernel` package with the `autogen` extra:
+
+```bash
+pip install semantic-kernel[autogen]
+```
+
+For an example of how to integrate an AutoGen Conversable Agent using the Semantic Kernel Agent abstraction, please refer to [`autogen_conversable_agent_simple_convo.py`](../../../samples/concepts/agents/autogen_conversable_agent/autogen_conversable_agent_simple_convo.py).

python/semantic_kernel/agents/autogen/__init__.py

@@ -0,0 +1,5 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+from semantic_kernel.agents.autogen.autogen_conversable_agent import AutoGenConversableAgent
+
+__all__ = ["AutoGenConversableAgent"]

python/semantic_kernel/agents/autogen/autogen_conversable_agent.py

@@ -0,0 +1,163 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+import logging
+from collections.abc import AsyncIterable, Callable
+from typing import TYPE_CHECKING, Any
+
+from autogen import ConversableAgent
+
+from semantic_kernel.agents.agent import Agent
+from semantic_kernel.contents.chat_message_content import ChatMessageContent
+from semantic_kernel.contents.function_call_content import FunctionCallContent
+from semantic_kernel.contents.function_result_content import FunctionResultContent
+from semantic_kernel.contents.text_content import TextContent
+from semantic_kernel.contents.utils.author_role import AuthorRole
+from semantic_kernel.exceptions.agent_exceptions import AgentInvokeException
+from semantic_kernel.functions.kernel_arguments import KernelArguments
+
+if TYPE_CHECKING:
+    from autogen.cache import AbstractCache
+
+    from semantic_kernel.kernel import Kernel
+
+logger: logging.Logger = logging.getLogger(__name__)
+
+
+class AutoGenConversableAgent(Agent):
+    """A Semantic Kernel wrapper around an AutoGen 0.2 `ConversableAgent`.
+
+    This allows one to use it as a Semantic Kernel `Agent`. Note: this agent abstraction
+    does not currently allow for the use of AgentGroupChat within Semantic Kernel.
+    """
+
+    conversable_agent: ConversableAgent
+
+    def __init__(self, conversable_agent: ConversableAgent, **kwargs: Any) -> None:
+        """Initialize the AutoGenConversableAgent.
+
+        Args:
+            conversable_agent: The existing AutoGen 0.2 ConversableAgent instance
+            kwargs: Other Agent base class arguments (e.g. name, id, instructions)
+        """
+        args: dict[str, Any] = {
+            "name": conversable_agent.name,
+            "description": conversable_agent.description,
+            "instructions": conversable_agent.system_message,
+            "conversable_agent": conversable_agent,
+        }
+
+        if kwargs:
+            args.update(kwargs)
+
+        super().__init__(**args)
+
+    async def invoke(
+        self,
+        *,
+        recipient: "AutoGenConversableAgent | None" = None,
+        clear_history: bool = True,
+        silent: bool = True,
+        cache: "AbstractCache | None" = None,
+        max_turns: int | None = None,
+        summary_method: str | Callable | None = ConversableAgent.DEFAULT_SUMMARY_METHOD,
+        summary_args: dict | None = {},
+        message: dict | str | Callable | None = None,
+        **kwargs: Any,
+    ) -> AsyncIterable[ChatMessageContent]:
+        """A direct `invoke` method for the ConversableAgent.
+
+        Args:
+            recipient: The recipient ConversableAgent to chat with
+            clear_history: Whether to clear the chat history before starting. True by default.
+            silent: Whether to suppress console output. True by default.
+            cache: The cache to use for storing chat history
+            max_turns: The maximum number of turns to chat for
+            summary_method: The method to use for summarizing the chat
+            summary_args: The arguments to pass to the summary method
+            message: The initial message to send. If message is not provided,
+                the agent will wait for the user to provide the first message.
+            kwargs: Additional keyword arguments
+        """
+        if recipient is not None:
+            if not isinstance(recipient, AutoGenConversableAgent):
+                raise AgentInvokeException(
+                    f"Invalid recipient type: {type(recipient)}. "
+                    "Recipient must be an instance of AutoGenConversableAgent."
+                )
+
+            chat_result = await self.conversable_agent.a_initiate_chat(
+                recipient=recipient.conversable_agent,
+                clear_history=clear_history,
+                silent=silent,
+                cache=cache,
+                max_turns=max_turns,
+                summary_method=summary_method,
+                summary_args=summary_args,
+                message=message,  # type: ignore
+                **kwargs,
+            )
+
+            logger.info(f"Called AutoGenConversableAgent.a_initiate_chat with recipient: {recipient}")
+
+            for message in chat_result.chat_history:
+                yield AutoGenConversableAgent._to_chat_message_content(message)  # type: ignore
+        else:
+            reply = await self.conversable_agent.a_generate_reply(
+                messages=[{"role": "user", "content": message}],
+            )
+
+            logger.info(f"Called AutoGenConversableAgent.a_generate_reply with recipient: {recipient}")
+
+            if isinstance(reply, str):
+                yield ChatMessageContent(content=reply, role=AuthorRole.ASSISTANT)
+            elif isinstance(reply, dict):
+                yield ChatMessageContent(**reply)
+            else:
+                raise AgentInvokeException(f"Unexpected reply type from `a_generate_reply`: {type(reply)}")
+
+    async def invoke_stream(
+        self,
+        message: str,
+        kernel: "Kernel | None" = None,
+        arguments: KernelArguments | None = None,
+        **kwargs: Any,
+    ) -> AsyncIterable[ChatMessageContent]:
+        """Invoke the agent with a stream of messages."""
+        raise NotImplementedError("The AutoGenConversableAgent does not support streaming.")
+
+    @staticmethod
+    def _to_chat_message_content(message: dict[str, Any]) -> ChatMessageContent:
+        """Translate an AutoGen message to a Semantic Kernel ChatMessageContent."""
+        items: list[TextContent | FunctionCallContent | FunctionResultContent] = []
+        role = AuthorRole(message.get("role"))
+        name: str = message.get("name", "")
+
+        content = message.get("content")
+        if content is not None:
+            text = TextContent(text=content)
+            items.append(text)
+
+        if role == AuthorRole.ASSISTANT:
+            tool_calls = message.get("tool_calls")
+            if tool_calls is not None:
+                for tool_call in tool_calls:
+                    items.append(
+                        FunctionCallContent(
+                            id=tool_call.get("id"),
+                            function_name=tool_call.get("name"),
+                            arguments=tool_call.get("function").get("arguments"),
+                        )
+                    )
+
+        if role == AuthorRole.TOOL:
+            tool_responses = message.get("tool_responses")
+            if tool_responses is not None:
+                for tool_response in tool_responses:
+                    items.append(
+                        FunctionResultContent(
+                            id=tool_response.get("tool_call_id"),
+                            result=tool_response.get("content"),
+                        )
+                    )
+
+        return ChatMessageContent(role=role, items=items, name=name)  # type: ignore

python/tests/unit/agents/autogen_conversable_agent/test_autogen_conversable_agent.py

@@ -0,0 +1,106 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+from autogen import ConversableAgent
+
+from semantic_kernel.agents.autogen.autogen_conversable_agent import AutoGenConversableAgent
+from semantic_kernel.contents.utils.author_role import AuthorRole
+from semantic_kernel.exceptions.agent_exceptions import AgentInvokeException
+
+
+@pytest.fixture
+def mock_conversable_agent():
+    agent = MagicMock(spec=ConversableAgent)
+    agent.name = "MockName"
+    agent.description = "MockDescription"
+    agent.system_message = "MockSystemMessage"
+    return agent
+
+
+async def test_autogen_conversable_agent_initialization(mock_conversable_agent):
+    agent = AutoGenConversableAgent(mock_conversable_agent, id="mock_id")
+    assert agent.name == "MockName"
+    assert agent.description == "MockDescription"
+    assert agent.instructions == "MockSystemMessage"
+    assert agent.conversable_agent == mock_conversable_agent
+
+
+async def test_autogen_conversable_agent_invoke_with_recipient(mock_conversable_agent):
+    mock_conversable_agent.a_initiate_chat = AsyncMock()
+    mock_conversable_agent.a_initiate_chat.return_value = MagicMock(
+        chat_history=[
+            {"role": "user", "content": "Hello from user!"},
+            {"role": "assistant", "content": "Hello from assistant!"},
+        ]
+    )
+    agent = AutoGenConversableAgent(mock_conversable_agent)
+    recipient_agent = MagicMock(spec=AutoGenConversableAgent)
+    recipient_agent.conversable_agent = MagicMock(spec=ConversableAgent)
+
+    messages = []
+    async for msg in agent.invoke(recipient=recipient_agent, message="Test message", arg1="arg1"):
+        messages.append(msg)
+
+    mock_conversable_agent.a_initiate_chat.assert_awaited_once()
+    assert len(messages) == 2
+    assert messages[0].role == AuthorRole.USER
+    assert messages[0].content == "Hello from user!"
+    assert messages[1].role == AuthorRole.ASSISTANT
+    assert messages[1].content == "Hello from assistant!"
+
+
+async def test_autogen_conversable_agent_invoke_without_recipient_string_reply(mock_conversable_agent):
+    mock_conversable_agent.a_generate_reply = AsyncMock(return_value="Mocked assistant response")
+    agent = AutoGenConversableAgent(mock_conversable_agent)
+
+    messages = []
+    async for msg in agent.invoke(message="Hello"):
+        messages.append(msg)
+
+    mock_conversable_agent.a_generate_reply.assert_awaited_once()
+    assert len(messages) == 1
+    assert messages[0].role == AuthorRole.ASSISTANT
+    assert messages[0].content == "Mocked assistant response"
+
+
+async def test_autogen_conversable_agent_invoke_without_recipient_dict_reply(mock_conversable_agent):
+    mock_conversable_agent.a_generate_reply = AsyncMock(
+        return_value={
+            "content": "Mocked assistant response",
+            "role": "assistant",
+            "name": "AssistantName",
+        }
+    )
+    agent = AutoGenConversableAgent(mock_conversable_agent)
+
+    messages = []
+    async for msg in agent.invoke(message="Hello"):
+        messages.append(msg)
+
+    mock_conversable_agent.a_generate_reply.assert_awaited_once()
+    assert len(messages) == 1
+    assert messages[0].role == AuthorRole.ASSISTANT
+    assert messages[0].content == "Mocked assistant response"
+    assert messages[0].name == "AssistantName"
+
+
+async def test_autogen_conversable_agent_invoke_without_recipient_unexpected_type(mock_conversable_agent):
+    mock_conversable_agent.a_generate_reply = AsyncMock(return_value=12345)
+    agent = AutoGenConversableAgent(mock_conversable_agent)
+
+    with pytest.raises(AgentInvokeException):
+        async for _ in agent.invoke(message="Hello"):
+            pass
+
+
+async def test_autogen_conversable_agent_invoke_with_invalid_recipient_type(mock_conversable_agent):
+    mock_conversable_agent.a_generate_reply = AsyncMock(return_value=12345)
+    agent = AutoGenConversableAgent(mock_conversable_agent)
+
+    recipient = MagicMock()
+
+    with pytest.raises(AgentInvokeException):
+        async for _ in agent.invoke(recipient=recipient, message="Hello"):
+            pass