-
-
Notifications
You must be signed in to change notification settings - Fork 523
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Implement TalkHier framework for hierarchical multi-agent content generation #783
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
The TalkHier framework introduces a sophisticated hierarchical multi-agent system for content generation and refinement. Here are the key points from analyzing the implementation:
- Implements a robust hierarchical structure in
/swarms/structs/Talk_Hier.py
with specialized agents (supervisor, generator, evaluator, revisor) communicating through structuredCommunicationEvent
objects - Utilizes
ThreadPoolExecutor
for parallel evaluation processing to improve performance when multiple evaluators assess content simultaneously - Adds dynamic criteria generation through a dedicated
CriteriaGenerator
agent that tailors evaluation metrics based on task requirements - Includes comprehensive error handling and fallback mechanisms for JSON parsing, though the current implementation may mask important errors by converting any text to a dict
- The example code contains a potential typo using model name "gpt-4o" which should be addressed for proper functionality
💡 (1/5) You can manually trigger the bot by mentioning @greptileai in a comment!
1 file(s) reviewed, 3 comment(s)
Edit PR Review Bot Settings | Greptile
talkhier = TalkHier( | ||
max_iterations=1, | ||
quality_threshold=0.8, | ||
model_name="gpt-4o", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
syntax: model_name='gpt-4o' appears to be a typo and should be 'gpt-4'
model_name="gpt-4o", | |
model_name="gpt-4", |
with ThreadPoolExecutor() as executor: | ||
evaluations = list(executor.map( | ||
lambda x: run_evaluator(*x), | ||
zip(self.evaluators, eval_inputs) | ||
)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style: ThreadPoolExecutor should specify max_workers to prevent resource exhaustion with many evaluators
json_match = re.search(r"\{.*\}", json_str, re.DOTALL) | ||
if (json_match): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logic: regex pattern r"{.}" is greedy and may match too much - use r"{.?}" for non-greedy matching
json_match = re.search(r"\{.*\}", json_str, re.DOTALL) | |
if (json_match): | |
json_match = re.search(r"\{.*?\}", json_str, re.DOTALL) | |
if (json_match): |
message=response.get("message", ""), | ||
background=response.get("background", ""), | ||
intermediate_output=response.get("intermediate_output", {}), | ||
sender=sender.agent_name, |
Check failure
Code scanning / Pyre
Incompatible parameter type Error
background=response.get("background", ""), | ||
intermediate_output=response.get("intermediate_output", {}), | ||
sender=sender.agent_name, | ||
receiver=receiver.agent_name, |
Check failure
Code scanning / Pyre
Incompatible parameter type Error
"content": current_content.get("content", {}).get( | ||
"main_body", "" | ||
), | ||
"final_score": evaluation["scores"]["overall"], |
Check failure
Code scanning / Pyre
Uninitialized local Error
"content_metadata": current_content.get( | ||
"content", {} | ||
).get("metadata", {}), | ||
"evaluation": evaluation, |
Check failure
Code scanning / Pyre
Uninitialized local Error
This PR introduces the
TalkHier
framework, a novel hierarchical multi-agent system designed for structured content generation and refinement.TalkHier
leverages a team of specialized agents, each with a distinct role, to collaboratively produce high-quality content through iterative communication and evaluation.Key Features:
TalkHier
employs a hierarchical structure with agents specializing in different aspects of content creation:CommunicationEvent
objects, ensuring clear and organized information exchange in JSON format. This includes messages, background context, intermediate outputs, sender/receiver information, and timestamps.TalkHier
dynamically generates evaluation criteria tailored to each specific content generation task, ensuring relevant and targeted feedback.ThreadPoolExecutor
to enable parallel content evaluation by multiple evaluator agents, speeding up the refinement process.max_iterations
,quality_threshold
, andmodel_name
to customize the content generation process.Conversation
object to log agent interactions and the overall content generation history for debugging and analysis.Benefits:
📚 Documentation preview 📚: https://swarms--783.org.readthedocs.build/en/783/