Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tutorial for evaluating LangGraph agents #1636

Conversation

sahusiddharth
Copy link
Contributor

This PR adds a detailed tutorial to guide users through building a ReAct agent using LangGraph. The tutorial also walks users through setting up an evaluation pipeline using Ragas to assess the agent's performance.

@dosubot dosubot bot added the size:XL This PR changes 500-999 lines, ignoring generated files. label Nov 6, 2024
Copy link
Member

@jjmachan jjmachan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall I think the tutorial is really good but I don't think this should be a getstarted blog - it's too specific to langgraph.

What I'm thinking is add a couple of these integration blogs to

  1. langgraph
  2. crewAI
  3. OpenAI's swarm

ideally for the same usecase and then we'll have enough experience to build a lighter getting started tutorial that is framework agnostic

For now I think you should add this into integration/langgraph

what do you guys think?

docs/getstarted/agent_evaluation.md Outdated Show resolved Hide resolved
@sahusiddharth
Copy link
Contributor Author

Yeah, that makes sense! Adding it to the integration/langgraph section for now sounds like a good to me.

docs/getstarted/agent_evaluation.jpg Outdated Show resolved Hide resolved
docs/getstarted/agent_evaluation.md Outdated Show resolved Hide resolved
@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels Nov 7, 2024
@jjmachan
Copy link
Member

jjmachan commented Nov 7, 2024

also could you add the integration to reference section: https://github.com/explodinggradients/ragas/blob/main/docs/references/integrations.md and provide a link to that in the docs

and add the integrations md file to

ragas/mkdocs.yml

Lines 90 to 92 in 895f9c0

- Integrations:
- howtos/integrations/index.md
- LlamaIndex: howtos/integrations/_llamaindex.md
so that it is part of the doc site

Copy link
Member

@jjmachan jjmachan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

I know this took a lot of back and forth but you did a great job with your fast responses 🙂 - looking forward to working with you more!

Copy link
Member

@shahules786 shahules786 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, @sahusiddharth! Thanks for patiently addressing the reviews and iterating swiftly ❤️

@shahules786 shahules786 merged commit 1d170d7 into explodinggradients:main Nov 8, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:XXL This PR changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

LangGraph Agent Evaluation Tutorial
3 participants