Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added code generation example #164

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

added code generation example #164

wants to merge 3 commits into from

Conversation

epinzur
Copy link
Collaborator

@epinzur epinzur commented Mar 17, 2025

No description provided.

@epinzur epinzur requested a review from bjchambers March 17, 2025 23:44
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@coveralls
Copy link

coveralls commented Mar 17, 2025

Pull Request Test Coverage Report for Build 13912090509

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 93.433%

Totals Coverage Status
Change from base Build 13811569492: 0.0%
Covered Lines: 1558
Relevant Lines: 1643

💛 - Coveralls

Copy link

github-actions bot commented Mar 17, 2025

Test Results

    8 files  ±0      8 suites  ±0   3m 7s ⏱️ -56s
  528 tests ±0    504 ✅ +128   24 💤 ±0  0 ❌ ±0 
2 112 runs  ±0  1 124 ✅ +128  988 💤 ±0  0 ❌ ±0 

Results for commit 1d0e76f. ± Comparison against base commit 94d2e5e.

♻️ This comment has been updated with latest results.

@@ -0,0 +1,817 @@
{
Copy link
Collaborator

@bjchambers bjchambers Mar 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An interesting alternative could be telling it to ask questions as needed, such as what kind of authentication to use, etc.


Reply via ReviewNB

@@ -0,0 +1,817 @@
{
Copy link
Collaborator

@bjchambers bjchambers Mar 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to update this with links to relevant helpers.


Reply via ReviewNB

@@ -0,0 +1,817 @@
{
Copy link
Collaborator

@bjchambers bjchambers Mar 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe strengthen the introduction with details from the conclusion -- "We show that using Graph RAG to navigate the documentation produces working code, while vector search on the same documents fails". This is an exciting result, and we should make sure that people understand that early.


Reply via ReviewNB

@@ -0,0 +1,817 @@
{
Copy link
Collaborator

@bjchambers bjchambers Mar 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strengthen the statement here -- this sounds "so-so". But, (a) other techniques *fail to generate usable code* and (b) this *works*.

Instead of "isn't perfect, but it works", "generates working code when the LLM only and standard Vector-based RAG fail".


Reply via ReviewNB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants