v0.0.22

What's Changed

• docs: comparing llms using ragas evaluations by @tinomaxthayil in #314
• convert context precision to json format by @shahules786 in #322
• fix: faithfulness.md by @kaiyikang in #328
• fix: structure faithfulness output by @shahules786 in #333
• fix: Column remap fix for Dataset evaluation by @tinomaxthayil in #331
• Fix OpenAIEmbeddings reference error in answer_relevancy by @ajinkyavbhandare in #329
• Fix: minor improvements across metrics by @shahules786 in #337
• fix wrong number by @Yongtae723 in #338
• fix: faithfulness for unrelated answering by @shahules786 in #340
• fix: load response as json by @shahules786 in #341
• docs: remove answer_relevancy from llm-docs by @tinomaxthayil in #343
• fix: bedrock llm error when evaluating rag qa validate_api_key by @arm-diaz in #350
• Added integration with Amazon API Gateway + bug fix in _answer_relevance.py by @brunopistone in #327
• feat: improve quality of answer correctness by @shahules786 in #339
• docs: add zeno visualization integration by @cabreraalex in #359
• fix(metrics): avoid getting none in context_recall by @yuukidach in #355
• fix: improved answer relevancy by @shahules786 in #346
• fix(types): fix pyright type issues with latest pyright version by @jjmachan in #366
• Retry for failed json loading by @shahules786 in #364
• fix: handle edge cases in prompt processing by @shahules786 in #374

New Contributors

• @kaiyikang made their first contribution in #328
• @ajinkyavbhandare made their first contribution in #329
• @brunopistone made their first contribution in #327
• @cabreraalex made their first contribution in #359
• @yuukidach made their first contribution in #355

Full Changelog: v0.0.21...v0.0.22

v0.0.21

What's Changed

• docs: Import AnswerSimilarity by @nelsonauner in #296
• FIx: testset_generator.py by @ferdinandl007 in #291
• Case-insensitive and whitespace handling for context_precision. by @Pauldevillers in #297
• Fix/tiny refactor by @Yongtae723 in #305
• fix: broken link to metrics by @younes-io in #311
• Convert context_recall to json format by @shahules786 in #312
• fix: add retry logic for OpenAI and Azure OpenAI by @jjmachan in #315
• fix: openai env var load after init and before score also by @jjmachan in #316
• import llama_index gracefully, solves #318 by @ceferisbarov in #319
• Make testgenerator output compatible with evaluate by @tinomaxthayil in #302

New Contributors

• @nelsonauner made their first contribution in #296
• @ferdinandl007 made their first contribution in #291
• @Pauldevillers made their first contribution in #297
• @younes-io made their first contribution in #311
• @ceferisbarov made their first contribution in #319

Full Changelog: v0.0.20...v0.0.21

v0.0.20

What's Changed

• docs: update how LLMs are customized in Ragas metrics by @jjmachan in #219
• fix: Add RagasLLM for customized test set generation by @chankeith96 in #242
• docs: add import statement in testset generation by @jjmachan in #249
• expose metrics prompts by renaming metric modules by @grauvictor in #243
• fix: added support for google vertexAI by @jjmachan in #252
• Renamed module names and made changes in init to reflect the change by @tinomaxthayil in #244
• added hint in synthetcic data generation by @tinomaxthayil in #259
• feat: RagasEmbeddings by @shahules786 in #232
• Fix: type hints by @shahules786 in #263
• Update issue templates by @shahules786 in #265
• Update evaluation.md typo by @whitewum in #275
• removed ragas score from result object by @tinomaxthayil in #276
• Custom embeddings from HuggingFace (sentence transformer) do not pass the assertion in src/ragas/metrics/_answer_similarity.py by @alebondarenko in #277
• fix: set temperate to near zero value by @shahules786 in #267
• feat: add native support for OpenAI and AzureOpenAI by @jjmachan in #261
• feat: make ragas pip more lighter by @jjmachan in #283
• docs: amazon bedrock (#241) by @arm-diaz in #246
• docs: added vertext ai support by @jjmachan in #284
• chore: pin OpenAI to >1 by @jjmachan in #285

New Contributors

• @chankeith96 made their first contribution in #242
• @grauvictor made their first contribution in #243
• @tinomaxthayil made their first contribution in #244
• @whitewum made their first contribution in #275
• @alebondarenko made their first contribution in #277

Full Changelog: v0.0.19...v0.0.20

v0.0.19

What's Changed

• docs: fix typo by @shahules786 in #214
• feat: context average precision by @shahules786 in #204
• fix: avoid division by zero by @shahules786 in #218
• fix: typecast in TestsetGeneration fails for python3.8 by @jjmachan in #215
• fix: make context_relevancy available by @jjmachan in #229
• feat: add support for llms on AWS Bedrock (#226) by @arm-diaz in #227
• tests: for metrics by @jjmachan in #233
• docs: add context_precision by @shahules786 in #236
• docs: add langfuse notebook by @jjmachan in #216
• Fix typos in langfuse.ipynb by @marcklingen in #237

New Contributors

• @arm-diaz made their first contribution in #227
• @marcklingen made their first contribution in #237

Full Changelog: v0.0.18...v0.0.19

v0.0.18

What's Changed

• fix: use llm from baseclass for metrics by @jjmachan in #189
• bugfix: context_precision by @shahules786 in #194
• Fix typos in Eval doc by @mmaysami in #198
• testset generation : Improvements by @shahules786 in #195
• feat: langchain documents support for TestsetGenerator by @jjmachan in #201
• docs: document loaders support by @shahules786 in #202
• feat: support for multiple n in llms by @jjmachan in #197
• feat: llamaIndex llm support by @jjmachan in #205
• Refactor testset_generator.py for newbies to understand easily by @Yongtae723 in #207
• fix: remove limit on pydantic 2.0 by @jjmachan in #209

New Contributors

• @mmaysami made their first contribution in #198

Full Changelog: v0.0.17...v0.0.18

v0.0.17

What's Changed

• Testset generation: add gaurdrails to chunk filtering by @shahules786 in #166
• Docs testset generation by @shahules786 in #165
• don't warn when context_precision is correctly called by @monstertruck in #168
• fixed chunk validation by @shahules786 in #169
• doc: init docs.ragas.io by @jjmachan in #170
• docs: Add references to ragas by @jjmachan in #171
• Docs: added contents by @shahules786 in #175
• Answer correctness metrics by @shahules786 in #161
• added imports to doc by @shahules786 in #178
• docs: added github and discord links by @jjmachan in #179
• Fix typos by @pitmonticone in #180
• docs: add refferences by @jjmachan in #177
• docs: support notebooks by @jjmachan in #182
• added emojis to docs by @shahules786 in #183
• Docs improvements by @shahules786 in #186
• testset generation: bug fixes by @shahules786 in #185

New Contributors

• @monstertruck made their first contribution in #168
• @pitmonticone made their first contribution in #180

Full Changelog: v0.0.16...v0.0.17

v0.0.16

What's Changed

• docs: azure openai quickstart by @jjmachan in #155
• fix: llamaIndex issues by @jjmachan in #156
• feat: changing context_relevancy to context_precision by @jjmachan in #157
• Test set generation paradigm by @shahules786 in #150
• Testset generation: documentation by @shahules786 in #158

Full Changelog: v0.0.15...v0.0.16

v0.0.15

What's Changed

• added contact info by @shahules786 in #142
• Add note about when ground truth labels are required by @tleyden in #146
• bug fix: embeddings -> embedding by @starrywheat in #134
• ZeroDivisionError in context_relevance by @DevanshuBrahmbhatt in #154

New Contributors

• @tleyden made their first contribution in #146
• @DevanshuBrahmbhatt made their first contribution in #154

Full Changelog: v0.0.14...v0.0.15

v0.0.14

What's Changed

• Update README.md by @emilesilvis in #139
• Fix remap_column_names by @yujonglee in #140

New Contributors

• @emilesilvis made their first contribution in #139
• @yujonglee made their first contribution in #140

Full Changelog: v0.0.13...v0.0.14

v0.0.13

General bug fixes and reliability improvements

What's Changed

• fix: spelling of context_relevancy by @jjmachan in #113
• update tutorial notebook by @Yongtae723 in #125
• Fix/column-map: fix bug in column_map argument of evaluation by @jjmachan in #121
• Update README with explanation of Ragas metrics by @Yongtae723 in #124
• Improve metrics by @shahules786 in #131
• add custom embeddings for answer relevance by @starrywheat in #123
• fix: remove dependency on protobuff by @jjmachan in #132
• fix: added support for azure LLM by @jjmachan in #133
• fix: update quickstart notebook by @jjmachan in #137
• fix: linting issues by @jjmachan in #138

New Contributors

• @Yongtae723 made their first contribution in #125
• @starrywheat made their first contribution in #123

Full Changelog: v0.0.12...v0.0.13