v0.1.15

What's Changed

• fix: typo in model name by @shahules786 in #1198
• metrics: add domain specific rubrics based scoring by @vaishakhRaveendran in #1189
• docs: rubrics based metrics by @shahules786 in #1200
• docs: add context utilisation to the index by @shahules786 in #1202
• Docs: fix typo in metrics by @shahules786 in #1203
• Implementation of Noise sensitivity metrics from RAGChecker by @sahusiddharth in #1190
• Tiny typo fix: it's -> its by @peterkaminski in #1219
• fix : solves #1215 - Inconsistency in the Value of EVALMODE_TO_COLUMNS[EvaluationMode.ca] by @jucgonzalezes in #1216
• chore: removed extra of requirements by @jjmachan in #1225
• Bugfix of evaluate function in llama_index.py by @vcim in #1223
• [FIX] - Fix for summarization edge case by @sky-2002 in #1201

New Contributors

• @peterkaminski made their first contribution in #1219
• @jucgonzalezes made their first contribution in #1216
• @vcim made their first contribution in #1223

Full Changelog: v0.1.14...v0.1.15

v0.1.14

What's Changed

• Add question to context recall doc by @shahules786 in #1173
• Add context utilization by @shahules786 in #1174
• doc: fix output for testset generation and missing files by @jjmachan in #1176
• Helicone Integration by @chitalian in #1178
• docs: minor fixes by @shahules786 in #1183
• solved issue that adaptation crashes when havig multiple nested promp… by @ssoima in #1181
• Bugfix Summarization Score by @emreds in #1164
• [DOCS] - Fix summarization score expression by @sky-2002 in #1184
• metrics: add faithfulness with HHEM model by @shahules786 in #1191
• Fix test command in development.md to match Makefile configuration by @sahusiddharth in #1195
• feat: deeper helicon integration by @jjmachan in #1196
• fix: added a test for prompts by @jjmachan in #1197

New Contributors

• @chitalian made their first contribution in #1178
• @ssoima made their first contribution in #1181
• @emreds made their first contribution in #1164
• @sahusiddharth made their first contribution in #1195

Full Changelog: v0.1.13...v0.1.14

v0.1.13

What's Changed

• Fix "global flags not at the start" raised in extract_text function by @vaishakhRaveendran in #1113
• Random seed fixed. by @Gwenn-LR in #1142
• feat: Usage tracking for evaluations by @jjmachan in #1129
• Fixing Demo Code - Compare Embeddings for retriever Docs by @matheusft in #1153
• fix: run_config post_init issue by @jjmachan in #1158
• Add save function for statement_prompt by @miyaketomoya in #1157
• Fix: adapt function to translate cached prompts in place. by @HerrIvan in #889
• fix: RunConfig documentation and improvements by @jjmachan in #1159
• Docs/run config by @jjmachan in #1160
• change segmenter language when language adaptaion by @miyaketomoya in #1163
• change default model to gpt4o-mini by @shahules786 in #1166
• Add tests to ragas.run_config.RunConfig rng attribute. by @Gwenn-LR in #1169
• feat: added track_completed decorator to track completed functions by @jjmachan in #1171
• docs: added development.md file by @jjmachan in #1115
• Allow Metric.score to work within an existing asyncio loop by @abramowskia in #1161

New Contributors

• @Gwenn-LR made their first contribution in #1142
• @matheusft made their first contribution in #1153
• @miyaketomoya made their first contribution in #1157
• @abramowskia made their first contribution in #1161

Full Changelog: v0.1.12...v0.1.13

v0.1.12

What's Changed

• [Fix bug] The adaptation for rewrite_invalid_question_prompt was not saved by @psydok in #1071
• [Fix bug] utils.py by @jrredondo in #1083
• Llama Index integration: optional RunConfig for evaluate() by @antono in #1085
• Create a custom directory loader for RAGAS by @vaishakhRaveendran in #1122
• Fix "sentence" -> "claim" the context recall metric docs by @artmatsak in #1126
• fix: temperature parameter in generate_text not ignored. by @HerrIvan in #887

New Contributors

• @psydok made their first contribution in #1071
• @jrredondo made their first contribution in #1083
• @antono made their first contribution in #1085
• @vaishakhRaveendran made their first contribution in #1122
• @artmatsak made their first contribution in #1126
• @HerrIvan made their first contribution in #887

Full Changelog: v0.1.11...v0.1.12

v0.1.11

What's Changed

• Testgeneration v3 by @shahules786 in #1024
• fix: import errors and make experimental runnable by @jjmachan in #1078
• chore: added analytics to count the num of testsets generated with experimental by @jjmachan in #1081
• Fix mutable dataclass defaults by @yasyf in #1082
• fix(executor): simplify executor by removing threading by @jjmachan in #1093
• chore: added ragas as a dependency by @jjmachan in #1095
• test: added more tests for executor by @jjmachan in #1110
• chore: remove context_relevancy from library by @jjmachan in #1111
• Update index.md by @IdaJess in #1117
• Improve Clarity and Structure of Faithfulness Explanation in Documentation by @aybruhm in #1114
• fix: remove is_async argument since everything is async now by @jjmachan in #1116

New Contributors

• @yasyf made their first contribution in #1082
• @IdaJess made their first contribution in #1117
• @aybruhm made their first contribution in #1114

Full Changelog: v0.1.10...v0.1.11

v0.1.10

What's Changed

• Update testset_generation.md - fix typo by @GTimothee in #1008
• bugfix: evaluation thread timeout by @NIL-zhuang in #858
• Update index.md by @shahules786 in #1023
• Add initial version of summarization metric by @sky-2002 in #990
• feat: added experimental package in ragas by @jjmachan in #1056
• Docs for summarization metric by @sky-2002 in #1026
• adding get_available_metrics method by @immortal3 in #1025
• chore: Fix typo in index.md by @anfindsen in #1058
• fix(testset): for generating testset with new docs by @Manav916 in #1001
• Fix counter in Runner and Executor in case of any exception by @baptiste-pasquier in #995
• Chore: Fix RuntimeError: ... got Future <..> attached to a different loop by @abetatos in #979
• fix: langchain EvaluatorChain OpenAI validation error by @danielfornarini in #1051
• chore: removed async time tests from benchmarks by @jjmachan in #1061
• Feature/updating prompt by @louky123 in #1050

New Contributors

• @GTimothee made their first contribution in #1008
• @NIL-zhuang made their first contribution in #858
• @immortal3 made their first contribution in #1025
• @anfindsen made their first contribution in #1058
• @Manav916 made their first contribution in #1001
• @baptiste-pasquier made their first contribution in #995
• @abetatos made their first contribution in #979
• @danielfornarini made their first contribution in #1051
• @louky123 made their first contribution in #1050

Full Changelog: v0.1.9...v0.1.10

v0.1.9

Feat/llamaindex: adding llamaindex (#999)

fixes: #557 


its been long 🙂

v0.1.8

What's Changed

• Devanshu's Blog Added in Doc's community folder. by @DevanshuBrahmbhatt in #869
• Update data_preparation doc link in evaluation.md by @ragul-kachiappan in #874
• docs: langchain.document_loaders is deprecated by @bioerrorlog in #881
• chores: change cal link by @shahules786 in #892
• chore: added issue manager by @jjmachan in #907
• fix: Feat/issue closer by @jjmachan in #909
• fix(docs): update document regarding langfuse usage by @jjmachan in #904
• docs: clean up integrations by @jjmachan in #919
• doc: Updated vertexai python notebook to use langchain wrapper approach for vertex ai by @shawnmittal in #915
• Add new RAG evaluation articles by @Yongtae723 in #932
• Fix/ci: fix the broken CI by @jjmachan in #933
• reproducibility PR1 : introduce ensembler by @shahules786 in #924
• Cleaned installation step in README by @andreped in #921
• reproducibility PR2: add new statement creation #930 by @shahules786 in #941
• Update generator.py - Redundant code by @omkar-334 in #942
• docs: improve vertex ai docs by @jjmachan in #951
• docs: small fixes in vertex ai docs by @jjmachan in #952
• Update discord invite link by @shahules786 in #953
• fix: sentence segmenter for multi languages by @shahules786 in #946
• answer-correctness : fix edge cases by @shahules786 in #970
• Fix ensembler : handle edge cases by @shahules786 in #972
• feat: adding new defaults to openai Model and embeddings by @jjmachan in #973
• Improve grammar and punctuation in critique prompts: addresses #971 by @ruankie in #974
• Fix testset generator issue on context selection by @Jovillios in #937
• feat: Ragas CI/CD by @jjmachan in #976
• fix: patch type issue in evolution parsing by @shahules786 in #980

New Contributors

• @ragul-kachiappan made their first contribution in #874
• @bioerrorlog made their first contribution in #881
• @shawnmittal made their first contribution in #915
• @andreped made their first contribution in #921
• @omkar-334 made their first contribution in #942
• @ruankie made their first contribution in #974
• @Jovillios made their first contribution in #937

Full Changelog: v0.1.7...v0.1.8

v0.1.7

What's Changed

• minor: fix llm expansion in docs by @glenacota in #847
• Update llama_index imports by @sestinj in #849
• feat: Change parsers and add custom critic model by @shahules786 in #838
• fix: change documentation to reflect accurate method name by @dbtreasure in #837
• Update answer_relevance.md by @carllapierre in #832
• fix ci: some linting issues by @jjmachan in #852

New Contributors

• @glenacota made their first contribution in #847
• @sestinj made their first contribution in #849
• @dbtreasure made their first contribution in #837
• @carllapierre made their first contribution in #832

Full Changelog: v0.1.6...v0.1.7

v0.1.6

What's Changed

• Fix: json format inconsistency in context recall (prompt for statemen… by @jokokojote in #776
• fix: Explanaitions typo on index.md by @dcamejo1 in #801
• feat(integrations): add support for langchain and langsmith by @jjmachan in #795
• Docs: fix confusion regarding getting started by @shahules786 in #815
• docs: add openlayer integration by @shahules786 in #827
• fix: Eventloop unclosed by @yunfwe in #796
• More robust JSON prompting and parsing by @mrtj in #807
• fix(integration): langchain integration and CI fixes by @jjmachan in #830
• Added more examples to the translation prompts by @baswenneker in #826
• fix: use aparse in all metrics by @shahules786 in #831

New Contributors

• @dcamejo1 made their first contribution in #801
• @yunfwe made their first contribution in #796
• @mrtj made their first contribution in #807
• @baswenneker made their first contribution in #826

Full Changelog: v0.1.5...v0.1.6