Skip to content

ASSERT-KTH/repairbench

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

87 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

RepairBench

Leaderboard of frontier models for program repair.

If you use RepairBench, please cite:

@techreport{repairbench,
  title={RepairBench: Leaderboard of Frontier Models for Program Repair}, 
  author={André Silva and Martin Monperrus},
  year={2024},
  url={https://arxiv.org/abs/2409.18952}, 
  number = {2409.18952},
  institution = {arXiv},
}

For the code to reproduce the benchmark, please refer to https://github.com/ASSERT-KTH/repairbench-framework

Structure

  • results includes all prompts, patches, and evaluation results
  • scripts contains scripts used to parse results into other formats
  • website contains the leaderboard's website code