Question: How are experiment results accessed when doing the paper writeup?

[neil-vqa]

Thank you for your contribution to the LLM open source community.

I've been going through the code for several hours now, so from what I understand, the experiment.py file will save the experiment outputs in a final_info.json file. So, I've been looking for how this final_info.json file will be accessed in the perform_writeup function, but I can't seem to see where it is, so that I can access the experimental results. Please can you point out or enlighten me how this works? How can I access the experimental results when performing the writeup? My goal is to tweak the writeup prompt but I need the experiment results. Thank you!

[conglu1997]

Via the notes.txt file :) Also via the aider context.

[ibrahimptester]

@conglu1997 @neil-vqa hello I'm trying to generate paper on a new topic but I'm confused about which files should I edit. can you help me out please? I know the custom templates says to edit the prompt.json and seed.json but is it enough to just change those two or we need to change the whole template, experiment.py altogether?

[ibrahimptester]

I've done
install miniconda
setup environment
git clone repository
pip install requirements.txt
export open_api key
python prepare scripts
now after these things should I edit the prompt.json then run the baseline and then launch experiment.py

please clear the confusion for me.