Listing per-method changes

[Mattdl]

I've been using Avalanche for my research and I have to say: great job all!

One thing I'm struggling with though, is that it's not really clear whenever fixes or updates of certain methods have been added.
I always try to work with the most recent version, but as research progresses, months pas by and I end up working with an outdated version. I don't want to update the whole avalanche codebase, as it might influence results I've previously obtained. But I do want to check if there were bugs being fixed in baselines (like recent JOINT/AGEM fixes).

Is there a way to work with patches for these baseline fixes so I don't have to update all Avalanche?
Right now, I'd have to manually check all the fixes being made, which makes it quite cumbersome.
An idea would be to keep logs to track the updates for each of the baselines. This could be on the website or in the code itself, with entries of when the update was made, an perhaps extra info like 'who' etc.
What do you think? Or does this already exist?

[AntonioCarta]

This is a critical issue. Right now we are still in a development phase where we do continuous changes even on the basic methods, which is not ideal for reproducibility.

IMO once we have versioning the easier thing to do would be to just work with a fixed version for every single project.
Maybe we could also use the loggers to save the avalanche version used for the experiment (or commit hash for the moment), so that it's easier to track whether an old experiment was using bugged strategies.

An idea would be to keep logs to track the updates for each of the baselines. This could be on the website or in the code itself, with entries of when the update was made, an perhaps extra info like 'who' etc.

The git history should have everything needed. In github, you can check the single files and see their relevant commits. Probably, you can apply a single commit to your local branch using some git commands.
We could have a separate place where we list changes in a more structured form. IMO, this is a changelog that should be updated at every release (not at every commit), so it doesn't help until we start making releases.

I'm happy to hear any other suggestions. I think is really a top priority, along with a battery of experiments that guarantees avalanche's methods do achieve the desired performance in some basic settings (e.g. reproducing paper's experiments).

PS Keep in mind that in Italy we are all on vacation in August, so don't expect any big changes soon ;)