You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Something I often find useful when writing my own samplers is to write the output directly to disk in a streaming (on-line) fashion.
It might sound silly, but I've worked in places where forced system updates/reboots have lost me days worth of work at a time, and saving sampler output directly to disk has let me at least recover some of the work (and use it, e.g., for helping to determine potential convergence issues with new/complex models, as prep for subsequent adaptive MCMC runs, etc.) There are also cases where piping the sampler output to some other program in real time (e.g., for plotting, tracking particular parameters of interest, etc.) has been helpful (a slightly different, but related, issue to piping to disk).
The text was updated successfully, but these errors were encountered:
I think we could probably do this by adding generic callback functions to sampling, such that you could have some kind of a plotting/saving/reporting/doing stuff functionality. This exists in one of my branches but it has not yet been formalized into good code -- I can add this in, because I think it's important functionality to have.
Currently this would take place upstream in AbstractMCMC, which I'm more than happy to take a look at.
As to the form that the storage should take, I think probably we could just save each Transition struct in a JLD2 file, and provide a little bit of code to allow for a quick resume of a chain that was terminated because of an error or something.
Something I often find useful when writing my own samplers is to write the output directly to disk in a streaming (on-line) fashion.
It might sound silly, but I've worked in places where forced system updates/reboots have lost me days worth of work at a time, and saving sampler output directly to disk has let me at least recover some of the work (and use it, e.g., for helping to determine potential convergence issues with new/complex models, as prep for subsequent adaptive MCMC runs, etc.) There are also cases where piping the sampler output to some other program in real time (e.g., for plotting, tracking particular parameters of interest, etc.) has been helpful (a slightly different, but related, issue to piping to disk).
The text was updated successfully, but these errors were encountered: