You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, tune uses foreach to facilitate parallel computing. As a group, we've discussed gradually deprecating our support for foreach in favor of the futureverse. The big ideas:
future does a better job identifying globals (i.e. no need for namespacing where we usually wouldn't, reassigning in strange situations), generating random numbers soundly, and handling errors, and comes with better progress support via progressr.
Newer R users are more likely to be familiar with the future user API (plan(multisession), etc) than the foreach one.
The futureverse is more actively developed/maintained than foreach.
Spent some time thinking through how this might come together this morning. The process could look something like this:
Timeline
User API
Developer API
Backend
Now
foreach
foreach
foreach
Proposed
future (foreach supported)
foreach (via doFuture)
future
2/3 Years Out
future
future (maybe via furrr)
future
By using doFuture as a backend, our codebase looks similar to how it does with foreach, which can help us with maintainability during the transition period where we support users using either the foreach user API (e.g. registerDoParallel()) or the future user API (i.e. plan(multisession)). Depending on the user's setup, we'd use foreach::`%do%`, foreach::`%doPar%`, or doFuture::`%doFuture%`where we currently just decided between the first two.
Currently, for error logging, tune uses ad-hoc machinery for logging when tuning in parallel and cli for logging sequentially. I'm not sure that we want we want to maintain a third logging system for parallelism with progressr during the transition period. Once we fully deprecate support for foreach, though, we should consider rewriting all of our error logging with progressr.
🚀🚀🚀
The text was updated successfully, but these errors were encountered:
Currently, tune uses foreach to facilitate parallel computing. As a group, we've discussed gradually deprecating our support for foreach in favor of the futureverse. The big ideas:
plan(multisession)
, etc) than the foreach one.Spent some time thinking through how this might come together this morning. The process could look something like this:
By using doFuture as a backend, our codebase looks similar to how it does with foreach, which can help us with maintainability during the transition period where we support users using either the foreach user API (e.g.
registerDoParallel()
) or the future user API (i.e.plan(multisession)
). Depending on the user's setup, we'd useforeach::`%do%`
,foreach::`%doPar%`
, ordoFuture::`%doFuture%`
where we currently just decided between the first two.Currently, for error logging, tune uses ad-hoc machinery for logging when tuning in parallel and cli for logging sequentially. I'm not sure that we want we want to maintain a third logging system for parallelism with progressr during the transition period. Once we fully deprecate support for foreach, though, we should consider rewriting all of our error logging with progressr.
🚀🚀🚀
The text was updated successfully, but these errors were encountered: