You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add a MergeModelCallback that merges the reference model with the current policy and optionally pushes the merged checkpoint to the Hub. This could be done on step/epoch end and/or the end of training. Implementation-wise, we could use Arcee's mergekit lib and include it as an optional dependency: https://github.com/arcee-ai/mergekit
Motivation
Various papers show that model merging can non-trivially improve performance, especially if the models belong to the same architecture:
Feature request
Add a
MergeModelCallback
that merges the reference model with the current policy and optionally pushes the merged checkpoint to the Hub. This could be done on step/epoch end and/or the end of training. Implementation-wise, we could use Arcee'smergekit
lib and include it as an optional dependency: https://github.com/arcee-ai/mergekitMotivation
Various papers show that model merging can non-trivially improve performance, especially if the models belong to the same architecture:
Your contribution
Open to the community!
The text was updated successfully, but these errors were encountered: