Update Scheduler to not persist SchedulingPlan and accept equivalent placements #2242
Comments
tillrohrmann
added a commit
to tillrohrmann/restate
that referenced
this issue
Nov 15, 2024
This commit adds an periodic SchedulingPlan check which allows multiple Schedulers to eventually agree on the same SchedulingPlan. Before, it could happen that there are two Schedulers that have different SchedulingPlans (e.g. one still having an older version). If those plans are contradicting, then the Schedulers would instruct nodes differently. The periodic update helps preventing this situation. Note: This logic can be removed once we allow the scheduler to accept equivalent partition processor placements. See restatedev#2242.
tillrohrmann
added a commit
to tillrohrmann/restate
that referenced
this issue
Nov 15, 2024
This commit adds an periodic SchedulingPlan check which allows multiple Schedulers to eventually agree on the same SchedulingPlan. Before, it could happen that there are two Schedulers that have different SchedulingPlans (e.g. one still having an older version). If those plans are contradicting, then the Schedulers would instruct nodes differently. The periodic update helps preventing this situation. Note: This logic can be removed once we allow the scheduler to accept equivalent partition processor placements. See restatedev#2242.
tillrohrmann
added a commit
to tillrohrmann/restate
that referenced
this issue
Nov 22, 2024
This commit adds an periodic SchedulingPlan check which allows multiple Schedulers to eventually agree on the same SchedulingPlan. Before, it could happen that there are two Schedulers that have different SchedulingPlans (e.g. one still having an older version). If those plans are contradicting, then the Schedulers would instruct nodes differently. The periodic update helps preventing this situation. Note: This logic can be removed once we allow the scheduler to accept equivalent partition processor placements. See restatedev#2242.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Right now the
Schedulerpersists itsSchedulingPlanto the metadata store as a way to synchronize on it (in case there are multiple CCs running). With #2238, this is no longer strictly required. Especially, we add support to theSchedulerto also accept equivalent placement decisions that were originally taken by a different cluster controller. That way it is not tragic if there is a short period where multiple CCs believe that they are the leader.The text was updated successfully, but these errors were encountered: