support reservation #62

lzlaa · 2024-08-29T12:17:23Z

No description provided.

binacs

Great work! LGTM overall just a few comments

binacs · 2024-09-06T11:24:02Z

cmd/scheduler/app/options/options.go

@@ -168,6 +168,7 @@ func (o *Options) Flags() (nfs cliflag.NamedFlagSets) {
 fs.Float32Var(&o.ComponentConfig.ClientConnection.QPS, "kube-api-qps", o.ComponentConfig.ClientConnection.QPS, "QPS to use while talking with kubernetes apiserver. This parameter overrides the value defined in config file, which is specified in --config.")
 fs.Int32Var(&o.ComponentConfig.ClientConnection.Burst, "kube-api-burst", o.ComponentConfig.ClientConnection.Burst, "burst to use while talking with kubernetes apiserver. This parameter overrides the value defined in config file, which is specified in --config.")

+ fs.Int64Var(&o.ComponentConfig.ReservationTimeOutSeconds, "reservation-ttl", o.ComponentConfig.ReservationTimeOutSeconds, "how long resources will be reserved (for resource reservation).")


nit: move this to #L185-L192 for more fine-grained configuration in the future

binacs · 2024-09-06T11:25:57Z

manifests/quickstart-feature-examples/godel-demo-default.yaml

@@ -4,5 +4,9 @@ name: godel-demo-default
 nodes:
 - role: control-plane
 image: kindest/node:v1.21.1
+ - role: worker


Do we need more workers for demo cluster?

Yes, reservation needs two worker nodes at least, and one single worker node may cause unexpected error.

binacs · 2024-09-06T11:31:39Z

pkg/binder/utils/reservation.go

+ newPrr := prr.DeepCopy()
+ // skip pod reservation request is not in reserved status.
+ if shouldSkipUpdateReservationStatus(prr) {
+ klog.V(4).InfoS("skip updating pod reservation request, pod reservation request is not in reserved status", "prr", podutil.GetReservationKey(prr))


nit: skip -> Skip

binacs · 2024-09-06T11:32:38Z

pkg/binder/utils/reservation.go

+ }
+ newPrr.Status.Phase = schedulingv1a1.ReservationMatched
+ } else {
+ // TODO: add to count and set in unavaliable status.


nit: no need to go on in this situation

binacs · 2024-09-06T11:40:21Z

pkg/scheduler/eventhandlers.go

@@ -35,12 +30,17 @@ import (
 "k8s.io/client-go/tools/cache"
 "k8s.io/klog/v2"

+ nodev1alpha1 "github.com/kubewharf/godel-scheduler-api/pkg/apis/node/v1alpha1"


Should packages from other repos (godel-scheduler-api, katalyst-api) be located in above?

binacs · 2024-09-06T11:42:04Z

pkg/scheduler/framework/handle/handle.go

@@ -84,4 +85,8 @@ type UnitFrameworkHandle interface {
 FindStore(commonstore.StoreName) commonstore.Store

 GetMaxWaitingDeletionDuration() time.Duration
+
+ // TODO: cleanup the following function methods.


Move theis line to #L87 considering that the GetMaxWaitingDeletionDuration should also be cleand up

binacs · 2024-09-06T11:46:24Z

pkg/util/resource_constants.go

+import v1 "k8s.io/api/core/v1"
+
+const (
+ ResourceBytedanceSocket v1.ResourceName = "bytedance.com/socket"


Please double check that whether bytedance.com may be claimed in open source projects

binacs

lgtm

binacs · 2024-10-10T02:57:30Z

docs/features/resource-reservation.md

+Please make sure that the Godel Controller Manager component has been deployed in the kind cluster along with other scheduler components as control plane components.
+
+```shell
+$ k get po -n godel-system


kubectl or alias k=kubectl

We should ensure that everyone can run these examples easily.

ok, replace "k" to "kubectl".

binacs · 2024-10-10T03:00:17Z

docs/features/resource-reservation.md

+godel.bytedance.com/reservation-ttl=7200
+```
+
+For example, if you configure the above annotation on a Pod or Deployment, the scheduler will automatically release the reserved resources after 24 hours if there is no consumer for the reserved resources.


What if both of the Deployment and Pod configured this field?

Priority needs to be clearly described.

added some notes about the priority

binacs

lgtm

zryfish · 2024-10-17T09:27:39Z

cmd/controller/app/options/insecure_serving.go

+ return
+ }
+
+ fs.StringVar(&o.BindAddress, "address", o.BindAddress, "DEPRECATED: the IP address on which to listen for the --port port (set to 0.0.0.0 for all IPv4 interfaces and :: for all IPv6 interfaces). See --bind-address instead.")


Better to remove those deprecated flags.

I think it is best to keep the behavior consistent with other scheduler components, retain the insecure server configuration, and clean it up later.

zryfish · 2024-10-17T09:29:48Z

cmd/controller/app/options/insecure_serving.go

+ return nil
+}
+
+func updateDeprecatedInsecureServingOptionsFromAddress(is *apiserveroptions.DeprecatedInsecureServingOptionsWithLoopback, addr string) error {


This function is unnecessary if deprecated flags are removed.

I think it is best to keep the behavior consistent with other scheduler components, retain the insecure server configuration, and clean it up later.

As the opinion shows, we need keep these flags.

zryfish · 2024-10-17T09:35:34Z

cmd/controller/app/options/leadermigration.go

+}
+
+// DefaultLeaderMigrationOptions returns a LeaderMigrationOptions with default values.
+func DefaultLeaderMigrationOptions() *LeaderMigrationOptions {


I don't think we need LeaderMigration. According to design doc https://github.com/kubernetes/enhancements/tree/master/keps/sig-cloud-provider/2436-controller-manager-leader-migration,

Support a migration process for large scale and highly available Kubernetes clusters using the in-tree cloud providers (via kube-controller-manager and kubelet) to their out-of-tree equivalents (via cloud-controller-manager).

Good idea, removed it.

zryfish · 2024-10-17T09:38:03Z

cmd/controller/app/options/reservation.go

+}
+
+// TODO
+func (opt *ReservationControllerOptions) Validate() error {


It's better to add some validations.

No validation requirement of ReservationControllerOptions, so just delete these method.

zryfish · 2024-10-17T09:40:19Z

docs/features/resource-reservation.md

+This document uses a kind cluster as an example to introduce how to enable and use the resource reservation capability of the Godel Scheduler.
+
+## Prerequisites
+


But how to deploy godel controller manager, is there any documentation, or links maybe?

Follow the docs/features/kind-cluster-setup.md, users can deploy godel controller manager in kind cluster.

zryfish · 2024-10-17T09:44:24Z

pkg/binder/controller/movement_controller.go

@@ -1,3 +1,19 @@
+/*
+Copyright 2023 The Godel Scheduler Authors.


Good catch. fix it.

lzlaa requested review from zryfish, XinyiSong, Liumeng-lambert, binacs, yuey002, lintongj and NickrenREN as code owners August 29, 2024 12:17

binacs reviewed Sep 6, 2024

View reviewed changes

lzlaa force-pushed the dev/support-reservation branch 2 times, most recently from 890575a to 60bfc95 Compare September 10, 2024 11:31

lzlaa force-pushed the dev/support-reservation branch 2 times, most recently from af5a097 to da3cc83 Compare September 19, 2024 09:27

binacs previously approved these changes Sep 19, 2024

View reviewed changes

lzlaa changed the title ~~WIP: support reservation~~ support reservation Sep 20, 2024

lzlaa added 2 commits September 23, 2024 16:38

support reservation

5e4ab3e

add resource reservation doc

958f98a

lzlaa dismissed binacs’s stale review via 958f98a September 23, 2024 08:39

lzlaa force-pushed the dev/support-reservation branch from da3cc83 to 958f98a Compare September 23, 2024 08:39

lzlaa requested a review from sxy390 as a code owner September 23, 2024 08:39

binacs reviewed Oct 10, 2024

View reviewed changes

update doc

15cdbd2

binacs previously approved these changes Oct 13, 2024

View reviewed changes

update license header

40324b1

lzlaa dismissed binacs’s stale review via 40324b1 October 14, 2024 11:48

zryfish reviewed Oct 17, 2024

View reviewed changes

remove leader migration configuration

fc7276c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

support reservation #62

support reservation #62

lzlaa commented Aug 29, 2024

binacs left a comment

binacs Sep 6, 2024

lzlaa Sep 10, 2024

binacs Sep 6, 2024

lzlaa Sep 10, 2024

binacs Sep 6, 2024

lzlaa Sep 10, 2024

binacs Sep 6, 2024

lzlaa Sep 10, 2024

binacs Sep 6, 2024

binacs Sep 6, 2024

lzlaa Sep 10, 2024

binacs Sep 6, 2024

binacs left a comment

binacs Oct 10, 2024

lzlaa Oct 10, 2024

binacs Oct 10, 2024

lzlaa Oct 10, 2024

binacs left a comment

zryfish Oct 17, 2024

lzlaa Oct 18, 2024

zryfish Oct 17, 2024

lzlaa Oct 18, 2024

zryfish Oct 17, 2024

lzlaa Oct 18, 2024

zryfish Oct 17, 2024

lzlaa Oct 17, 2024 •

edited

Loading

zryfish Oct 17, 2024

lzlaa Oct 18, 2024

zryfish Oct 17, 2024

lzlaa Oct 18, 2024

		This document uses a kind cluster as an example to introduce how to enable and use the resource reservation capability of the Godel Scheduler.

		## Prerequisites

		@@ -1,3 +1,19 @@
		/*
		Copyright 2023 The Godel Scheduler Authors.

support reservation #62

Are you sure you want to change the base?

support reservation #62

Conversation

lzlaa commented Aug 29, 2024

binacs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

binacs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

binacs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lzlaa Oct 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lzlaa Oct 17, 2024 •

edited

Loading