Updating affinity in job is destructive

[pawelbeza]

Hi there 👋

not sure if that's expected behavior but changing Affinity is destructive and it'causig in-place update. It's a weird behavior considering that there is no update on Constraint change

nomad/scheduler/annotate.go

Line 191 in a113258

case "Service", "Constraint":

Would you consider changing this behavior and skipping in-place update when affinity is changed? I could help with the PR

[tgross]

Hi @pawelbeza!

Just to refine your bug report a bit, what you're seeing is a destructive update. An in-place update is one where we update the definition of the allocation without touching any running tasks (any easy example of this might be incrementing the group.count). An affinity, spread, or constraint update are all supposed to be in-place updates, because we update the definition of the job without replacing any of the allocations. This logic was make explicit in #6703 (shipped in Nomad 0.10.2).

But, as you note, it's not behaving as expected! Here's an example of what that looks like with a job update that only updates an affinity block:

$ nomad job plan example.nomad.hcl
+/- Job: "example"
+/- Task Group: "group" (1 create/destroy update)
  + Affinity {
    + LTarget: "${attr.os.name}"
    + Operand: "="
    + RTarget: "ubuntu"
    + Weight:  "75"
    }
  - Affinity {
    - LTarget: "${attr.os.name}"
    - Operand: "="
    - RTarget: "ubuntu"
    - Weight:  "50"
    }
    Task: "task"
...

And if I run that, I get a destructive update with a new allocation, which isn't right.

Would you consider changing this behavior and skipping in-place update when affinity is changed? I could help with the PR

I'd be happy to review a PR! We should fix this for spread blocks too while we're here.

[pawelbeza]

Thanks for clarifying the issue. I may have misused the term in-place update. You can find the fix here #25109 :)