add cluster flushslot command.

[wuranxx]

add cluster flushslot command. #1133

background

We want to implement the ability to SYNC or ASYNC free all data from a single slot. This is useful in two cases:

1. When a primary loses ownership of a slot, it will temporarily freeze while it synchronously frees the slot.
2. When doing resharding, you may prefer to forcibly delete all the data from a slot and assign it to another node.

behavior

1. Add CLUSTER FLUSHSLOT <slot> [ASYNC|SYNC] command.

2. Modify the function signature of delKeysInSlot, adding a lazy parameter to decide whether to delete keys SYNC or ASYNC.

[codecov]

Codecov Report

Attention: Patch coverage is 86.36364% with 3 lines in your changes missing coverage. Please review.

Project coverage is 70.98%. Comparing base (7fc958d) to head (4db74af).

Files with missing lines	Patch %	Lines
src/cluster_legacy.c	86.36%	3 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##           unstable    #1384      +/-   ##
============================================
+ Coverage     70.80%   70.98%   +0.17%     
============================================
  Files           121      121              
  Lines         65132    65149      +17     
============================================
+ Hits          46118    46243     +125     
+ Misses        19014    18906     -108     

Files with missing lines	Coverage Δ
src/commands.def	100.00% <ø> (ø)
src/cluster_legacy.c	86.93% <86.36%> (-0.42%)	⬇️

... and 11 files with indirect coverage changes

[wuranxx]

Please help review this PR. Thank you. @hwware @madolson @PingXie @zuiderkwast

[hwware]

There are too much information in issue #1133. To save some of the reviewer energy, could you pls update the top comment to describe the background why we want to this new command? You could reference PR #1392.

Thanks a lot

[PingXie]

LGTM overall. Thanks @wuranxx!

[PingXie]

we should fail the c->argc > 4 case I think

[wuranxx]

In the call site of clusterCommandFlushslot, I have already performed a check on the number of parameters. Is it still necessary to validate the number of parameters again within the function?

 else if (!strcasecmp(c->argv[1]->ptr, "flushslot") && (c->argc == 3 || c->argc == 4)) {
        /* CLUSTER FLUSHSLOT <slot> [ASYNC|SYNC] */
        clusterCommandFlushslot(c);
    }

[wuranxx]

Before I rebased to the latest unstable (old commit is 32f7541) , the Tcl tests and manual command tests ran fine locally.

After I rebased to the latest unstable 6df376d, the command failed when executed locally.

It is possible that changes in the code during this period caused the inconsistent behavior. Until I confirm the cause, I have marked this PR as WIP (Work In Progress).

[wuranxx]

Before I rebased to the latest unstable (old commit is 32f7541) , the Tcl tests and manual command tests ran fine locally.

After I rebased to the latest unstable 6df376d, the command failed when executed locally.

It is possible that changes in the code during this period caused the inconsistent behavior. Until I confirm the cause, I have marked this PR as WIP (Work In Progress).

Fixed. For details, see #1399.

[zuiderkwast]

@enjoy-binbin @murphyjacob4 Do you want this command for atomic slot migration, for example when a slot migration is cancelled, the importing node can send CLUSTER FLUSHSLOT to its replicas to clean up the slot?

[murphyjacob4]

I think we will have a protocol for cancelling the slot migration, and this will trigger the drop for all slot ranges being migrated locally. But we should be able to use the async delKeysInSlot functionality to implement this

[zuiderkwast]

I think we will have a protocol for cancelling the slot migration, and this will trigger the drop for all slot ranges being migrated locally. But we should be able to use the async delKeysInSlot functionality to implement this

@murphyjacob4 The replicas are not involved in the slot migration protocol IIUC. So when a primary cleans up a cancelled slot import, do you just replicate a lot of DEL commands to the replicas? Replicating a single FLUSHSLOT may save some work and bandwidth...

[murphyjacob4]

do you just replicate a lot of DEL commands to the replicas? Replicating a single FLUSHSLOT may save some work and bandwidth...

Yeah that's a good idea. Currently, this is what delKeysInSlot does, and it could result in a lot of bandwidth for big slots. Disabling the replication of the individual keys in delKeysInSlot and replacing with a single FLUSHSLOT makes sense to me. This path is also executed in the dirty slot case, so it won't just be an improvement for atomic slot migration (although I anticipate it will be executed more frequently there).