nsqd: discover topic/channel paused state on new topic discovery

[jehiah]

I'd like to provide an experience report from introducing new nsqd nodes to a cluster while operational activities which involve pausing channels are underway.

---

Currently nsqd discovers the topic/channel topology from nqlookupd when it receives a new message. This ensures that if you have say 10 channels on a topic, all 10 channels get a copy of the first message.

If a channel is in a paused state when the first message is published to a new nsqd node it's naturally expected that the new node would also end up with channels in an appropriately paused states. That doesn't happen unfortunately because nsqlookupd has no context about the paused state of a channel, and so it can't provide it when queried by nsqd on a new topic creation. (There also isn't technically anything that requires all hosts to be paused or unpaused together, even though that's obviously most common)

---

Feature Request: nsqd propagate topic/channel pause states to nsqlookupd. On discovery, nsqlookupd can advise if the majority of clients for a topic/channel are paused and if so, nqsd will create a pre-paused topic or channel. If there are no clients for a topic/channel connected to nsqd at the time of topic discovery, nsqd should create a topic and channel un-paused. When nsqd queries multiple nqslookupd if any response indicates paused, the result should be a paused state. i.e. paused is preferred.

Note: nsqlookupd response changes to support this should be backwards compatible.

[mreiferson]

SGTM

[maplessssy]

@jehiah @mreiferson any update here? I think i can spend a weekend to finish it.

[maplessssy]

hi, anyone can help to take a look at this?
first step: Command [Register] need support to deliver topic/channel paused state.
see go-nsq pr