-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use state to avoid re-sending data to observers #554
Comments
We might need to wait after the end of April (BTC++ conf) to take on this one |
To be honest, I don't know exactly how this one should be approached. But here are some notes: We already have this to some degree when a Chainhook node is restarted. When a predicate is in Streaming mode, we update the status of the predicate after every block, storing the last evaluated block. When a predicate is in Scanning mode, we update the status after every match between a block + predicate. With these values stored, whenever Chainhook is restarted, the service can load the predicates from redis, evaluate each predicate's status to determine the last block evaluated, and continue where it left off. This prevents a restart of the Chainhook node from re-sending data to observers. This issue, however, is for another case (which happened recently). If a Stacks node has to be reset, say to chain tip - 100 blocks, each of those 100 blocks will have already been evaluated by Chainhook. However, when the block is received by the event observer there currently is no way to determine if the predicate has already been evaluated against that block (the Chainhook SDK does not have access to the redis store of predicate status). Furthermore, even if the predicate has a It would be nice to have some guarantee that a predicate will never be triggered twice from the same block, but I think it will be quite difficult and is currently handled for the general case. The recent events on the testnet that caused us to reset the chain will probably be very rare, so maybe we aren't as concerned about handling it. I wouldn't say I recommend closing this issue at this point, but maybe it can be revisited in a few months |
When a chainhook-node is re-processing blocks (for whatever reason), it should avoid re-sending blocks to observers when they have been sent in the past.
Data from Redis could help.
The text was updated successfully, but these errors were encountered: