da-store: inconsistent last_block and submit state after rollback

[popcnt1]

Rooch version

v0.8.4

What did you do?

1. rooch db rollback to {tx_order_a}
2. rooch server start

What did you see happen?

before rollback:

{"da_info":{"last_avail_block_number":"1670","last_avail_block_update_time":"1734660472","last_avail_tx_order":"197    29032","last_block_number":"1670","last_block_update_time":"1734657377","last_tx_order":"19729032"},"root_state":{"    size":"1062","state_root":"0xeae16d871f882fa984b60157973385811ff2e5213df137fe15b7b52418696c5e"},"sequencer_info":{"    last_accumulator_info":{"accumulator_root":"0xa7c1a29c04a806d3e85b601760c4b035c91de4062f129cd80f645bbfc1c90654","fr    ozen_subtree_roots":["0x46e1531bae26ef91b036cb29bf9d9f25c95f044cca6f38272200edcaf3f20b22","0x11ad783b26b2cba87634b4    87afb5755499d6520a3928b90b83a436b31bb81de3","0x9c01c2302640147529544055bfd444a0baebd7a003de792546e112b0b2e493a3","0    x28522f10c8123f35e0b3bdcc40f1c62ba9f40270e24c8d3291154b8225bd7690","0xe30f2571e077174da1b285feb87bad1101f86caeb7e2d    059b46d70348544e217","0x2e8429f9f7b57d659d2e0187a244f474e6d586a30c64a9c5c20ab6fada6870be","0x8f799a8da6f457c69ebc51    824e57600ca5c0e1bd397fc609a80d954238c5352d","0x67391fc4e99c66b52b8789b39b573ef9a3c1bb28f238069c01db4b8f947180b4","0    x0b4ddb43abedafc3918848f79eeb8fab49b17697f81419ac17a0c280c37f9ac5","0x721aba747187595148cbddb7df335eea5507e48e919b9    8f1efc5c4841bcda171","0xbb7ed6f9649625eece6107dc07ea0a2d12018bd8d2a00080a25ef6fa599aa2ba"],"num_leaves":"19729290",    "num_nodes":"39458569"},"last_order":"19729289"}}

after rollback and rooch server start:

2024-12-20 11:30:52.796
2024-12-20T03:30:52.761102Z ERROR rooch_rpc_server: block submit state not found for block: 1670
2024-12-20 11:30:52.746
2024-12-20T03:30:52.746298Z  INFO rooch_rpc_server: RPC Server proposer address: rooch19p2jn4laz0lum2wcnnf9pdqztw5jymqw9etltj3awwwtydkuykwsas4mgf(0x285529d7fd13ffcda9d89cd250b4025ba9226c0e2e57f5ca3d739cb236dc259d)



2024-12-20 11:30:52.101
2024-12-20T03:30:52.101761Z  INFO rooch_rpc_server: DA meta issues: 13, fixed: 13

What did you expect to see?

rooch server start automatically succeed after rollback, no last_block and submit state searching issue.

[popcnt1]

every block should have its own submit state. Need to find out which operation causes inconsistence

• try] try to reproduce local to reproduce local: caused by proposer startup: da-store: inconsistent last_block and submit state after rollback #3073 (comment)
• add test
• fix

[popcnt1]

in server startup:

1. fix da meta, start da
2. start proposer, proposer request submit state to da meta with last_proposer_block_number
3. if rollback happened, last_proposer_block_number may be beyond than da meta