fix: update miner mempool iterator query to consider both nonces and fee rates

[rafaelcr]

Problem

The current query used by Stacks miners to iterate over pending mempool transactions sorts them only by fee_rate in descending order. This approach creates the following problems:

1. Since it does not consider transaction nonces, non-mineable transactions with high fees but old (or very future) nonces go to the top of the list and get visited every time, thus wasting block construction time budget.
2. Transactions with null fee rates are sent to the bottom of the list even if their nonces should be evaluated next, forcing the algorithm to query them separately with some probability of inclusion.

Solution

This PR adds a new mempool walk strategy (NextNonceWithHighestFeeRate) which prioritizes transactions that can be confirmed as fast as possible (next expected nonces for both origin and sponsor) with the highest fee rates as possible. This means that even if it doesn't select the transactions with the highest global fees first, it will select those that can be mined immediately therefore optimizing block building time and allowing the miner to fit in more transactions in total within the allotted time. This strategy also mixes in transactions with null fee rates in results by simulating a fee rate for the purposes of ordering only.

For easier testing, a mempool walk strategy enum value was added to the MinerConfig, which includes:

• GlobalFeeRate: Current global fee rate sorting behavior. Enabled by default.
• NextNonceWithHighestFeeRate: The new strategy added by this PR

[CLAassistant]

All committers have signed the CLA.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[jcnelson]

Thanks for taking this on @rafaelcr!

This is something that I think is going to need somewhat extensive real-world testing before we make it the default behavior. Do you think you could add this new mempool walk logic as an opt-in alternative, which the node operator can opt to use? This would give miners a chance to compare/contrast the new behavior with the old behavior in an easy-to-rollback manner. A subsequent PR could make the new behavior the default.

What do you think?

[rafaelcr]

Great idea @jcnelson and thanks for the feedback! I'll get on it.

[jcnelson]

From the call:

• We need @jbencin's try_mine PR so we can verify that this improves the time to mine blocks.
• Is the nonce cache preserved between blocks in Nakamoto? Not caching this will have a high regressive performance impact on Nakamoto that wasn't there in Stacks 2.x.

[jbencin]

@jcnelson @rafaelcr I have a PR that adds Nakamoto support to stacks-inspect try-mine (#5584), feel free to try it out. For benchmarking, I'd suggest building both versions as release binaries:

cargo build --release --bin=stacks-inspect

And using hyperfine (you can cargo install it) to compare the two. Example:

hyperfine -w 3 -r 20 "command 1" "command 2"

[rafaelcr]

@obycode @jcnelson I just updated the code to

1. Create the new mempool walk strategy, disabled by default, configurable via miner config
2. Consider sponsor nonces as well in selection query
3. Expand unit test to mix in sponsored and not sponsored transactions with fee variety and cross-sponsor variety

Can you take another look? Local tests look very good, including the query plan which uses only indexes

Also looking forward to trying this out with @jbencin 's tools once you think it's in good shape