Deniability - a tool to automatically improve coin ownership privacy

[denavila]

This new feature is an implementation of the ideas in Paul Sztorc's blog post "Deniability - Unilateral Transaction Meta-Privacy"(https://www.truthcoin.info/blog/deniability/).

In short, the idea is to periodically split coins and send them to yourself, making it look like common "spend" transactions, such that blockchain ownership analysis becomes more difficult, and thus improving the user's privacy.

This is the GUI portion of the PR (bitcoin-core/gui). The core functionality PR ( bitcoin/bitcoin#27792 ) is in the main repo (bitcoin/bitcoin).

This PR implements an additional "Deniability" wallet view. The majority of the GUI code is in a new deniabilitydialog.cpp/h source files containing a new DeniabilityDialog class, hooked up via the WalletView class. 

On startup and on notable events (new blocks, new transactions, etc), we evaluate the privacy of all coins in the wallet, and we build a "deniabilization" candidate list. UTXOs that share the same destination address are grouped together into a single candidate (see DeniabilityDialog::updateCoins and DeniabilityDialog::updateCoinTable).

We inspect the blockchain data to find out if we have performed "deniabilization" transactions already, and we count how many "cycles" (self-sends) have been performed for each coin (see DeniabilityDialog::calculateDeniabilizationStats). Since we infer the state entirely from the blockchain data, even if the wallet is restored from a backup, the state would not be lost. This also means that if the user has performed manual self-sends that have improved the ownership privacy, they will be counted too.

The user has a couple of controls for the "deniabillization" process - a Frequency selector and a Budget spinner, which respectively determine how often to perform a cycle (see below) and how much total BTC to spend on transaction fees.

The user can initiate the "deniabillization" process by pressing the Start button (DeniabilityDialog::startDeniabilization).
The first time the button is pressed, we offer an information / confirmation box to explain to the user how Deniability works.

The process periodically perform a "deniabilization" cycle (DeniabilityDialog::deniabilizeProc). In each such cycle we do the following:
A coin is selected form the candidate list. The more a coin is "deniabilized", the less likely it is to be selected. Smaller coins are also less likely to be selected. If a coin is selected, we prepare and broadcast a transaction, which splits the coin into a pair of new wallet addresses (DeniabilityDialog::deniabilizeCoin). 

If Bitcoin Core is left running continuously, the cycles would be performed at the selected frequency (with some randomization). If Bitcoin Core is shutdown, the "deniabilization" process will resume at the next restart, and if more time has elapsed than the selected frequency, it will perform a single cycle. We deliberately don't "catch up" all missed cycles, since that would expose the process to blockchain analysis. The state is saved and restored via QSettings (DeniabilityDialog::loadSettings and DeniabilityDialog::saveSettings).

We monitor each broadcasted transaction and we automatically attempt a fee bump if the transaction is still in the memory pool since the previous cycle (DeniabilityDialog::bumpDeniabilizationTx). We don't issue any other deniabilization transactions until the previous transaction is confirmed (or abandoned/dropped).

The process ends when the budget is exhausted or if there's no candidates left. The user can also stop the process manually by pressing a Stop button (DeniabilityDialog::stopDeniabilization).

External signers are supported in a "best effort" way - since the user needs to manually sign, we postpone the deniabilization processing till the external signer is connected and use some additional UI to get the user's attention to sign (see the codepath predicated on hasExternalSigner). This is not ideal, so I'm looking for ideas if we can improve this in some way.

Watch-only wallets are partially supported, where we display the candidate list, but we don't allow any processing (since we don't have the private keys to issue transactions).

I've tested all this functionality on regtest, testnet, signet and mainnet. I've also added some unit tests (under WalletTests) to exercise the main functions.

This is my first change and PR for Bitcoin Core, and I did my best to validate everything against the guidelines and documentation and to follow the patterns in the existing code, but I'm sure there may be things I've missed, so I'm looking forward to your feedback.
In particular one thing I'm not particularly happy with - the save/restore of state via QSettings makes me a bit nervous as we store wallet specific data keyed on wallet name, however wallets can be erased and new ones created with the same name, so this is not ideal. I made an effort to validate all data on load, but it would be great if we could improve this somehow - either by storing the settings based on some wallet identity signature, or by storing the state in the wallet database (however that doesn't seem accessible via the interfaces::Wallet API). Please let me know your thoughts and suggestions.

Thank you.

[DrahtBot]

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	katesalazar

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #815 (Bugfix on TransactionsView - Disable if privacy mode is set during wallet selection by pablomartin4btc)
• #700 (Dialog for allowing the user to choose the change output when bumping a tx by achow101)
• #bitcoin/bitcoin/28710 (Remove the legacy wallet and BDB dependency by achow101)
• #bitcoin/bitcoin/27865 (wallet: Track no-longer-spendable TXOs separately by achow101)
• #bitcoin/bitcoin/27286 (wallet: Keep track of the wallet's own transaction outputs in memory by achow101)
• #bitcoin/bitcoin/24963 (RPC/Wallet: Convert walletprocesspsbt to use options parameter by luke-jr)
• #bitcoin/bitcoin/21283 (Implement BIP 370 PSBTv2 by achow101)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

[hebasto]

This PR is an implementation of the ideas in Paul Sztorc's blog post "Deniability - Unilateral Transaction Meta-Privacy"(truthcoin.info/blog/deniability).

Has that idea been discussed somewhere already?

[denavila]

Has that idea been discussed somewhere already?

The idea was published quite a while ago, but I learned about it from some Bitcoiner friends just recently.
We looked around to see if there were existing discussions or implementations, but we didn't find anything on bitcoin-dev or the repo here.
There's an old implementation in a Bitcoin fork (Drivechain) which I used as a starting point, but ended up with a pretty different approach and implementation.
We were thinking of perhaps writing down the spec into a Process BIP, if this would make it easier to review on a conceptual level?

[VladdyC]

This is really cool and I'd love to have it as an option in Bitcoin Core. However, most CoinJoin implementations are more advanced in this regard – for example, WabiSabi performs both divisions and consolidations. And it makes me wonder to which extent it's possible to decentralize coordination to the point where clients randomly take turns in doing it.

Anyway, this is just my curiosity/wish list. I believe that anything is better than having no privacy options at all, so I'd be happy with some more basic deniability too :)

[achow101]

It seems like a feature like this would be better to integrate into the wallet directly rather than something that is done through and managed by the GUI.

[denavila]

This is really cool and I'd love to have it as an option in Bitcoin Core. However, most CoinJoin implementations are more advanced in this regard – for example, WabiSabi performs both divisions and consolidations. And it makes me wonder to which extent it's possible to decentralize coordination to the point where clients randomly take turns in doing it.

Yes, CoinJoin is definitely more powerful, however it has some drawbacks.
In particular, if it's possible to detect that an UTXO has come from a coinjoin, it may be considered "tainted" by CEXs and they could refuse processing it (eg demand source of funds etc).

That's the "meta-privacy" that Paul talks about in his blog post. The idea that the knowledge of a secret has to also be kept a secret.

Here's another one of Paul's blog posts where he talks about drawbacks of other privacy methods: https://www.truthcoin.info/blog/expensive-privacy/

At any rate, I'm definitely open to suggestions or ideas we can borrow from CoinJoin.
In particular if there's a way we can re-merge UTXOs in a privacy-preserving way. Or at bare minimum, use "deniability-aware" logic in the automatic coin selection during user-initiated spends to minimize the privacy reduction.

Anyway, this is just my curiosity/wish list. I believe that anything is better than having no privacy options at all, so I'd be happy with some more basic deniability too :)

Thank you. I believe so too.

[denavila]

It seems like a feature like this would be better to integrate into the wallet directly rather than something that is done through and managed by the GUI.

Do you mean moving some of the core logic inside the interfaces::Wallet API, or the CWallet implementation?

Yes, certainly, that could have benefits. I had to workaround certain things that weren't in the Wallet API (namely address reservation, bump fee handling etc). So that would clean up the wallet side for sure.

Since this was my first Bitcoin Core change I decided to be more conservative and avoid API changes.
Of course, if this PR is accepted, I'd be happy to move functionality into the API.

[achow101]

Do you mean moving some of the core logic inside the interfaces::Wallet API, or the CWallet implementation?

Inside of CWallet.

Since this was my first Bitcoin Core change I decided to be more conservative and avoid API changes.
Of course, if this PR is accepted, I'd be happy to move functionality into the API.

The API isn't public, so changing it isn't a big concern. Really the only consumer of it is the GUI.

Having it inside of CWallet will probably make this feature more likely to be accepted. It will be easier to write tests (adding an RPC allows for functional tests, and implementing in CWallet allows unit tests to better access the data) and it can be split up into at least 2 chunks - the CWallet and RPC only implementation, and then the GUI and interface changes. This makes it more likely to be accepted.

[denavila]

The API isn't public, so changing it isn't a big concern. Really the only consumer of it is the GUI.

Having it inside of CWallet will probably make this feature more likely to be accepted. It will be easier to write tests (adding an RPC allows for functional tests, and implementing in CWallet allows unit tests to better access the data) and it can be split up into at least 2 chunks - the CWallet and RPC only implementation, and then the GUI and interface changes. This makes it more likely to be accepted.

Ah, I see. I had assumed the opposite - that not touching the API would be easier to accept.
Ok, I can look into refactoring the PR and moving the core functionality into CWallet.

I'd need to open a PR against the main repo in that case, right?
Should I keep this PR and leave the GUI bits here, and open another PR in the main repo with just the CWallet changes?
Or is it better to close this PR and open a new one in the main repo with all the changes?

[achow101]

I'd need to open a PR against the main repo in that case, right?

Yes

Should I keep this PR and leave the GUI bits here, and open another PR in the main repo with just the CWallet changes?
Or is it better to close this PR and open a new one in the main repo with all the changes?

You can leave this open and rebase it onto the main repo PR, just mention it in the description.

[denavila]

You can leave this open and rebase it onto the main repo PR, just mention it in the description.

All right, I'll start working on this refactor.
Btw, do we have some docs on how to setup git locally so I can have dependent changes across the "bitcoin" and the "gui" repos? So far I was moving changes via patches but that doesn't seem too convenient.

[denavila]

I refactored the code and updated the PR. The new code is split between two branches:

1. deniability-api - new API and implementation of core "deniabilization" functions. I'll open a PR in the main repo once I add RPC and unit tests.
2. deniability - the GUI portion of the Deniability dialog which uses the new deniability-api functions.

[jonatack]

Didn't conceptually review yet. While rebasing, the clang-tidy CI highlighted these issues to be addressed:

wallet/spend.cpp:1539:9: error: function 'CalculateDeniabilizationFeeEstimate' is within a recursive call chain [misc-no-recursion,-warnings-as-errors]
 1539 | CAmount CalculateDeniabilizationFeeEstimate(const CScript& shared_script, CAmount total_value, unsigned int num_utxos, unsigned int deniabilization_cycles, const CFeeRate& fee_rate)
      |         ^
wallet/spend.cpp:1539:9: note: example recursive call chain, starting from function 'CalculateDeniabilizationFeeEstimate'

wallet/spend.cpp:1552:40: note: Frame #1: function 'CalculateDeniabilizationFeeEstimate' calls function 'CalculateDeniabilizationFeeEstimate' here:
 1552 |     CAmount futureDeniabilizationFee = CalculateDeniabilizationFeeEstimate(shared_script, total_value / NUM_DENIABILIZATION_OUTPUTS, 1, deniabilization_cycles + 1, fee_rate) * NUM_DENIABILIZATION_OUTPUTS;
      |                                        ^
wallet/spend.cpp:1552:40: note: ... which was the starting point of the recursive call chain; there may be other cycles

wallet/spend.cpp:1561:31: error: function 'CalculateDeniabilizationCycles' is within a recursive call chain [misc-no-recursion,-warnings-as-errors]
 1561 | std::pair<unsigned int, bool> CalculateDeniabilizationCycles(CWallet& wallet, const COutPoint& outpoint)
      |                               ^
wallet/spend.cpp:1561:31: note: example recursive call chain, starting from function 'CalculateDeniabilizationCycles'

wallet/spend.cpp:1634:23: note: Frame #1: function 'CalculateDeniabilizationCycles' calls function 'CalculateDeniabilizationCycles' here:
 1634 |     auto inputStats = CalculateDeniabilizationCycles(wallet, txIn.prevout);

[denavila]

Yes, it looks like recursive functions were disallowed recently.
I'll rework the code to not use recursion.

[DrahtBot]

🚧 At least one of the CI tasks failed.
_{Debug: https://github.com/bitcoin-core/gui/runs/29695577162}

Hints

Make sure to run all tests locally, according to the documentation.

The failure may happen due to a number of reasons, for example:

• Possibly due to a silent merge conflict (the changes in this pull request being
  incompatible with the current code in the target branch). If so, make sure to rebase on the latest
  commit of the target branch.

• A sanitizer issue, which can only be found by compiling with the sanitizer and running the
  affected test.

• An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

[DrahtBot]

🚧 At least one of the CI tasks failed.
_{Debug: https://github.com/bitcoin-core/gui/runs/30344213036}

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

• Possibly due to a silent merge conflict (the changes in this pull request being
  incompatible with the current code in the target branch). If so, make sure to rebase on the latest
  commit of the target branch.

• A sanitizer issue, which can only be found by compiling with the sanitizer and running the
  affected test.

• An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

[1440000bytes]

@denavila any update on 3 and 4 from #733 (comment)?

[achow101]

This PR does not seem to have attracted much attention from reviewers. As such, it does not seem important enough right now to keep it sitting idle in the list of open PRs.

Closing due to lack of interest.