Improve On-Boarding for a better TTP (Time To new Position)

[bonomat]

Right now it is quite hard to use our app in various ways and we should improve the on-boarding experience, and overall, reduce the time to open a position.

There might be some low hanging fruits and some more complex tasks. This discussion is meant to be used to collect and describe them.

[klochowicz]

shall we try to create a metric in grafana to check TTP for new users? otherwise, it might be hard to see whether any actions increase / decrease such metric?

[klochowicz]

We'd need essentially two timestamps:

• first one, when a new taker id is online,
• second one, when that taker id takes an offer successfully, creating a CFD.

We should be able to scrape both from the maker logs.

It's still an open question how we could visualise this in Grafana, and whether it would be beneficial to continue tracking TTP between subsequent positions (probably not that useful, as it would not help us with anything really).

[bonomat]

I'm not sure one can build such a metric based on logs over a longer timespan.

Imho we would need to build a metric into the daemon.
This is still hard because what does first seen mean? Since the restart? Overall? For the latter one we'd need to remember the information between restarts.

Collecting information on when a position was opened is doable, but similar to the other statement, how do we know it was the first one?

[bonomat]

I thought about this more and started writing a metric which would look like this:

p2p_connection_online_info{daemon_version="0.4.11-118-gbd190de",taker_id="c5364d168544eb05460d1fa3b7ed270cf0f2b056ea715869d77c9fbb2b25526a"} 1

The problem is that the number of metrics grows with the users we have. Prometheus discourages the usage of using labels with high cardinality because this can lead to performance issues.

CAUTION: Remember that every unique combination of key-value label pairs represents a new time series, which can dramatically increase the amount of data stored. Do not use labels to store dimensions with high cardinality (many different label values), such as user IDs, email addresses, or other unbounded sets of values.

We need to come up with a better way of tracking this. Ideas are welcome :)

Imho we should store this information in the database and we query this manually when needed, e.g.

user	first_seen	first_position
uuid	timestamp	timestamp

[luckysori]

I like the idea of storing this information in a database that can be queried. I would expect us to then be able to calculate the average TTFP for the past 7 days and compare it to the previous 7 days.

This data also doubles as a source for measuring unique users: each column represents a new "active" user (as long as they don't tinker with their taker_id).

[bonomat]

Note from @luckysori in today's call: this metric is only useful for new users - who download the app once we have this feature activated. We could retrospectively check the logs and manually insert entries in the db though. Let's see if this is necessary.

Ticket created here: #1895

[bonomat]

Connect to external wallet

Umbrel comes with a wallet which is also used for opening channels. Contrary to what we initially thought, the internal wallet is based on LND's internal wallet. (I've confirmed this by depositing funds and using the cli to check if the balance on the lnd wallet changed).

Either way, we should be able to tap into this wallet using LND's RPC interface: https://api.lightning.community/. It might be a bit more work as expected as we are not yet familiar with their API and the library landscape is rather limited.

However, LND's RPC interface seems to support our needed functions such as fundpsbt and later on finalize psbt.

How should this new feature behave?

There are a few options for how we can support this new feature. I'll mention all I can imagine below:

internal wallet: the wallet provided in ItchySats
external wallet: the wallet outside of ItchySats, e.g. the wallet provided by Umbrel (LND's wallet)

1. Support internal wallet and external wallet in parallel
   Some existing users will have funds already in the internal wallet as well as open positions which will end up in the internal wallet eventually. Removing the internal wallet completely could end in losing funds.
   We don't want that, hence, we could offer to use both wallets in parallel: the internal wallet and the external wallet:

   1. we provide wallet management for internal wallet only
   2. we refer to the wallet management for the external wallet to the Umbrel interface.
   3. do we need to show both balances to the user or only the internal one?

The advantage of this is that we can keep using our wallet which is derived from an internal key. If users were to move their installation, the wallet balance would move with them provided they use the same seed.
Another example is that there would be no change for users not running on Umbrel.

The disadvantage of this solution is complexity: we would need to coordinate between the two wallets and ask the user for input on which wallet they would want to use the funds from and receive the funds to.

2. Make external wallet optional and use over internal wallet if present

   This could simplify our life as we could refer to Umbrel's wallet management screen instead of having to provide our own.
   The challenge here is what we do with existing balance and balance still stuck in open positions:

   1. We could offer a seamless upgrade and use the new wallet when rollingover a position.
   2. We could make use of PSBT and use funds from internal and external wallets.
      This could be a preferable option, however, for existing positions, funds may still end up in the internal wallet from which the user will need to withdraw manually ending up in a similar situation as in 1.

3. Support external wallet only and remove internal wallet
   The advantage of this would be that we could get rid of some code and the internal wallet. We would remove quite some responsibility here.
   This is probably the least preferred option as it would require users not using Umbrel to run a LND.

Own opinion

I would go for 2.

• add additional cli argument to access LND's wallet
• keep external wallet management out of ItchySats
• use external wallet over internal wallet if provided
• seamless upgrade and use external wallet's address when rollingover existing positions
• when opening the UI and we notice that the internal wallet still has funds: ask the user if he wants to withdraw all funds to external wallet if provided or withdraw to arbitrary address

Outlook

The reason why I called the wallet external wallet is because if we keep it in mind that this is an external wallet, we should be able to add other wallets in the future which support psbts.
Additionally, given we attach LND to ItchySats, are we still that far away from supporting Lightning? 😬

[bonomat]

yes, that's the idea :)

[thomaseizinger]

How hard would it be to combine 2 and 3 by building two binaries? Removing conditional code on when to use what could greatly simplify the codebase. We could easily release the version that users the new wallet as a new app on umbrel and then we don't have to worry about any of the migration stuff.

[klochowicz]

How hard would it be to combine 2 and 3 by building two binaries? Removing conditional code on when to use what could greatly simplify the codebase. We could easily release the version that users the new wallet as a new app on umbrel and then we don't have to worry about any of the migration stuff.

I love the idea. then the decision is in the user hands. We'd need to either rename the old app to 'legacy' or something, and of course approach the umbrel team about it; saves a lot of headache though. we could also add a FAQ entry for binary users that they can retrieve their money if they downgrade the app (which is trivial) - unless you would like to rename the app on docker too?

[bonomat]

TODO: make a poll to see if this feature is wanted

• twitter
• umbrel telegram group
• our telegram group

[bonomat]

Ticket created: https://github.com/itchysats/maker-automation-rs/issues/236

[bonomat]

On-boarding tour:

An onboarding tour is a guided walkthrough through our application. It guides users step-by-step through what they can do and have to do to for the important actions:

• deposit funds
• open a position
• close a position

The hard part here is to highlight what the user wants to see, not just what we think he wants to see... Meaning, we will likely need to adapt this constantly and ask for feedback.

I was able to find various libraries:

• intro-js: looks like the best rated one. Has a free mode for non-commercial and paid for commercial applications. Not so sure about react support: https://github.com/usablica/intro.js/
  • demo: https://introjs.com/
• intro-js is a wrapper around intro-js: https://github.com/HiDeoo/intro.js-react
  • demo: https://codesandbox.io/embed/o2A4gwXE3?hidenavigation=1
• react-joyride: looks well rated according to github stars. https://github.com/gilbarbara/react-joyride
  • demo: https://react-joyride.com/
• react-sheperd: looks a bit stale: https://github.com/shipshapecode/react-shepherd
  • demo: https://shipshapecode.github.io/react-shepherd/

[bonomat]

Note from @thomaseizinger in today's call: we could make use of an on-boarding checklist, e.g.

• deposit funds ✓
• open position ✓
• close position to make profit ❌

[bonomat]

Ticket opened here: #1897

[klochowicz]

FAQ

• a blog entry explaining most confusing parts of the system
• link in the corner of the app, so it's easy to spot for newcomers
• can get updated "as we go", when users are asking us a question that would be obvious to us, but not to a first-time user
• can also point to some popular knowledge bases where people can learn about CFDs etc.

Example:
Q: Can I use my existing wallet / umbrel wallet?
A: Not yet. We've got it on our roadmap, for the time being the app has its own inbuilt wallet into which you can transfer funds. It is available in the "Wallet" tab.

[luckysori]

I'm all for adding an FAQ.

[da-kami]

I think that's a good idea and a somewhat low-hanging fruit. The only thing I'm worried about is maintaining it - that will require clear responsibility.

[bonomat]

Note from @bonomat in today's call:
hosting the FAQ on medium is suboptimal because medium has a max-read counter.
Best place would be to have it on the homepage (which is managed in Git)

Note from @klochowicz :
we could host the FAQs in form of discussions on Github.

[bonomat]

ticket created here: #1898

[da-kami]

Mainnet / Testnet Switch

I was thinking about what @bonomat suggested here #1841 (comment) in the past and came to the conclusion that it would mean quite an effort to get this right and keep it maintained.

A different solution might be to give the user a proper "playground" by allowing the user to switch between mainnet and testnet in the UI. By default (on first startup) ItchySats could be on testnet (against a public testnet node and not Umbrel). We could add a small guide to show how to transfer funds in through a faucet. Then the user can just play on testnet and whenever ready switch to mainnet and go.

How to implement this?

• We have to be able to "toggle" the monitor from testnet to mainnet
• We have to be able to "toggle" the wallet from testnet to mainnet
• We need a toggle switch in the UI and an endpoint to handle switching the monitor and wallet
• We could consider adding different routes for mainnet and testnet

Advantages:

• Users might appreciate being able to try ItchtSats without using real BTC
• This feature might (eventually) help us with a testnet-setup that involves Umbrel
• The boundaries might be clearer clearer defined than Improve On-Boarding for a better TTP (Time To new Position) #1841 (comment)

Disadvantages:

• TTP for mainnet might actually go up
• Might be more technical challenges for separating the network properly

[klochowicz]

Before we venture far into this way, I would write a blogpost on how to quickly change it for Umbrel (step-by-step instructions). This is a low-hanging fruit, and from 0.4.12 anyone doing this should be able to connect to our testnet maker by default.

[klochowicz]

Docker-compose repo

For Docker users, or people who'd like to have ItchySats taker running as a service (or on their VPS/cloud), a docker-compose repo could be really handy. I've used many of these out-of-the-box configs myself.
It could also have sane defaults as for the seed etc.
This would also allow people to trivially fork it or raise issues. I suspect it might be easier for people to write there than on our main repo, as it would be more targeted towards deploying it.

Example: https://github.com/AzagraMac/PiHoleDocker

pros:

• making it easier to use it with Docker (docker-compose up)
• easy upgrade mechanism for Docker users - git pull could bump the version to latest "stable"

cons:

• one more step after release - bump the version in the repo.