gaby: support local testing mode

[hyangah]

Since CL 605576, gaby always needs database in Cloud.
For read-only features, we can do

cd internal/gaby; go run . -project <gcp-project-name> -firestoredb <devel|prod>

But it's tricky to locally test write functionalities (e.g. cron, etc)

We can consider to revive the local mode.

An alternative idea from @jba:

For read-only operations, use the database in Cloud.
For write-involving operations, write to the local database. That may be useful when we want to run new features or A/B testing against the production data without fully syncing the production database.

[randall77]

It might be nice to have an overlay database package which is backed by the read-only devel db but does writes to the overlay db (and serves up those writes in response to subsequent reads).
Then when the test is over we just throw away the overlay db.

[jba]

@randall77 That's better than what I was thinking, which is to adapt the code for 2 dbs.
However, why read from devel when we can read from prod?

[jba]

I'm going to write the overlay DB package as a fun distraction.

[hyangah]

However, why read from devel when we can read from prod?

To read the data back written by the local instance. That is not in the prod by design. For example, let's say I added extra data to devel to improve the search page result. I want to visit the search page to check the extra data actually helps.

[randall77]

The overlay DB would just be a local, in-memory database. If you added extra data to it, it would be accessible just by that instance of the Oscar process. So search on that instance would get the additional data, but search on a separate instance also pointed at devel (or also pointed at prod) wouldn't.

[jba]

My implementation (https://go-review.googlesource.com/c/oscar/+/626002) always reads and writes from different DBs.

I suppose we want to write to A but read from A first and then B if not found. The problem is, if you delete, you want to observe the deletion. That requires some sort of persistent tombstone in the general case. If the overlay is always in-memory, though, that is easy.

My problem with it being in-memory is that you can't easily inspect it ad-hoc, as with internal/devtools/cmd/dbedit. You have to write code or an HTML endpoint to get its contents.

[randall77]

That requires some sort of persistent tombstone in the general case.

Just keep a set of deleted keys as part of the overlay DB?
Could just be a [][]byte that contains all the deleted keys. Check them all each time. Or convert them to strings and put them in a map.

My problem with it being in-memory is that you can't easily inspect it ad-hoc, as with internal/devtools/cmd/dbedit. You have to write code or an HTML endpoint to get its contents.

True.

[randall77]

It's just a test database - always expose an HTML endpoint for it, at a known high port #?