reproducible usage

[zooko]

Dear libottery person/people:

I like your idea! Thank you for working on this. One thing that I find very useful, maybe even important, is having a reproducible PRNG. A good use for that is generating and comparing test vectors of different implementations, or of the same implementation built and deployed on different platforms. Another good use is testing and debugging — it is often very useful to tweak the code (perhaps by adding debugging printfs, for example) and rerun it with the same random numbers.

So my preferred API would be that the caller provide a seed to libottery — or even better to a specific instance of the libottery state, so that I can have multiple libottery states at once in the same process.

This is directly in tension with what I infer to be the "forward secrecy" goals of the design, but they do not have to be in conflict! Maybe we can have both, or at least have an option.

What do you think?

[nmathewson]

Hi, Zooko!

Libottery is just me right now.

Internally, for testing, the code does do some work on reproducible seeding. test_spec.c is an example of this.

Note that the relevant APIs ( ottery_config_set_urandom_device_ and ottery_config_disable_entropy_sources_ ) are exactly the sort of thing I'm talking about in the readme when I say that I'm going to be breaking APIs. That goes double for non-public APIs.

For a public API, I've got some ideas, but there needs to be a lot of thinking done first. One thing I've noticed that a lot of secure PRNG designs fail to do is to distinguish between "the user has some more bits that they think might be random" and "the user has some bits and insists that they are a strong entropy source."

My main concern in designing APIs like this is to minimize the number of ways you can shoot yourself in the foot without knowing it.

[eternaleye]

I wonder if one option might be something along the lines of

ottery_config_add_source_buffer( buf[], len ); // returns some sort of handle to the source
ottery_config_trust_source( sourcehandle ); // 'trusts' the source to provide strong randomness
ottery_config_restrict_sources( sources[], count ); // force libottery to only use the enumerated sources
ottery_config_untrust_source( sourcehandle ); // stops trusting a source, like if you know you're in early bootup on an embedded system
ottery_config_remove_source( sourcehandle ); // stops trying to use a source

That makes more dangerous steps progressively more work - adding a source, trusting it to provide good entropy, and using only that source. It'd probably be a good idea to require more trusted than untrusted sources in restrict(). I suspect that it'd be better on the 'scary' scale with restrict_random_input than restrict_sources, but that's getting long.

That also decouples the type of source, allowing things like

ottery_config_add_source_fd( int fd );
ottery_config_add_source_file( char* path );

Having a #define OTTERY_DEFAULT_SOURCE 0 or something would allow the untrust/remove parts to work more naturally (i.e. ottery_config_untrust_source( OTTERY_DEFAULT_SOURCE )), although some form of enumeration (and possibly reset-to-default) might prove necessary.

EDIT: Thinking about it further, it might be better to skip restrict() and require anyone who wants to pick an explicit set of sources to manually drop the default/recommended ones via untrust(OTTERY_DEFAULT_SOURCE)+remove(OTTERY_DEFAULT_SOURCE).

[nmathewson]

Well, it's not crazy. I think something more or less along those lines might work, especially given the way that sources are modular now.

Callback functions are better choices than buffers IMO, since they're actually useful for non-deterministic sources as well. Files and fds also a decent idea.

[eternaleye]

In terms of callback functions, what are you thinking regarding API? No arguments, pthread_create void* style, registration has a (...) prototype and passes the va_list down, something else?

In any of the latter "user can pass args" cases, I'd be very tempted to implement fd()/file()/etc via callbacks internally, so that actually getting randomness from any source only has one path of execution in the core libottery code. That would make fd()/file()/etc nothing more than convenient sugar.

[nmathewson]

I think that a void* or two is usually best. It's going to need a corresponding cleanup function, too. You'd also need to make sure to set flags on it, since the FAST and SECURE flags are pretty important for how ottery's entropy logic works.

This would also require a little refactoring of how ottery_entropy*.c works, but it could be worthwhile to do if it makes there be only a single path.

[eternaleye]

The refactoring could probably be done in stages and stay runnable throughout:

1.) Change the get fn prototype for the current sources so that cfg and state are void*s

2.) Change ottery_randbytes_source from a (get fn, flags) tuple to a (get fn, cleanup fn, const void* cfg, void* state, flags) tuple, and use per-source config and state storage.

3.) Move RAND_SOURCES into the config struct and convert it to a linked list, to allow addition/deletion at runtime. Since the operations on it will generally be iteration (frequent), appending (infrequent), and deletion (rare) I think that's likely to be the best fit data-structure wise. Holding onto the list tail would allow an O(1) append, and a singly-linked list would likely be sufficient.

4.) Expose a function to register callbacks. I'd suggest not allowing the user to pass flags straight, and rather use trust() to set SECURE. It may be worthwhile to have a flags parameter that gets privileged flags like SECURE masked out, though.

Using (const void_)const list_entry_ as the handle/cookie would also simplify things, as deleting/altering the flags of an item would come down to iterating the list while comparing the parameter against current->next.