Use arc4random_buf() as the best practice method for obtaining randomness on OpenBSD (examples)

[alpn]

The method above is suggested instead of the current getentropy(), which is not intended to be used by user code.
See OpenBSD's man page:

getentropy() is not intended for regular code; use the arc4random(3) family of functions instead.

[real-or-random]

getentropy() was chosen deliberately over arc4random() when this was introduced, see #748 (comment) for the discussion.

getentropy(), which is not intended to be used by user code. See OpenBSD's man page:

getentropy() is not intended for regular code; use the arc4random(3) family of functions instead.

Hm, I don't think "regular code" means "user code". This is a syscall after all, if it is not supposed to be called by user code who else should call it?

getentropy() fills a buffer with high-quality entropy, which can be used as input for process-context pseudorandom generators like arc4random(3).

Together with the above quote, I read this as "Use getentropy() for seeding PRGs because it's high-quality, but if you want PRG output, then use arc4random(). But this distinction is a bit arbitrary, to be honest. And it's wrong because it contradicts the implementation of getentropy(), which has always used arc4random().

[alpn]

Hey Tim, thank you for the links and the quick response.

OpenBSD actually has two functions named arc4random_buf().
The one you linked to is the kernel side one, which as you saw is used in the implementation of the getentropy() syscall.
The one in this PR (arc4random_buf(3)) is part of libc, and somewhat confusingly, is itself implemented using that same getentropy() syscall, but additionally applies some ChaCha based per-process stirring:

https://github.com/openbsd/src/blob/7c9f822481c4d6823c0a1b1741f55500f10f1341/lib/libc/crypt/arc4random.c#L198

The linked discussion mentions that OpenSSL uses getentropy(), but LibreSSL, OpenBSD’s fork thereof, is using arc4random_buf() to generate its private keys, e.g:

https://github.com/libressl/openbsd/blob/c5e95c4ee6991643e587f9e4b8f23c7dbb5a66fc/src/lib/libcrypto/curve25519/curve25519.c#L4905

https://github.com/openbsd/src/blob/7c9f822481c4d6823c0a1b1741f55500f10f1341/lib/libcrypto/bn/bn_rand.c#L153

[real-or-random]

Sigh ok, all of this is a huge mess.

OpenBSD actually has two functions named arc4random_buf().
The one you linked to is the kernel side one, which as you saw is used in the implementation of the getentropy() syscall.
The one in this PR (arc4random_buf(3)) is part of libc, and somewhat confusingly, is itself implemented using that same getentropy() syscall, but additionally applies some ChaCha based per-process stirring:

openbsd/src@7c9f822/lib/libc/crypt/arc4random.c#L198

Oh, you're right. The kernel one is here:
https://github.com/openbsd/src/blob/deb51e2ea407a9dc0b2340231f789bb752bc00ed/sys/dev/rnd.c#L538

Internally, it performs the same ChaCha "stirring" as the libc one.

So IIUC the entire stack is as follows. There's

1. an entropy_pool in the kernel which is used to seed
2. the kernel arc4random_buf which produces randomness for
3. the getentropy syscall which seeds
4. the libc arc4random_buf which produces randomness (and hopefully handles process forks) for
5. the user without a need for a syscall

I think this confirms my theory that randomness output by getentropy is considered "scarcer" and is recommended to seed software RNGs. arc4random_get is then recommended for users that just want random data, but without syscalls.

By the way: The way they get rid of error statues is as follows: If libc's call to getentropy fails, they try to kill the process.

Another interesting question one can ask: What does Bitcoin Core do? They have changed to arc4random_buf as a result of our previous discussion , even though we ended up with a different decision. /cc @theStack
It also contradicts the above theory. If getentropy is supposed to seed software RNGs, then Bitcoin Core should use that perhabs.

Okay, what does all of this mean for us?

• What we currently (getentropy) do is fine.
• What this PR proposes (arc4random_buf) is what Bitcoin Core does, and it is fine. It is probably a bit more appropriate for the use case.

But currently all methods in random.h (except maybe on Windows) use syscalls, and it's fine to recommend this. Syscalls are more direct than relying on libc (with its fork handling etc.) and the performance penalty is not huge and certainly okay for example code... So to be honest, I'm a bit reluctant to support this PR because it seems arbitrary. It's changes good code to code which is not clearly better.

I think a better approach would be to sync our code with Bitcoin Core. There could be a library/header file which is used by both, and then we don't need to maintain two versions of this mess. (Of course, requirements are bit different, e.g., getting randomness vs seeding RNGs vs just getting randomness, but in the end, none of this really matters.)