Modern CRNG on Windows

[aloraman]

The go-to implementation for cryptographic random number generator on .NET is System.Security.Cryptography.RandomNumberGenerator.

On Windows, it is implemented via call to BCryptGenRandom function from BCrypt.dll here, which is a well-known approach, e.g. used in SecurityDriven.Inferno's CryptoRandom and multiple other places.

Interop.BCrypt.BCryptGenRandom(IntPtr.Zero, pbBuffer, count, Interop.BCrypt.BCRYPT_USE_SYSTEM_PREFERRED_RNG)

I've known about this approach for about ten years, so recently I've started wondering if there is a faster and more modern API on Windows.
Initially, I found a recommendation to prefer a pseudo-handle over flags like this:

 Interop.BCrypt.BCryptGenRandom(Interop.BCrypt.BCRYPT_RNG_ALG_HANDLE, pbBuffer, count, 0)

I didn't find an actual source for this recommendation, just direct references to documentation, so I guess it was generated by some LLM.
I did, however, found another API function by looking for what is used by other programming platforms. Which is ProcessPrng from BCryptPrimitives.dll.

Indeed, Golang, Chromium, BoringSSL and Rust have moved away from BCryptGenRandom and RtlGenRandom to ProcessPrng, to achieve better performance and stability in some environments. Though, there seems to be some problem with static linking to that function (that's not a problem for dotnet programs, is it?)

I've cooked up some dirty benchmark to check this:

public unsafe class Benchmarks
{
    [DllImport("BCrypt.dll")]
    internal static extern int BCryptGenRandom(IntPtr hAlgorithm, byte* pbBuffer, int cbBuffer, int dwFlags);
    [DllImport("BCryptPrimitives.dll")]
    internal static extern int ProcessPrng(byte* pdData, int cbData);

    public delegate void Rng(byte* ptr);

    public static long Cycle(Rng action)
    {
        byte* buffer = stackalloc byte[8];
        long acc = 0;
        for (int i = 0; i < 10000; i++)
        {
            action(buffer);
            acc ^= Unsafe.ReadUnaligned<long>(buffer);
        }
        return acc;
    }

    [Benchmark]
    public long ProcessPrng() => Cycle(ptr => ProcessPrng(pdData: ptr, cbData: 8));
    [Benchmark]
    public long BCryptGenRandomPseudoHandle() => Cycle(ptr => BCryptGenRandom(hAlgorithm: 0x81 /*BCRYPT_RNG_ALG_HANDLE*/, pbBuffer: ptr, cbBuffer: 8, dwFlags: 0));
    [Benchmark(Baseline = true)]
    public long BCryptGenRandomFlags() => Cycle(ptr => BCryptGenRandom(hAlgorithm: 0, pbBuffer: ptr, cbBuffer: 8, dwFlags: 2 /*BCRYPT_USE_SYSTEM_PREFERRED_RNG*/));
}

Not the best benchmark possible, just small enough to paste here and to check if there is a difference. Judging from results:

BenchmarkDotNet v0.15.6, Windows 11 (10.0.26200.7462)
13th Gen Intel Core i9-13900 2.00GHz, 1 CPU, 32 logical and 24 physical cores
[Host] : .NET 10.0.2 (10.0.2, 10.0.225.61305), X64 RyuJIT x86-64-v3

Method	Mean	Error	StdDev	Ratio
ProcessPrng	257.8 us	1.19 us	1.11 us	0.68
BCryptGenRandomPseudoHandle	352.0 us	4.10 us	3.83 us	0.93
BCryptGenRandomFlags	379.7 us	3.90 us	3.65 us	1.00

the difference between Flags and PseudoHandle is neglible (and not very stable), but ProcessPrng is indeed noticeably faster. I've tried juggling it a bit, e.g. manually expanding Cycle delegate into benchmark methods, trying to fill up larger structs - the difference is still there.

Considering all of the above, I have two questions:

1. Would it be a good idea to update BCL implementation (RandomNumberGeneratorImplementation.Windows.cs) to use ProcessPrng or BCryptGetRandom with pseudo-handle?
2. Are there any pitfalls in using ProcessPrng (or BCryptGenRandom with pseudo-handle) in another library, which targets older .NET version or even .NET Framework?

[bartonjs]

I had a chat with a compatriot on the OS crypto team.

• Right now, our (internal) company guidance is to use BCRYPT_USE_SYSTEM_PREFERRED_RNG, but they say that's probably just that no one remembered to update it, so they're looking to get it changed.
• BCRYPT_USE_SYSTEM_PREFERRED_RNG requires hitting the registry, so using the pseudo-handle is better.
  • Previously we needed the code to work on Windows 7, so this wasn't really available to us.
• ProcessPrng is OK (from their perspective), but that it should go through the API-set DLL, not directly to BCryptPrimitives. I don't see the API-set DLL (theoretically ext-ms-win-cng-rng-l1) on my computer, so it's probably one of those "if you reference it you should carry it with you" things, or it's just magically handled by LoadLibrary.

Given that the API-set is not being super-obvious, the recommendation I got was to use the pseudo-handle (if we want to make a change).

"But it could be faster if we just jump to ProcessPrng anyways!" Sure... but the team that owns it asked us not to (at least not "yet"). And is the CSPRNG really the limiting factor on your hotpath? I would imagine most of that perf delta is the function call overhead, so for better bytes/second you'd want to bulk-fetch, not fetch 8 bytes at a time.

[aloraman]

Ah, API-set DLL. Guess it's the same story as with _fpreset, you can import from msvcr.dll without shipping dependencies - but it's frowned upon and not officially supported, you're expected to import from msvcrXX.dll and ship corresponding redistributable with your app. Too bad :)

As for performance factors - it's a part of cost analysis. Optimization via bulk-fetching/batching is great, but large batches have their downsides too (see caching in #123540 for example). So it's kinda balancing between "how fast is the call" and "how large is the batch". Thus, I'm testing the limits.