Re: [PATCH 1/3] Make /dev/urandom scalable

[Austin S Hemmelgarn <ahferroin7@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4111 bytes --]

On 2015-09-25 15:07, Austin S Hemmelgarn wrote:
> On 2015-09-25 07:41, Austin S Hemmelgarn wrote:
>> On 2015-09-24 16:14, Theodore Ts'o wrote:
>>> On Thu, Sep 24, 2015 at 03:11:23PM -0400, Austin S Hemmelgarn wrote:
>>>>> That is a startling result.  Please say what architecture, kernel
>>>>> version, dieharder version and commandline arguments you are using to
>>>>> get 10% WEAK or FAILED assessments from dieharder on /dev/urandom.
>>>>
>>>> I do not remember what exact dieharder version or command-line
>>>> arguments
>>>> (this was almost a decade ago), except that I compiled it from source
>>>> myself, I do remember it was a 32-bit x86 processor (as that was
>>>> sadly all I
>>>> had to run Linux on at the time), and an early 2.6 series kernel
>>>> (which if I
>>>> remember correctly was already EOL by the time I was using it).
>>>
>>> It might have been nice if you had said this from the beginning
>>> instead of making an unqualified statement with the assumption that it
>>> was applicable to kernels likely to be used today in non-obsolete
>>> systems.  Otherwise it risks generating a click-bait article on
>>> Phoronix that would get people really worried for no good reason...
>> I sincerely apologize about this, I should have been more specific right
>> from the beginning (I need to get better about that when talking to
>> people, I'm so used to dealing with some of my friends who couldn't
>> event tell you the difference between RAM and a hard drive, think a bus
>> is only something you use for transportation, and get confused when I
>> try to properly explain even relatively simple CS and statistics
>> concepts).
>>>
>>> There was a bug a long, long time ago (which where we weren't doing
>>> sufficient locking and if two processes raced reading from
>>> /dev/urandom at the same time, it was possible that the two processes
>>> would get the same value read out from /dev/urandom).  This was fixed
>>> a long time ago, though, and in fact the scalability problem which
>>> Andi is trying to fix was caused by that extra locking that was
>>> added.  :-)
>>>
>>> It's possible that is what you saw.  I don't know, since there was no
>>> reproduction information to back up your rather startling claim.
>> I don't think this was what I hit, I'm pretty sure I had serialized the
>> dieharder runs.
>>>
>>> If you can reproduce consistent Dieharder failures, please do let us
>>> know with detailed reproduction instructures.
>> Will do.
> OK, just started a couple of runs in parallel using different generators
> using the following command line:
> dieharder -a -m 32 -k 1 -Y 1 -g XXX
> with one each for:
> /dev/urandom (502)
> AES_OFB (205)
> glibc random() (039)
> mt19937 (013)
> The above command line will run all dieharder tests with 12800 psamples,
> using a higher than default precision, and re-running tests that return
> WEAK until it gets a PASS or FAIL.  Even on the relatively fast (at
> least, fast for a desktop) system I'm running them on, I expect it will
> take quite some time to finish (although regardless of that I'm probably
> not going to be getting back to it until Monday).
>
> Interestingly, based on what dieharder is already saying about
> performance, /dev/urandom is slower than AES_OFB (at least, on this
> particular system, happy to provide hardware specs if someone wants).
>
Apologies for not replying yesterday like I said I would.

I actually didn't get a chance to run the tests to completion as the 
wifi card in the system I was running the tests on lost it's mind about 
55 hours in and I had to cold reboot the system to reset it.  I would 
give the results here, except that I have a feeling that people probably 
don't want 110kb of data in the e-mail body, and thunderbird is for some 
reason choking on trying to attach files.  In general, the results were 
pretty typical of a good PRNG, performance differences not withstanding. 
  In other words, don't use /dev/urandom except for seeding other 
PRNG's, but because of the speed, not the quality.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3019 bytes --]