Re: Kernel 3.0: Instant kernel crash when mounting CIFS (also crashes with linux-3.1-rc2

[Justin Piszcz <jpiszcz@lucidpixels.com>]

On Thu, 18 Aug 2011, Steve French wrote:

> On Thu, Aug 18, 2011 at 12:16 PM, Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
>>
>>
>> On Thu, 18 Aug 2011, Jeff Layton wrote:
>>
>>> On Thu, 18 Aug 2011 09:15:36 -0400 (EDT)
>>> Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
>>>
>>>>
>>>>
>>>> On Thu, 18 Aug 2011, Jeff Layton wrote:
>>>>
>>>>> On Thu, 18 Aug 2011 08:22:44 -0400 (EDT)
>>>>> Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
>>>>>
>>>>>> Justin.
>>>>>>
>>>>>
>>>>> To be clear -- incoming in this case is reads or writes?
>>>>
>>>> Reading from the CIFS share (Windows 7).
>>>>
>>>>>
>>>>> Up until 3.0 cifs.ko didn't parallelize writes from a single thread. In
>>>>> 3.0 I added a patchset to increase the allowable wsize and to allow the
>>>>> kernel to issue writes in parallel.
>>>>
>>>> Ahh, good to know, have not tried writes yet.
>>>>
>>>>>
>>>>> Reads still suffer from the same problem however. I'm working on a
>>>>> patchset that should do the same thing for them, but it requires a
>>>>> fairly substantial overhaul of the receive codepaths.
>>>>
>>>> Ok, that explains it then, thanks.
>>
>>
>> Hi,
>>
>> Watching the rsync, it ran for a while, then:
>>
>> rsync: send_files failed to open "/cifs/w1/r1/data/hs12/f4_0.JPG": Cannot
>> allocate memory (12)
>> rsync: send_files failed to open "/cifs/w1/r1/data/hs12/f4_1.JPG": Cannot
>> allocate memory (12)
>> rsync: send_files failed to open "/cifs/w1/r1/data/hs12/f4_2.JPG": Cannot
>> allocate memory (12)
>> rsync: send_files failed to open "/cifs/w1/r1/data/hs12/f4_0.JPG": Cannot
>> allocate memory (12)
>
> When we were testing async write to Windows 7 Pavel mentioned to me
> another WIndows 7 bug - which may be what you are hitting.
>
> Under stress of simultaneous operations, Windows 7 server will sometimes start
> responding with STATUS_INSUFF_SERVER_RESOURCES error code
> (mapped to posix error ENOMEM by the Linux cifs kernel client)
> He solved it by setting MaxWorkItems to 4096 in the Windows 7 registry.
>
> If anyone knows whether Microsoft has fixed this or has a bug #, let us know
> because it is easier to hit with Linux kernel 3.0 and later (to
> Windows 7 server).

Ok, aside from sharing a few files, is there any recommended protocol to
transfer millions of files at 10GbE speeds (500MiB/s) in both directions
between Windows and Linux?

Specifically from Linux..

>From Windows, it achieves 500MiB/s no issues (From Linux/Samba).

The other direction:
- 30mb/s over 10GbE
- Cannot allocate memory

This should be good for moving a small subset of files, but for larger datasets
it will not work.  Any other filesystems/services/etc to try?

Justin.