* dma_alloc_coherent() sets __GFP_NORETRY ? [was: Re: [PATCH 1/2] dpt_i2o: 64 bit support (take 4)]
[not found] ` <1209144552.3087.43.camel@localhost.localdomain>
@ 2008-05-20 0:24 ` Miquel van Smoorenburg
0 siblings, 0 replies; only message in thread
From: Miquel van Smoorenburg @ 2008-05-20 0:24 UTC (permalink / raw)
To: James Bottomley; +Cc: linux-scsi, linux-kernel
On Fri, 2008-04-25 at 12:29 -0500, James Bottomley wrote:
> On Thu, 2008-04-24 at 23:33 +0200, Miquel van Smoorenburg wrote:
>
> >
> > memset(msg, 0, sizeof(msg));
> > - buf = kmalloc(80,GFP_KERNEL|ADDR32);
> > + buf = pci_alloc_consistent(pHba->pDev, 80, &addr);
>
> You probably want to use dma_alloc_coherent here ... it's identical to
> pci_alloc_consistent in almost every way, except that it allows you to
> pass in the GFP_KERNEL flag (pci_alloc_consistent has to assume
> GFP_ATOMIC and thus you can get unexpected failures if SLUB is having a
> bad day) and you have to call it on &pHba->pDev->dev and use the
> corresponding dma_free_coherent().
I actually did that in the next patch, but I have been looking a bit
deeper into this and it might not be such a good idea. That, or there is
a bug in pci-dma_64.c.
In arch/x86/kernel/pci-dma_64.c , dma_alloc_coherent() adds
__GFP_NORETRY to the gfp flags before it calls __get_free_pages (through
dma_alloc_pages).
That means dma_alloc_coherent() -> __get_free_pages() can fail quite
easily on x86_64 with GFP_KERNEL.
If in __get_free_pages() try_to_free_pages() fails once, and
__GFP_NORETRY is set, there is .. well .. no retry :)
But why does dma_alloc_coherent() on x86_64 set __GFP_NORETRY ? It says
"don't invoke OOM killer" but I think it has more side affects than
that: easier failure.
Now I think I know why the 3ware management utility tw_cli crashes a lot
on my 64-bit boxes with a large diskwrite load ... I've fixed that now
by commenting out gfp |= __GFP_NORETRY .
Note that pci-dma_32.c in 2.6.25 does not do this, but in 2.6.26-rc3 the
two have been merged and __GFP_NORETRY is set for x86_32 as well now. Is
that a good idea ? Perhaps a __GFP_NO_OOMKILL ?
Mike.
^ permalink raw reply [flat|nested] only message in thread