mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: "Torsten Kaiser" <just.for.lkml@googlemail.com>
To: "Herbert Xu" <herbert@gondor.apana.org.au>
Cc: "Andrew Morton" <akpm@linux-foundation.org>,
	linux-kernel@vger.kernel.org, "Neil Brown" <neilb@suse.de>,
	"J. Bruce Fields" <bfields@fieldses.org>,
	netdev@vger.kernel.org
Subject: Re: 2.6.24-rc6-mm1
Date: Sun, 30 Dec 2007 04:34:36 +0100	[thread overview]
Message-ID: <64bb37e0712291934o77a3d365h56c9c31ac8437469@mail.gmail.com> (raw)
In-Reply-To: <20071230013021.GA13603@gondor.apana.org.au>

On Dec 30, 2007 2:30 AM, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> On Sat, Dec 29, 2007 at 05:51:13PM +0100, Torsten Kaiser wrote:
> >
> > > > The cause, why I am resending this: I just got a crash with
> > > > 2.6.24-rc6-mm1, again looking network related:
> > > >
> > > > [93436.933356] WARNING: at include/net/dst.h:165 dst_release()
> > > > [93436.936685] Pid: 8079, comm: konqueror Not tainted 2.6.24-rc6-mm1 #11
> > > > [93436.939292]
> > > > [93436.939293] Call Trace:
> > > > [93436.939304]  [<ffffffff80531d2d>] skb_release_all+0xdd/0x110
> > > > [93436.939307]  [<ffffffff80531311>] __kfree_skb+0x11/0xa0
> > > > [93436.939309]  [<ffffffff805313b7>] kfree_skb+0x17/0x30
> > > > [93436.939312]  [<ffffffff805a0b48>] unix_release_sock+0x128/0x250
> > > > [93436.939315]  [<ffffffff805a0c91>] unix_release+0x21/0x30
> > > > [93436.939318]  [<ffffffff8052b144>] sock_release+0x24/0x90
> > > > [93436.939320]  [<ffffffff8052b656>] sock_close+0x26/0x50
> > > > [93436.939324]  [<ffffffff8029f921>] __fput+0xc1/0x230
> > > > [93436.939327]  [<ffffffff8029fe46>] fput+0x16/0x20
> > > > [93436.939329]  [<ffffffff8029c576>] filp_close+0x56/0x90
> > > > [93436.939331]  [<ffffffff8029de46>] sys_close+0xa6/0x110
> > > > [93436.939335]  [<ffffffff8020b57b>] system_call_after_swapgs+0x7b/0x80
> >
> > >From code inspection I would blame the patch "[SKBUFF]: Free old skb
> > properly in skb_morph" from Herbert Xu. (CC added)
>
> I doubt it.  skb_morph is only used on IP fragments so I don't see how
> you could attribute an error from a Unix domain socket to this patch.

That's why I wrote that I do not know much about the network core...

> In any case, Unix socket packets should not have a dst at all so the
> very fact that you're in that path means that you have some sort of
> memory corruption.

... I did not know about the fact that there should not have been an dst.

Its just that this warning was the first nice clue about the memory
corruption related to networking that I see since 2.6.24-rc3-mm2.
The time of the patch (Mon, 26 Nov 2007 15:11:19) even fits into the
window between -rc3-mm1 and -rc3-mm2.

I doubt that the memory corruption is a hardware problem, because the
system in question is using ECC ram and I did not see any messages
about corrected/detected errors.

> Is this the very first OOPS/warning that you see? If not you should
> ignore all but the very first one as that may have left your system
> in an inconsistent state which may render all subsequent OOPSes and
> warnings useless.

I looked into the log in question and the only other warning was a
circular locking dependency that lockdep detected around 1.5 hour
before this warning.

As reported in my original mail immeadeatly after the warning the
system OOPSed and hang:
[93436.947241] general protection fault: 0000 [1] SMP
-> first OOPS
[93436.947243] last sysfs file:
/sys/devices/pci0000:00/0000:00:0f.0/0000:01:00.1/irq
[93436.947245] CPU 1
[93436.947246] Modules linked in: radeon drm nfsd exportfs w83792d
ipv6 tuner tea5767 tda8290 tuner_xc2
028 tda9887 tuner_simple mt20xx tea5761 tvaudio msp3400 bttv ir_common
compat_ioctl32 videobuf_dma_sg v
ideobuf_core btcx_risc tveeprom usbhid videodev v4l2_common hid
v4l1_compat pata_amd sg i2c_nforce2
[93436.947257] Pid: 8079, comm: konqueror Not tainted 2.6.24-rc6-mm1 #11
-> not tainted by a previous OOPS
[93436.947259] RIP: 0010:[<ffffffff80531438>]  [<ffffffff80531438>]
skb_drop_list+0x18/0x30
[93436.947262] RSP: 0018:ffff810005f4fda8  EFLAGS: 00010286
[93436.947263] RAX: ab1ed5ca5b74e7de RBX: ab1ed5ca5b74e7de RCX: 000000000000d135
[93436.947265] RDX: ffff81011d089a80 RSI: 0000000000000001 RDI: ffff81011d089a88
[93436.947266] RBP: ffff810005f4fdb8 R08: 0000000000000001 R09: 0000000000000006
[93436.947268] R10: 0000000000000000 R11: 0000000000000000 R12: ffff8100de02c500
[93436.947269] R13: ffff81011c188a00 R14: 0000000000000001 R15: ffff81011c189198
[93436.947271] FS:  00007fb5bde0d700(0000) GS:ffff81007ff22000(0000)
knlGS:0000000000000000
[93436.947273] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[93436.947274] CR2: 00007fb5bdd76000 CR3: 00000000664d5000 CR4: 00000000000006e0
[93436.947276] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[93436.947277] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[93436.947279] Process konqueror (pid: 8079, threadinfo
ffff810005f4e000, task ffff8100a1dec000)
[93436.947281] Stack:  ffff810005f4fdd8 ffff810116c86140
ffff810005f4fdd8 ffffffff805314ae
[93436.947284]  ffff810116c86140 ffff8100de02c500 ffff810005f4fdf8
ffffffff80531cf0
[93436.947286]  ffff8100de02c500 ffff81011c188b48 ffff810005f4fe18
ffffffff80531311
[93436.947288] Call Trace:
[93436.947290]  [<ffffffff805314ae>] skb_release_data+0x5e/0xa0
[93436.947293]  [<ffffffff80531cf0>] skb_release_all+0xa0/0x110
[93436.947295]  [<ffffffff80531311>] __kfree_skb+0x11/0xa0
[93436.947297]  [<ffffffff805313b7>] kfree_skb+0x17/0x30
[93436.947299]  [<ffffffff805a0b48>] unix_release_sock+0x128/0x250
[93436.947302]  [<ffffffff805a0c91>] unix_release+0x21/0x30
[93436.947304]  [<ffffffff8052b144>] sock_release+0x24/0x90
[93436.947307]  [<ffffffff8052b656>] sock_close+0x26/0x50
[93436.947309]  [<ffffffff8029f921>] __fput+0xc1/0x230
[93436.947312]  [<ffffffff8029fe46>] fput+0x16/0x20
[93436.947314]  [<ffffffff8029c576>] filp_close+0x56/0x90
[93436.947316]  [<ffffffff8029de46>] sys_close+0xa6/0x110
[93436.947319]  [<ffffffff8020b57b>] system_call_after_swapgs+0x7b/0x80
[93436.947322]
[93436.947322]
[93436.947323] Code: 48 8b 18 48 89 c7 e8 5d ff ff ff 48 85 db 75 ed 48 83 c4 08
[93436.947328] RIP  [<ffffffff80531438>] skb_drop_list+0x18/0x30
[93436.947330]  RSP <ffff810005f4fda8>
[93436.947332] ---[ end trace befb7cc3528ab3b1 ]---

Your patch just fit so "good" to my problems:
* it had the correct time frame for 2.6.24-rc3-mm2
* it looked guilty at changing the refcounting of __refcnt because of
the added dst_release()
* it added other release / freeing operations so that a use-after-free
memory corruption seemed possible

I just have no better idea to what caused this OOPS and the other
hangs in -rc3-mm2.

Torsten

  reply	other threads:[~2007-12-30  3:34 UTC|newest]

Thread overview: 146+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-12-23  7:30 2.6.24-rc6-mm1 Andrew Morton
2007-12-23 11:04 ` 2.6.24-rc6-mm1 Ingo Molnar
2007-12-23 11:10   ` 2.6.24-rc6-mm1 Ingo Molnar
2007-12-23 11:34     ` 2.6.24-rc6-mm1 Andrew Morton
2007-12-23 11:57       ` 2.6.24-rc6-mm1 Ingo Molnar
2007-12-23 12:12         ` 2.6.24-rc6-mm1 Christoph Hellwig
2007-12-23 12:35 ` 2.6.24-rc6-mm1 Rafael J. Wysocki
2007-12-23 13:00   ` 2.6.24-rc6-mm1 Ingo Molnar
2007-12-23 13:48     ` 2.6.24-rc6-mm1 Rafael J. Wysocki
2007-12-23 13:53       ` 2.6.24-rc6-mm1 Rafael J. Wysocki
2007-12-23 20:09         ` 2.6.24-rc6-mm1 Sam Ravnborg
2007-12-23 22:44           ` 2.6.24-rc6-mm1 Rafael J. Wysocki
2008-01-02 20:05             ` [patch] x86 compat_binfmt_elf, Makefile fixes (was: Re: 2.6.24-rc6-mm1) Ingo Molnar
2008-01-02 20:32               ` Rafael J. Wysocki
2008-01-02 20:43                 ` Ingo Molnar
2007-12-23 23:09   ` 2.6.24-rc6-mm1 H. Peter Anvin
2007-12-23 16:27 ` 2.6.24-rc6-mm1 Torsten Kaiser
2007-12-23 20:39   ` 2.6.24-rc6-mm1 Andrew Morton
2007-12-27 11:42     ` 2.6.24-rc6-mm1 Torsten Kaiser
2007-12-27 14:30       ` 2.6.24-rc6-mm1 Torsten Kaiser
2007-12-28 22:53   ` 2.6.24-rc6-mm1 Torsten Kaiser
2007-12-28 23:07     ` 2.6.24-rc6-mm1 Andrew Morton
2007-12-29 16:51       ` 2.6.24-rc6-mm1 Torsten Kaiser
2007-12-30  1:30         ` 2.6.24-rc6-mm1 Herbert Xu
2007-12-30  3:34           ` Torsten Kaiser [this message]
2007-12-30  5:41             ` 2.6.24-rc6-mm1 Randy Dunlap
2007-12-31 20:15             ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-01 12:04               ` 2.6.24-rc6-mm1 Herbert Xu
2008-01-01 12:59                 ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-01 18:29                   ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-02 18:29                 ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-02 21:51                   ` 2.6.24-rc6-mm1 Herbert Xu
2008-01-02 21:57                     ` 2.6.24-rc6-mm1 J. Bruce Fields
2008-01-03  5:02                       ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-03 15:37                       ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-03 18:52                         ` 2.6.24-rc6-mm1 J. Bruce Fields
2008-01-04 10:23                     ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-04 13:30                       ` 2.6.24-rc6-mm1 Jarek Poplawski
2008-01-04 15:21                         ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-04 21:24                           ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-05  0:07                           ` 2.6.24-rc6-mm1 Jarek Poplawski
2008-01-05  8:01                             ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-05 10:13                               ` 2.6.24-rc6-mm1 Jarek Poplawski
2008-01-05 14:52                                 ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-05 22:10                                   ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-06  1:25                                     ` 2.6.24-rc6-mm1 Andrew Morton
2008-01-06  3:28                                       ` 2.6.24-rc6-mm1 FUJITA Tomonori
2008-01-06 10:41                                         ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-06 11:23                                           ` 2.6.24-rc6-mm1 FUJITA Tomonori
2008-01-06 11:35                                             ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-06 13:33                                               ` 2.6.24-rc6-mm1 FUJITA Tomonori
2008-01-06 20:03                                                 ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-07  6:16                                                   ` 2.6.24-rc6-mm1 FUJITA Tomonori
2008-01-08 15:59                                                     ` 2.6.24-rc6-mm1 Ingo Molnar
2008-01-08 23:57                                                       ` 2.6.24-rc6-mm1 FUJITA Tomonori
2008-01-09  0:27                                                         ` 2.6.24-rc6-mm1 Andrew Morton
2008-01-09  0:54                                                           ` 2.6.24-rc6-mm1 FUJITA Tomonori
2008-01-09  1:07                                                             ` 2.6.24-rc6-mm1 Andrew Morton
2008-01-09  9:04                                                         ` 2.6.24-rc6-mm1 Jarek Poplawski
2008-01-10  0:54                                                           ` 2.6.24-rc6-mm1 FUJITA Tomonori
2008-01-25 21:06                                                     ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-06  3:16                                     ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-06  8:27                                   ` 2.6.24-rc6-mm1 Jarek Poplawski
2008-01-06 10:30                                     ` 2.6.24-rc6-mm1 Torsten Kaiser
2008-01-06 14:52                                       ` 2.6.24-rc6-mm1 Jarek Poplawski
2007-12-30 21:24       ` 2.6.24-rc6-mm1 J. Bruce Fields
2007-12-30 21:35         ` 2.6.24-rc6-mm1 Torsten Kaiser
2007-12-31 13:17           ` 2.6.24-rc6-mm1 Torsten Kaiser
2007-12-23 22:54 ` 2.6.24-rc6-mm1: suspend broken on HP nx6325 due to cpufreq changes Rafael J. Wysocki
2007-12-23 22:50   ` Andrew Morton
2007-12-23 22:56     ` Dave Jones
2007-12-24 11:21     ` Takashi Iwai
2007-12-24 14:13       ` Rafael J. Wysocki
2007-12-25 21:51 ` 2.6.24-rc6-mm1 Andreas Mohr
2007-12-26  5:43 ` 2.6.24-rc6-mm1 - git-lblnet.patch and networking horkage Valdis.Kletnieks
2007-12-26  7:34   ` James Morris
2007-12-26  8:25     ` Valdis.Kletnieks
2007-12-26  8:52       ` James Morris
2007-12-26 14:16         ` James Morris
2007-12-26 22:46           ` Andrew Morton
2007-12-26 23:39             ` 2.6.24-rc6-mm1 - e1000 breakage James Morris
2007-12-26 16:44         ` 2.6.24-rc6-mm1 - git-lblnet.patch and networking horkage Valdis.Kletnieks
2007-12-26  8:37 ` 2.6.24-rc6-mm1 Dave Young
2007-12-26 12:29 ` 2.6.24-rc6-mm1: some section mismatches on sparc64 Mariusz Kozlowski
2007-12-27  3:05   ` David Miller
2007-12-28 23:22     ` Adrian Bunk
2007-12-29  8:14       ` David Miller
2007-12-29  8:27         ` David Miller
2007-12-29  8:48         ` Adrian Bunk
2007-12-29  8:54           ` David Miller
2007-12-29  9:06             ` Adrian Bunk
2007-12-29  9:18               ` David Miller
2007-12-29  9:53                 ` Adrian Bunk
2007-12-29  9:15             ` David Miller
2007-12-27  2:21 ` 2.6.24-rc6-mm1: __raw_spin_is_contended undefined Joseph Fannin
2007-12-27  5:21   ` Nick Piggin
2007-12-27  6:04 ` 2.6.24-rc6-mm1 - drivers/char/tpm/tpm_bios.c oddness? Valdis.Kletnieks
2007-12-27  8:49 ` 2.6.24-rc6-mm1 Kernel panics at different functions () Kamalesh Babulal
2007-12-27  9:54   ` Andrew Morton
2007-12-27 10:25     ` Kamalesh Babulal
2007-12-28  9:11     ` Kamalesh Babulal
2007-12-28 13:03       ` Dhaval Giani
2007-12-27  9:16 ` 2.6.24-rc6-mm1 - power_supply driver dmesg weirdness? Valdis.Kletnieks
2007-12-27 17:54 ` 2.6.24-rc6-mm1 - crash in tick_sched_timer/update_process_times Valdis.Kletnieks
2007-12-28  7:34   ` Andrew Morton
2007-12-29 16:02   ` Valdis.Kletnieks
2007-12-31 18:05 ` [PATCH -mm] crypto: scatterwalk.h needs sched.h Randy Dunlap
2007-12-31 22:31   ` Herbert Xu
2007-12-31 18:05 ` [PATCH -mm] driver core: build with SYSFS=n Randy Dunlap
2007-12-31 18:18 ` 2.6.24-rc6-mm1 (build problem: v4l / i2c) Randy Dunlap
2007-12-31 19:01   ` [v4l-dvb-maintainer] " Michael Krufky
2007-12-31 19:08     ` Michael Krufky
2007-12-31 18:18 ` 2.6.24-rc6-mm1 (build problem: gpio_keys) Randy Dunlap
2007-12-31 18:40   ` David Brownell
2007-12-31 19:10     ` [PATCH -mm] gpio: fix x86 build problem: gpio_keys Randy Dunlap
2008-01-01 15:32       ` Ingo Molnar
2007-12-31 18:19 ` 2.6.24-rc6-mm1 (build problem: gpio/W1) Randy Dunlap
2008-01-05 15:29   ` Evgeniy Polyakov
2008-01-05 16:16     ` Ville Syrjälä
2008-01-05 17:18       ` Randy Dunlap
2007-12-31 20:11 ` 2.6.24-rc6-mm1 (driver core/sysfs) Randy Dunlap
2008-01-12  1:05   ` Greg KH
2008-01-14 16:11     ` [patch] auto-qa Kconfig Ingo Molnar
2008-01-15 22:13       ` Pavel Machek
2008-01-06 10:03 ` 2.6.24-rc6-mm1: sparc64: undefined reference to `vmemmap_table' Mariusz Kozlowski
2008-01-06 10:15   ` Andrew Morton
2008-01-07  1:18     ` David Miller
2008-01-07 13:50 ` [PATCH -mm/x86] revert i386: handle an initrd in highmem (Was Re: 2.6.24-rc6-mm1) Dhaval Giani
2008-01-07 13:53   ` Thomas Gleixner
2008-01-07 14:56     ` Ingo Molnar
2008-01-07 15:19       ` Dhaval Giani
2008-01-07 16:22       ` Randy Dunlap
2008-01-07 16:32         ` Dhaval Giani
2008-01-07 16:50         ` Thomas Gleixner
2008-01-07 22:15           ` Randy Dunlap
     [not found] <9DtBq-2jD-3@gated-at.bofh.it>
     [not found] ` <476EAF98.6040004@yahoo.fr>
     [not found]   ` <20071223121410.0d572e03.akpm@linux-foundation.org>
2007-12-24  1:25     ` 2.6.24-rc6-mm1 Herbert Xu
2007-12-30 13:10       ` 2.6.24-rc6-mm1 Ingo Molnar
2008-01-02 10:31         ` 2.6.24-rc6-mm1 Nick Piggin
2008-01-02 11:01           ` 2.6.24-rc6-mm1 Peter Zijlstra
2008-01-02 11:12             ` 2.6.24-rc6-mm1 Nick Piggin
2008-01-02 11:24               ` 2.6.24-rc6-mm1 Peter Zijlstra
2008-01-02 12:19                 ` 2.6.24-rc6-mm1 Ingo Molnar
2008-01-02 13:26                   ` 2.6.24-rc6-mm1 Alan Cox
2008-01-02 16:18                     ` 2.6.24-rc6-mm1 Ingo Molnar
2008-01-02 22:49                       ` 2.6.24-rc6-mm1 Alan Cox
2008-01-02 11:08           ` 2.6.24-rc6-mm1 Ingo Molnar

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=64bb37e0712291934o77a3d365h56c9c31ac8437469@mail.gmail.com \
    --to=just.for.lkml@googlemail.com \
    --cc=akpm@linux-foundation.org \
    --cc=bfields@fieldses.org \
    --cc=herbert@gondor.apana.org.au \
    --cc=linux-kernel@vger.kernel.org \
    --cc=neilb@suse.de \
    --cc=netdev@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®