Re: list corruption in the last few days. (block ? crypto ?)

[Xiaotian Feng]

On Mon, Aug 8, 2011 at 2:38 PM, Pekka Enberg <penberg@xxxxxxxxxx> wrote:
> On Mon, Aug 8, 2011 at 9:31 AM, Xiaotian Feng <xtfeng@xxxxxxxxx> wrote:
>> On Mon, Aug 8, 2011 at 2:18 PM, Pekka Enberg <penberg@xxxxxxxxxx> wrote:
>>> On Mon, Aug 8, 2011 at 9:14 AM, Xiaotian Feng <xtfeng@xxxxxxxxx> wrote:
>>>> On Mon, Aug 8, 2011 at 1:18 PM, Pekka Enberg <penberg@xxxxxxxxxx> wrote:
>>>>> Hi Linus,
>>>>>
>>>>> On Sun, Aug 7, 2011 at 11:58 AM, Pekka Enberg <penberg@xxxxxxxxxx> wrote:
>>>>>>>
>>>>>>> Christoph, I've been reading the code and spotted two potential issues in
>>>>>>> __slab_free(). The first one seems like an off-by-one where our
>>>>>>> comparison
>>>>>>> in deactivate_slab() doesn't match __slab_free.
>>>>>>>
>>>>>>> The other one is remove_full() call in __slab_free() that can get called
>>>>>>> even if cache debugging is not enabled.
>>>>>>>
>>>>>>> Hmm?
>>>>>
>>>>> On Sun, 7 Aug 2011, Linus Torvalds wrote:
>>>>>>
>>>>>> I'd like to do -rc1 today, regardless of whether this fixes things or
>>>>>> not (-rc1 is already a few days delayed).
>>>>>>
>>>>>> The patch seems to be a good fix, and a likely candidate for the
>>>>>> corruption. Commit log and sign-off? I assume you've given it some
>>>>>> testing, even if you couldn't reproduce the original issue?
>>>>>
>>>>> No, I haven't tested the patch myself but here's one in proper format in
>>>>> case someone wants to test it.
>>>>
>>>> I applied it and I'm still seeing the corruption warning start from this :(
>>>>
>>>> Â[ 3674.255030] ------------[ cut here ]------------
>>>> Â[ 3674.255040] WARNING: at lib/list_debug.c:53 __list_del_entry+0xa1/0xd0()
>>>> Â[ 3674.255042] Hardware name: 42424XC
>>>> Â[ 3674.255045] list_del corruption. prev->next should be
>>>> ffffea0000493420, but was ffffea0001ab0520
>>>> Â[ 3674.255047] Modules linked in: ip6table_filter ip6_tables
>>>> ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4
>>>> xt_state nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp
>>>> iptable_filter ip_tables x_tables binfmt_misc bridge stp parport_pc
>>>> ppdev snd_hda_codec_conexant snd_hda_intel snd_hda_codec snd_hwdep
>>>> snd_pcm thinkpad_acpi arc4 cryptd aes_x86_64 aes_generic snd_seq_midi
>>>> i915 snd_rawmidi snd_seq_midi_event snd_seq joydev iwlagn btusb
>>>> snd_timer snd_seq_device drm_kms_helper bluetooth uvcvideo snd
>>>> videodev drm mac80211 cfg80211 i2c_algo_bit soundcore tpm_tis tpm
>>>> tpm_bios lp snd_page_alloc psmouse serio_raw v4l2_compat_ioctl32 nvram
>>>> video parport usbhid hid sdhci_pci firewire_ohci ahci libahci sdhci
>>>> firewire_core crc_itu_t e1000e
>>>> Â[ 3674.255129] Pid: 3, comm: ksoftirqd/0 Not tainted 3.1.0-rc1+ #38
>>>> Â[ 3674.255131] Call Trace:
>>>> Â[ 3674.255138] Â[<ffffffff8106db3f>] warn_slowpath_common+0x7f/0xc0
>>>> Â[ 3674.255143] Â[<ffffffff8106dc36>] warn_slowpath_fmt+0x46/0x50
>>>> Â[ 3674.255147] Â[<ffffffff81331851>] __list_del_entry+0xa1/0xd0
>>>> Â[ 3674.255151] Â[<ffffffff81331891>] list_del+0x11/0x40
>>>> Â[ 3674.255156] Â[<ffffffff81179631>] __slab_free+0x3d1/0x3e0
>>>> Â[ 3674.255162] Â[<ffffffff811c5b06>] ? bvec_free_bs+0x26/0x40
>>>> Â[ 3674.255166] Â[<ffffffff8117ab27>] ? kmem_cache_free+0x97/0x220
>>>> Â[ 3674.255170] Â[<ffffffff811c5b06>] ? bvec_free_bs+0x26/0x40
>>>> Â[ 3674.255174] Â[<ffffffff811c5b06>] ? bvec_free_bs+0x26/0x40
>>>> Â[ 3674.255179] Â[<ffffffff8117ac9f>] kmem_cache_free+0x20f/0x220
>>>
>>> Can you please also run
>>>
>>> Âaddr2line -e vmlinux
>>> Â0xffffffff81179631
>>>
>>
>> /opt/upstream/linux-3.0/mm/slub.c:1061
>>
>>
>> /*
>> Â* list_lock must be held.
>> Â*/
>> static void remove_full(struct kmem_cache *s, struct page *page)
>> {
>> Â Â Â Âif (!(s->flags & SLAB_STORE_USER))
>> Â Â Â Â Â Â Â Âreturn;
>>
>> Â Â Â Âlist_del(&page->lru); Â Â Â <=== here
>> }
>
> OK, so it's the full list. Looking at deactivate_slab() we actually
> can access the list without n->list_lock held. Does the attached patch
> help?

I'm still facing a hang with this patch, but I didn't catch the original log :(
Will try it again and get you updated, thanks.

>
> Â Â Â Â Â Â Â Â Â Â Â ÂPekka
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/