From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1PYSSJ-0002uM-UN for garchives@archives.gentoo.org; Fri, 31 Dec 2010 00:02:52 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id B1F37E07BD for ; Fri, 31 Dec 2010 00:02:50 +0000 (UTC) Received: from atoth.sote.hu (atoth.sote.hu [195.111.75.211]) by pigeon.gentoo.org (Postfix) with ESMTP id 28341E0773 for ; Thu, 30 Dec 2010 23:16:03 +0000 (UTC) Received: from atoth.sote.hu (apache@localhost [127.0.0.1]) by atoth.sote.hu (8.14.4/8.14.4/atoth@atoth.sote.hu) with ESMTP id oBUNFxeG011592 for ; Fri, 31 Dec 2010 00:16:01 +0100 X-DKIM: Sendmail DKIM Filter v2.8.3 atoth.sote.hu oBUNFxeG011592 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=atoth.sote.hu; s=dwokfur; t=1293750963; bh=a5MRWsTeRIrpfuK8JAjUYKD5TJD2BR1Ec/DO21FbyH8=; l=7727; h=Message-ID:In-Reply-To:References:Date:Subject:From:To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=Cik+xx2hDNIMf3mqnD/7V8jWYmWaz4DyFC/0th83WPe2ZqLZiaFWZEMAgX1qX/bJD lAJb2Qzmbz643v3QMxMfcwUIUPtApauGZN4N71oGiy5lsnStiMvHdWkXv5iu4/24w7 hALl6jDo1+ivLnQj4nYxGMTEsOPB1SZQRJxUmcas= X-DomainKeys: Sendmail DomainKeys Filter v1.0.2 atoth.sote.hu oBUNFxeG011592 DomainKey-Signature: a=rsa-sha1; s=dwokfur; d=atoth.sote.hu; c=nofws; q=dns; h=x-virus-status:x-virus-scanned:received:message-id: in-reply-to:references:date:subject:from:to:user-agent:mime-version: content-type:content-transfer-encoding:x-priority:importance: x-spam-status:x-spam-checker-version:x-list-milter:x-dcc-debian-metrics; b=lUF++txrKUmRhx+A6GTPtVlF6WoxyPIWzD4z3E/c5rgdiZPH65VCLqsM3XLZa2bx0 LUvn+zsDn3A348tsJB754W3ru2Kn/xo+k1GpgDpeOo9k/s+TE80zSVzuHnW4A/kc0a8 orb5b323l4r4DXhTiUnj/VHnHcGoxJotnNYKTCU= X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.96.5 at atoth Received: from 193.6.26.133 (SquirrelMail authenticated user atoth) by atoth.sote.hu with HTTP; Fri, 31 Dec 2010 00:16:01 +0100 Message-ID: <1034a62c0684fdae00a744a7e9d08cbf.squirrel@atoth.sote.hu> In-Reply-To: <4D1CFB14.30540.4766625B@pageexec.freemail.hu> References: <4D16E7BA.3030508@orlitzky.com>, <4D17AE55.5680.32B29833@pageexec.freemail.hu>, <0e4f0110b8b35e9b3f1be11a69cc45ba.squirrel@atoth.sote.hu> <4D1CFB14.30540.4766625B@pageexec.freemail.hu> Date: Fri, 31 Dec 2010 00:16:01 +0100 Subject: Re: [gentoo-hardened] Disappearing root on 2.6.36-hardened-r6 upgrade From: =?utf-8?B?IlTDs3RoIEF0dGlsYSI=?= To: gentoo-hardened@lists.gentoo.org User-Agent: SquirrelMail/1.4.21 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-hardened@lists.gentoo.org Reply-to: gentoo-hardened@lists.gentoo.org MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 X-Priority: 3 (Normal) Importance: Normal X-Spam-Status: No, score=-99.2 required=5.0 tests=ALL_TRUSTED,AWL, DKIM_ADSP_ALL,USER_IN_WHITELIST autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on atoth.sote.hu X-List-Milter: local mail X-DCC-debian-Metrics: atoth; whitelist Content-Transfer-Encoding: quoted-printable X-Archives-Salt: 62ef074a-4c42-4302-bd64-4b802f351928 X-Archives-Hash: 4c5b6cbb8666ab2778155414f49939ef 2010.December 30.(Cs) 21:35 id=C5=91pontban pageexec@freemail.hu ezt =C3=AD= rta: > On 30 Dec 2010 at 20:29, "T=C3=B3th Attila" wrote: > >> There were two screen shots attached. The older one was outdated relat= ed >> to 2.6.32 kernel. >> >> But the other was a recent panic. > > unfortunately this one had the first oops scroll away already, so i can= 't > tell > much about it... I took a look at on the logs again. You are right. First came this: Dec 30 19:43:33 szk-simor kernel: PAX: suspicious general protection fault: 0000 [#1] DEBUG_PAGEALLOC Dec 30 19:43:33 szk-simor kernel: last sysfs file: /sys/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A03:00/device:00/PNP0C09:00/PNP0= C0A:00/power_supply/BAT0/energy_full Dec 30 19:43:33 szk-simor kernel: Modules linked in: i2c_dev tp_smapi thinkpad_ec lib80211_crypt_wep lib80211_crypt_tkip lib80211_crypt_ccmp radeon ttm drm_kms_helper ehci_h cd ipw2200 libipw yenta_socket i2c_i801 uhci_hcd Dec 30 19:43:33 szk-simor kernel: Dec 30 19:43:33 szk-simor kernel: Pid: 1400, comm: kjournald Not tainted 2.6.36-hardened-r6 #1 1830W7F/1830W7F Dec 30 19:43:33 szk-simor kernel: EIP: 0060:[<0014d697>] EFLAGS: 00010216 CPU: 0 Dec 30 19:43:33 szk-simor kernel: EIP is at journal_commit_transaction+0x6f7/0xd00 Dec 30 19:43:33 szk-simor kernel: EAX: 00e89222 EBX: cc7ae76d ECX: 00000000 EDX: 00000000 Dec 30 19:43:33 szk-simor kernel: ESI: 00000005 EDI: 00000000 EBP: e9c1c4c0 ESP: f695bf04 Dec 30 19:43:33 szk-simor kernel: DS: 0068 ES: 0068 FS: 0000 GS: 00e0 SS: 0068 Dec 30 19:43:34 szk-simor kernel: Process kjournald (pid: 1400, ti=3Df695a000 task=3Df70bb0b0 task.ti=3Df695a000) Dec 30 19:43:34 szk-simor kernel: Stack: Dec 30 19:43:34 szk-simor kernel: 000028cd 26626a70 00029eaf f6ae6800 00000000 00000005 e3834c1c e10ed03c Dec 30 19:43:34 szk-simor kernel: <0> 00000fc4 00000001 f55b1000 f6ae68c0 ebe65e9e 0000681e f7076064 00000000 Dec 30 19:43:34 szk-simor kernel: <0> 10fe2a49 e62673c0 000293ec 000028ce e3e13910 f70bb0b0 005bb206 00000003 Dec 30 19:43:34 szk-simor kernel: Call Trace: Dec 30 19:43:34 szk-simor kernel: [<000028cd>] ? copy_thread+0x1d/0x140 Dec 30 19:43:34 szk-simor kernel: [<00029eaf>] ? switched_to_idle+0x1f/0x= 60 Dec 30 19:43:34 szk-simor kernel: [<0000681e>] ? write_ldt+0x10e/0x2d0 Dec 30 19:43:34 szk-simor kernel: [<000293ec>] ? finish_task_switch.clone.120.clone.124+0x2c/0x90 Dec 30 19:43:34 szk-simor kernel: [<000028ce>] ? copy_thread+0x1e/0x140 Dec 30 19:43:34 szk-simor kernel: [<005bb206>] ? schedule+0x146/0x3e0 Dec 30 19:43:34 szk-simor kernel: [<0014fb99>] ? kjournald+0x99/0x1b0 Dec 30 19:43:34 szk-simor kernel: [<00046ad0>] ? autoremove_wake_function+0x0/0x40 Dec 30 19:43:34 szk-simor kernel: [<0014fb00>] ? kjournald+0x0/0x1b0 Dec 30 19:43:34 szk-simor kernel: [<000466a4>] ? kthread+0x74/0x80 Dec 30 19:43:34 szk-simor kernel: [<00046630>] ? kthread+0x0/0x80 Dec 30 19:43:34 szk-simor kernel: [<0000455e>] ? kernel_thread_helper+0x6/0x18 Dec 30 19:43:34 szk-simor kernel: Code: 00 b9 03 00 00 00 89 ea e8 67 f7 ff ff 89 d8 ba 17 00 00 00 e8 1b 94 ef ff 89 d8 e8 f4 f3 f7 ff e9 ef fc f= f ff 83 6d 40 01 8b 03 40 34 71 04 ff 48 34 ce 8b 03 80 48 02 01 8b 44 24 4c 89 da Dec 30 19:43:34 szk-simor kernel: EIP: [<0014d697>] journal_commit_transaction+0x6f7/0xd00 SS:ESP 0068:f695bf04 Dec 30 19:43:34 szk-simor kernel: ---[ end trace 0f9efa514b41f93a ]--- and there came this: Dec 30 19:49:30 szk-simor kernel: PAX: suspicious general protection fault: 0000 [#2] DEBUG_PAGEALLOC Dec 30 19:49:30 szk-simor kernel: last sysfs file: /sys/devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A03:00/device:00/PNP0C09:00/PNP0= C0A:00/power_supply/BAT0/energy_full Dec 30 19:49:30 szk-simor kernel: Modules linked in: i2c_dev tp_smapi thinkpad_ec lib80211_crypt_wep lib80211_crypt_tkip lib80211_crypt_ccmp radeon ttm drm_kms_helper ehci_h cd ipw2200 libipw yenta_socket i2c_i801 uhci_hcd Dec 30 19:49:30 szk-simor kernel: Dec 30 19:49:30 szk-simor kernel: Pid: 12897, comm: shutdown Tainted: G = =20 D 2.6.36-hardened-r6 #1 1830W7F/1830W7F Dec 30 19:49:30 szk-simor kernel: EIP: 0060:[<000bdf8e>] EFLAGS: 00010206 CPU: 0 Dec 30 19:49:30 szk-simor kernel: EIP is at iput+0x4e/0x1f0 Dec 30 19:49:30 szk-simor kernel: EAX: e393467c EBX: e393467c ECX: 00000001 EDX: 48000200 Dec 30 19:49:30 szk-simor kernel: ESI: f68ab201 EDI: e3934abc EBP: e3934a14 ESP: e6ccfef0 Dec 30 19:49:30 szk-simor kernel: DS: 0068 ES: 0068 FS: 0000 GS: 00e0 SS: 0068 Dec 30 19:49:30 szk-simor kernel: Process shutdown (pid: 12897, ti=3De6cce000 task=3Df70c37f0 task.ti=3De6cce000) Dec 30 19:49:30 szk-simor kernel: Stack: Dec 30 19:49:30 szk-simor kernel: f68ab270 e3934a14 000c7217 7fffffff f68ab200 00000001 00000000 e6ccff0c Dec 30 19:49:30 szk-simor kernel: <0> e6ccff0c e6ccff18 00000000 e6ccff1c e6ccff1c f68ab200 00000001 000e6f00 Dec 30 19:49:30 szk-simor kernel: <0> 000cae20 000cadff f68ab200 f7205400 f68ab23c 000ab9f0 e6ccff60 11ee9e94 Dec 30 19:49:30 szk-simor kernel: Call Trace: Dec 30 19:49:30 szk-simor kernel: [<000c7217>] ? sync_inodes_sb+0xb7/0x10= 0 Dec 30 19:49:30 szk-simor kernel: [<000e6f00>] ? dquot_quota_sync+0x0/0x2= a0 Dec 30 19:49:30 szk-simor kernel: [<000cae20>] ? sync_one_sb+0x0/0x20 Dec 30 19:49:30 szk-simor kernel: [<000cadff>] ? __sync_filesystem+0x7f/0= xa0 Dec 30 19:49:30 szk-simor kernel: [<000ab9f0>] ? iterate_supers+0x50/0x90 Dec 30 19:49:30 szk-simor kernel: [<000cad42>] ? sync_filesystems+0x12/0x= 20 Dec 30 19:49:30 szk-simor kernel: [<000caea8>] ? sys_sync+0x18/0x40 Dec 30 19:49:30 szk-simor kernel: [<005bce39>] ? syscall_call+0x7/0xb Dec 30 19:49:30 szk-simor kernel: Code: c0 75 0a 5b 5e c3 8d b4 26 00 00 00 00 8b b3 9c 00 00 00 8b 46 20 85 c0 0f 84 3f 01 00 00 8b 50 10 85 d2 0= f 84 34 01 00 00 89 d8 d2 85 c0 0f 85 9c 00 00 00 f6 83 10 01 00 00 87 75 26 8b 53 Dec 30 19:49:30 szk-simor kernel: EIP: [<000bdf8e>] iput+0x4e/0x1f0 SS:ES= P 0068:e6ccfef0 Dec 30 19:49:30 szk-simor kernel: ---[ end trace 0f9efa514b41f93b ]--- Dec 30 19:49:35 szk-simor kernel: SysRq : Emergency Sync Dec 30 19:49:35 szk-simor kernel: Emergency Sync complete Dec 30 19:49:39 szk-simor kernel: SysRq : Emergency Remount R/O > >> So here is another one. This time I could paste it from the log: > > this is gain some fs/journaling code trying to increment some seemingly > invalid > pointer (in eax), there's probably some memory corruption going on here > and it'd > be important to try both vanilla and -r7. Now I'm running -r7. I may have time for vanilla. But I cannot reliably reproduce it. I'll give memtest a spin overnight. Last time it was OK. I also have a feeling of a possible memory corruption, but why it would always result i= n a file system error? I have no other symptoms. > >> It happens during IO activity. I wouldn't say heavy IO. The memory is >> OK, >> the harddrive is perfect. >> I can dd the whole hdd to my backup booting on a gentoo CD. > > is the filesystem ok as well (fsck)? > Because of these recurrent fs problems I reverted my mount options to use data=3Djournal and barrier=3D1. That is the most conservative and the slo= west. Fortunately I'm not a speed-freak. That way the systems survives these events without loosing fs consistency. But it happened before, that I had to restore some of my partitions from backup. It is interesting to note, that hardened-sources-2.6.32-r20 was more stable, than the other version I've met since than. It used grsec-2.2.0-2.6.32.24-201010021153. May be the memory handling of that kernel is different and that keeps it from triggering some memory problems... Thx: Dw. --=20 dr T=C3=B3th Attila, Radiol=C3=B3gus, 06-20-825-8057 Attila Toth MD, Radiologist, +36-20-825-8057