From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id E0FCD1381F3 for ; Sun, 9 Jun 2013 07:46:58 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 0A2E6E0898; Sun, 9 Jun 2013 07:46:52 +0000 (UTC) Received: from mail-wg0-f46.google.com (mail-wg0-f46.google.com [74.125.82.46]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id B3AB9E081D for ; Sun, 9 Jun 2013 07:46:50 +0000 (UTC) Received: by mail-wg0-f46.google.com with SMTP id c11so617904wgh.13 for ; Sun, 09 Jun 2013 00:46:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=acHW16vce7iQatdz/3mkguhKSnBiGtzoan8WdelnQwM=; b=fJ6hmtvc+2JeRCUpISLBYxc2nf7ay+NRrqnTqe9xPltc1b2UIJowp4FHMbYU6oXndz 2AGUhOONb7QrsdDnT36IdkAKyrlPAjb54X3mNy+3PL1PVGtS6aJTQVV8yp48obJVu3F4 +C+gOQwUq4HTkCPPQOmD9GTy6ywX2q1GFjLJt0QCDYOSMuKJJeUAjtGwDHBal3EU+tav QxXlv6Tmv+ULIrdjgaA4mQdqTDkY/HzI57pGQBlXgw1FM5olvf44GMXej7NQ7Lgx/iFh sAakt06WJfim0qXDA0f333VdWOESwuCqIJh/5o44A4a9qsNshrj4e78vXMSbBQD1acqZ tgdQ== X-Received: by 10.180.149.171 with SMTP id ub11mr2191948wib.40.1370764009394; Sun, 09 Jun 2013 00:46:49 -0700 (PDT) Received: from [172.20.0.41] (196-210-127-214.dynamic.isadsl.co.za. [196.210.127.214]) by mx.google.com with ESMTPSA id b19sm4927100wik.10.2013.06.09.00.46.47 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 09 Jun 2013 00:46:48 -0700 (PDT) Message-ID: <51B432A3.9090903@gmail.com> Date: Sun, 09 Jun 2013 09:45:39 +0200 From: Alan McKinnon User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130518 Thunderbird/17.0.6 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-user@lists.gentoo.org Reply-to: gentoo-user@lists.gentoo.org MIME-Version: 1.0 To: gentoo-user@lists.gentoo.org Subject: Re: [gentoo-user] Help running fsck on reiserfs lvm /var on production server? References: <51B3A409.7050701@libertytrek.org> In-Reply-To: <51B3A409.7050701@libertytrek.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Archives-Salt: 9007b3ec-ca85-40ba-b10c-40f3c5cd2ef7 X-Archives-Hash: 43769fe5ced108f07275cc34b6b2e293 On 08/06/2013 23:37, Tanstaafl wrote: > Hi everyone, > > What is best practice for doing this? > > If I reboot in single user mode, will my lvm volumes (ie, /var) be > available for fsck'ing, or do I have to mount them first? > > The current problem started after a different problem required me to do > a hard reset on the server - had to do with a mounted QNAP device being > unavailable when I initiated a reboot, and everything just hung. > > Ever since I did this hard reset, the server hangs at unmounting /var. > I've let it sit there for at least an hour, and it never goes past that. > > Then after I hard reset it, it fsck's /var partition again, maybe fixes > minor problems very quickly, and everything works fine until I have to > reboot or shutdown again. > > This became a major problem this weekend when we had one extended power > outage (about 8 hours) yesterday evening, then another one (about 4 > hours) this morning right after I got everything back up and running > from last nights outage. > > Anyway, I need to do this this weened if at all possible, so... > > Anyone have any pointers to detailed docs and or willing to hold my hand > through this a little? fsck'ing that filesystem should be no different from any other fsck - it should find what it finds and fix what it can. The fs must be unmounted of course which means you have to do it in single-user mode, or from booting a rescue system (I prefer the second, I find it easier as none of the production filesystems are required to be mounted). fsck.resiserfs has several modes, IIRC there's --rebuild-tree or similar that does an extensive checks but takes ages. I needed to do this 2 or 3 times when I was still using reiser. There's also an option to do not writes if you want a sanity check first. I'm not convinced a power outage broke the fs so that you now can't umount it, I'm having a hard time imaging how that would happen. More likely some other script file elsewhere is damaged and leaves files open when the system wants to umount /var. You have some options: This requires considerable downtime, easily an hour or more. You can dd /var somewhere to get a copy you can experiment on with another host. At least you will then know how much downtime to schedule. You should do a full check and repair on all filesystems to be 100% certain. For the umount issues, that is trickier as you won't have log files in /var after the fact. Any clues on the Alt-F12 console whilst shutting down? Try configure your syslogger to send logs to another host, you might be lucky enough to get some logs that way that describe what is going on. -- Alan McKinnon alan.mckinnon@gmail.com