From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1RqO9Q-0008Da-F1 for garchives@archives.gentoo.org; Thu, 26 Jan 2012 12:10:00 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 1675EE07D3 for ; Thu, 26 Jan 2012 12:09:59 +0000 (UTC) Received: from gw1.transmode.se (gw1.transmode.se [195.58.98.146]) by pigeon.gentoo.org (Postfix) with ESMTP id BF6B1E0675 for ; Thu, 26 Jan 2012 11:11:42 +0000 (UTC) Received: from mail1.transmode.se (mail1.transmode.se [192.168.201.18]) by gw1.transmode.se (Postfix) with ESMTP id 199742581E1 for ; Thu, 26 Jan 2012 12:11:42 +0100 (CET) In-Reply-To: <4F2001D5.40109@wildgooses.com> References: <1324835572.6935.5.camel@here> <4F1DAA65.3000408@wildgooses.com> <20120124140629.GR26939@ns1.bonedaddy.net> <4F2001D5.40109@wildgooses.com> Subject: Re: [gentoo-embedded] quickpkg and PKG_INSTALL_MASK X-KeepSent: 1A73DD24:940A20DF-C1257991:002E47FA; type=4; name=$KeepSent To: gentoo-embedded@lists.gentoo.org X-Mailer: Lotus Notes Release 8.5.2FP3 Aug 10, 2010 Message-ID: From: Joakim Tjernlund Date: Thu, 26 Jan 2012 12:11:41 +0100 X-MIMETrack: Serialize by Router on mail1/Transmode(Release 8.5.3|September 15, 2011) at 01/26/2012 12:11:41 Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-embedded@lists.gentoo.org Reply-to: gentoo-embedded@lists.gentoo.org MIME-Version: 1.0 Content-type: text/plain; charset=US-ASCII X-Archives-Salt: 670f646c-26cf-493e-b856-50d0fb0d6cd0 X-Archives-Hash: 5cd7fbfabfbf6d6b993238e403bfffc8 Ed W wrote on 2012/01/25 14:21:25: > > On 24/01/2012 14:06, Todd Goodman wrote: > > * Joakim Tjernlund [120124 02:18]: > >> Ed W wrote on 2012/01/23 19:43:49: > > [ .. ] > >>> I'm doing something like this using aufs. The performance seems "not > >>> bad", but you get a couple MB or so memory hit. (I'm using squashfs as > >>> well, so unsure which causes the main memory increase). > >> hmm, not sure how aufs would work out. One would like to permanently delete the > > aufs is A Union FileSystem. You can have a RO "branch" (say SquashFS) > > and a r/w "branch" (say JFFS2 or UBIFS.) When you write it will write > > to the r/w branch and when you read it will read from the r/w branch if > > it exists, else the r/o. > > > > So, if you want to permanently delete the old software. Write it to the > > filesystem r/w filesystem (initially or after you have the aufs mounts > > mounted.) Then you can delete from the r/w filesystem and it's gone for > > good. > > Agreed. Just to augment that explanation: Sorry for the late reply, got sidetracked with high prio stuff. > > - The RO branch would be at the bottom (can be multiple of these) > - The RW branch goes over the top (*can* be multiple of these, but one > is more normal) > - You can effectively delete stuff from the RO branches because Aufs has > the concept of "white out" files. So for all intents and purposes the > top RW layer can create any end result you like, including that of > completely masking out some lower layer > - With some knowledge of how the whiteout files work you can also "undo" > changes to the RO files. Eg directly mounting the RW layer and erasing > all files (from the RW layer) leaves you back with just the stacked RO > layers again. I find this helpful for development where I can basically > work live on the last released build and then by inspection the RW layer > has all the changes needed to apply to the next RO layer release! When you update the RO layer you will be back to a single copy, the previous release which is only in the overlay is gone, right? It also implies 2 different SW update methods, one for updating the overlay copy and one for updating the RO layer. For development it works well though. > > I believe recent kernels also have a much simpler "Overlay Filesystem" > that has fewer features. Also the big alternative to aufs is Unionfs 2 > - most distros use aufs, but both seem viable? Dunno, I haven't tried any overlay FS yet. > > > Finally note that you don't need to use aufs for the entire filesystem. > A common setup might be to use a bunch of bind mounts where you know you > don't need overlay features, eg /usr might be a bunch of overlays, /home > might be a bind mount to writeable storage, /var might be a ram drive > which is initialised from some fixed template, etc? In my case I have > an overlay over most things, but /usr/lib/modules is a bind mount to a > RO filesystem (you can't write to it), /home is mounted to my writeable > storage (not layered), the main OS dirs are layered and /var is a mess... > > > You still want to pick a r/w branch with a filesystem that handles power > > cuts well. You can continue to use JFFS2. > > Thought: Is there any evidence that one modern filesystem is better than > another with regards to sudden power removal? You probably need to > speak to filesystem experts at this point and define the kind of thing > you are trying to protect against? Sounds like you have raw flash > storage here, so that constrains your choices somewhat? AFAIK, only FSes which are designed for flash(JFFS2, UBIFS, YAFFS etc. ) are safe w.r.t power cuts. > > Just note that with aufs you can use quite a few filesystems for the > different layers. So for example you could have a base RW layer which > is a DM snapshot, overlaid with a loopback mount to a DVD iso, overlay > that with a squashfs, and finally overlay an Ext4 RW mount... (And of > course each of the RO layers might be stored on varied filesystems > themselves - check distributions such as Slax which allow you to overlay > a squashfs that is itself inside some loopback mounted file...) hmm, DM snapshots might be something. Have to look closer on that. > http://aufs.sourceforge.net/aufs2/report/sq/ > > http://aufs.sourceforge.net/ > > I believe for most cheapo consumer flash storage where the underlying > flash filesystem isn't exposed, its quite susceptable to *complete* > failure with sudden poweroff? The issue is the invisible, underlying > flash filesystem gets corrupted during a partial write and that can be > the end of your flash drive - you don't even get to see it again to > recover from it... I don't believe partitioning protects you from this, > but of course separating read/write concerns to physically separate > devices would help? I presume this isn't what you are using though? We are not using NAND flash yet but our next product will. I do have the impression that any block emulating device such as SSD are unreliable w.r.t power cuts. I would love to be proven wrong though :) Jocke