From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) by finch.gentoo.org (Postfix) with ESMTP id 26BD313888F for ; Thu, 22 Oct 2015 11:59:13 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id B7A9421C026; Thu, 22 Oct 2015 11:59:10 +0000 (UTC) Received: from smtp.transmode.se (smtp.transmode.se [31.15.61.139]) by pigeon.gentoo.org (Postfix) with ESMTP id 19BE821C021 for ; Thu, 22 Oct 2015 11:59:09 +0000 (UTC) Received: from exch1.transmode.se (exch1.transmode.se [192.168.201.16]) by smtp.transmode.se (Postfix) with ESMTP id 43296118709D for ; Thu, 22 Oct 2015 13:59:07 +0200 (CEST) Received: from exch1.transmode.se (192.168.201.16) by exch1.transmode.se (192.168.201.16) with Microsoft SMTP Server (TLS) id 15.0.1076.9; Thu, 22 Oct 2015 13:59:07 +0200 Received: from exch1.transmode.se ([fe80::3029:ce14:7d42:cc5]) by exch1.transmode.se ([fe80::3029:ce14:7d42:cc5%17]) with mapi id 15.00.1076.000; Thu, 22 Oct 2015 13:59:07 +0200 From: Joakim Tjernlund To: "gentoo-portage-dev@lists.gentoo.org" Subject: Re: [gentoo-portage-dev] Re: How to have several gentoo repos on one machine? Thread-Topic: [gentoo-portage-dev] Re: How to have several gentoo repos on one machine? Thread-Index: AQHRDLykpVCS5SfKgkGQi+7cPKMrvZ53RxoA Date: Thu, 22 Oct 2015 11:59:06 +0000 Message-ID: <1445515146.31293.88.camel@transmode.se> References: <1445425682.5222.12.camel@transmode.se> <1445496485.31293.42.camel@transmode.se> In-Reply-To: Accept-Language: en-US, sv-SE Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: Evolution 3.16.5 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [192.168.200.4] Content-Type: text/plain; charset="iso-8859-15" Content-ID: Content-Transfer-Encoding: quoted-printable Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-portage-dev@lists.gentoo.org Reply-to: gentoo-portage-dev@lists.gentoo.org MIME-Version: 1.0 X-Archives-Salt: 51beb9e1-4188-457b-8dfd-7b2db21bd587 X-Archives-Hash: ed274d702fb7fe441179c656a6218b14 On Thu, 2015-10-22 at 11:26 +0000, Duncan wrote: > Joakim Tjernlund posted on Thu, 22 Oct 2015 06:48:06 +0000 as excerpted: >=20 > > On Thu, 2015-10-22 at 02:29 +0000, Duncan wrote: > > > Joakim Tjernlund posted on Wed, 21 Oct 2015 11:08:02 +0000 as > > > excerpted: > > >=20 > > > > I need to more than one gentoo repo in my computer. >=20 >=20 > > > > this did not work as "portageq repositories_configuration /" > > > > complains: > > > > !!! Section 'tm-cusfpv3' in repos.conf has name different from > > > > repository name 'gentoo' set inside repository > > > >=20 > > > > I figured the name in repos.conf would just override > > > > /usr/local/portage/tm-cusfpv3/profiles/repo_name ? > > >=20 > > > While it's not quite clear to me either why you'd need two identical > > > gentoo repos[...] > >=20 > > I use one for my host and the other for cross building our products roo= t > > FS and they are not in sync. That rules out the aliases I guess? >=20 > I think so, yes. However, as a user I'd really like to understand=20 > aliases, their purpose, and at high level how they work, and the current= =20 > manpage doesn't help so much there. Without that I really don't know=20 > enough about aliases to say anything further. >=20 > But meanwhile, I was sort of in your situation for awhile as I was=20 > building for my main amd64 system and in a 32-bit chroot for a 32-bit- > only netbook, with a separate portage config for each, and while in my=20 > case they both pointed at the same gentoo repo and overlays using bind- > mounts into the 32-bit chroot, without those bind-mounts it would have=20 > been two parallel and separate portage installations, one configured for= =20 > 32-bit x86 in the chroot, one configured for amd64 outside the chroot. >=20 > And that's what I'd use in your case, two separate portage installations,= =20 > which could then of course have separate configs. I don't need separate portage installations but I do need separate configs = and separate gentoo ebuild repos as the embedded target has a less frequent upd= ate cycle and I want to be able to rebuild the exact same pkg at any time. >=20 > That said, while I understand the principle of stability, and if it's=20 > private there shouldn't be legal issues, I still wonder at the idea. One= =20 > of the reasons I could and did use bind-mounts and thus literally the=20 > same repos in my case, was that the gentoo repo is the gentoo repo, and=20 > other than the possibility of snapshotting it for archiving purposes (and= =20 > of using one of those snapshots should it be needed, say because I left=20 > the netbook unupgraded for too long and it could no longer jump from the= =20 > version on it to current), I considered the gentoo repo the gentoo repo,= =20 > and a local copy that wasn't synced would no longer represent the present= =20 > state of the gentoo repo. >=20 > If I were to un-sync for other than very temporary recovery purposes, I'd= =20 > thus want to call the repo something other than gentoo, since it would no= =20 > longer represent the current state of the true gentoo repo. >=20 > And if I made changes to that unsynced repo, say to stabilize it further= =20 > (and if I wasn't doing so, what would be the purpose of keeping it=20 > unsynced for so long), that'd be even /more/ reason to call it something= =20 > other than gentoo, because then it would no longer properly represent=20 > that state of the true gentoo repo at /any/ time. >=20 > But having the git repo available changes the way that works=20 > dramatically, see below... >=20 > > I don't plan on renaming anything in the repo_name file, it should just > > be ignored and the name I have select in repos.conf should used. > >=20 > > I don't see any value in repo_name file now that we have the new > > repos.conf, possibly it could be a fallback only for PORTDIR users. >=20 > The portage devs are welcome to contradict me if they like, but AFAIK, it= =20 > still serves the useful purpose of double-checking that you don't for=20 > instance have two repos accidently syncing to the same place, and that=20 > the names used to refer to the repo stay consistent. (Again, part of the= =20 > need for consistency would be due to the metadata and thus metadata cache= =20 > being repo-specific, automatically invalidating the cache if the remote=20 > name and local name don't agree. Locally regenerating the metadata cache= =20 > will go a long way to avoiding that problem, but it's an expensive=20 > operation that most users won't want to do, and keeping the names in sync= =20 > helps avoid inadvertent cache invalidation.) In any case, the use of repo_name must be the same for all tools, now emerg= e and portageq are different. >=20 > > > I actually use gentoo's git-based usersync > > > repo on github, now, and thus don't rsync any repos all any more, her= e, > > > and git of course has its git-ignore feature/files, which I use now.= =20 > > > But I used rsync's exclude as suggested above, for years. Worked fin= e. > > > =3D:^) > >=20 > > Nice, I am heading the same was, using git all the way but I not there > > yet. > > One problem is that using git is disk space I think. Files are just > > ignored but still present in the repo so syncing to our embedded target > > will take a lot more space. > > Any thoughts on that? >=20 > Well, at least once your trailing target (presumably the embedded repo)=20 > is safely past the git repo's epoc (the date imported from cvs, for our=20 > purposes), git flexibility will let you checkout older versions on- > demand, then checkout HEAD once again. >=20 > In a scenario where both copies aren't likely to be used at once, you can= =20 > use a single local git repo and just checkout the version of it you want= =20 > dynamically. Hmm, that is an good ide but for now I do want to keep them separate as the host repo is managed by IT and the other is R&D and having to coordinat= e this common repo with IT is not something I want ATM. My space concerns are a bit different, currently I exclude lots of categories from the embedded target, (No X11, games etc.) and I do this using rsync exclude on the server side. >=20 > In a concurrent-use scenario, there's a few ways you could go. What I'd= =20 > probably do would be two git repos, one synced to gentoo-remote,=20 > presumably with full git history (or at least git history back to the=20 > other checkout), the other locally checked out from the "current" repo,=20 > at the checkout of interest. >=20 > If you're doing this sort of thing then the sort of space the git repo=20 > takes up shouldn't be a big concern, but in case it is, it's worth noting= =20 > that given the right filesystem and dedup tools, there will only actually= =20 > be the one copy of "common" data on-storage, with each of those two git=20 > repos reflinking (think a lower-level hard-link) data that's common=20 > between them, which will be pretty much everything in the earlier one=20 > since the current one will have the earlier one as history. >=20 > I'm a regular on the btrfs list, for instance, and on btrfs, a very space= =20 > efficient solution would be to originally do an initial git checkout of=20 > the older, presumably embedded target repo, create a btrfs snapshot out=20 > of it, and then (in the working copy, not the snapshot) git-pull from the= =20 > remote to update to current. The btrfs snapshot will have locked in=20 > place the older version in the snapshot, while the git pull in the=20 > working copy will create any new files, delete any remote-deleted ones=20 > (but they'll still be in the btrfs snapshot), reflink any old files, and= =20 > reflink but then cow (copy-on-write) any updated files. For this=20 > scenario you wouldn't even need any additional dedup tools, tho if you=20 > had them, they'd probably save even more space (multiple versions of the= =20 > same package often have very nearly the same ebuilds, for instance,=20 > differing in little more than name, and dedup would catch and dedup these= =20 > as well, while the pure native btrfs snapshot method probably wouldn't). We use BTRFS too :) this is the perfect FS for safe upgrades and rollback. However, we create a btrfs snapshot and do the upgrades into the snapshot. Once we a happy we set default subvolume to the snapshot and reboot. We feel this is much safer than upgrading into the running system, much can= go wrong halfway through an upgrade and our systems sits in the bush somewhere= . Even if the upgrade goes well, you may end up with a system you cannot logi= n to if pam was upgraded until you reboot. Jocke=