[gentoo-dev] Managing updates on many identical Gentoo systems

[gentoo-dev] Managing updates on many identical Gentoo systems

[Anthony G. Basile]

Hi everyone,

I'm trying to design an update system for many identical Gentoo systems.
 Using a binhost is obvious, but there are still problems with this
approach.

Unless there's some magic I don't know about (and this is why I'm
sending this email) each machine still needs to have the portage tree
installed locally (1.5 GB) or somehow mounted by a network filesystem
(which is not practical if the machines are not on a local network).
Furthermore, each machine would have to run emerge locally to do the
calculation of what packages need updating.

This procedure is redundant because each machine is housing the same
data and doing the same dependence-tree calculation.  It should be
possible to do this calculation on a centralized binhost and simply
communicate the update information to the remote machines.  They would
then only have to download the .tbz2's and install them, keeping a tidy
/var/db/pkg.  Thus they avoid having to house the portage tree and
burning cpu cycles that just calculate redundant information.

I'm inspired here by OpenBSD's pkg_add which doesn't require all of
ports to be installed, and mender which is a

Any ideas?

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] Managing updates on many identical Gentoo systems

[Joakim Tjernlund]

On Thu, 1970-01-01 at 00:00 +0000, Anthony G. Basile wrote:
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> 
> 
> Hi everyone,
> 
> I'm trying to design an update system for many identical Gentoo systems.
>  Using a binhost is obvious, but there are still problems with this
> approach.
> 
> Unless there's some magic I don't know about (and this is why I'm
> sending this email) each machine still needs to have the portage tree
> installed locally (1.5 GB) or somehow mounted by a network filesystem
> (which is not practical if the machines are not on a local network).
> Furthermore, each machine would have to run emerge locally to do the
> calculation of what packages need updating.
> 
> This procedure is redundant because each machine is housing the same
> data and doing the same dependence-tree calculation.  It should be
> possible to do this calculation on a centralized binhost and simply
> communicate the update information to the remote machines.  They would
> then only have to download the .tbz2's and install them, keeping a tidy
> /var/db/pkg.  Thus they avoid having to house the portage tree and
> burning cpu cycles that just calculate redundant information.
> 
> I'm inspired here by OpenBSD's pkg_add which doesn't require all of
> ports to be installed, and mender which is a
> 
> Any ideas?

No ideas but I am interested in solutions. I am thinking/looking at updating
embedded devices.

Also, I guess you have the common problem with changes in /etc/ files which 
needs to be kept in sync.

 Jocke

Re: [gentoo-dev] Managing updates on many identical Gentoo systems

[Lars Wendler]

[-- Attachment #1: Type: text/plain, Size: 2114 bytes --]

Hi Anthony,

On Thu, 18 Jan 2018 06:46:53 -0500 Anthony G. Basile wrote:

>Hi everyone,
>
>I'm trying to design an update system for many identical Gentoo
>systems.
> Using a binhost is obvious, but there are still problems with this
>approach.
>
>Unless there's some magic I don't know about (and this is why I'm
>sending this email) each machine still needs to have the portage tree
>installed locally (1.5 GB) or somehow mounted by a network filesystem
>(which is not practical if the machines are not on a local network).
>Furthermore, each machine would have to run emerge locally to do the
>calculation of what packages need updating.
>
>This procedure is redundant because each machine is housing the same
>data and doing the same dependence-tree calculation.  It should be
>possible to do this calculation on a centralized binhost and simply
>communicate the update information to the remote machines.  They would
>then only have to download the .tbz2's and install them, keeping a tidy
>/var/db/pkg.  Thus they avoid having to house the portage tree and
>burning cpu cycles that just calculate redundant information.
>
>I'm inspired here by OpenBSD's pkg_add which doesn't require all of
>ports to be installed, and mender which is a
>
>Any ideas?
>

well, I never did anything like that but regarding the dependency
calculation... how about something like

emerge -1OKanv $(qlist -CISq)

(--oneshot --nodeps --usepkgonly --ask --noreplace --verbose)

which simply omits dependency calculations, only takes into account
available binary packages and doesn't replace same versions?
Of course this requires all installed packages really being available as
binpkgs.
Since all the installations are the same, as long as you provide a sane
set of binpkgs, dependency calculation should not matter anyway.
The only issue I can think of is that a system might become broken if
the update gets interrupted before all packages have been updated.

Kind regards

-- 
Lars Wendler
Gentoo package maintainer
GPG: 21CC CF02 4586 0A07 ED93  9F68 498F E765 960E 9B39

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] Managing updates on many identical Gentoo systems

[NP-Hardass]

[-- Attachment #1.1: Type: text/plain, Size: 1205 bytes --]

On 01/18/2018 07:34 AM, Lars Wendler wrote:
> Hi Anthony,
> 
[...]
> 
> well, I never did anything like that but regarding the dependency
> calculation... how about something like
> 
> emerge -1OKanv $(qlist -CISq)
> 
> (--oneshot --nodeps --usepkgonly --ask --noreplace --verbose)
> 
> which simply omits dependency calculations, only takes into account
> available binary packages and doesn't replace same versions?
> Of course this requires all installed packages really being available as
> binpkgs.
> Since all the installations are the same, as long as you provide a sane
> set of binpkgs, dependency calculation should not matter anyway.
> The only issue I can think of is that a system might become broken if
> the update gets interrupted before all packages have been updated.
> 
> Kind regards
> 

TTBOMK, this will fail also if there is a slot change.  For instance,
with the recent autoconf-2.69 move from :0 to :2.69, dependency
resolution would have resulted softblock, thus removing :0 and then
merging :2.69 automatically. with a --nodeps, :2.69 would attempt to
merge without removing :0 for this package, resulting in an emerge failure.

-- 
NP-Hardass


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] Managing updates on many identical Gentoo systems

[Martin Gysel]

Am Donnerstag, 18. Januar 2018, 12:46:53 CET schrieb Anthony G. Basile:
> Hi everyone,
> 
> I'm trying to design an update system for many identical Gentoo systems.
>  Using a binhost is obvious, but there are still problems with this
> approach.
> 
> Unless there's some magic I don't know about (and this is why I'm
> sending this email) each machine still needs to have the portage tree
> installed locally (1.5 GB) or somehow mounted by a network filesystem
> (which is not practical if the machines are not on a local network).
> Furthermore, each machine would have to run emerge locally to do the
> calculation of what packages need updating.
> 
> This procedure is redundant because each machine is housing the same
> data and doing the same dependence-tree calculation.  It should be
> possible to do this calculation on a centralized binhost and simply
> communicate the update information to the remote machines.  They would
> then only have to download the .tbz2's and install them, keeping a tidy
> /var/db/pkg.  Thus they avoid having to house the portage tree and
> burning cpu cycles that just calculate redundant information.
> 
> I'm inspired here by OpenBSD's pkg_add which doesn't require all of
> ports to be installed, and mender which is a
> 
> Any ideas?

what about app-portage/portage-utils? AFAIK doesn't need a local portage 
tree...

[gentoo-dev] Re: Managing updates on many identical Gentoo systems

[Duncan]

Anthony G. Basile posted on Thu, 18 Jan 2018 06:46:53 -0500 as excerpted:

> I'm trying to design an update system for many identical Gentoo systems.
>  Using a binhost is obvious, but there are still problems with this
> approach.
> 
> Unless there's some magic I don't know about (and this is why I'm
> sending this email) each machine still needs to have the portage tree
> installed locally (1.5 GB) or somehow mounted by a network filesystem
> (which is not practical if the machines are not on a local network).
> Furthermore, each machine would have to run emerge locally to do the
> calculation of what packages need updating.
> 
> This procedure is redundant because each machine is housing the same
> data and doing the same dependence-tree calculation.

I had a gen-1.5 32-bit netbook for a number of years, that I updated 
using rsync from a 32-bit chroot on my main machine, no portage tree on 
the target netbook, tho I didn't worry about separating out build deps 
from run deps.

That was a single machine config, but it should be even easier if you're 
running nearly identical machines and thus don't need the separate chroot 
build image.

If you have temporary networking you can rsync directly machine to 
machine, as I did after I was fully setup, but at first I was sneaker-
netting it, rsyncing to a thumb drive from the build machine, that I 
would then plug into the target and rsync thumb drive to target.

The thumb drive was bootable, and I used it to do the first gentoo boots 
on the target as well, testing my config and updating as necessary as I 
went.  When I got everything I initially wanted booting from the thumb 
drive, I booted the thumb drive, wiped the initial Pingus Linux on the 
netbook and setup the partitioning, etc, then rsynced selected bits into 
the appropriate place on the target.

For multiple nearly identical machines you can exclude the non-identical 
bits from the primary rsync image, keeping the specific bits in 
individual images synced on top of the primary.  Of course you can sync 
in reverse as well to keep the non-identical bits updated, giving you a 
nice backup of each one as well. =:^)

Alternatives would include simply creating the thumb drive once and then 
cloning it enough times to give every machine a bootable thumb drive copy 
(using symlinking and/or mounts to handle the non-identical stuff, so 
simply toggling a symlink lets you switch machine layouts), or if the 
machines have enough memory, setting up a single thumb drive to boot and 
put everything in a tmpfs for the machine to run from, so you can use the 
same thumb drive to boot them all, effectively the sneakernet version of 
net-boot.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: Managing updates on many identical Gentoo systems

[Bill Kenworthy]

On 18/01/18 23:36, Duncan wrote:
> Anthony G. Basile posted on Thu, 18 Jan 2018 06:46:53 -0500 as excerpted:
> 
>> I'm trying to design an update system for many identical Gentoo systems.
>>  Using a binhost is obvious, but there are still problems with this
>> approach.
>>

I'd suggest go for a semi diskless OS - boot them from one central image
with an individual overlay filesystem with local customisations.  NFS
mount the common directories.

you just have a one central host to build for and don't need to worry
about portage everywhere.

Worked ok with a small number of mythtv frontends.

BillK

Re: [gentoo-dev] Re: Managing updates on many identical Gentoo systems

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 848 bytes --]

On Thu, Jan 18, 2018 at 5:13 PM, Bill Kenworthy <billk@iinet.net.au> wrote:

> On 18/01/18 23:36, Duncan wrote:
> > Anthony G. Basile posted on Thu, 18 Jan 2018 06:46:53 -0500 as excerpted:
> >
> >> I'm trying to design an update system for many identical Gentoo systems.
> >>  Using a binhost is obvious, but there are still problems with this
> >> approach.
> >>
>
> I'd suggest go for a semi diskless OS - boot them from one central image
> with an individual overlay filesystem with local customisations.  NFS
> mount the common directories.
>
> you just have a one central host to build for and don't need to worry
> about portage everywhere.
>
> Worked ok with a small number of mythtv frontends.
>

It doesn't work if you have a WAN; NFS needs low latencies between the NFS
server and the client or you will have a bad time.


>
> BillK
>
>

[-- Attachment #2: Type: text/html, Size: 1435 bytes --]

Re: [gentoo-dev] Re: Managing updates on many identical Gentoo systems

[Anthony G. Basile]

On 1/19/18 9:45 AM, Alec Warner wrote:
> On Thu, Jan 18, 2018 at 5:13 PM, Bill Kenworthy <billk@iinet.net.au> wrote:
> 
>> On 18/01/18 23:36, Duncan wrote:
>>> Anthony G. Basile posted on Thu, 18 Jan 2018 06:46:53 -0500 as excerpted:
>>>
>>>> I'm trying to design an update system for many identical Gentoo systems.
>>>>  Using a binhost is obvious, but there are still problems with this
>>>> approach.
>>>>
>>
>> I'd suggest go for a semi diskless OS - boot them from one central image
>> with an individual overlay filesystem with local customisations.  NFS
>> mount the common directories.
>>
>> you just have a one central host to build for and don't need to worry
>> about portage everywhere.
>>
>> Worked ok with a small number of mythtv frontends.
>>
> 
> It doesn't work if you have a WAN; NFS needs low latencies between the NFS
> server and the client or you will have a bad time.
> 
> 

Zac pretty much nailed the requirements in bug #644990.  You should not
need the portage tree at all, neither locally nor via any network
filesystem.  He mentions there that it is currently possible via "a
dummy profile", but I'm not sure what he means by that yet or how to set
one up.  I'll read his bug #640318 and try to figure it out.

Thanks guys, I'm glad people at least recognized the usefulness of such
a possibility.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] Re: Managing updates on many identical Gentoo systems

[Zac Medico]

[-- Attachment #1.1: Type: text/plain, Size: 2437 bytes --]

On 01/19/2018 07:03 AM, Anthony G. Basile wrote:
> On 1/19/18 9:45 AM, Alec Warner wrote:
>> On Thu, Jan 18, 2018 at 5:13 PM, Bill Kenworthy <billk@iinet.net.au> wrote:
>>
>>> On 18/01/18 23:36, Duncan wrote:
>>>> Anthony G. Basile posted on Thu, 18 Jan 2018 06:46:53 -0500 as excerpted:
>>>>
>>>>> I'm trying to design an update system for many identical Gentoo systems.
>>>>>  Using a binhost is obvious, but there are still problems with this
>>>>> approach.
>>>>>
>>>
>>> I'd suggest go for a semi diskless OS - boot them from one central image
>>> with an individual overlay filesystem with local customisations.  NFS
>>> mount the common directories.
>>>
>>> you just have a one central host to build for and don't need to worry
>>> about portage everywhere.
>>>
>>> Worked ok with a small number of mythtv frontends.
>>>
>>
>> It doesn't work if you have a WAN; NFS needs low latencies between the NFS
>> server and the client or you will have a bad time.
>>
>>
> 
> Zac pretty much nailed the requirements in bug #644990.  You should not
> need the portage tree at all, neither locally nor via any network
> filesystem.  He mentions there that it is currently possible via "a
> dummy profile", but I'm not sure what he means by that yet or how to set
> one up.

The dummy profile consists of making a /etc/portage/make.profile/
directory that is nearly empty, containing only a make.defaults file
with minimal settings like these:

ARCH="amd64"
ACCEPT_KEYWORDS="**"
ACCEPT_LICENSE="*"

You also need a big IUSE_IMPLICIT setting in there if you don't have
this patch that's in portage-2.3.19:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=2382082dcfce5e6c2be7c4dd6aed7c32e1d20616

With the dummy profile, you can install all of the packages available in
the binhost. Currently, there's not a really good way to tell emerge to
install all packages from the binhost, but you can parse them all from
${PORTAGE_BINHOST}/Packages and write the list to /var/lib/portage/world
or something similar. Then `emerge -vuGD @world`.

I've described my currently planned sequence of steps here:

    https://bugs.gentoo.org/644990#c3

If you try the dummy profile approach, then if there are any in-place
package moves in the binhost then you may have problems with file
collisions, but I've described how I plan to handle that in the above
bug comment.
-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 224 bytes --]

Re: [gentoo-dev] Re: Managing updates on many identical Gentoo systems

[Anthony G. Basile]

On 1/19/18 10:03 AM, Anthony G. Basile wrote:
> On 1/19/18 9:45 AM, Alec Warner wrote:
>> On Thu, Jan 18, 2018 at 5:13 PM, Bill Kenworthy <billk@iinet.net.au> wrote:
>>
>>> On 18/01/18 23:36, Duncan wrote:
>>>> Anthony G. Basile posted on Thu, 18 Jan 2018 06:46:53 -0500 as excerpted:
>>>>
>>>>> I'm trying to design an update system for many identical Gentoo systems.
>>>>>  Using a binhost is obvious, but there are still problems with this
>>>>> approach.
>>>>>
>>>
>>> I'd suggest go for a semi diskless OS - boot them from one central image
>>> with an individual overlay filesystem with local customisations.  NFS
>>> mount the common directories.
>>>
>>> you just have a one central host to build for and don't need to worry
>>> about portage everywhere.
>>>
>>> Worked ok with a small number of mythtv frontends.
>>>
>>
>> It doesn't work if you have a WAN; NFS needs low latencies between the NFS
>> server and the client or you will have a bad time.
>>
>>
> 
> Zac pretty much nailed the requirements in bug #644990.  You should not
> need the portage tree at all, neither locally nor via any network
> filesystem.  He mentions there that it is currently possible via "a
> dummy profile", but I'm not sure what he means by that yet or how to set
> one up.  I'll read his bug #640318 and try to figure it out.
> 
> Thanks guys, I'm glad people at least recognized the usefulness of such
> a possibility.
> 

Okay, I have a workable solution to my question.  I was able to get
binhost working with a portage tree containing ONLY /profiles and
/eclass.  That's 12MB and 2.8MB in size, respectively, and I can
probably dump a bunch of the unused profile directories slimming that
down.  With just those two directories in PORTDIR, emerge -K pulls down
the update packages from BINHOST and installs them.

@zac any comments about this approach?  Is it likely to break?

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-dev] Re: Managing updates on many identical Gentoo systems

[Zac Medico]

[-- Attachment #1.1: Type: text/plain, Size: 1755 bytes --]

On 01/20/2018 07:34 AM, Anthony G. Basile wrote:
> On 1/19/18 10:03 AM, Anthony G. Basile wrote:
>>
>> Zac pretty much nailed the requirements in bug #644990.  You should not
>> need the portage tree at all, neither locally nor via any network
>> filesystem.  He mentions there that it is currently possible via "a
>> dummy profile", but I'm not sure what he means by that yet or how to set
>> one up.  I'll read his bug #640318 and try to figure it out.
>>
>> Thanks guys, I'm glad people at least recognized the usefulness of such
>> a possibility.
>>
> 
> Okay, I have a workable solution to my question.  I was able to get
> binhost working with a portage tree containing ONLY /profiles and
> /eclass.  That's 12MB and 2.8MB in size, respectively, and I can
> probably dump a bunch of the unused profile directories slimming that
> down.  With just those two directories in PORTDIR, emerge -K pulls down
> the update packages from BINHOST and installs them.
> 
> @zac any comments about this approach?  Is it likely to break?

It's desirable to rely exclusively on the BINHOST as a single source of
truth, since otherwise you have to keep multiple data sources in
consistent states.

You should not need the eclasses, since portage uses the eclass code
from environment.bz2 that is embedded in each binary package.

Using /profiles can cause problems because things like package.mask and
package.keywords have to be consistent with the BINHOST.

For the above reasons, I use a dummy profile. I also sync
/profiles/updates so that emerge can apply package moves, but I intend
to eliminate that as part of bug #644990 since keeping /profiles/updates
consistent with the binhost is not practical.
-- 
Thanks,
Zac


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 224 bytes --]

Re: [gentoo-dev] Managing updates on many identical Gentoo systems

[Alec Warner]

[-- Attachment #1: Type: text/plain, Size: 3107 bytes --]

On Thu, Jan 18, 2018 at 6:46 AM, Anthony G. Basile <blueness@gentoo.org>
wrote:

> Hi everyone,
>
> I'm trying to design an update system for many identical Gentoo systems.
>  Using a binhost is obvious, but there are still problems with this
> approach.
>
> Unless there's some magic I don't know about (and this is why I'm
> sending this email) each machine still needs to have the portage tree
> installed locally (1.5 GB) or somehow mounted by a network filesystem
> (which is not practical if the machines are not on a local network).
>

+Zac

I don't believe this is true; with the correct binhost configuration
portage should:

1) Contact the binhost to get a dump of packages available on it.
2) Use that dump to create a 'virtual portage tree' (bindb).
3) Use the bindb for package discovery (is foo available) and dependency
calculation.


> Furthermore, each machine would have to run emerge locally to do the
> calculation of what packages need updating.
>

The good news is that with a bindb; the package tree itself is much
smaller. ::gentoo is 23000 packages.
But you bindb is only as large as you build binpkgs for and publish them.
So say stage3 + some other stuff.
Around 500 packages perhaps.

A couple of exciting problems might include:

1) How to expose the bindb to machines?
2) How to secure bindb updates (HTTPS?)


> This procedure is redundant because each machine is housing the same
> data and doing the same dependence-tree calculation.  It should be
> possible to do this calculation on a centralized binhost and simply
> communicate the update information to the remote machines.  They would
> then only have to download the .tbz2's and install them, keeping a tidy
> /var/db/pkg.  Thus they avoid having to house the portage tree and
> burning cpu cycles that just calculate redundant information.
>

Emerge used to have support for emerge package.tar.gz; I wonder if it still
works?
I haven't seen anyone use it in forever?

antarus@woodpecker ~ $ emerge --help
emerge: command-line interface to the Portage system
Usage:
   emerge [ options ] [ action ] [ ebuild | tbz2 | file | @set | atom ] [ ... ]
   emerge [ options ] [ action ] < @system | @world >

Note that 'tbz2' is still listed ;)


>
> I'm inspired here by OpenBSD's pkg_add which doesn't require all of
> ports to be installed, and mender which is a
>

An alternative is that your central build hosts builds images instead of
packages.
You then boot your devices into a new image to get updates. This is a
pretty common pattern for embedded devices.
You need a secure method to deliver new images and probably a sane method
for rollback of a bad image.
Typically this is done with a bootloader and 3 image slots:

SLOT A: Current image.
SLOT B: Proposed update image.
SLOT C: Recovery Image # usually RO.

Bootloader boots A normally; but during update boots B. If A and B fail to
boot, Boots C.


>
> Any ideas?
>
> --
> Anthony G. Basile, Ph.D.
> Gentoo Linux Developer [Hardened]
> E-Mail    : blueness@gentoo.org
> GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
> GnuPG ID  : F52D4BBA
>
>

[-- Attachment #2: Type: text/html, Size: 4735 bytes --]

Re: [gentoo-dev] Managing updates on many identical Gentoo systems

[Zac Medico]

On 01/18/2018 08:13 AM, Alec Warner wrote:
> On Thu, Jan 18, 2018 at 6:46 AM, Anthony G. Basile <blueness@gentoo.org
> <mailto:blueness@gentoo.org>> wrote:
> 
>     Hi everyone,
> 
>     I'm trying to design an update system for many identical Gentoo systems.
>      Using a binhost is obvious, but there are still problems with this
>     approach.
> 
>     Unless there's some magic I don't know about (and this is why I'm
>     sending this email) each machine still needs to have the portage tree
>     installed locally (1.5 GB) or somehow mounted by a network filesystem
>     (which is not practical if the machines are not on a local network).
> 
> 
> +Zac
> 
> I don't believe this is true; with the correct binhost configuration
> portage should:
> 
> 1) Contact the binhost to get a dump of packages available on it.
> 2) Use that dump to create a 'virtual portage tree' (bindb).
> 3) Use the bindb for package discovery (is foo available) and dependency
> calculation.

It's already possible if you use PORTAGE_BINHOST and create a dummy
profile to satisfy portage. I've filed this bug to create a convenient
option for this:

https://bugs.gentoo.org/644990

> 
>     Furthermore, each machine would have to run emerge locally to do the
>     calculation of what packages need updating.
> 
> 
> The good news is that with a bindb; the package tree itself is much
> smaller. ::gentoo is 23000 packages.
> But you bindb is only as large as you build binpkgs for and publish
> them. So say stage3 + some other stuff.
> Around 500 packages perhaps.

Doing the calculation on the client side is fine, since binary package
calculations are much simpler than source-based ebuild calculations.
-- 
Thanks,
Zac

Re: [gentoo-dev] Managing updates on many identical Gentoo systems

[Alexander Tsoy]

В Чт, 18/01/2018 в 06:46 -0500, Anthony G. Basile пишет:
> Hi everyone,
> 
> I'm trying to design an update system for many identical Gentoo
> systems.
>  Using a binhost is obvious, but there are still problems with this
> approach.
> 
> Unless there's some magic I don't know about (and this is why I'm
> sending this email) each machine still needs to have the portage tree
> installed locally (1.5 GB) or somehow mounted by a network filesystem
> (which is not practical if the machines are not on a local network).
> Furthermore, each machine would have to run emerge locally to do the
> calculation of what packages need updating.

AFAIK each machine only needs "profiles" and "eclass" directories and
not the full repo. eclass dir is needed because of the check below in
ebuild.sh:


inherit()
...
    [[ -z ${location} ]] && die "${1}.eclass could not be found by
inherit()"

And this check actually should be skipped for binary packages. They are
already contain full environment.

Re: [gentoo-dev] Managing updates on many identical Gentoo systems

[Alexander Tsoy]

В Пт, 19/01/2018 в 01:30 +0300, Alexander Tsoy пишет:
> В Чт, 18/01/2018 в 06:46 -0500, Anthony G. Basile пишет:
> > Hi everyone,
> > 
> > I'm trying to design an update system for many identical Gentoo
> > systems.
> >  Using a binhost is obvious, but there are still problems with this
> > approach.
> > 
> > Unless there's some magic I don't know about (and this is why I'm
> > sending this email) each machine still needs to have the portage
> > tree
> > installed locally (1.5 GB) or somehow mounted by a network
> > filesystem
> > (which is not practical if the machines are not on a local
> > network).
> > Furthermore, each machine would have to run emerge locally to do
> > the
> > calculation of what packages need updating.
> 
> AFAIK each machine only needs "profiles" and "eclass" directories and
> not the full repo. eclass dir is needed because of the check below in
> ebuild.sh:
> 
> 
> inherit()
> ...
>     [[ -z ${location} ]] && die "${1}.eclass could not be found by
> inherit()"
> 
> And this check actually should be skipped for binary packages. They
> are
> already contain full environment.

Oops. Forget the last part about inherit(). You need only profiles dir.
I experimented on a host with local overlay thus I got inherit
errors. :)

Re: [gentoo-dev] Managing updates on many identical Gentoo systems

[R0b0t1]

Hello,

There's been a lot of relevant points but I'm not sure how to stitch
together all of the posts so I will reply to the OP.

On Thu, Jan 18, 2018 at 5:46 AM, Anthony G. Basile <blueness@gentoo.org> wrote:
> Hi everyone,
>
> I'm trying to design an update system for many identical Gentoo systems.
>  Using a binhost is obvious, but there are still problems with this
> approach.
>

How are the hosts administered now? Which problems are you referring to?

It's my understanding that Debian/Fedora/etc. based datacenters have
all of their machines crunching dependencies individually. There may
be local mirrors, but it looks like people have not seen fit to
centralize dependency information.

> Unless there's some magic I don't know about (and this is why I'm
> sending this email) each machine still needs to have the portage tree
> installed locally (1.5 GB) or somehow mounted by a network filesystem
> (which is not practical if the machines are not on a local network).
> Furthermore, each machine would have to run emerge locally to do the
> calculation of what packages need updating.
>
> This procedure is redundant because each machine is housing the same
> data and doing the same dependence-tree calculation.  It should be
> possible to do this calculation on a centralized binhost and simply
> communicate the update information to the remote machines.  They would
> then only have to download the .tbz2's and install them, keeping a tidy
> /var/db/pkg.  Thus they avoid having to house the portage tree and
> burning cpu cycles that just calculate redundant information.
>

Is it necessary to retain information that would be used to administer
the machines on the machines? You could treat your servers as embedded
appliances. The typical method used in that case is to push an entire
root to them at once. For safety's sake a separate system partition
can receive the system image; system data is kept elsewhere.

This would require some small customization by way of update code to
unpack the new system and modify bootloader entries. All configuration
of the system apart from that would have to be handled upstream when
the release was prepared, or stored on the data partition.

This is done for lots of devices (phones?) and works well even when
bandwidth is fairly limited.

Cheers,
     R0b0t1