From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 49E32138206 for ; Thu, 18 Jan 2018 23:00:51 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 3EBDCE096D; Thu, 18 Jan 2018 23:00:46 +0000 (UTC) Received: from mail-yw0-x243.google.com (mail-yw0-x243.google.com [IPv6:2607:f8b0:4002:c05::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id D5E06E08F9 for ; Thu, 18 Jan 2018 23:00:45 +0000 (UTC) Received: by mail-yw0-x243.google.com with SMTP id v139so12623927ywg.4 for ; Thu, 18 Jan 2018 15:00:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=84W+z9NBFyW+yDN6vj0bHjd7+HEbjXIYlbM+119JcBI=; b=DBQ47fayLPdTBAUH1BFcDPiW4c8XlIEVRm6BHXVpYflWiGa2P324PGv2dZAZnE/GfY VEEj44d5rqbdJFY4RvQSLxtJQXMAClqAvjc/2TZGp+oShKgejbEyDprJtdZjkA89WZ0P nGQUKtVdosLHfYZ/JZ25RH6ha3LfiEKf7wpjzuY4f2wnd+++u8FU2Xa18Sj6o8BmV9ME 9PNReZTlRCZzneSsx6WVa8WYu6FkNiULfx5H8wFmJRhZ0YPjHNsy4Mp6/DARWrc0jaAI fZ/xdk+YW+IC1I/ZhRF9QAYfaEPRFPnlYc/tcXHclY5NOXX2vwR6r2TnxnIjyz3gGmx6 IWJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=84W+z9NBFyW+yDN6vj0bHjd7+HEbjXIYlbM+119JcBI=; b=tB3XDDnd2fFzlEiGf+q4sQYzfzPHgqOXLTefBvmR9otXugnq74QD3BovmjZgySxduH AlVXwp9KcnzIlxgZb9Cpiwf2ZIG+Wbkv/yCHzG++QsD8YXwikueJpsvrDsvmBJ8Dw0EZ a2YAVXi/kg2XkUky/WNnjk867U6nT27Sp0Cm3FIUazOeIrSJkhqTVyMsu/LxKVOYAITS GMYFU3EfBG+Wb2SRBe5eZRXI9UWyOblQRyaFSaO8OGrv0Db+Pv8miGT6eIz9LsGoWRaT tkb9EIyju6kXpxvWdJzlHAhAfGimjq2B2U6CInh/rYhHNrLxiwl0d74uC0KWg3BqlfeK baQQ== X-Gm-Message-State: AKwxytcMuZjznVxdu4VgJf7MzOfT+PlkkxwYO2H3w/8cgPyyvqUDQerY Cx39Yf2jA1dG2Bfiw5X/NJ8fggwlPZ3X2k5YiqE= X-Google-Smtp-Source: ACJfBotcuFjENyrsQsNXWAoZJfG/9McuMTvzdku5xaEWFlRF540wtc6SWbPR5FWjxbj0p4+4ZjmkMZxqH6+IGQt9MvA= X-Received: by 10.37.144.142 with SMTP id t14mr26486105ybl.105.1516316444121; Thu, 18 Jan 2018 15:00:44 -0800 (PST) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org MIME-Version: 1.0 Received: by 10.129.84.85 with HTTP; Thu, 18 Jan 2018 15:00:43 -0800 (PST) In-Reply-To: <2686de8e-334c-084b-4828-6109b10dd536@gentoo.org> References: <2686de8e-334c-084b-4828-6109b10dd536@gentoo.org> From: R0b0t1 Date: Thu, 18 Jan 2018 17:00:43 -0600 Message-ID: Subject: Re: [gentoo-dev] Managing updates on many identical Gentoo systems To: gentoo-dev@lists.gentoo.org Content-Type: text/plain; charset="UTF-8" X-Archives-Salt: 9a59685c-3ded-4c72-89ad-5675a6c1fff3 X-Archives-Hash: 2663e9bdcbe2b7569f46887f46a49113 Hello, There's been a lot of relevant points but I'm not sure how to stitch together all of the posts so I will reply to the OP. On Thu, Jan 18, 2018 at 5:46 AM, Anthony G. Basile wrote: > Hi everyone, > > I'm trying to design an update system for many identical Gentoo systems. > Using a binhost is obvious, but there are still problems with this > approach. > How are the hosts administered now? Which problems are you referring to? It's my understanding that Debian/Fedora/etc. based datacenters have all of their machines crunching dependencies individually. There may be local mirrors, but it looks like people have not seen fit to centralize dependency information. > Unless there's some magic I don't know about (and this is why I'm > sending this email) each machine still needs to have the portage tree > installed locally (1.5 GB) or somehow mounted by a network filesystem > (which is not practical if the machines are not on a local network). > Furthermore, each machine would have to run emerge locally to do the > calculation of what packages need updating. > > This procedure is redundant because each machine is housing the same > data and doing the same dependence-tree calculation. It should be > possible to do this calculation on a centralized binhost and simply > communicate the update information to the remote machines. They would > then only have to download the .tbz2's and install them, keeping a tidy > /var/db/pkg. Thus they avoid having to house the portage tree and > burning cpu cycles that just calculate redundant information. > Is it necessary to retain information that would be used to administer the machines on the machines? You could treat your servers as embedded appliances. The typical method used in that case is to push an entire root to them at once. For safety's sake a separate system partition can receive the system image; system data is kept elsewhere. This would require some small customization by way of update code to unpack the new system and modify bootloader entries. All configuration of the system apart from that would have to be handled upstream when the release was prepared, or stored on the data partition. This is done for lots of devices (phones?) and works well even when bandwidth is fairly limited. Cheers, R0b0t1