From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 7738 invoked by uid 1002); 18 Aug 2003 00:34:37 -0000 Mailing-List: contact gentoo-dev-help@gentoo.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@gentoo.org Received: (qmail 15621 invoked from network); 18 Aug 2003 00:34:35 -0000 Message-ID: <3F401F15.9060204@sentuny.com.au> Date: Mon, 18 Aug 2003 10:34:29 +1000 From: Ron O'Hara User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: chris.rs@xtra.co.nz Cc: gentoo-dev@gentoo.org References: <3F3FFCFF.2080804@sentuny.com.au> <200308181026.44148.chris.rs@xtra.co.nz> In-Reply-To: <200308181026.44148.chris.rs@xtra.co.nz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [gentoo-dev] Large scale deployments - and portage X-Archives-Salt: f7b840c4-aa7c-4e8d-bd10-1dd0aa824db2 X-Archives-Hash: 622843325830f7852c0a9848b02780b2 Hi Chris Chris Smith wrote: [chop] >I think the real trick to it, is having a central server for you. There is a >great post on it in the forums somewhere. Having one box for 500 gentoo boxes >downloading all the distfiles, and doing all the rsyncs is a big bandwidth >saver! > > > >>Now the only question is - did I miss something and this already exists? >> >> > >Not really. Here is that forum post if you are interested: >http://forums.gentoo.org/viewtopic.php?t=59134 > Thanks for the link. Ok I've read it and along with you, a few others have explained how they are using their own mirror of files (in one form or other) .. I am already assumming that that is the approach that every large site would setup. My issue is more about version control and change control in a complex large deployment - the bandwidth issue is solved by the local mirror, but versioning is not. I put a lot more about this in a reply to Stuart and Cc'd the list. In essence, big sites want things to operate at a known software level for all components. Gentoo gives brilliant support for deployment and upgrades, but does not directly support locking things down at a 'specific release'. The local mirror is just a hack for that aspect (it is still needed for performance reasons) One thing I didn't point out in the earlier mail, is another consequence of change control issues in large deployments. Many huge sites ALWAYS run multiple release levels of all sorts of things, for the simple reason that logistics prevent them ever having all systems running at the same level. Change windows are constrained and there are multiple teams of developers and admins working through changes all the time. Organized hell!! Eg. BT in the Uk deploys a new Cisco router about every 30 minutes... every day of the year!! Thats an operation with SCALE. By definition, their routers are never ALL running the same release of IOS.. they are always managing the operation of multiple versions in production - and multiple migration paths are being tested all the time. - Continuous organized hell!! A similar situation exists for their Solaris deployments - multiple versions in production. You also get the situation where one business application is only certified to run on version x.y.z with patches a+b+c of an operating system, and another (also essential) business application is only certifed to run on some other version with some other set of patches. How do you support that using the current Gentoo? Unfortunately not as easily as you can with a distro like RedHat. But I suspect that if some form of 'tag' or 'label' becomes supported through the portage tree, then Gentoo will in fact offer the equivelant support in this area and have a superior ability in the upgrade/security fix area. As for how this could be done... hmmmm. I suspect putting the portage tree into CVS and then using something like 'cvsfs' (http://www.gnu.org/directory/sysadmin/vc/cvsfs.html) to present it to rsync would be good enough.. Thinking out loud:::: backward compatibility - current rsync updates would continue unchanged so 'migration' is a non-event. New features: If you mirrored the CVS repository to your local miror (I think the cvs client can do that?), then your local mirror can offer different views of the portage tree at any of the labels it holds. Multiple concurrent versions could be presented under different 'host names' for the internal rsync process. In fact you could use that to achieve the same thing for the global situation too. Eg. Assume I have set up a fake host called 'Aug2003.acme.com' as one way to access my local rsync mirror. In this host name, rsyncd runs chrooted to the CVS view of the files against the 'tag' for August-2003. The same can happen for any other 'tag' to allow multiple externally accessible views of the portage tree via rsync against my local mirror. All I need to do for my Aug2003 version gentoo machines is point them at that fake host name. The same goes for other versions. Ok - where is that different to running multiple local mirrors? It's different in one important way (and there is a saving on disk space/bandwidth too). The primary difference is that the 'tags' are publicly defined by the Gentoo project. That means that external vendors can take a 'tag' and QA their software against it. Their customers can then install using the predefined 'tag' and use that vendors software ... Plus if, the gentoo project choses to offer the same type of central repository, then a 'release' also implies the creation of an rsync hostname for accessing that release (and only that release). It also provides a way to remove 'supported' access to that release by removing the hostname -- simple DNS changes. For those companies that take a mirror - there is no need to remove the internal CVS labels so old 'releases' in the portage tree can live for a long time after they are publicly accessible from Gentoo. end of thinking through my hat!! more ramblings from me.... but still - worth discussing to help bring Gentoo into the corporate realm Regards Ron O'Hara http://www.openconsulting.us -- gentoo-dev@gentoo.org mailing list