[gentoo-dev] The release of 1.4 and its impact on our mirrors

[Kurt Lieber <klieber@gentoo.org>]

[-- Attachment #1: Type: text/plain, Size: 2370 bytes --]

Folks --

One issue that came out of today's manager meeting was the amount of space
that GRP and some of the new livecds will require for 1.4_release.  As I
understand it, that number hovers right around 10GB at the moment.

We have already received numerous complaints from our mirror admins about
the amount of disk space we chew up now.  For reference, here is a break
down:

9.7G    ./releases
139M    ./snapshots
17G     ./distfiles
6.6G    ./experimental

We can clean out some space in the /releases directory by deleting some of
the old 1.4_rc* stuff, but we will still have a significant overall size
increase as a part of 1.4_final.  We will be working on reducing the size
of /distfiles as well, but this will be nothing more than a temporary,
tactical fix.  We still need a longer term strategic fix.

I'd like to come up with a solution that allows each arch the flexibility
it requires to create new livecds, GRP packages, etc.  But I'd also like to
be mindful and respectful of our mirrors.  

I'd like to hear suggestions from you all on the best way to achieve this.
Some ideas that I've heard so far include:

* creating separate directories that are optional for our mirrors to 
  support.  GRP would be a prime candidate for this given the space it
  requires.  Mirrors short on disk space could choose not to mirror these
  files.
* assigning each arch a quota for the files it may use outside of
  /distfiles.  This quota would apply to liveCDs, things in /experimental
  as well as everything within /releases (as well as /grp if we make that a
  separate top level directory)  To be effective, this quota would need to
  be around the ~3GB level given the number of arches we support.

Those are simply two ideas that have been proposed so far.  I'm sure you
guys will have more.  The only requirements at this point are:

* We cannot expect our system of mirrors to give us an unlimited amount of
  disk space. (they've already started to complain)
* We cannot afford to lose many mirrors.  Especially in North America,
  source mirrors are in short supply.
* We must expect /distfiles to continue growing as we continue to add new
  ebuilds. So solutions involving "reducing the size of /distfiles" will
  not be feasible except as a short-term, stop-gap measure.

Thoughts?  Ideas?

--kurt

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]