[gentoo-releng] space allocation for releases on the mirrors

[gentoo-releng] space allocation for releases on the mirrors

[Kurt Lieber]

[-- Attachment #1: Type: text/plain, Size: 2864 bytes --]

In order to get a better handle on space utilization for our mirrors, I'm
going to put in a mostly-firm-but-somewhat-flexible cap on overall space
used for each release.  Taking a look at the amount of space used for each
arch in 2004.1, we have:

2.9G    ./ppc/2004.1
4.5G    ./x86/2004.1
1.3G    ./amd64/2004.1
2.4G    ./sparc/2004.1

If you go on the recently-heard "we're only releasing x86 and 686" for x86,
then the space there works out to ~2.2GB.

There are other projects (like hardened) that want to get their stuff on
the mirror as well, and I expect we're going to have at least one or two
new arches releasing this time around that missed the 2004.1 deadline.  As
such, I'm proposing a 2.5GB per arch/major-top-level project cap on file
size for the mirror.

Now, lest anyone get up in arms at this, I only care about the total size
of all the files used for a release.  So, when it comes time to put 2004.2
stuff up on the mirror, I'm going to look at the number of arches + major
top level projects (currently, only hardened) and multiple that by 2.5 to
get a total size cap.  As long as the total size is under that number,
that's fine with me.

Some notes:

* If an arch isn't releasing for 2004.2, then no space is allocated for
  them.
* If an arch is planning on releasing for 2004.2 and cancels at the last
  minute, then no space is allocated for them.
* I would greatly appreciate a heads up at least 2-3 weeks prior to a
  release about arches that *will* be releasing for 2004.2.  So far, I'm
  assuming x86, ppc, amd64 and sparc will all be releasing from an arch POV
  and I also assume hardened will release something as well.  So, that's a
  total of 12.5GB of data that I'm expecting.  If more arches will be
  releasing, please let me know as soon as possible so I can give a heads
  up to our mirrors.
* sub-projects, such as GameCDs, will need to fit into the existing space
  requirements.  Arches like AMD64 are unlikely to need a full 2.5GB of
  space, so this should allow some extra space for GameCDs and other
  special projects.  If there are other unique circumstances that may
  warrant special consideration, please feel free to discuss with me on
  IRC.

So, at this point, you have 12.5GB of available space to fill up with
whatever you want.  I don't care if it's 12.5GB devoted to one arch and
everyone else is left out in the cold -- that's up to you guys to figure
out.  I just need a firm number that I can communicate to our mirror
admins so they can plan accordingly. (again, if more arches are releasing,
please let me know ASAP)

--kurt

P.S. We will also have a torrent server available for distributing ISOs and
stages that might not make it into the above size cap.  At this point, we
have no alternate means of distributing this data (such as http or ftp) so
please plan what goes in the 12.5GB carefully.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-releng] space allocation for releases on the mirrors

[John Davis]

Kurt Lieber wrote:
> In order to get a better handle on space utilization for our mirrors, I'm
> going to put in a mostly-firm-but-somewhat-flexible cap on overall space
> used for each release.  Taking a look at the amount of space used for each
> arch in 2004.1, we have:
> 

I have no objections - 12.5 GB is tight, but it should meet our needs. 
How flexible are you willing to be on that cap? +500MB, +250MB, +0MB? We 
will try our best, but there is always the possibility of running over 
the limit.

We will be utilizing BT this release for (at least) p3, p4, and 
athlon-xp GRP as well as one or two x86/ AMD64 GameCDs.

An up-to-date list of arches releasing for 2004.2 can be found at 
http://www.gentoo.org/proj/en/releng/release/2004.2/2004.2.xml.

Regards,
-- 
John Davis
Gentoo Linux Developer
<http://dev.gentoo.org/~zhen>

----
GnuPG Public Key: <http://dev.gentoo.org/~zhen/zhen_pub.asc>
Fingerprint: 4F9E 41F6 D072 5C1A 636C  2D46 B92C 4823 E281 41BB

--
gentoo-releng@gentoo.org mailing list

Re: [gentoo-releng] space allocation for releases on the mirrors

[Kurt Lieber]

[-- Attachment #1: Type: text/plain, Size: 518 bytes --]

On Tue, Jun 22, 2004 at 07:03:53PM -0400 or thereabouts, John Davis wrote:
> I have no objections - 12.5 GB is tight, but it should meet our needs. 
> How flexible are you willing to be on that cap? +500MB, +250MB, +0MB? We 
> will try our best, but there is always the possibility of running over 
> the limit.

500MB over is a bit high.  I think 250MB is probably within reason.  Again,
the primary purpose is to allow our mirrors some sort of idea of what to
expect so they can plan accordingly.

--kurt

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-releng] space allocation for releases on the mirrors

[John Davis]

Kurt Lieber wrote:
> On Tue, Jun 22, 2004 at 07:03:53PM -0400 or thereabouts, John Davis wrote:
> 
>>I have no objections - 12.5 GB is tight, but it should meet our needs. 
>>How flexible are you willing to be on that cap? +500MB, +250MB, +0MB? We 
>>will try our best, but there is always the possibility of running over 
>>the limit.
> 
> 
> 500MB over is a bit high.  I think 250MB is probably within reason.  Again,
> the primary purpose is to allow our mirrors some sort of idea of what to
> expect so they can plan accordingly.
> 
> --kurt

Sounds good to me - thanks Kurt.
Regards,
-- 
John Davis
Gentoo Linux Developer
<http://dev.gentoo.org/~zhen>

----
GnuPG Public Key: <http://dev.gentoo.org/~zhen/zhen_pub.asc>
Fingerprint: 4F9E 41F6 D072 5C1A 636C  2D46 B92C 4823 E281 41BB

--
gentoo-releng@gentoo.org mailing list

Re: [gentoo-releng] space allocation for releases on the mirrors

[Seemant Kulleen]

[-- Attachment #1: Type: text/plain, Size: 392 bytes --]

> We will be utilizing BT this release for (at least) p3, p4, and 
> athlon-xp GRP as well as one or two x86/ AMD64 GameCDs.
I thought the number of subarches for x86 was getting trimmed?
-- 
Seemant Kulleen
http://dev.gentoo.org/~seemant

Public Key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x3458780E
Key fingerprint = 23A9 7CB5 9BBB 4F8D 549B 6593 EDA2 65D8 3458 780E


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-releng] space allocation for releases on the mirrors

[John Davis]

Seemant Kulleen wrote:
>>We will be utilizing BT this release for (at least) p3, p4, and 
>>athlon-xp GRP as well as one or two x86/ AMD64 GameCDs.
> 
> I thought the number of subarches for x86 was getting trimmed?


For the main mirrors, it is. We are going to keep the other subarches 
around on BT because the store sells quite a bit of them I have been 
told (which means that our users do in fact still use them).

It would be nice to have them all on our mirrors, but given our limited 
resources we have to make due with what we have.

Kurt - is our main mirror limitation our own HD space or our official 
mirrors' HD space?

Cheers,
-- 
John Davis
Gentoo Linux Developer
<http://dev.gentoo.org/~zhen>

----
GnuPG Public Key: <http://dev.gentoo.org/~zhen/zhen_pub.asc>
Fingerprint: 4F9E 41F6 D072 5C1A 636C  2D46 B92C 4823 E281 41BB

--
gentoo-releng@gentoo.org mailing list

Re: [gentoo-releng] space allocation for releases on the mirrors

[Kurt Lieber]

[-- Attachment #1: Type: text/plain, Size: 438 bytes --]

On Wed, Jun 23, 2004 at 12:57:40PM -0400 or thereabouts, John Davis wrote:
> It would be nice to have them all on our mirrors, but given our limited 
> resources we have to make due with what we have.
> 
> Kurt - is our main mirror limitation our own HD space or our official 
> mirrors' HD space?

Both.  We have lots of disk space internally, but not much of it is SCSI,
which is needed to do any serious mirroring. 

--kurt

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]