[gentoo-dev] Policy on RESTRICT=[no]mirror and use of mirror://foo/

[gentoo-dev] Policy on RESTRICT=[no]mirror and use of mirror://foo/

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1139 bytes --]

Hi,

I've tried to dig through our policy documents, to find the policy
regarding RESTRICT=mirror/fetch. I don't find anything in the htdocs
tree from CVS, beyond a few brief mentions in the handbook.

RESTRICT=[no]fetch
------------------
Should be used if the license prohibits unattended/automated download
(eg click-through licenses), or the upstream author wants all downloads
to be manually done from their site (I know of at least one package
where this is true).

RESTRICT=[no]mirror
-------------------
- Files too large for the mirrors? (What is the size limit?)
- License issues?
- anything else?

Unless there is something I've missed, why do we have ebuilds with
RESTRICT=nomirror, having a GPL-2 license, and distributed via
sourceforge? Wouldn't RESTRICT=primaryuri be much better? Most fetches
would go to SF first, and we'd still have a copy available on our
mirrors.

-- 
Robin Hugh Johnson
E-Mail     : robbat2@orbis-terrarum.net
Home Page  : http://www.orbis-terrarum.net/?l=people.robbat2
ICQ#       : 30269588 or 41961639
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 241 bytes --]

Re: [gentoo-dev] Policy on RESTRICT=[no]mirror and use of mirror://foo/

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1207 bytes --]

On Thu, 5 May 2005 14:04:20 -0700 "Robin H. Johnson"
<robbat2@gentoo.org> wrote:
| I've tried to dig through our policy documents, to find the policy
| regarding RESTRICT=mirror/fetch. I don't find anything in the htdocs
| tree from CVS, beyond a few brief mentions in the handbook.

Some of it's in the "not policy" documents:

http://www.firedrop.org.uk/devmanual/general-concepts/mirrors/

Contributions welcomed if someone knows exactly what the rules are :)

| RESTRICT=[no]mirror
| -------------------
| - Files too large for the mirrors? (What is the size limit?)
| - License issues?
| - anything else?

AFAIK it's now just licence issues.

| Unless there is something I've missed, why do we have ebuilds with
| RESTRICT=nomirror, having a GPL-2 license, and distributed via
| sourceforge? Wouldn't RESTRICT=primaryuri be much better? Most fetches
| would go to SF first, and we'd still have a copy available on our
| mirrors.

Now that primaryuri exists, yes. That's a fairly recent feature,
however.

-- 
Ciaran McCreesh : Gentoo Developer (Vim, Shell tools, Fluxbox, Cron)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Policy on RESTRICT=[no]mirror and use of mirror://foo/

[Corey Shields]

On Thursday 05 May 2005 02:04 pm, Robin H. Johnson wrote:
> Unless there is something I've missed, why do we have ebuilds with
> RESTRICT=nomirror, having a GPL-2 license, and distributed via
> sourceforge? Wouldn't RESTRICT=primaryuri be much better? Most fetches
> would go to SF first, and we'd still have a copy available on our
> mirrors.

At one point in time someone suggested that we may as well utilize other 
people's mirror networks that are already out there, rather than our own.  
This then got implemented for stuff like sourceforge packages.

Unfortunately this plays hell for lots of people.  If you have a local mirror, 
then you get files fast and without using your outside bandwidth.  Having to 
wait for a file from some mirror in zimbabwe because "that package already 
has a mirror structure" defeats the purpose of running a local mirror.

On top of that, if you have a private network (RFC 1918) with a gentoo mirror 
on the inside, you can't reach the SRC_URI on restrict=nomirror packages, and 
they don't end up on the mirror anyway.  We had this problem on dozens of 
servers at IU..  At one point in time even gentoo-sources was marked 
nomirror, since kernel.org had their own network.

I think that nomirror should be used as seldom as possible.  Licensing issues 
are legit.  We've had one case where mirror admins complained about a file 
that was too big (a 2 gig neverwinter nights file), but otherwise I have 
talked with them before about releasing "nomirror" wherever we can, and they 
didn't mind.

If someone wants to put the "Official" stamp on the licensing/nomirror policy, 
that would be great.  Just as important, I think we need to backtrack and 
clear out that restriction wherever it is unnecessarily used.

Cheers,

-C

-- 
Corey Shields
Gentoo Linux Infrastructure Team and Devrel Team
Gentoo Foundation Board of Trustees
http://www.gentoo.org/~cshields
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Policy on RESTRICT=[no]mirror and use of mirror://foo/

[Mike Frysinger]

On Thursday 05 May 2005 05:04 pm, Robin H. Johnson wrote:
> RESTRICT=[no]fetch
> ------------------
> Should be used if the license prohibits unattended/automated download
> (eg click-through licenses), or the upstream author wants all downloads
> to be manually done from their site (I know of at least one package
> where this is true).

correct

> RESTRICT=[no]mirror
> -------------------
> - Files too large for the mirrors? (What is the size limit?)

yes ...

> - License issues?

right, some people dont allow us to redistribute (like some games)

> - anything else?
>
> Unless there is something I've missed, why do we have ebuilds with
> RESTRICT=nomirror, having a GPL-2 license, and distributed via
> sourceforge? Wouldn't RESTRICT=primaryuri be much better? Most fetches
> would go to SF first, and we'd still have a copy available on our
> mirrors.

because a while back someone got the big idea that if someone provides a 
mirroring system (like sourceforge), we should let users abuse those mirrors 
instead of gentoo mirrors ... subsequently, people started sticking RESTRICT 
into the ebuilds ...

we have since decided that was a dumb idea (which i agree with) but no one 
cared to go through portage and revert the RESTRICT ... i just do it when i 
happen to notice it updating old ebuilds
-mike
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Policy on RESTRICT=[no]mirror and use of mirror://foo/

[Lance Albertson]

[-- Attachment #1: Type: text/plain, Size: 1613 bytes --]

Mike Frysinger wrote:

[snip]

>>RESTRICT=[no]mirror
>>-------------------
>>- Files too large for the mirrors? (What is the size limit?)
> 
> 
> yes ...

We should probably have a hard definition for this or expectations to the rule
too. Let me look through our current distfiles and see how big we really get on
a few things. I'd say anything more than a few hundred megs is a bit too much,
but thats just my rough guess.

[snip]

>>Unless there is something I've missed, why do we have ebuilds with
>>RESTRICT=nomirror, having a GPL-2 license, and distributed via
>>sourceforge? Wouldn't RESTRICT=primaryuri be much better? Most fetches
>>would go to SF first, and we'd still have a copy available on our
>>mirrors.
> 
> 
> because a while back someone got the big idea that if someone provides a 
> mirroring system (like sourceforge), we should let users abuse those mirrors 
> instead of gentoo mirrors ... subsequently, people started sticking RESTRICT 
> into the ebuilds ...
> 
> we have since decided that was a dumb idea (which i agree with) but no one 
> cared to go through portage and revert the RESTRICT ... i just do it when i 
> happen to notice it updating old ebuilds

We could make a couple of bugs for each category and get those folks to fixing
them. But then that would require a lot of work and I know we're all just a
bunch of lazy bums anyways ;)

-- 
Lance Albertson <ramereth@gentoo.org>
Gentoo Infrastructure | Operational Manager

---
Public GPG key:  <http://www.ramereth.net/lance.asc>
Key fingerprint: 0423 92F3 544A 1282 5AB1  4D07 416F A15D 27F4 B742

ramereth/irc.freenode.net

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 187 bytes --]

Re: [gentoo-dev] Policy on RESTRICT=[no]mirror and use of mirror://foo/

[Corey Shields]

[-- Attachment #1: Type: text/plain, Size: 496 bytes --]

On Thursday 05 May 2005 03:06 pm, Lance Albertson wrote:
> We could make a couple of bugs for each category and get those folks to
> fixing them. But then that would require a lot of work and I know we're all
> just a bunch of lazy bums anyways ;)

I smell a new dev requirement..   "Fix 10 nomirror'ed ebuilds, then report to 
your mentor"

;)

-- 
Corey Shields
Gentoo Linux Infrastructure Team and Devrel Team
Gentoo Foundation Board of Trustees
http://www.gentoo.org/~cshields

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Policy on RESTRICT=[no]mirror and use of mirror://foo/

[Lance Albertson]

[-- Attachment #1: Type: text/plain, Size: 723 bytes --]

Corey Shields wrote:
> On Thursday 05 May 2005 03:06 pm, Lance Albertson wrote:
> 
>>We could make a couple of bugs for each category and get those folks to
>>fixing them. But then that would require a lot of work and I know we're all
>>just a bunch of lazy bums anyways ;)
> 
> 
> I smell a new dev requirement..   "Fix 10 nomirror'ed ebuilds, then report to 
> your mentor"

Oohhh,  You're good :-D

"You won't become a full dev till you fine 10 nomirror'd ebuilds and fix0r them"

-- 
Lance Albertson <ramereth@gentoo.org>
Gentoo Infrastructure | Operational Manager

---
Public GPG key:  <http://www.ramereth.net/lance.asc>
Key fingerprint: 0423 92F3 544A 1282 5AB1  4D07 416F A15D 27F4 B742

ramereth/irc.freenode.net

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 187 bytes --]

Re: [gentoo-dev] Policy on RESTRICT=[no]mirror and use of mirror://foo/

[Alin Nastac]

[-- Attachment #1: Type: text/plain, Size: 754 bytes --]

Corey Shields wrote:

>At one point in time someone suggested that we may as well utilize other 
>people's mirror networks that are already out there, rather than our own.  
>This then got implemented for stuff like sourceforge packages.
>
>Unfortunately this plays hell for lots of people.  If you have a local mirror, 
>then you get files fast and without using your outside bandwidth.  Having to 
>wait for a file from some mirror in zimbabwe because "that package already 
>has a mirror structure" defeats the purpose of running a local mirror.
>  
>
I know only one mirror network who could worth the hassle: cpan.
Perl has a nice geographically distributed network of mirrors. Too bad
portage can't automatically select the closest cpan mirror. :(

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: [gentoo-dev] Policy on RESTRICT=[no]mirror and use of mirror://foo/

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 717 bytes --]

On Fri, May 06, 2005 at 07:37:07AM +0300, Alin Nastac wrote:
> I know only one mirror network who could worth the hassle: cpan.
> Perl has a nice geographically distributed network of mirrors. Too bad
> portage can't automatically select the closest cpan mirror. :(
You can already put in specific mirrors to use via /etc/portage/mirrors

So just write up some changes to mirrorselect that can populate that
file by selecting the optimum choices from profiles/thirdpartymirrors.

-- 
Robin Hugh Johnson
E-Mail     : robbat2@orbis-terrarum.net
Home Page  : http://www.orbis-terrarum.net/?l=people.robbat2
ICQ#       : 30269588 or 41961639
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 241 bytes --]

Re: [gentoo-dev] Policy on RESTRICT=[no]mirror and use of mirror://foo/

[Brian Harring]

On Thu, May 05, 2005 at 10:53:03PM -0700, Robin H. Johnson wrote:
> On Fri, May 06, 2005 at 07:37:07AM +0300, Alin Nastac wrote:
> > I know only one mirror network who could worth the hassle: cpan.
> > Perl has a nice geographically distributed network of mirrors. Too bad
> > portage can't automatically select the closest cpan mirror. :(
> You can already put in specific mirrors to use via /etc/portage/mirrors
> 
> So just write up some changes to mirrorselect that can populate that
> file by selecting the optimum choices from profiles/thirdpartymirrors.
For third party mirrors, might I suggest /etc/portage/mirrors?

Bit saner then jamming a cpan specific mirror into GENTOO_MIRRORS...
~brian
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Policy on RESTRICT=[no]mirror and use of mirror://foo/

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 878 bytes --]

On Thu, 2005-05-05 at 14:30 -0700, Corey Shields wrote:
> I think that nomirror should be used as seldom as possible.  Licensing issues 
> are legit.  We've had one case where mirror admins complained about a file 
> that was too big (a 2 gig neverwinter nights file), but otherwise I have 
> talked with them before about releasing "nomirror" wherever we can, and they 
> didn't mind.

We actually use nomirror pretty extensively on the games team.  We have
quite a few mods/games that come in at over 500MB.  There's even a
couple at over 1GB.  I can tell you that if we loosened the nomirror on
those, we would need at least another 20GB of mirror space just for the
added games-* distfiles.

So how about we leave it a bit more open for interpretation?

-- 
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Policy on RESTRICT=[no]mirror and use of mirror://foo/

[Lance Albertson]

[-- Attachment #1: Type: text/plain, Size: 1439 bytes --]

On Fri, 2005-05-06 at 17:08 -0400, Chris Gianelloni wrote:
> On Thu, 2005-05-05 at 14:30 -0700, Corey Shields wrote:
> > I think that nomirror should be used as seldom as possible.  Licensing issues 
> > are legit.  We've had one case where mirror admins complained about a file 
> > that was too big (a 2 gig neverwinter nights file), but otherwise I have 
> > talked with them before about releasing "nomirror" wherever we can, and they 
> > didn't mind.
> 
> We actually use nomirror pretty extensively on the games team.  We have
> quite a few mods/games that come in at over 500MB.  There's even a
> couple at over 1GB.  I can tell you that if we loosened the nomirror on
> those, we would need at least another 20GB of mirror space just for the
> added games-* distfiles.
> 
> So how about we leave it a bit more open for interpretation?

Yeah, thanks for not putting those on our mirrors :) See my previous
email about a possible way to determine what goes on the mirrors. Maybe
we have a limit at which those devs should ask our mirror admins if the
file can be added or not (say, anything over 200M or 300M). Just an
idea. Mainly, use common sense!

Cheers,

-- 
Lance Albertson <ramereth@gentoo.org>
Gentoo Infrastructure | Operations Manager

---
GPG Public Key:  <http://www.ramereth.net/lance.asc>
Key fingerprint: 0423 92F3 544A 1282 5AB1  4D07 416F A15D 27F4 B742

ramereth/irc.freenode.net

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]