[gentoo-dev] keep a gen 2013 snapshot on mirrors

[gentoo-dev] keep a gen 2013 snapshot on mirrors

[Francesco R.]

long story short
having a  portage-20130126.tar.bz2 snapshot (before the EAPI 5 switch)
greatly simplified the upgrade of an old server on a client.

Why not keep a copy on the servers? I mean
http://distfiles.gentoo.org/snapshots/

thanks,
Francesco Riosa

Re: [gentoo-dev] keep a gen 2013 snapshot on mirrors

[Rich Freeman]

On Wed, Nov 13, 2013 at 1:58 PM, Francesco R. <vivo75@gmail.com> wrote:
>
> long story short
> having a  portage-20130126.tar.bz2 snapshot (before the EAPI 5 switch)
> greatly simplified the upgrade of an old server on a client.
>
> Why not keep a copy on the servers? I mean
> http://distfiles.gentoo.org/snapshots/
>

Going back in time with portage is the easy part - it is in CVS.

The real problem is all the distfiles themselves, especially things
like out-of-tree patch tarballs hosted by devs.

Rich

Re: [gentoo-dev] keep a gen 2013 snapshot on mirrors

[Rick "Zero_Chaos" Farina]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/13/2013 01:58 PM, Francesco R. wrote:
> 
> long story short having a  portage-20130126.tar.bz2 snapshot
> (before the EAPI 5 switch) greatly simplified the upgrade of an old
> server on a client.
> 
> Why not keep a copy on the servers? I mean 
> http://distfiles.gentoo.org/snapshots/
> 
The goal is to be able to update a device for a year.  Not updating at
least once a year is not supportable, and should be discouraged.  I'm
sorry for your pain, I really am, but I hope that it pushes you to
update twice a year instead of zero times.

Thanks,
Zero

> thanks, Francesco Riosa
> 
> 
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJSg9AVAAoJEKXdFCfdEflK+OMQAICBP5TQOdOCIi1SZqnyfPgH
y6UhoLLbvbpuhjLmGFlbVi9uj+zeSArGI8dY795ayS34LDjXgMPkAnFRN/xOiaPq
3SQpRc20YB6dD9GmB3f+7DNGZJFORft9eINsSgFlgiDu4fOsk3vjNuqEHgBI+ld2
8gXcEMsiAK2yq0FXdSyaeoyOpBquVYJTsP1qVFayEFNIo6ZWe7gGex5g+CMpJ/XN
j5cWqtRiaQ1dQKUASs8sxfN0WG/hVY4hLuBnhgblTkrvtK69el0nWRm3e47g14SL
bh2ZG3ViBlNqGBhAZqWNlwll3PbU0SVp1B8k1KTHfaXBwW+n3xO7KJw3ablV7tri
x+wUsbKozwRv1XruL+b0+WNBXBFOuLwq8Acomq7nXELzyLOz5smrzA6eYuz7/ck0
sZbWwbTVNsMm8EE3ZXZZRpbNc6cy2SFAV+UFSt6IJD+19dK6rARTM88yE/ACAhn7
V7EL+KKk1ilOGZXV6vrsBv9jm5aAueauiMDz6uE/yeVPddl1Wb7M1rgiGDQ2R5Ie
CKNKZjkU1s0a644BeaNOX2Mow9rQp68Tvb0s+sTiT1PZnStBmUuuocYCbAvDhbt3
QvuIUKA3b6ddbJ1ynNg7FYWmLXtcZGbf7zMOZSLZLMybonvBsrfE+adVLEb+11Z8
3UeBxk5EF3KrtXSHLyM5
=Rg4r
-----END PGP SIGNATURE-----

Re: [gentoo-dev] keep a gen 2013 snapshot on mirrors

[Roy Bamford]

[-- Attachment #1: Type: text/plain, Size: 471 bytes --]

On 2013.11.13 19:12, Rich Freeman wrote:

[snip]

> Going back in time with portage is the easy part - it is in CVS.
> 
> The real problem is all the distfiles themselves, especially things
> like out-of-tree patch tarballs hosted by devs.
> 
> Rich
> 

Rich,

The GPL obliges us to keep such patches around for three years, iirc.
Don't we do that ?

-- 
Regards,

Roy Bamford
(Neddyseagoon) a member of
elections
gentoo-ops
forum-mods
trustees

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] keep a gen 2013 snapshot on mirrors

[Rich Freeman]

On Wed, Nov 13, 2013 at 3:49 PM, Roy Bamford <neddyseagoon@gentoo.org> wrote:
> The GPL obliges us to keep such patches around for three years, iirc.
> Don't we do that ?

Why?  We own the copyright on the patches (to whatever degree that
they're copyrightable), so we don't need a license to distribute them.
 If somebody else wants to redistribute our patches they need our
permission or they need to comply with whatever license we issue them
under (likely the same as the upstream license so that our users don't
have bindist issues).

The only thing we might need a license to redistribute are the parts
of the patch that we didn't change, and upstream already provides
those.

I don't think patches are a derivative work.  The result of applying
the patches to the original source is a derivative work, but we don't
distribute that - it only exists in a user's /var/tmp.

At least, that's my understanding of copyright.

Rich

Re: [gentoo-dev] keep a gen 2013 snapshot on mirrors

[Tom Wijsman]

[-- Attachment #1: Type: text/plain, Size: 1035 bytes --]

On Wed, 13 Nov 2013 14:12:24 -0500
Rich Freeman <rich0@gentoo.org> wrote:

> On Wed, Nov 13, 2013 at 1:58 PM, Francesco R. <vivo75@gmail.com>
> wrote:
> >
> > long story short
> > having a  portage-20130126.tar.bz2 snapshot (before the EAPI 5
> > switch) greatly simplified the upgrade of an old server on a client.
> >
> > Why not keep a copy on the servers? I mean
> > http://distfiles.gentoo.org/snapshots/
> >
> 
> Going back in time with portage is the easy part - it is in CVS.
> 
> The real problem is all the distfiles themselves, especially things
> like out-of-tree patch tarballs hosted by devs.

The mirroring system could be adjusted to keep around @system distfiles
for a longer time, or alternatively snapshot them when the above
snapshot is made; which should be able to solve this problem.

-- 
With kind regards,

Tom Wijsman (TomWij)
Gentoo Developer

E-mail address  : TomWij@gentoo.org
GPG Public Key  : 6D34E57D
GPG Fingerprint : C165 AF18 AB4C 400B C3D2  ABF0 95B2 1FCD 6D34 E57D

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] keep a gen 2013 snapshot on mirrors

[Johann Schmitz]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>> long story short having a  portage-20130126.tar.bz2 snapshot 
>> (before the EAPI 5 switch) greatly simplified the upgrade of an
>> old server on a client.


I have done the switch to the current profile+portage on many server
recently and i don't think that it is neccessary to keep an old
portage snapshot around. Updating from old portage versions or
profiles isn't fun but it basically boils down to

- - Update python to latest 2.7/3.2
- - Run python-updater and select 2.7/3.2 as the main python version
- - Drop old python version
- - Update Portage, layman, eix
- - Switch to the current profile
- - Update the rest

If you switched to the new profile with an old portage too early
(happens to me *twice*), you will get at some point many blockers due
to EAPI=5 and have to hack either the ebuilds or update portage manually.

> The goal is to be able to update a device for a year.  Not updating
> at least once a year is not supportable, and should be discouraged.
> I'm sorry for your pain, I really am, but I hope that it pushes you
> to update twice a year instead of zero times.

I think that even half a year is a very long period in Gentoo.
Updating the core packages (portage, openrc, udev, etc.) on a
(bi-)monthly basis makes sure that you don't run in such troubles.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJShFOwAAoJEKCEBkJ3xQHtVB8H/jTp17a1nuQP8N8wajkG90/o
pc353PvxW/JgRYBKBrdmWEqW0dS1wRsdbPVnZuOkWpWw4CXbXc7fTJ6gualYUx8L
6FgwsGwWRu/WpSVIvu9gYmtc+yWSPv3YBVmdi0d0KfzUU10sAZn74+0tHUKaSglE
TkunxAB6YSGRYj4VC6ZfaFK38gq5owgyzNjLa90rEPY4QFuTum16Y7Jr6RKtQGa1
LuJygufXPGxasW001dUmVTyQUH+vOxF3nWWaaj92zgGf33t67GeY6MMSnwFVXnnr
AYlHykYQr0RmPGmZShAnhr8B3HL7vDdm0CXzmg23noyiT67RqRXOqDJxwTw+29o=
=jQEK
-----END PGP SIGNATURE-----

Re: [gentoo-dev] keep a gen 2013 snapshot on mirrors

[Francesco R.]

Il 14/11/2013 05:38, Johann Schmitz ha scritto:
> >> long story short having a  portage-20130126.tar.bz2 snapshot
> >> (before the EAPI 5 switch) greatly simplified the upgrade of an
> >> old server on a client.
>
>
>  Updating from old portage versions or
> profiles isn't fun but it basically boils down to
Sorry for the snip, but all you said is valid only if:
you remember to update portage _before_ an emerge --sync
_and_ you are able to do it.

basically what I did was to remove the old snapshot find an old snapshot
but recent enough to have EAPI5 portage.
yes some wget of some distfiles was needed but it made the whole thing
possible.

Alternatively build a completely new system and then switch is a
possibility,
nothing deadly but as said a simple copy of a january snapshot would
have made some paths simpler

cheers,
Francesco Riosa

Re: [gentoo-dev] keep a gen 2013 snapshot on mirrors

[Francesco R.]

Il 13/11/2013 20:12, Rich Freeman ha scritto:
> On Wed, Nov 13, 2013 at 1:58 PM, Francesco R. <vivo75@gmail.com> wrote:
>> long story short
>> having a  portage-20130126.tar.bz2 snapshot (before the EAPI 5 switch)
>> greatly simplified the upgrade of an old server on a client.
>>
>> Why not keep a copy on the servers? I mean
>> http://distfiles.gentoo.org/snapshots/
>>
> Going back in time with portage is the easy part - it is in CVS.
>
> The real problem is all the distfiles themselves, especially things
> like out-of-tree patch tarballs hosted by devs.
>
> Rich
Rich, that made me smile, none of my remote machine has cvs since a
_very_ long time say 2006.
We are speaking of box that have troubles to emerging anything new, plus
me and most of the internet barely remember cvs up  :)

I do highly appreciate the suggestion to keep a @system distfiles
snapshot (once a year + portage snapshot would be a bone), but that it's
not strictly needed, just a 40MB bzipped files on a public directory and
maybe some change to the cron that wipe old files.

cheers,
Francesco R.

Re: [gentoo-dev] keep a gen 2013 snapshot on mirrors

[Lars Wendler]

[-- Attachment #1: Type: text/plain, Size: 1209 bytes --]

Am Wed, 13 Nov 2013 14:12:24 -0500
schrieb Rich Freeman <rich0@gentoo.org>:

> On Wed, Nov 13, 2013 at 1:58 PM, Francesco R. <vivo75@gmail.com>
> wrote:
> >
> > long story short
> > having a  portage-20130126.tar.bz2 snapshot (before the EAPI 5
> > switch) greatly simplified the upgrade of an old server on a client.
> >
> > Why not keep a copy on the servers? I mean
> > http://distfiles.gentoo.org/snapshots/
> >
> 
> Going back in time with portage is the easy part - it is in CVS.
> 
> The real problem is all the distfiles themselves, especially things
> like out-of-tree patch tarballs hosted by devs.

IMHO not even the worst problems as long as devs stop removing files
they consider as "deprecated" or "old". I keep everything in my
dev-space I ever put in an ebuild that landed in our official portage
tree. Of course this only works as long as you reference that place in
the affected ebuilds. Once you go the route to use mirror://gentoo in
ebuilds as SRC_URI people are screwed as soon as the ebuild vanishes
from portage.
I'd love to see this (mis-)behavior being more vigorously discouraged
in Gentoo-land...

Cheers
-- 
Lars Wendler
Gentoo package maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] keep a gen 2013 snapshot on mirrors

[Rich Freeman]

On Thu, Nov 14, 2013 at 8:17 AM, Francesco R. <vivo75@gmail.com> wrote:
> Rich, that made me smile, none of my remote machine has cvs since a
> _very_ long time say 2006.
> We are speaking of box that have troubles to emerging anything new, plus
> me and most of the internet barely remember cvs up  :)

You don't need to run cvs on the box that you're trying to update.
Just run it on a different box and create a tarball of the resulting
tree.

I couldn't tell you what the syntax for doing that is offhand, but I
imagine that it wouldn't take you much longer than it would take me to
read the manpage and figure it out.  If you're really stuck I'm sure
somebody would help you.

>
> I do highly appreciate the suggestion to keep a @system distfiles
> snapshot (once a year + portage snapshot would be a bone), but that it's
> not strictly needed, just a 40MB bzipped files on a public directory and
> maybe some change to the cron that wipe old files.

Personally I'd prefer to see a more standardized way of handling
non-upstream distfiles as well (probably not a tarball).  However,
Gentoo really isn't one of those distros that is ever going to be easy
to upgrade in-place if you only do it once a year or less often.
There are ways to do it, but they're going to be painful.

Rich

Re: [gentoo-dev] keep a gen 2013 snapshot on mirrors

[Ian Stakenvicius]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 14/11/13 08:19 AM, Lars Wendler wrote:

> .... Once you go the route to use mirror://gentoo in ebuilds as
> SRC_URI people are screwed as soon as the ebuild vanishes from
> portage. I'd love to see this (mis-)behavior being more vigorously
> discouraged in Gentoo-land...

Me too; at least until such time as infra starts running out of space,
and i'm sure they can let us know when that's about to happen....




-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iF4EAREIAAYFAlKFKVkACgkQ2ugaI38ACPD7GgD+O0Of0R0BeOmGmKNKHwaLt/Oh
63xVewB5tQKdTeqn7dIBAKZgrPu2ScMfPSyOgSQNXpveUGWntsin3HqOifGLCTvQ
=o4tM
-----END PGP SIGNATURE-----

[gentoo-dev] Re: keep a gen 2013 snapshot on mirrors

[Duncan]

Rich Freeman posted on Wed, 13 Nov 2013 16:18:51 -0500 as excerpted:

> On Wed, Nov 13, 2013 at 3:49 PM, Roy Bamford <neddyseagoon@gentoo.org>
> wrote:
>> The GPL obliges us to keep such patches around for three years, iirc.
>> Don't we do that ?
> 
> Why?  We own the copyright on the patches (to whatever degree that
> they're copyrightable), so we don't need a license to distribute them.
> If somebody else wants to redistribute our patches they need our
> permission or they need to comply with whatever license we issue them
> under (likely the same as the upstream license so that our users don't
> have bindist issues).

IANAL of course, but as someone who cares about such things because he 
knows they protect his rights, and who thus follows discussions such as 
this rather closer than most...

For the GPL, the biggest concern is binary distribution, covered under 
section three of the GPL (v2 at least), and this is where the three-year 
clause appears.  Because gentoo in-the-main only distributes sources and 
any patches are generally distributed under the same license as the 
primary package sources, that clause in general doesn't apply.

There are, however, two big exceptions to gentoo's sources-only 
distribution, the installers and stage tarballs, and the packages CD/DVD 
images.  It is with these that there is some concern, and where gentoo 
has in the past had GPL violations.

However, if you look at section three, there are three alternative 
clauses.  Complying with only one of the three is necessary (my rewording 
of the GPLv2 license from the file in the tree, see the license itself 
for the literal wording if there's any doubt):

3a) Accompany binaries/object code with complete source-code.

3b) Accompany them with a written offer for source code, valid for at 
least three years from date of last distribution of binary/object code.

3c) Pass on the 3b offer you received from upstream.  (Non-commercial 
only, only applies to code unmodified from upstream and thus not to the 
case under discussion.)

Since 3c is out for modified code, that leaves 3a or 3b.  But the three 
years of 3b ends up being a long time to track such things!  Luckily, 
there's still 3a, which if properly complied with, lets gentoo off the 
hook for 3b.

What that means is this:  Every time and place gentoo distributes 
binaries, we must make available sources as well.  If we're giving away 
install-CDs at a conference, we better have a few copies of the parallel 
sources CD, including our patches, available as well.  (The stack of 
sources CDs could of course be smaller, provided we're willing to remove 
the stack of installer CDs until we burn a few more source CDs if they 
ever run out.)

Similarly on the net, if we're distributing stage tarballs, we should 
ensure that we have the sources available for download at the same time 
as well.

Where gentoo screwed up in the past is that for quite awhile, we kept 
past installer images available as well, "for historical interest".  At 
some point we were made aware of the fact that this was a violation of 
the GPL if the corresponding sources, INCLUDING OUR PATCHES, were not 
either directly available as well, or available on request for three 
years, and we had to take those historical copies down.

But IIRC that was well over three years ago now, so AFAIK, we're out of 
violation on at least the major scale, tho it remains possible that 
someone representing gentoo at a conference might up and fail to ensure 
those source CDs are made available with the installer CDs, for instance, 
thus obligating us to a three-year-clock we might have problems actually 
following up on, should someone make that request.  I would hope that the 
PR project folks properly brief all conference booth volunteers to ensure 
that doesn't happen, however.

> The only thing we might need a license to redistribute are the parts of
> the patch that we didn't change, and upstream already provides those.
> 
> I don't think patches are a derivative work.

To the extent patches are larger than the rather blurry "trivial" level, 
I believe there's no question that they ARE derivative.  In the case of 
literal patches, literally and provably so, due to the context-diff which 
literally includes lines from the original from which it is derived.

However, fair-use laws often allow "trivial" use, and while that's a 
somewhat blurry line and fair-use laws definitely differ from country to 
country, I found a reference in the FSF FAQ that says they use a 300-line 
benchmark (tho that's in the "whole work" context):

http://www.gnu.org/licenses/gpl-faq.html#WhatIfWorkIsShort

While I think most would agree that a one-line patch is "trivial", I'm 
not so sure about a 50-line patch tho it's nowhere near that 300-line 
threshold, which itself is just the FSF benchmark, tho I'm not sure if 
that FAQ context is applicable to patches or not.

However, given the above 3a clause and gentoo's efforts to comply with 
it, I suppose the whole question of triviality becomes moot.

> At least, that's my understanding of copyright.

Likewise mine. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: keep a gen 2013 snapshot on mirrors

[Rich Freeman]

On Fri, Nov 15, 2013 at 8:17 AM, Duncan <1i5t5.duncan@cox.net> wrote:
> To the extent patches are larger than the rather blurry "trivial" level,
> I believe there's no question that they ARE derivative.  In the case of
> literal patches, literally and provably so, due to the context-diff which
> literally includes lines from the original from which it is derived.

Ok, to illustrate, let's consider the bit I just quoted the original
work.  This would be a derivative work:

To the extent patches are larger than the rather blurry "trivial" level,
I believe there's no question that they ARE derivative.  In the case of
literal patches, literally, and figuratively, and provably so, due to
the context-diff which
literally includes lines from the original from which it is derived.

This would not be a derivative work:
and figuratively,

This isn't a derivative work:
On line three insert the characters "and figuratively, " after the second comma.

This is more fuzzy, but is probably fair use to the extent that it is
a derivative work:
I believe there's no question that they ARE derivative.  In the case of
-literal patches, literally, and provably so, due to the context-diff which
+literal patches, literally, and figuratively, and provably so, due to
the context-diff which
literally includes lines from the original from which it is derived.

That's what I'm getting at.  The actual changes themselves aren't a
derivative work - it is the result of applying them that is.

Rich

[gentoo-dev] Re: keep a gen 2013 snapshot on mirrors

[Duncan]

Rich Freeman posted on Fri, 15 Nov 2013 08:38:20 -0500 as excerpted:

> That's what I'm getting at.  The actual changes themselves aren't a
> derivative work - it is the result of applying them that is.

I can (cautiously) agree with that, tho I'm sure there are those who 
would take an opposing position, certainly if they're lawyers paid to do 
so!

Meanwhile, I found your (paraphrased) "on the third line, after the 
comma" example interesting, both since I saw a similar argument on one of 
the pages that came up in my google, and because it's basically what a sed 
script (as opposed to a patch) does in practice.  The point being that 
simple instructions saying /how/ to do it, without incorporating any of 
the original work, can hardly be said to be derived.

(The example I saw in my google was text instructions for cutting a face 
out of a larger photo and putting a red frame around it.  It's quite 
clear that those text instructions are not derivative of a particular 
photo, while a screen-grab illustrating the process may well be 
considered derivative of the original photo, since it includes part of 
it.  A sed script is arguably similar, tho s// regex replacement can 
certainly be borderline, but for the triviality test.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: keep a gen 2013 snapshot on mirrors

[Peter Stuge]

Duncan wrote:
> 3a) Accompany binaries/object code with complete source-code.
..
> What that means is this:  Every time and place gentoo distributes 
> binaries, we must make available sources as well.

"accompany" !== "make available"


> If we're giving away install-CDs at a conference, we better have a
> few copies of the parallel sources CD, including our patches,
> available as well.  (The stack of sources CDs could of course be
> smaller, provided we're willing to remove the stack of installer
> CDs until we burn a few more source CDs if they ever run out.)
> 
> Similarly on the net, if we're distributing stage tarballs, we should 
> ensure that we have the sources available for download at the same time 
> as well.

I think those two are nowhere near good enough. Accompanied means
that both go together AFAIU, though I'm not a native english speaker.


//Peter