[gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Ed W]

Hi, how do others handle open source licence compliance when building 
some base system using gentoo?

In particular I guess simply capturing the ebuilds is not sufficient and 
it's necessary to capture and distribute all the source and patch files 
used to create a build.  The emerge tool doesn't obviously give a way to 
capture this stuff.  I looked in the eclasses, particularly the epatch 
file and I'm not clear that I can easily hook into that.

At the moment I'm using a bashrc file to grab everything from the build 
directory.  This seems reasonably robust for source files.  However, for 
patches I have considered creating a fake patch utility which would 
record all the files it operates on.  Any other suggestions?  Perhaps 
catalyst already has done something like that - not familiar with it though?

Whilst the above is largely targeting GPL type licences, are there other 
things I should consider for other licences? Other things I need to 
ensure I distribute for GPL? Any pointers to (simple) documentation on 
how one can be a compliant open source citizen..?

Thanks

Ed W

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Peter Stuge]

Ed W wrote:
> Hi, how do others handle open source licence compliance when building
> some base system using gentoo?

Review the packages that get built, and adhere to their licenses. It
can be a fair bit of work.


> In particular I guess simply capturing the ebuilds is not sufficient
> and it's necessary to capture and distribute all the source and
> patch files used to create a build.

How do you build your system? If you use catalyst, the open source
gold standard citizen publishes spec files, snapshot, toolchain and
toolchain source.


> The emerge tool doesn't obviously give a way to capture this stuff.

First step is to analyze licenses. emerge does know the license for a
package, and it is available in /var/db/pkg/ after install.


> I looked in the eclasses, particularly the epatch file and I'm not
> clear that I can easily hook into that.

If you have patches which use a different license than the package
they modify then you have more work to do. Portage doesn't help here.
A good start would be to add record of all patches applied by emerge.
Indeed add it into the epatch command.


> Whilst the above is largely targeting GPL type licences, are there
> other things I should consider for other licences?

Yes. You obviously need to adhere to all licenses used by the
packages in your system. :)


> Other things I need to ensure I distribute for GPL?

Read the licenses, really.


> Any pointers to (simple) documentation on how one can be a
> compliant open source citizen..?

It's not simple. You have to learn the requirements of each license
and see if and how they allow themselves to be combined. There are
businesses doing exactly that. If you want to DIY I think you just
have to start by reading the licenses. You may or may not want an
IP lawyer sitting beside you while doing it.


//Peter

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Ed W]

Hi

> It's not simple. You have to learn the requirements of each license
> and see if and how they allow themselves to be combined. There are
> businesses doing exactly that. If you want to DIY I think you just
> have to start by reading the licenses. You may or may not want an
> IP lawyer sitting beside you while doing it.

This is the kind of unhelpful answer that I can find plenty of examples 
of through google...

Consider that all software comes with some kind of licence.  Generally 
if you ask a non opensource company about licensing costs then even the 
sales droid can help you out. I do find it quite baffling that on 
average if you question an opensource user then their answer on 
licensing is that one should redirect the question to one of the most 
expensive and opaque professions on earth...  If your mate gave you that 
answer in the pub when you asked what price for a beer you would 
immediately cotton on that they don't really know and are bluffing...

The bit people seem to miss is that legal documents are for forcing 
arbitration in the event of dispute - in the meantime people are 
supposed to rub along in a cooperative manner.  That many OSS advocates 
seem to feel that employing expensive lawyers is the only way to talk to 
them shows that they are probably missing the bigger picture...

On a more constructive note: I think I do understand the key terms of 
the main software licences we use, from my understanding they are not 
all that onerous.  So can we perhaps move this topic onto tips, 
suggestions and practical matters about moving forward?  I'm not sure 
that one of the most expensive type of lawyers is best employed talking 
scripting tips?

> If you have patches which use a different license than the package
> they modify then you have more work to do. Portage doesn't help here.
> A good start would be to add record of all patches applied by emerge.
> Indeed add it into the epatch command.

OK, so this is what I asked the list.  Please don't turn it back at me...

Firstly can we not assume that the patches in gentoo *are* in 
compliance, otherwise gentoo's various packaged binaries would cause 
Gentoo to be out of compliance?  (I'm going to assume that human error 
will cause at least some mistakes, but lets hope that just like Gentoo 
isn't being sued right now, copyright holders are actually going to be 
cooperative in fixing minor issues...!)


So, back to the problem: one of the bigger challenges seems to be how to 
actually capture the absolute list of patches applied?  Any 
suggestions?  I already suggested creating my own "patch" utility which 
saves it's input - seems ugly - other suggestions?

I'm not using catalyst.  Any tips from others on capturing, presenting, 
managing and deploying GPL code?

Hoping for useful answers here rather than "talk to some really 
expensive professional who knows nothing about programming".

Gentoo seems very attractive for building embedded system - however, 
there seem to be some missing steps to help with deployment.  I thought 
that was ontopic for this list?  Any tips from others who are building 
things?


Cheers

Ed W

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Peter Stuge]

Ed W wrote:
>> It's not simple. You have to learn the requirements of each license
>> and see if and how they allow themselves to be combined. There are
>> businesses doing exactly that. If you want to DIY I think you just
>> have to start by reading the licenses. You may or may not want an
>> IP lawyer sitting beside you while doing it.
>
> This is the kind of unhelpful answer that I can find plenty of examples of 
> through google...
>
> Consider that all software comes with some kind of licence.  Generally if 
> you ask a non opensource company about licensing costs then even the sales 
> droid can help you out. I do find it quite baffling that on average if you 
> question an opensource user then their answer on licensing is that one 
> should redirect the question to one of the most expensive and opaque 
> professions on earth...

That's not what I wrote, I wrote that *you* should read the licenses.

I wouldn't have mentioned an IP lawyer at all had it not been for the
fact that I know that you are in the US. :)

Lawyer or not depends on how much self education one is interested
in. I've given open source law advice to customers and have had some
contact guiding IP lawyers to better understanding. But it really is
a big field, and you weren't clear on if you already had a good
understanding of the requirements in the various licenses.


> If your mate gave you that answer in the pub when you asked what
> price for a beer you would immediately cotton on that they don't
> really know and are bluffing...

No bluff.


> The bit people seem to miss is that legal documents are for forcing 
> arbitration in the event of dispute - in the meantime people are supposed 
> to rub along in a cooperative manner.  That many OSS advocates seem to feel 
> that employing expensive lawyers is the only way to talk to them shows that 
> they are probably missing the bigger picture...

Mh. A pretty picture, but the reason IP lawyers can live off open
source alone is that there are people who do not and do not want to
understand the licenses themselves. They may or may not desire to
behave well. If they do want to behave well, they may want IP lawyers
to not have to learn themselves, or because they don't trust
themselves or their understanding of the legal system. If they don't
want to behave well the IP lawyers will be working for the opposition
and have a job hunting them down. :)

I do not and have never advocated immediately talking to a lawyer
over first trying to understand licenses oneself.

That said, the legal systems of the world are such that it actually
doesn't matter what the own understanding is, the only thing that
matters is the opinions of the legal systems. And that's where the
lawyers have experience.

OTOH, I think that by now, there are enough documented cases that
allow also developers themselves to understand the issues if they
want to, and I very much encourage this.


> On a more constructive note: I think I do understand the key terms of the 
> main software licences we use, from my understanding they are not all that 
> onerous.

Sounds good. It is important to have gotten this right, since it is
what drives everything else.


> So can we perhaps move this topic onto tips, suggestions and 
> practical matters about moving forward?  I'm not sure that one of
> the most expensive type of lawyers is best employed talking
> scripting tips?

No, and I never said they were.


>> If you have patches which use a different license than the package
>> they modify then you have more work to do. Portage doesn't help here.
>> A good start would be to add record of all patches applied by emerge.
>> Indeed add it into the epatch command.
>
> OK, so this is what I asked the list.  Please don't turn it back at me...
>
> Firstly can we not assume that the patches in gentoo *are* in compliance, 
> otherwise gentoo's various packaged binaries would cause Gentoo to be out 
> of compliance?

Hm, this last sentence is inconsistent, but I guess it should be
without the first "not" based on the rest you write.

If you dare assume that all patches use the same license as the
package they apply to, then I would say that it's actually easy
to do an inventory of the packages and licenses your system uses.

The package/license inventory is the first step.


> So, back to the problem: one of the bigger challenges seems to be how to 
> actually capture the absolute list of patches applied?  Any suggestions?

I think you will have to extend portage. You can have a look at PMS
(emerge app-doc/pms) to find what is currently possible. I think the
A environment variable (Table 12.1 page 46) is the closest to what
you want, but it does not seem to cover patches.


> I already suggested creating my own "patch" utility which saves it's
> input - seems ugly - other suggestions?

I gave it already in the last mail; you should modify the epatch
function to also do some bookkeeping.


> I'm not using catalyst.

Ok, then you have more manual work to do, because catalyst already
does some of the things required by e.g. GPL for you, or at least it
makes them easier to do.


> Any tips from others on capturing, presenting, managing and
> deploying GPL code?

This is a little like asking "Any tips from others on building a car
from scratch?" It's not a very practical question; it's much too
broad. I think it would be good to focus on something specific.


> Hoping for useful answers here rather than "talk to some really
> expensive professional who knows nothing about programming".

The programming involved is trivial IMO, although there are of course
pricey commercial products for software inventory and license
management out there. The difficult part is to understand the legal
requirements that create technical requirements for your distribution
of open source software.

That process obviously has nothing to do with programming, and if you
need legal advice it really is a good idea to talk to lawyers, not
programmers.

At the same time, I do advocate studying and discussing licenses.
They are not magical, I actually find them pretty straightforward.


> Gentoo seems very attractive for building embedded system - however, there 
> seem to be some missing steps to help with deployment.  I thought that was 
> ontopic for this list?  Any tips from others who are building things?

I use catalyst, and I control what gets deployed with custom ebuilds
and snapshots. The fewer packages in the final system the better;
less stuff to track.


//Peter

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Ed W]

Hi

> I wouldn't have mentioned an IP lawyer at all had it not been for the
> fact that I know that you are in the US. :)

I'm in the UK

> I use catalyst, and I control what gets deployed with custom ebuilds
> and snapshots. The fewer packages in the final system the better;
> less stuff to track.

Whilst I guess it should be possible to tear apart catalyst and find out 
how they do it, does anyone happen to know or have a heads up on the 
code for catalyst?  It must be a solved problem so I should think others 
have solved this in various ways?

Thanks

Ed W

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Peter Stuge]

Hey,

Ed W wrote:
>> I wouldn't have mentioned an IP lawyer at all had it not been for the
>> fact that I know that you are in the US. :)
>
> I'm in the UK

Ha! Awesome. :) Sorry, must have mixed you up then!


>> I use catalyst, and I control what gets deployed with custom ebuilds
>> and snapshots. The fewer packages in the final system the better;
>> less stuff to track.
>
> Whilst I guess it should be possible to tear apart catalyst and find out 
> how they do it, does anyone happen to know or have a heads up on the code 
> for catalyst?

The catalyst code has no part in this, but it takes a portage snapshot
as one of it's inputs, and if you maintain a custom snapshot (with
only packages you need) then you know what gets used.


> It must be a solved problem so I should think others have solved
> this in various ways?

I'm not sure it is a solved problem. If you want a different solution
than basically maintaining your own portage snapshot then the easiest
way to track patches is (third time now) to add bookkeeping in the
epatch function.


//Peter

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[wireless]

On 02/29/12 09:46, Ed W wrote:

> Whilst the above is largely targeting GPL type licences, are there other
> things I should consider for other licences? Other things I need to
> ensure I distribute for GPL? Any pointers to (simple) documentation on
> how one can be a compliant open source citizen..?

Ed,

It maybe worth the effort to ask your questions to other embedded lists
too, as my reading of all of these responses, makes me wonder, has not
someone else already discovered and publish a list at some point in time.

For example maybe at Linux From Scratch they advise on what softwares
and codes to use, depending on what you are building up. Or maybe
open embedded ?

It just seems like this question should be solved and already documented
somewhere? With dozens (hundreds) of commercial linux distros, surely
they list licenses for codes they include therein?

Maybe some research on what google has published on the licenses it 
encounters via the Android packages? Some of the BSD embedded
projects might also be a good source of information.

hth,
James

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Peter Stuge]

wireless wrote:
> It just seems like this question should be solved and already documented
> somewhere? With dozens (hundreds) of commercial linux distros, surely
> they list licenses for codes they include therein?

The license question is easy to answer using what goes into /var/db/pkg.

The next step is obviously to make sure that you are in compliance.
Part of that means being able to sometimes provide source code for
the binaries you have distributed. If those binaries include patches
added by portage build scripts then it's not sufficient to provide
the upstream tarball.


//Peter

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Ed W]

Hi

>> Whilst I guess it should be possible to tear apart catalyst and find out
>> how they do it, does anyone happen to know or have a heads up on the code
>> for catalyst?
> The catalyst code has no part in this, but it takes a portage snapshot
> as one of it's inputs, and if you maintain a custom snapshot (with
> only packages you need) then you know what gets used.
>

But not all the patches are in the portage tree?  Trivial example might 
be the kernel where the ebuild is tiny and references an http location 
for the patches?  My understanding is that for a GPL licence one should 
provide a copy of these patches in the "code dump", not just an http 
link?  Is that your understanding?

So by implication it's not clear that catalyst does satisfy your GPL 
requirements for distribution?

I suspect something more is probably happening, eg some of the linked 
patches probably get included into the source download location and 
probably you can pick them up there - however, there are now a LOT of 
ways to fetching source and patches and it would be hard to be sure of 
100% coverage?

Has someone done some actual probing on this?  Peter what does catalyst 
provide for say gcc/kernel sources in it's source output?  All the patches?

Cheers

Ed W

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Peter Stuge]

Ed W wrote:
>>> Whilst I guess it should be possible to tear apart catalyst and find out
>>> how they do it, does anyone happen to know or have a heads up on the code
>>> for catalyst?
>>
>> The catalyst code has no part in this, but it takes a portage snapshot
>> as one of it's inputs, and if you maintain a custom snapshot (with
>> only packages you need) then you know what gets used.
>
> But not all the patches are in the portage tree?  Trivial example might
> be the kernel where the ebuild is tiny and references an http location
> for the patches?

Then you would change the kernel ebuild in your snapshot, so that it
becomes self-contained.

For the specific example of the kernel you could of course just pick
vanilla-sources, but the issue is real.


> My understanding is that for a GPL licence one should provide a
> copy of these patches in the "code dump", not just an http link?
> Is that your understanding?

I think your understanding is incomplete, and I recommend that you
read through the license again.

There isn't just a single way to provide the source, but yes, if you
have downloaded and included a patch in your binary, then you have to
provide that patch yourself, because if you refer to someone else and
they stop providing the patch you would no longer be in compliance.


> So by implication it's not clear that catalyst does satisfy your GPL
> requirements for distribution?

I never say it did. I said that it helps with some things.


> I suspect something more is probably happening, eg some of the linked 
> patches probably get included into the source download location and 
> probably you can pick them up there - however, there are now a LOT of
> ways to fetching source and patches and it would be hard to be sure
> of 100% coverage?

Fourth time: Add bookkeeping into the epatch function.

Downloading is irrelevant, especially since sometimes many more
patches are downloaded than are actually applied.


> Has someone done some actual probing on this?  Peter what does catalyst 
> provide for say gcc/kernel sources in it's source output?  All the patches?

It's the other way around:

You provide a snapshot to catalyst, and catalyst builds kernel from
that. You say what you want catalyst to build, and you create the
package.

You may end up doing more ebuild maintenance, but you likely want to
do just that anyway, in order to keep track of what actually goes
into your system.


//Peter

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1075 bytes --]

On Wednesday 29 February 2012 09:46:57 Ed W wrote:
> In particular I guess simply capturing the ebuilds is not sufficient and
> it's necessary to capture and distribute all the source and patch files
> used to create a build.  The emerge tool doesn't obviously give a way to
> capture this stuff.

file a bug report to add a feature to do this ... something like "buildsrcpkg".  
it'd automatically bundle up all the eclasses the pkg is using as well as all 
of $CATEGORY/$PN/.

> At the moment I'm using a bashrc file to grab everything from the build
> directory.  This seems reasonably robust for source files.  However, for
> patches I have considered creating a fake patch utility which would
> record all the files it operates on.  Any other suggestions?  Perhaps
> catalyst already has done something like that - not familiar with it
> though?

if you capture all of the $PORTDIR/$CATEGORY/$PN/ and $A, then there should be 
no need to manually hook into epatch to capture the patches.  there's really 
no other place these could come from.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Peter Stuge]

[-- Attachment #1: Type: text/plain, Size: 392 bytes --]

Mike Frysinger wrote:
> if you capture all of the $PORTDIR/$CATEGORY/$PN/ and $A, then
> there should be no need to manually hook into epatch

The point of hooking into epatch would be to only have exactly those
patches which get applied. Some ebuilds come with a huge set of
patches, but only few may be applied depending on USE and version.
It's nice to have just the right ones.


//Peter

[-- Attachment #2: Type: application/pgp-signature, Size: 190 bytes --]

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Bertrand Jacquin]

On 02.03.2012 15:35, Peter Stuge wrote:
> Mike Frysinger wrote:
>> if you capture all of the $PORTDIR/$CATEGORY/$PN/ and $A, then
>> there should be no need to manually hook into epatch
>
> The point of hooking into epatch would be to only have exactly those
> patches which get applied. Some ebuilds come with a huge set of
> patches, but only few may be applied depending on USE and version.
> It's nice to have just the right ones.

epatch is not the only necessary thing, some ebuilds do 'sed -i' on 
files. I don't really know for autotools files.

An extension about Mike mind can be an automagic diff between all 
SRC_URI freshly src_unpack(ed) and after install+distclean or after 
src_prepare. Assuming that not any other code is modified during 
src_(compile|install).

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Ed W]

Hi

>> But not all the patches are in the portage tree?  Trivial example might
>> be the kernel where the ebuild is tiny and references an http location
>> for the patches?
> Then you would change the kernel ebuild in your snapshot, so that it
> becomes self-contained.

That's clearly not a practical suggestion because there are many such 
ebuilds with this behaviour and the suggestion to "rewrite all your 
ebuilds" kind of defeats the benefit of using gentoo?


>> My understanding is that for a GPL licence one should provide a
>> copy of these patches in the "code dump", not just an http link?
>> Is that your understanding?
> I think your understanding is incomplete, and I recommend that you
> read through the license again.

?? Why all the stupid hints rather than just stating the answer!

Under what circumstances do you claim that it's not necessary to 
actually supply the code for a patch which has been made to a GPL 
licenced code base??  I think you are implying that it's satisfactory to 
"supply" code by having a twisted and nested chain of source locations 
for all the code, some of which may not be under my control? As you 
hint, I then have the risk of servers outside of my control causing my 
compliance failure.  However, this is all moot because my whole question 
is about accurately capturing all the upstream source so that I can 
maintain my own cache?


I'm not sure why GPL seems to attract such special behaviour.  In every 
other industry one will usually provide both a legal licence and also a 
non legal "summary of intent".  For some reason the open source 
advocates seem to excel in leaping on any minor misunderstanding of 
their licensing agreements, but then enjoy confounding the situation 
with "nah that's not it, but I can't give you any hints as to why I 
*think* you are wrong...".  Look it's just a straightforward licence - 
we don't need to be lawyers to have a stab at complying with it and 
generally helping with understanding it's nuances...

The big thing which annoys me is that one can comply with to the letter 
of the GPL with a big code dump that, and lets be honest here, benefits 
absolutely no one really (what do you do with a lump of undocumented and 
obfuscated hacky code.  There are several open letters on the internet 
discussing this, but what you are looking for is people to get involved 
with the *spirit* of working within the open source process and sharing 
in a useful way, not just code dumping.

The piece we are discussing here is really the boring compliance piece 
which personally I think is largely unhelpful, last chance saloon kind 
of code dump.  All the useful pieces of code I try to push upstream.  
For sure the GPL provision at least means you get the code even if *I* 
don't try and push it upstream and am uncooperative, but really, for the 
vast majority of code, it's just boiler plate reproduction of stuff that 
you would get from upstream if you needed it anyway...


>> So by implication it's not clear that catalyst does satisfy your GPL
>> requirements for distribution?
> I never say it did. I said that it helps with some things.

What "some things"?  Previously I asked for help capturing the source 
code tree and you implied that it would be correctly captured by 
catalyst - however, now it seems to be becoming clear that catalyst 
doesn't capture all the patches either?  So we seem to be back to the 
original question again and catalyst seem to be just a detour that 
hasn't advanced us?

With that in mind if you are using only catalyst, how do *you* make sure 
you are GPL compliant and provide all patches/sources, etc?  (Not a 
challenge, just genuinely trying to learn from how others are doing things?)



>> I suspect something more is probably happening, eg some of the linked
>> patches probably get included into the source download location and
>> probably you can pick them up there - however, there are now a LOT of
>> ways to fetching source and patches and it would be hard to be sure
>> of 100% coverage?
> Fourth time: Add bookkeeping into the epatch function.

No, it's not "fourth time".  It was my idea in the original email!  
However, patching portage is unsatisfactory in that it's fragile and 
easily overwritten accidently.  By all means if you have a way of 
patching which is less fragile, eg if there is some way to patch the 
eclass using some overlay in /usr/local/portage then I would be grateful 
for *that* information.

you are just saying "do it" like having the idea is the easy bit!  
Actually the implementation seems hacky to me.  Wrapping the patch 
utility seems more robust to me, but it's still not ideal...

> Downloading is irrelevant, especially since sometimes many more
> patches are downloaded than are actually applied.

I'm not sure I follow?  My understanding is that we need to supply 
patches that are applied, not just every patch to every ebuild - I think 
we are agreeing on that?


> It's the other way around:
>
> You provide a snapshot to catalyst, and catalyst builds kernel from
> that. You say what you want catalyst to build, and you create the
> package.
>
> You may end up doing more ebuild maintenance, but you likely want to
> do just that anyway, in order to keep track of what actually goes
> into your system.

Hmm, that's a very superficial description of what is done, but I can 
infer some of what you mean.

You might be saying that you figure out every ebuild that you need in 
your solution, then manually patch them all to use source pulled down 
from your own server, plus sync all the sources from gentoo to yoru own 
server?  However, this seems like a desperate amount of work?

You might be saying you just snapshot the gentoo portage tree, however, 
I don't see how that helps you capture all the sources and patches 
correctly?


Can you please clarify how you generate your portage snapshot for 
catalyst and how you create your own offline snapshot of all sources 
(including downloaded patches) - this is I think is what I'm looking to 
learn?

Thanks

Ed W

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Ed W]

[-- Attachment #1: Type: text/plain, Size: 2003 bytes --]

On 02/03/2012 15:22, Bertrand Jacquin wrote:
> On 02.03.2012 15:35, Peter Stuge wrote:
>> Mike Frysinger wrote:
>>> if you capture all of the $PORTDIR/$CATEGORY/$PN/ and $A, then
>>> there should be no need to manually hook into epatch
>>
>> The point of hooking into epatch would be to only have exactly those
>> patches which get applied. Some ebuilds come with a huge set of
>> patches, but only few may be applied depending on USE and version.
>> It's nice to have just the right ones.
>
> epatch is not the only necessary thing, some ebuilds do 'sed -i' on 
> files. I don't really know for autotools files.

Hmm, this is an interesting thought.

My instinct would be to consider this under the heading of "build 
recipe" since it's arguably similar to what the makefile and other 
pre-processors are doing.  I don't disagree that someone could argue 
this all kinds of ways, but I think you would have to be fairly bloody 
minded to try for an infringement claim if the ebuild were provided 
(since arguably the patch is there)?

It also occurs to me that it's safer to include all of $ FILESDIR, or at 
least everything without a .patch extension, since there is also "cp 
$FILESDIR/somefile $S" to watch for?

> An extension about Mike mind can be an automagic diff between all 
> SRC_URI freshly src_unpack(ed) and after install+distclean or after 
> src_prepare. Assuming that not any other code is modified during 
> src_(compile|install).

A very simple solution is to diff the code - however, I claim this is 
*incredibly* unhelpful to the whole notion of open source.  If I were 
the copyright holder then I would far rather have a cooperative, but 
accidentally non compliant distributor who has genuinely made an effort, 
than someone who just provides a massive code diff...  (Yeah, probably 
some corner case in that argument, but the point is that patches like 
"fix segfault on touching some file" are infinitely more useful than a 
massive diff...)


Thanks for the feedback

Ed W


[-- Attachment #2: Type: text/html, Size: 3025 bytes --]

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Ed W]

On 02/03/2012 06:37, Mike Frysinger wrote:
> On Wednesday 29 February 2012 09:46:57 Ed W wrote:
>> In particular I guess simply capturing the ebuilds is not sufficient and
>> it's necessary to capture and distribute all the source and patch files
>> used to create a build.  The emerge tool doesn't obviously give a way to
>> capture this stuff.
> file a bug report to add a feature to do this ... something like "buildsrcpkg".
> it'd automatically bundle up all the eclasses the pkg is using as well as all
> of $CATEGORY/$PN/.
>

Submitted

https://bugs.gentoo.org/show_bug.cgi?id=406811

Thanks

Ed W

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Todd Goodman]

* Ed W <lists@wildgooses.com> [120303 13:29]:
> Hi
> 
[..]

> >> My understanding is that for a GPL licence one should provide a
> >> copy of these patches in the "code dump", not just an http link?
> >> Is that your understanding?
> > I think your understanding is incomplete, and I recommend that you
> > read through the license again.
> 
> ?? Why all the stupid hints rather than just stating the answer!

I'm sure it's just frustration but your emails really make you sound
like an a-hole with a sense of entitlement.

They're legal licenses.  As with anything involved with lawyers and the
legal system you really need to decide for yourself what needs to be
done (most people to be safe would contact a lawyer.)

If you're at a company releasing a product then the company most likely
has a legal dept or legal consultant.  They certainly would here in the
US (I know you said you're not in the US so perhaps that's not the
case.)

Have you looked at http://www.gnu.org/licenses/gpl-faq.html?

I'm not a lawyer (by a far shot) but what's the problem with creating a
script that when run pulls the upstream files from
/usr/portage/distfiles, the files and ebuilds from /usr/portage for
whatever packages you have installed on whatever you're releasing?

If I were releasing commercial software I'd want all that on a local
mirror (in source control too) so that I could recreate any released
versions.

Todd

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1144 bytes --]

On Thursday 01 March 2012 14:05:36 Peter Stuge wrote:
> Ed W wrote:
> > My understanding is that for a GPL licence one should provide a
> > copy of these patches in the "code dump", not just an http link?
> > Is that your understanding?
> 
> I think your understanding is incomplete, and I recommend that you
> read through the license again.
> 
> There isn't just a single way to provide the source, but yes, if you
> have downloaded and included a patch in your binary, then you have to
> provide that patch yourself, because if you refer to someone else and
> they stop providing the patch you would no longer be in compliance.

Peter's understanding seems to match my own.  pointing someone to a URL that 
provides the source satisfies the GPL requirements.  obviously if the linked 
source is incomplete or outdated, that's another matter, but if the full 
source is there, then the obligations have been met.

alternatively, you could create a bundle of all the sources and provide it 
directly.  companies tend to do this more because they modify the releases 
directly rather than a cleaner build approach.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-embedded] Licence compliance - capturing all source files used to make a build?

[Ed W]

Hi

> They're legal licenses.  As with anything involved with lawyers and the
> legal system you really need to decide for yourself what needs to be
> done (most people to be safe would contact a lawyer.)

Thanks for your feedback - however, again you are mentioning lawyers.  
My understanding is that even in the USA, lawyers aren't the best people 
write build scripts?

The (main) question is a technical one, not a legal one.  I think we are 
all clear that the general provision is to supply all code to downstream 
users.  If you check my original question, I'm looking for technical 
suggestions to wrap portage and ensure that all patches, source files, 
etc are supplied in a neat and useful form (I'm trying to avoid the big 
code dump and keep things as well documented as possible)

That said I'm grateful for the clarifications on interpretations of the 
GPL such as with regards to linking to other sources of code.  Thanks 
(Mike/Peter?)

> If you're at a company releasing a product then the company most likely
> has a legal dept or legal consultant.  They certainly would here in the
> US (I know you said you're not in the US so perhaps that's not the
> case.)

In the UK you would generally be quite a large company to have an 
expensive person such as a lawyer on your book full time (or have some 
special reason to be needing one every day...). Generally we just rent 
them as they are needed.  Same also for accountants, you only need them 
once a year, so rent them rather than owning them full time...


> I'm not a lawyer (by a far shot) but what's the problem with creating a
> script that when run pulls the upstream files from
> /usr/portage/distfiles, the files and ebuilds from /usr/portage for
> whatever packages you have installed on whatever you're releasing?

Please see original question.  Yes, this is basically what I'm trying to do

However, it's not quite as straightforward as you state.  For a build 
process you would need to clear distfiles at some point, then scrape it 
later in order to infer what some package had used.

However, yes, you are on the right lines for the kind of thing I'm 
looking for.  Note that it's the corner cases which are important, hence 
the reason for my question (I really don't want to learn about how many 
lawyers every US company owns...)


> If I were releasing commercial software I'd want all that on a local
> mirror (in source control too) so that I could recreate any released
> versions.

Agreed.  See - my question was sensible!  I'm using git for all that 
right now.

However, I think you are being a bit handwaving about the details.  For 
example, how would you handle this in practice, for example we need to 
clean distfiles before we start building a fresh image otherwise we 
don't know what is new, on the flip side we don't want to download some 
existing file which is unchanged and already in source control?  So a 
kind of two level distfiles would be helpful...

At the moment I'm tackling it a slightly different way by grabbing $A 
and the ./files dir during the build, via a bashrc script (basically as 
suggested by Mike Frysinger).  I think I'm coming to the conclusion that 
this is the most complete solution, but obviously grabs additional 
pieces that might not be used.

However, please, if anyone else has tackled this and has some notes then 
please share?  Some technical challenges have been exposed from some of 
the answers so far.

Cheers

Ed W