[gentoo-dev] FYI: Rules for distro-friendly packages

[gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

Hi folks,


I'm currently collecting a set of rules which upstream developers
should follow to make distro maintainer's life easier.

Comments welcomed :)


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Alistair Bush]

> Hi folks,
> 
> 
> I'm currently collecting a set of rules which upstream developers
> should follow to make distro maintainer's life easier.
> 
> Comments welcomed :)
> 

Is this language specific?  would you be interested in comments about java, 
ruby, python, etc, etc, etc or are you only interested in good old C/C++, etc

> 
> cu

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Alistair Bush <ali_bush@gentoo.org> schrieb:

> Is this language specific?  

I'll try to separate it into generic and language specific 
rules step by step (same for various build systems, etc).

> would you be interested in comments about java, ruby, python, 
> etc, etc, etc or are you only interested in good old C/C++, etc

Just give me everything you've got :)


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Krzysztof Pawlik]

[-- Attachment #1: Type: text/plain, Size: 497 bytes --]

On 06/25/10 21:17, Enrico Weigelt wrote:
> I'm currently collecting a set of rules which upstream developers
> should follow to make distro maintainer's life easier.
> 
> Comments welcomed :)

Take a look at this page:
http://overlays.gentoo.org/proj/java/wiki/How_to_be_a_good_upstream - it is Java
specific mostly, but some general points can be reused :)

-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, apache, ppc, vim, kernel, python...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Krzysztof Pawlik <nelchael@gentoo.org> schrieb:

> Take a look at this page:
> http://overlays.gentoo.org/proj/java/wiki/How_to_be_a_good_upstream - it is Java
> specific mostly, but some general points can be reused :)

Hmm, this document suggests something, I just forgot to prohibit:

"Release the source archives along with whatever binary archives you may have."
                                                ^^^^^^^^^^^^^^^^^

cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Krzysztof Pawlik]

[-- Attachment #1: Type: text/plain, Size: 1146 bytes --]

On 06/26/10 19:51, Enrico Weigelt wrote:
> * Krzysztof Pawlik <nelchael@gentoo.org> schrieb:
> 
>> Take a look at this page:
>> http://overlays.gentoo.org/proj/java/wiki/How_to_be_a_good_upstream - it is Java
>> specific mostly, but some general points can be reused :)
> 
> Hmm, this document suggests something, I just forgot to prohibit:
> 
> "Release the source archives along with whatever binary archives you may have."
>                                                 ^^^^^^^^^^^^^^^^^

You intend to "prohibit" releasing binary packages for upstream? Thats...
something you don't see everyday. Why would you do that?

It's quite common for Java projects to release pre-built JAR files or for C/C++
(or other) based projects to release debs or rpms - it's standard practice.

What's important in quoted sentence is: "Release the source archives along with
binary archives" - the "with" is very important - in short: don't release only
binary versions and force us to grab them from VCS.

-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, apache, ppc, vim, kernel, python...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Krzysztof Pawlik <nelchael@gentoo.org> schrieb:

> > Hmm, this document suggests something, I just forgot to prohibit:
> > 
> > "Release the source archives along with whatever binary archives you may have."
> >                                                 ^^^^^^^^^^^^^^^^^
> 
> You intend to "prohibit" releasing binary packages for upstream? 

No, I'm talking about precompiled stuff within the source tree.
Such things should _never ever_ happen.

In Java world it seems to be quite common to bundle precompiled 
imported libraries within the source tree. Needless to say that
this can easily get hell for any package-based distro.
(actually, that mostly comes from the windows front, where 
concepts like package management are quite unknown ... ;-o)

> What's important in quoted sentence is: "Release the source archives along with
> binary archives" - the "with" is very important - in short: don't release only
> binary versions and force us to grab them from VCS.

Okay, perhaps I misunderstood this sentence.

BTW: if upstream has an proper VCS and an canonical tagging 
scheme, they don't actually have to create release tarballs,
just hack up a little script which creates them on-the-fly
from an canonical URL scheme (eg. oss-qm does exactly that).


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 567 bytes --]

On Sat, 26 Jun 2010 21:46:39 +0200
Enrico Weigelt <weigelt@metux.de> wrote:
> BTW: if upstream has an proper VCS and an canonical tagging 
> scheme, they don't actually have to create release tarballs,
> just hack up a little script which creates them on-the-fly
> from an canonical URL scheme (eg. oss-qm does exactly that).

Down that path lies madness. There's no guarantee that you'll get the
same tarball if you request the same URL twice in a row, particularly
if you're using one of those new-fangled new compression schemes.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Krzysztof Pawlik]

[-- Attachment #1: Type: text/plain, Size: 805 bytes --]

On 06/26/10 20:59, Ciaran McCreesh wrote:
> On Sat, 26 Jun 2010 21:46:39 +0200
> Enrico Weigelt <weigelt@metux.de> wrote:
>> BTW: if upstream has an proper VCS and an canonical tagging 
>> scheme, they don't actually have to create release tarballs,
>> just hack up a little script which creates them on-the-fly
>> from an canonical URL scheme (eg. oss-qm does exactly that).
> 
> Down that path lies madness. There's no guarantee that you'll get the
> same tarball if you request the same URL twice in a row, particularly
> if you're using one of those new-fangled new compression schemes.

I agree with Ciaran here, to add one more thing: tags can be mutable.

-- 
Krzysztof Pawlik  <nelchael at gentoo.org>  key id: 0xF6A80E46
desktop-misc, java, apache, ppc, vim, kernel, python...


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 554 bytes --]

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Krzysztof Pawlik <nelchael@gentoo.org> schrieb:

> > Down that path lies madness. There's no guarantee that you'll get the
> > same tarball if you request the same URL twice in a row, particularly
> > if you're using one of those new-fangled new compression schemes.
> 
> I agree with Ciaran here, to add one more thing: tags can be mutable.

Thats a matter of proper VCS configuration (hooks, etc)


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Ciaran McCreesh <ciaran.mccreesh@googlemail.com> schrieb:
> On Sat, 26 Jun 2010 21:46:39 +0200
> Enrico Weigelt <weigelt@metux.de> wrote:
> > BTW: if upstream has an proper VCS and an canonical tagging 
> > scheme, they don't actually have to create release tarballs,
> > just hack up a little script which creates them on-the-fly
> > from an canonical URL scheme (eg. oss-qm does exactly that).
> 
> Down that path lies madness. There's no guarantee that you'll get the
> same tarball if you request the same URL twice in a row, particularly
> if you're using one of those new-fangled new compression schemes.

Well, with git this works. (I'll yet have to run some automatic
stress tests, but at all my manual tests worked really fine).
*If* this might not work somewhere, the little generator script
could create the tarball only once (on first request) and use
it all the time. Or add some commit/push hook, etc, etc.


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 379 bytes --]

On Sat, 26 Jun 2010 22:09:09 +0200
Enrico Weigelt <weigelt@metux.de> wrote:
> Well, with git this works. (I'll yet have to run some automatic
> stress tests, but at all my manual tests worked really fine).

You assume that, given the same input and program options, a
compression program will always produce the same output. This is not
the case.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Ciaran McCreesh <ciaran.mccreesh@googlemail.com> schrieb:
> On Sat, 26 Jun 2010 22:09:09 +0200
> Enrico Weigelt <weigelt@metux.de> wrote:
> > Well, with git this works. (I'll yet have to run some automatic
> > stress tests, but at all my manual tests worked really fine).
> 
> You assume that, given the same input and program options, a
> compression program will always produce the same output. This is not
> the case.

Well, at least for tar, I've experienced no problem here yet.
But: true, it might change between tar versions.

A more sophisticated approach, IMHO, would be: directly fetching
from the VCS, but that would probably mean a major design change.
(my Briegel buildsystem goes this way ... it doesnt even patch
on itself anymore, thats done entirely in git)


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 574 bytes --]

On Sun, 27 Jun 2010 12:34:44 +0200
Enrico Weigelt <weigelt@metux.de> wrote:
> > You assume that, given the same input and program options, a
> > compression program will always produce the same output. This is not
> > the case.
> 
> Well, at least for tar, I've experienced no problem here yet.
> But: true, it might change between tar versions.

The main offender is the compression program, not tar.

> A more sophisticated approach, IMHO, would be: directly fetching
> from the VCS

Yes, upstreams are going to love you for that...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Ciaran McCreesh <ciaran.mccreesh@googlemail.com> schrieb:

> > Well, at least for tar, I've experienced no problem here yet.
> > But: true, it might change between tar versions.
> 
> The main offender is the compression program, not tar.

hmm, I'm exclusively using bzip2 and never had these problems
yet. maybe it depends on the compressor type.
 
> > A more sophisticated approach, IMHO, would be: directly fetching
> > from the VCS
> 
> Yes, upstreams are going to love you for that...

Why ? Because of traffic ? Of course, distros should work 
from their own clones (yes, yes, I'm assuming modern VCS'es, 
not toys like svn ;-p)


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 626 bytes --]

On Sun, 27 Jun 2010 13:08:58 +0200
Enrico Weigelt <weigelt@metux.de> wrote:
> > > Well, at least for tar, I've experienced no problem here yet.
> > > But: true, it might change between tar versions.
> > 
> > The main offender is the compression program, not tar.
> 
> hmm, I'm exclusively using bzip2 and never had these problems
> yet. maybe it depends on the compressor type.

Your problem is that you are thinking in terms of "well I've not seen
it break", not "I can prove that it will always work".

You may find the "Compressed output may vary" section of "man 1 xz" to
be useful.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Brian Harring]

[-- Attachment #1: Type: text/plain, Size: 817 bytes --]

On Sun, Jun 27, 2010 at 01:08:58PM +0200, Enrico Weigelt wrote:
> * Ciaran McCreesh <ciaran.mccreesh@googlemail.com> schrieb:
> 
> > > Well, at least for tar, I've experienced no problem here yet.
> > > But: true, it might change between tar versions.
> > 
> > The main offender is the compression program, not tar.
> 
> hmm, I'm exclusively using bzip2 and never had these problems
> yet. maybe it depends on the compressor type.

http://www.gentoo.org/proj/en/glep/glep-0025.html#the-problem-in-detail

Note also that bzip2 had another change in output after that 
release- my memory is failing me a bit, but it was roughly a 
a reduction of their hash size to fix a CVE- either way, same thing, 
differing output.

It happens, and further, is a well known potential for compressors.

~harring

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Patrick Lauer]

On 06/27/10 20:33, Brian Harring wrote:
> On Sun, Jun 27, 2010 at 01:08:58PM +0200, Enrico Weigelt wrote:
>> * Ciaran McCreesh <ciaran.mccreesh@googlemail.com> schrieb:
>>
>>>> Well, at least for tar, I've experienced no problem here yet.
>>>> But: true, it might change between tar versions.
>>>
>>> The main offender is the compression program, not tar.
>>
>> hmm, I'm exclusively using bzip2 and never had these problems
>> yet. maybe it depends on the compressor type.
> 
> http://www.gentoo.org/proj/en/glep/glep-0025.html#the-problem-in-detail
> 
> Note also that bzip2 had another change in output after that 
> release- my memory is failing me a bit, but it was roughly a 
> a reduction of their hash size to fix a CVE- either way, same thing, 
> differing output.
Are you thinking about
"bzip2 version 1.0.2 worked with huffman codes with a length of up to 20
bits. Unfortunately the Author of bzip2, Julian Seward, changed this in
the new version 1.0.3 - it uses only a maximum length of 17 bits now. "
(Quoted from http://linux01.gwdg.de/~nlissne/bugsandproblems.html ) ?

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Brian Harring <ferringb@gmail.com> schrieb:

> > hmm, I'm exclusively using bzip2 and never had these problems
> > yet. maybe it depends on the compressor type.
> 
> http://www.gentoo.org/proj/en/glep/glep-0025.html#the-problem-in-detail
> 
> Note also that bzip2 had another change in output after that 
> release- my memory is failing me a bit, but it was roughly a 
> a reduction of their hash size to fix a CVE- either way, same thing, 
> differing output.

Okay, you've convinced me :)

Meanwhile I've reworked my Briegel buildsystem [1] to support
direct git checkouts (including a repo cache). Next step will be
a mechanism to check tag signatures.


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Jacob Godserv]

[-- Attachment #1: Type: text/plain, Size: 625 bytes --]

On Tue, 17 Aug 2010 19:04:03 +0200
Enrico Weigelt <weigelt@metux.de> wrote:

> Meanwhile I've reworked my Briegel buildsystem [1] to support
> direct git checkouts (including a repo cache). Next step will be
> a mechanism to check tag signatures.

You have a footnote, but no link, and I'm curious. :)

-- 
    Jacob

    "For then there will be great distress, unequaled
    from the beginning of the world until now — and never
    to be equaled again. If those days had not been cut
    short, no one would survive, but for the sake of the
    elect those days will be shortened."

    Are you ready?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Jacob Godserv <jacobgodserv@gmail.com> schrieb:
> On Tue, 17 Aug 2010 19:04:03 +0200
> Enrico Weigelt <weigelt@metux.de> wrote:
> 
> > Meanwhile I've reworked my Briegel buildsystem [1] to support
> > direct git checkouts (including a repo cache). Next step will be
> > a mechanism to check tag signatures.
> 
> You have a footnote, but no link, and I'm curious. :)

ups, I probably thinking and typing got out of sync ;-O

http://www.metux.de/index.php/en/component/content/article/57.html


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Nirbheek Chauhan]

On Sun, Jun 27, 2010 at 1:29 AM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Sat, 26 Jun 2010 21:46:39 +0200
> Enrico Weigelt <weigelt@metux.de> wrote:
>> BTW: if upstream has an proper VCS and an canonical tagging
>> scheme, they don't actually have to create release tarballs,
>> just hack up a little script which creates them on-the-fly
>> from an canonical URL scheme (eg. oss-qm does exactly that).
>
> Down that path lies madness. There's no guarantee that you'll get the
> same tarball if you request the same URL twice in a row, particularly
> if you're using one of those new-fangled new compression schemes.
>

Or if they generate the tarball on-the-fly with no caching, which
results in differing timestamps each time. Hence, each time you fetch
it, you get a tarball with a different hash.

No, don't ask where I saw such a thing :p

-- 
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Nirbheek Chauhan <nirbheek@gentoo.org> schrieb:

> Or if they generate the tarball on-the-fly with no caching, which
> results in differing timestamps each time. Hence, each time you fetch
> it, you get a tarball with a different hash.

Does portage check the timestamps ?


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Nirbheek Chauhan]

On Sun, Jun 27, 2010 at 4:05 PM, Enrico Weigelt <weigelt@metux.de> wrote:
> * Nirbheek Chauhan <nirbheek@gentoo.org> schrieb:
>
>> Or if they generate the tarball on-the-fly with no caching, which
>> results in differing timestamps each time. Hence, each time you fetch
>> it, you get a tarball with a different hash.
>
> Does portage check the timestamps ?
>

I think you need to sit down and understand how tarballs are made,
what they contain, and what a "hash" means, and how it's calculated.


-- 
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Hans de Graaff]

[-- Attachment #1: Type: text/plain, Size: 639 bytes --]

On Sat, 2010-06-26 at 21:46 +0200, Enrico Weigelt wrote:

> BTW: if upstream has an proper VCS and an canonical tagging 
> scheme, they don't actually have to create release tarballs,
> just hack up a little script which creates them on-the-fly
> from an canonical URL scheme (eg. oss-qm does exactly that).

This breaks badly when the upstream tarball generation is changed. This
happened to github a few months ago and broke all our manifests. To
their credit they quickly reverted what turned out to be a gratuitous
change, but if they ever need to make a real change we get to redigest
everything.

Kind regards,

Hans

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Petteri Räty]

[-- Attachment #1: Type: text/plain, Size: 377 bytes --]

On 06/25/2010 11:17 PM, Enrico Weigelt wrote:
> 
> Hi folks,
> 
> 
> I'm currently collecting a set of rules which upstream developers
> should follow to make distro maintainer's life easier.
> 
> Comments welcomed :)
> 


There should be useful stuff here:
http://video.fosdem.org/2010/devrooms/distributions/How_to_be_a_good_upstream.ogv

Regards,
Petteri


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Petteri Räty <betelgeuse@gentoo.org> schrieb:

> There should be useful stuff here:
> http://video.fosdem.org/2010/devrooms/distributions/How_to_be_a_good_upstream.ogv

#1 he says nothing about that - if upstream has a VCS (and properly 
uses it ;-o) - the distros should use it, so eg. set their branches 
ontop the upstream's release tags instead of manually maintaining
patches against tarballs ...

#2 One point i don't agree is the "dont add -Werror" rule. actually,
i'm thinking of making -Wall and -Werror mandatory. if some
package doenst build fine, it's simply broken. period.

#3 When he's talking about "disabling dependencies", he most likely
has the totally wrong concept in mind (as probably most Gentoo
folks ;-p): you naturally dont want to enable/disable dependencies,
but features (which then imply certain dependencies). Once you 
use terms like "zlib support", you're already on the wrong path.

#4 The discussion about "how to be a good downstream" (started by
the ffmpeg guy) is quite interesting ... that's exactly the 
kind of situations what the OSS-QM (see my recent posts) is for.
(and all goes sooo easy w/ tools like git ;-p)


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 381 bytes --]

On Sat, 26 Jun 2010 21:39:15 +0200
Enrico Weigelt <weigelt@metux.de> wrote:
> #2 One point i don't agree is the "dont add -Werror" rule. actually,
> i'm thinking of making -Wall and -Werror mandatory. if some
> package doenst build fine, it's simply broken. period.

Uhm. No. Certain compilers will give you warnings for f(g(a), g(b)) if
you -Wall.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Ciaran McCreesh <ciaran.mccreesh@googlemail.com> schrieb:
> On Sat, 26 Jun 2010 21:39:15 +0200
> Enrico Weigelt <weigelt@metux.de> wrote:
> > #2 One point i don't agree is the "dont add -Werror" rule. actually,
> > i'm thinking of making -Wall and -Werror mandatory. if some
> > package doenst build fine, it's simply broken. period.
> 
> Uhm. No. Certain compilers will give you warnings for f(g(a), g(b)) if
> you -Wall.

Warn on what exactly ? Which compilers do that ?


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 479 bytes --]

On Sat, 26 Jun 2010 21:57:33 +0200
Enrico Weigelt <weigelt@metux.de> wrote:
> > Uhm. No. Certain compilers will give you warnings for f(g(a), g(b))
> > if you -Wall.
> 
> Warn on what exactly ?

That f's arguments are evaluated in an unspecified order.

> Which compilers do that ?

For all you know, gcc 4.7.

New gcc releases regularly issue lots of new warnings for correct code,
particularly with -Wall. Other compilers are even worse.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-dev] Re: FYI: Rules for distro-friendly packages

[Nikos Chantziaras]

On 06/26/2010 11:12 PM, Ciaran McCreesh wrote:
> On Sat, 26 Jun 2010 21:57:33 +0200
> Enrico Weigelt<weigelt@metux.de>  wrote:
>>> Uhm. No. Certain compilers will give you warnings for f(g(a), g(b))
>>> if you -Wall.
>>
>> Warn on what exactly ?
>
> That f's arguments are evaluated in an unspecified order.
>
>> Which compilers do that ?
>
> For all you know, gcc 4.7.
>
> New gcc releases regularly issue lots of new warnings for correct code,
> particularly with -Wall. Other compilers are even worse.

Did it actually occur to anyone that warnings are not errors?  You can 
have them for correct code.  A warning means you might want to look at 
the code to check whether there's some real error there.  It doesn't 
mean the code is broken.

Re: [gentoo-dev] Re: FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Nikos Chantziaras <realnc@arcor.de> schrieb:

> Did it actually occur to anyone that warnings are not errors?  You can 
> have them for correct code.  A warning means you might want to look at 
> the code to check whether there's some real error there.  It doesn't 
> mean the code is broken.

In my personal experience, most times a warning comes it, the
code *is* broken (but *might* work in most situations).
So, developers should always enable it and think ten times
if some warning is acceptable.

But: you all conviced me. These flags should be controlled by the 
distro buildsystem, not the individual package. Developers just
should take care that there're no preventable warnings.


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

[gentoo-dev] Re: FYI: Rules for distro-friendly packages

[Nikos Chantziaras]

On 06/27/2010 01:47 PM, Enrico Weigelt wrote:
> * Nikos Chantziaras<realnc@arcor.de>  schrieb:
>
>> Did it actually occur to anyone that warnings are not errors?  You can
>> have them for correct code.  A warning means you might want to look at
>> the code to check whether there's some real error there.  It doesn't
>> mean the code is broken.
>
> In my personal experience, most times a warning comes it, the
> code *is* broken (but *might* work in most situations).

That's the key to it: most times.  Granted, without -Wall (or any other 
options that tweaks the default warning level) we can be very sure that 
the warning is the result of a mistake by the developer.  But with 
-Wall, many warnings are totally not interesting ("unused parameter") 
and some even try to outsmart the programmer even though he/she knows 
better ("taking address of variable declared register").  In that last 
example, fixing it would even be wrong when you consider the optimizer 
and the fuzzy meaning of "register" which the compiler is totally free 
to ignore.

Re: [gentoo-dev] Re: FYI: Rules for distro-friendly packages

[Harald van Dijk]

On Sun, Jun 27, 2010 at 02:56:33PM +0300, Nikos Chantziaras wrote:
> On 06/27/2010 01:47 PM, Enrico Weigelt wrote:
> > * Nikos Chantziaras<realnc@arcor.de>  schrieb:
> >
> >> Did it actually occur to anyone that warnings are not errors?  You can
> >> have them for correct code.  A warning means you might want to look at
> >> the code to check whether there's some real error there.  It doesn't
> >> mean the code is broken.
> >
> > In my personal experience, most times a warning comes it, the
> > code *is* broken (but *might* work in most situations).
> 
> That's the key to it: most times.  Granted, without -Wall (or any other 
> options that tweaks the default warning level) we can be very sure that 
> the warning is the result of a mistake by the developer.  But with 
> -Wall, many warnings are totally not interesting ("unused parameter") 
> and some even try to outsmart the programmer even though he/she knows 
> better ("taking address of variable declared register").  In that last 
> example, fixing it would even be wrong when you consider the optimizer 
> and the fuzzy meaning of "register" which the compiler is totally free 
> to ignore.

The compiler is not totally free to ignore the register keyword. Both
the C and the C++ standards require that the compiler complain when
taking the address of a register variable. Other compilers will issue a
hard error for it. Fixing the code to not declare the variable as
register would be the correct thing to do.

Make sure you *understand* warnings, and then you can decide whether to
fix the code, if there is anything to fix, or to ignore.

Re: [gentoo-dev] Re: FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Harald van D??k <truedfx@gentoo.org> schrieb:

> The compiler is not totally free to ignore the register keyword. Both
> the C and the C++ standards require that the compiler complain when
> taking the address of a register variable. Other compilers will issue a
> hard error for it. Fixing the code to not declare the variable as
> register would be the correct thing to do.

BTW: is the register keyword still so useful nowadays ?
Shouldn't a modern (optimizing) compiler be clever enough to 
find out when it can use registers instead of RAM ?

> Make sure you *understand* warnings, and then you can decide whether to
> fix the code, if there is anything to fix, or to ignore.

hmm, is there a (portable) way to prevent a specific warning
in an specific place ? (some kind of #pragma ?)


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [gentoo-dev] Re: FYI: Rules for distro-friendly packages

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 224 bytes --]

On Sun, 27 Jun 2010 14:25:39 +0200
Enrico Weigelt <weigelt@metux.de> wrote:
> hmm, is there a (portable) way to prevent a specific warning
> in an specific place ? (some kind of #pragma ?)

No.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[gentoo-dev] Re: FYI: Rules for distro-friendly packages

[Nikos Chantziaras]

On 06/27/2010 03:23 PM, Harald van Dijk wrote:
> On Sun, Jun 27, 2010 at 02:56:33PM +0300, Nikos Chantziaras wrote:
>> On 06/27/2010 01:47 PM, Enrico Weigelt wrote:
>>> * Nikos Chantziaras<realnc@arcor.de>   schrieb:
>>>
>>>> Did it actually occur to anyone that warnings are not errors?
>>>> You can have them for correct code.  A warning means you might
>>>> want to look at the code to check whether there's some real
>>>> error there.  It doesn't mean the code is broken.
>>>
>>> In my personal experience, most times a warning comes it, the
>>> code *is* broken (but *might* work in most situations).
>>
>> That's the key to it: most times.  Granted, without -Wall (or any
>> other options that tweaks the default warning level) we can be very
>> sure that the warning is the result of a mistake by the developer.
>> But with -Wall, many warnings are totally not interesting ("unused
>> parameter") and some even try to outsmart the programmer even
>> though he/she knows better ("taking address of variable declared
>> register").  In that last example, fixing it would even be wrong
>> when you consider the optimizer and the fuzzy meaning of "register"
>> which the compiler is totally free to ignore.
>
> The compiler is not totally free to ignore the register keyword.
> Both the C and the C++ standards require that the compiler complain
> when taking the address of a register variable. Other compilers will
> issue a hard error for it. Fixing the code to not declare the
> variable as register would be the correct thing to do.

No, it would not be the correct thing to do, because of the following. 
(This is part of a discussion between me and someone quite smarter than 
me, who explained the issue in detail.)

The basic issue is that the code takes the address of the variable in
question in expressions passed as parameters to certain function calls.
These function calls all happen to be in-linable functions, and it
happens that in each function, the address operator is always canceled
out by a '*' dereference operator - in other words, we have '*&p', which
the compiler can turn into just plain 'p' when the calls are in-lined,
eliminating the need to actually take the address of 'p'.

A compiler is always free to ignore 'register' declarations *anyway*,
even if enregistration is possible.  Therefore a warning that it's not
possible to obey 'register' is unnecessary, because it's explicit in the
language definition that 'register' is not binding.  It simply is not
possible for an ignored 'register' attribute to cause unexpected
behavior.  Warnings really should only be generated for situations where
it is likely that the programmer expects different behavior than the
compiler will deliver; in the case of an ignored 'register' attribute,
the programmer is *required* to expect that the attribute might be
ignored, so a warning to this effect is superfluous.

Now, I understand why they generate the warning - it's because the
compiler believes that the program code itself makes enregistration
impossible, not because the compiler has chosen for optimization
purposes to ignore the 'register' request.  However, as we'll see
shortly, the program code doesn't truly make enregistration impossible;
it is merely impossible in some interpretations of the code.  Therefore
we really are back to the compiler choosing to ignore the 'register'
request due to its own optimization decisions; the 'register' request is
made impossible far downstream of the actual decisions that the compiler
makes (which have to do with in-line vs out-of-line calls), but it
really is compiler decisions that make it impossible, not the inherent
structure of the code.

When a function is in-lined, the compiler is not required to generate
the same code it would generate for the most general case of the same
function call, as long as the meaning is the same.

For example, suppose we have some code that contains a call to a
function like so:

    a = myFunc(a + 7, 3);

In the general out-of-line case, the compiler must generate some
machine-code instructions like this:

    push #3
    mov [a], d0
    add #7, d0
    push d0
    call #myFunc
    mov d0, [a]

The compiler doesn't have access to the inner workings of myFunc, so it
must generate the appropriate code for the generic interface to an
external function.

Now, suppose the function is defined like so:

   int myFunc(int a, int b) { return a - 6; }

and further suppose that the compiler decides to in-line this function.
In-lining means the compiler will generate the code that implements the
function directly in the caller; there will be no call to an external
linkage point.  This means the compiler can implement the linkage to the
function with a custom one-off interface for this particular invocation
- every in-line invocation can be customized to the exact context where
it appears.  So, for example, if we call myFunc right now and registers
d1 and d2 happens to be available, we can put the parameters in d1 and
d2, and the generated function will refer to those registers for the
parameters rather than having to look in the stack.  Later on, if we
generate a separate call to the same function, but registers d3 and d7
are the ones available, we can use those instead.  Each generated copy
of the function can fit its exact context.

Furthermore, looking at this function and at the arguments passed, we
can see that the formal parameter 'b' has no effect on the function's
results, and the actual parameter '3' passed for 'b' has no side
effects.  Therefore, the compiler is free to completely ignore this
parameter - there's no need to generate any code for it at all, since we
have sufficient knowledge to see that it has no effect on the meaning of
the code.

Further still, we can globally optimize the entire function.  So, we can
see that myFunc(a+7, 3) is going to turn into the expression (a+7-6).
We can fold constants to arrive at (a+1) as the result of the function.
We can therefore generate the entire code for the function's invocation
like so:

    inc [a]

Okay, now let's look at the &p case.  In the specific examples in
vmrun.cpp, we have a bunch of function invocations like this:

   register const char *p;
   int x = myfunc(&p);

In the most general case, we have to generate code like this:

   lea [p], d0        ; load effective address
   push d0
   call #myfunc
   mov d0, [x]

So, in the most general case of a call with external linkage, we need
'p' to have a main memory address so that we can push it on the stack as
the parameter to this call.  Registers don't have main memory addresses,
so 'p' can't go in a register.

However, we know what myfunc() looks like:

   char myfunc(const char **p)
   {
       char c = **p;
       *p += 1;
       return c;
   }

If the compiler chooses to in-line this function, it can globally
optimize its linkage and implementation as we saw earlier.  So, the
compiler can rewrite the code like so:

   register const char *p;
   int x = **(&p);
   *(&p) += 1;

which can be further rewritten to:

   register const char *p;
   int x = *p;
   p += 1;

Now we can generate the machine code for the final optimized form:

   mov [p], a0         ; get the *value* of p into index register 0
   mov.byte [a0+0], d0 ; get the value index register 0 points to
   mov.byte d0, [x]    ; store it in x
   inc [p]             ; inc the value of p

do we need a main memory address for p.  This means the compiler
can keep p in a register, say d5:

   mov d5, a0
   mov.byte [a0+0], d0
   mov.byte d0, [x]
   inc d5

And this is indeed exactly what the code that comes out of most 
compilers looks like (changed from my abstract machine to 32-bit x86, of 
course).

So: if the compiler chooses to in-line the functions that are called
with '&p' as a parameter, and the compiler performs the available
optimizations on those calls once they're in-lined, then a memory
address for 'p' is never needed.  Thus there is a valid interpretation
of the code where 'register p' can be obeyed.  If the compiler doesn't
choose to in-line the functions or make those optimizations, then the
compiler will be unable to satisfy the 'register p' request and will be
forced to put 'p' in addressable main memory.  But it really is entirely
up to the compiler whether to obey the 'register p' request; the
program's structure does not make the request impossible to satisfy.
Therefore there is no reason for the compiler to warn about this, any
more than there would be if the compiler chose not to obey the 'register
p' simply because it thought it could make more optimal use of the
available registers.  That GCC warns is understandable, in that a
superficial reading of the code would not reveal the optimization
opportunity; but the warning is nonetheless unnecessary, and the
'register' does provide useful optimization hinting.


OK, long read, but the the conclusion is that "fixing the code to not 
declare the variable as register would be the correct thing to do" it 
*not* the correct thing to do.  The correct thing to do is to ignore the 
warning, which is not possible if warnings are turned into errors.

You also mentioned that "other compilers will issue a hard error for 
it."  That sounds rather strange, and I wonder which compilers that 
might be; someone should file a bug report against them ;)

Re: [gentoo-dev] Re: FYI: Rules for distro-friendly packages

[Harald van Dijk]

On Sun, Jun 27, 2010 at 05:46:28PM +0300, Nikos Chantziaras wrote:
> On 06/27/2010 03:23 PM, Harald van Dijk wrote:
> > The compiler is not totally free to ignore the register keyword.
> > Both the C and the C++ standards require that the compiler complain
> > when taking the address of a register variable. Other compilers will
> > issue a hard error for it. Fixing the code to not declare the
> > variable as register would be the correct thing to do.
> 
> No, it would not be the correct thing to do, because of the following. 
> (This is part of a discussion between me and someone quite smarter than 
> me, who explained the issue in detail.)
>
> [snip]

That explanation seems to be written by someone who does not know that
taking the address of a register variable is simply not allowed.

> OK, long read, but the the conclusion is that "fixing the code to not 
> declare the variable as register would be the correct thing to do" it 
> *not* the correct thing to do.  The correct thing to do is to ignore the 
> warning, which is not possible if warnings are turned into errors.

And which is not possible if the warning is a hard error in the first place.

> You also mentioned that "other compilers will issue a hard error for 
> it."  That sounds rather strange, and I wonder which compilers that 
> might be; someone should file a bug report against them ;)

Well, let's start with gcc; that's quite an important one for Gentoo...

[gentoo-dev] Re: FYI: Rules for distro-friendly packages

[Nikos Chantziaras]

On 06/27/2010 08:14 PM, Harald van Dijk wrote:
> On Sun, Jun 27, 2010 at 05:46:28PM +0300, Nikos Chantziaras wrote:
>> On 06/27/2010 03:23 PM, Harald van Dijk wrote:
>>> The compiler is not totally free to ignore the register keyword.
>>> Both the C and the C++ standards require that the compiler complain
>>> when taking the address of a register variable. Other compilers will
>>> issue a hard error for it. Fixing the code to not declare the
>>> variable as register would be the correct thing to do.
>>
>> No, it would not be the correct thing to do, because of the following.
>> (This is part of a discussion between me and someone quite smarter than
>> me, who explained the issue in detail.)
>>
>> [snip]
>
> That explanation seems to be written by someone who does not know that
> taking the address of a register variable is simply not allowed.

It is allowed.  Section 7.1.1, Paragraphs 2 and 3 of the C++ standard:

The register specifier [...] specifies that the named variable has 
automatic storage duration (3.7.3). A variable declared without a 
storage-class-specifier at block scope or declared as a function 
parameter has automatic storage duration by default.

A register specifier is a hint to the implementation that the variable 
so declared will be heavily used. [ Note: the hint can be ignored and in 
most implementations it will be ignored if the address of the variable 
is taken. This use is deprecated (see D.4). — end note ]


>> OK, long read, but the the conclusion is that "fixing the code to not
>> declare the variable as register would be the correct thing to do" it
>> *not* the correct thing to do.  The correct thing to do is to ignore the
>> warning, which is not possible if warnings are turned into errors.
>
> And which is not possible if the warning is a hard error in the first place.
>
>> You also mentioned that "other compilers will issue a hard error for
>> it."  That sounds rather strange, and I wonder which compilers that
>> might be; someone should file a bug report against them ;)
>
> Well, let's start with gcc; that's quite an important one for Gentoo...

That code compiles just fine.  I don't know of any GCC version that 
issues an error for this rather than just a warning.

[gentoo-dev] Re: FYI: Rules for distro-friendly packages

[dev-random]

On Sun, Jun 27, 2010 at 08:48:25PM +0300, Nikos Chantziaras wrote:
> ...
> It is allowed.  Section 7.1.1, Paragraphs 2 and 3 of the C++ standard:
> ...

Not in C.
ISO/IEC 9899:1999 (aka C99), section 6.7.1, note 101:
> The implementation may treat any register declaration simply as an auto
> declaration. However, whether or not addressable storage is actually
> used, the address of any part of an object declared with storage-class
> specifier register cannot be computed, either explicitly (by use of the
> unary & operator as discussed in 6.5.3.2) or implicitly (by converting
> an array name to a pointer as discussed in 6.3.2.1). Thus, the only
> operator that can be applied to an array declared with storage-class
> specifier register is sizeof.

[gentoo-dev] Re: FYI: Rules for distro-friendly packages

[Nikos Chantziaras]

On 06/27/2010 09:10 PM, dev-random@mail.ru wrote:
> On Sun, Jun 27, 2010 at 08:48:25PM +0300, Nikos Chantziaras wrote:
>> ...
>> It is allowed.  Section 7.1.1, Paragraphs 2 and 3 of the C++ standard:
>> ...
>
> Not in C.
> ISO/IEC 9899:1999 (aka C99), section 6.7.1, note 101:
>> The implementation may treat any register declaration simply as an auto
>> declaration. However, whether or not addressable storage is actually
>> used, the address of any part of an object declared with storage-class
>> specifier register cannot be computed, either explicitly (by use of the
>> unary&  operator as discussed in 6.5.3.2) or implicitly (by converting
>> an array name to a pointer as discussed in 6.3.2.1). Thus, the only
>> operator that can be applied to an array declared with storage-class
>> specifier register is sizeof.

Wasn't aware of the difference here.  But anyway, the warning is issued 
by GCC for C++ too, not just C.

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Sergei Trofimovich]

On Sat, 26 Jun 2010 21:57:33 +0200
Enrico Weigelt <weigelt@metux.de> wrote:

> > > #2 One point i don't agree is the "dont add -Werror" rule. actually,
> > > i'm thinking of making -Wall and -Werror mandatory. if some
> > > package doenst build fine, it's simply broken. period.
> > 
> > Uhm. No. Certain compilers will give you warnings for f(g(a), g(b)) if
> > you -Wall.
> 
> Warn on what exactly ? Which compilers do that ?

I suggest you to try latest available dev-lang/icc (11.1.072).
This thing is really paranoid:

remark #2259: non-pointer conversion from "int" to "unsigned char" may lose significant bits
      unsigned char BlinkerPhase = 0;
      ...
      BlinkerPhase = (BlinkerPhase + 1) & 3;

remark #981: operands are evaluated in unspecified order (tons of them)
      return strcmp( left.c_str(), right.c_str() ) > 0;

-- 

  Sergei

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Sergei Trofimovich <slyfox@gentoo.org> schrieb:

> I suggest you to try latest available dev-lang/icc (11.1.072).
> This thing is really paranoid:
> 
> remark #2259: non-pointer conversion from "int" to "unsigned char" may lose significant bits
>       unsigned char BlinkerPhase = 0;
>       ...
>       BlinkerPhase = (BlinkerPhase + 1) & 3;

In this case, the compiler is wrong: it should convert to
int and back to char here ;-p

> remark #981: operands are evaluated in unspecified order (tons of them)
>       return strcmp( left.c_str(), right.c_str() ) > 0;

I'm not sure if this really qualifies an warning, since - AFAIK -
C spec never said, that there is an evaluation order for 
function parameters.


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Richard Freeman]

On 06/27/2010 06:52 AM, Enrico Weigelt wrote:
>> remark #981: operands are evaluated in unspecified order (tons of them)
>>        return strcmp( left.c_str(), right.c_str() )>  0;
>
> I'm not sure if this really qualifies an warning, since - AFAIK -
> C spec never said, that there is an evaluation order for
> function parameters.
>

I could see how somebody might make that assumption (incorrectly), and 
get burned by this.  However, creating local variables just to hold 
intermediate results so as to not embed them in function calls seems to 
be a lot of overhead - certainly in terms of readability, and I can't 
think of a situation where the compiler would have to do it on its own. 
  I guess religiously doing this might make the code less likely to 
contain very subtle bugs, but perhaps it is a bit over the top for 
anybody who wouldn't be otherwise developing in ADA.

Rich

[gentoo-dev] Re: FYI: Rules for distro-friendly packages

[Nikos Chantziaras]

On 06/26/2010 10:39 PM, Enrico Weigelt wrote:
> * Petteri Räty<betelgeuse@gentoo.org>  schrieb:
>
>> There should be useful stuff here:
>> http://video.fosdem.org/2010/devrooms/distributions/How_to_be_a_good_upstream.ogv
>
>[...[
> #2 One point i don't agree is the "dont add -Werror" rule. actually,
> i'm thinking of making -Wall and -Werror mandatory. if some
> package doenst build fine, it's simply broken. period.

That's the most hilarious thing I've read in ages. xD

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Rémi Cardona]

Le 26/06/2010 21:39, Enrico Weigelt a écrit :
> #2 One point i don't agree is the "dont add -Werror" rule. actually,
> i'm thinking of making -Wall and -Werror mandatory. if some
> package doenst build fine, it's simply broken. period.

You're obviously new here...

Just take a stroll through bugzilla to see how much we _fight_ against
-Werror. Let's see why you obviously have not thought through this
completely before writing this :

We currently offer 11 different slots of GCC, 3 of gcc-apple, each with
multiple versions, 3 versions of llvm-gcc, 2 versions of clang, 7
versions of icc, so in all 26 *major* versions. You do well know that
each compiler prints out different warnings for the *same* code...

We also offer 10 versions of glibc, 8 versions of uclibc, and 7 versions
of klibc. Each version may have header bugs, so may trigger warnings for
perfectly good code.

And finally we offer 5 unmasked versions of binutils (newer ones even
have a brand new linker - gold) and 5 versions of binutils-apple. Again,
different tools, different warnings...

If you want to make -Werror mandatory, you *MUST* test all combinations
above as *THEY ARE ALL SUPPORTED*.

Otherwise, packages will break for no good reason and users will hate us.

-Werror is a perfectly fine *developer* feature. For example, Gnome
autoconf macros enable it for development snapshots, but never ever
enable it for stable releases.

So please, if you want to write nonsense, don't write it in the name of
Gentoo.

Rémi

PS, Diego (flameeyes) is already having enough issues with his tinderbox
running *ONE* compiler/linker/libc combination...

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Rémi Cardona <remi@gentoo.org> schrieb:

> We currently offer 11 different slots of GCC, 3 of gcc-apple, each with
> multiple versions, 3 versions of llvm-gcc, 2 versions of clang, 7
> versions of icc, so in all 26 *major* versions. You do well know that
> each compiler prints out different warnings for the *same* code...

hmm, perhaps I'll hack some little scripts which create a jail/container
for each compiler version and triggers emerge within them.

> We also offer 10 versions of glibc, 8 versions of uclibc, and 7 versions
> of klibc. Each version may have header bugs, so may trigger warnings for
> perfectly good code.

Well, if there're header bugs, shouldn't they get fixed before these
libs are stabelized ? ;-O

> And finally we offer 5 unmasked versions of binutils (newer ones even
> have a brand new linker - gold) and 5 versions of binutils-apple. Again,
> different tools, different warnings...

Okay, adds *10 to the test matrix. Still not yet an too complex 
problem (still linear complexity).

> -Werror is a perfectly fine *developer* feature. For example, Gnome
> autoconf macros enable it for development snapshots, but never ever
> enable it for stable releases.

Okay, aggreed. I've reworked my rule, now:

"package build systems MUST NOT enable "-Wall -Werror" (unless explicitly 
asked), but developers SHOULD use them for their test builds"


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Patrick Lauer]

On 06/27/10 13:02, Enrico Weigelt wrote:
[snip]
>> We also offer 10 versions of glibc, 8 versions of uclibc, and 7 versions
>> of klibc. Each version may have header bugs, so may trigger warnings for
>> perfectly good code.
> 
> Well, if there're header bugs, shouldn't they get fixed before these
> libs are stabelized ? ;-O

And there we have the thin line between actual bug and fuzzy
specification. Sometimes things fail just because two people assumed
something and thus the code disagrees in a really funny way.
Or just the GNUisms that tend to leak in through glibc+gcc ... *shudder*

>> And finally we offer 5 unmasked versions of binutils (newer ones even
>> have a brand new linker - gold) and 5 versions of binutils-apple. Again,
>> different tools, different warnings...
> 
> Okay, adds *10 to the test matrix. Still not yet an too complex 
> problem (still linear complexity).

So just to keep up with the current package update speed in gentoo you'd
be saturating about a dozen quadcore machines. For amd64 only.
And that's not even checking all reverse dependencies of every single
update (which can easily push it up by another factor of 100)

Unless you have a strategic supply of gold bars it won't be possible to
check all permutations (and we haven't even started toggling useflags
for fun!)

>> -Werror is a perfectly fine *developer* feature. For example, Gnome
>> autoconf macros enable it for development snapshots, but never ever
>> enable it for stable releases.
> 
> Okay, aggreed. I've reworked my rule, now:
> 
> "package build systems MUST NOT enable "-Wall -Werror" (unless explicitly 
> asked), but developers SHOULD use them for their test builds"
I'd say they can enable it, but SHOULD NOT

If people want to run headfirst into a wall let them do it a few times,
they'll learn ...

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Patrick Lauer <patrick@gentoo.org> schrieb:

> > Well, if there're header bugs, shouldn't they get fixed before these
> > libs are stabelized ? ;-O
> 
> And there we have the thin line between actual bug and fuzzy
> specification. Sometimes things fail just because two people assumed
> something and thus the code disagrees in a really funny way.
> Or just the GNUisms that tend to leak in through glibc+gcc ... *shudder*

hmm, do you have some concrete examples ?

> > Okay, adds *10 to the test matrix. Still not yet an too complex 
> > problem (still linear complexity).
> 
> So just to keep up with the current package update speed in gentoo you'd
> be saturating about a dozen quadcore machines. For amd64 only.

Well, I'll have to investigate further.

Maybe it's time for a distributed build project: a generic container
image, which gets distributed to dozens of machines and runs build
tests coordinated by some server ... a bit like seti@home ;-)

Enough CPU is available all around the world, just heavily distributed.


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 479 bytes --]

On Sun, 27 Jun 2010 14:22:53 +0200
Enrico Weigelt <weigelt@metux.de> wrote:
> Maybe it's time for a distributed build project: a generic container
> image, which gets distributed to dozens of machines and runs build
> tests coordinated by some server ... a bit like seti@home ;-)
> 
> Enough CPU is available all around the world, just heavily
> distributed.

PHP and mplayer both have 100 USE flags. There's not enough CPU power
in the world.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Rémi Cardona]

Le 27/06/2010 14:33, Ciaran McCreesh a écrit :
> On Sun, 27 Jun 2010 14:22:53 +0200
> Enrico Weigelt <weigelt@metux.de> wrote:
>> Maybe it's time for a distributed build project: a generic container
>> image, which gets distributed to dozens of machines and runs build
>> tests coordinated by some server ... a bit like seti@home ;-)
>>
>> Enough CPU is available all around the world, just heavily
>> distributed.
> 
> PHP and mplayer both have 100 USE flags. There's not enough CPU power
> in the world.
> 

The *other* problem with -Werror is that the build will stop on the
first warning, hiding all the others.

Enrico, if you actually want to fix this, better capture the build.log
of a *full* build, enabling all the warnings you could think of and then
fix whatever makes sense.

Cheers,

Rémi

Re: [gentoo-dev] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Ciaran McCreesh <ciaran.mccreesh@googlemail.com> schrieb:

> PHP and mplayer both have 100 USE flags. There's not enough 
> CPU power in the world.

We don't have to try *all* possible combinations, but only those
differing in interfaces (eg. if some libfoo changes its exported
interface on a userflag, the clients have to be built against
that two versions). It involves some interface analysis and a bit
of graph theory. Should be possible to do it incrementally. 


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------

[gentoo-dev] Re: [gentoo-user] FYI: Rules for distro-friendly packages

[Enrico Weigelt]

* Alan McKinnon <alan.mckinnon@gmail.com> schrieb:

> My biggest beef by far when packaging apps is automagic dependencies.
> 
> e17 is full of them - if package A is present, the app will configure itself 
> to use it. Usually you cannot switch this kind of thing off even if you have 
> valid reasons to do so.

That's one of the reasons I'm building everything by a sysroot'ed
crosscompiler (when not using Gentoo ;-)). This way these things
(normally) can't happen.

> I want explicit --enable-<package> features in the ./configure step for 
> everything that might be optional. Because I often do have that lib on my 
> system and the app's usage of it is buggy, so I should be able to disable that 
> support.

Can you live with the current formulation ?
http://www.metux.de/index.php/de/component/content/article/57.html

(see and of the text)


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------