[gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Enrico Weigelt]

Hi folks,


YFYI: yet another of my ebuilds kicked-down.

It's an improved version of procmail, which automatically creates
missing maildir directories.


cu

----- Forwarded message from bugzilla-daemon@gentoo.org -----

From: bugzilla-daemon@gentoo.org
Subject: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs
To: weigelt@metux.de
Reply-To: DO NOT REPLY <devnull@localhost.invalid>
Date: Wed,  7 Jul 2010 20:06:53 +0000 (UTC)

DO NOT REPLY TO THIS EMAIL. Also, do not reply via email to the person
whose email is mentioned below. To comment on this bug, please visit:

Clear-Text: http://bugs.gentoo.org/show_bug.cgi?id=322157
Secure: https://bugs.gentoo.org/show_bug.cgi?id=322157


ssuominen@gentoo.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |WONTFIX




------- Comment #2 from ssuominen@gentoo.org  2010-07-07 20:06 0000 -------
The SRC_URI is pointing to something unofficial, so this is a WONTFIX.

-- 
Configure bugmail: https://bugs.gentoo.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You reported the bug, or are watching the reporter.

----- End forwarded message -----

-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Samuli Suominen]

On 07/08/2010 01:56 AM, Enrico Weigelt wrote:
> 
> Hi folks,
> 
> 
> YFYI: yet another of my ebuilds kicked-down.
> 
> It's an improved version of procmail, which automatically creates
> missing maildir directories.

Provide your patches as plain/text attachments like everyone else does,
can't expect people to download patch(ed?) tarballs from your site. And
specially not using them in ebuilds as if the software got forked or
something.

> 
> 
> cu
> 
> ----- Forwarded message from bugzilla-daemon@gentoo.org -----
> 
> From: bugzilla-daemon@gentoo.org
> Subject: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs
> To: weigelt@metux.de
> Reply-To: DO NOT REPLY <devnull@localhost.invalid>
> Date: Wed,  7 Jul 2010 20:06:53 +0000 (UTC)
> 
> DO NOT REPLY TO THIS EMAIL. Also, do not reply via email to the person
> whose email is mentioned below. To comment on this bug, please visit:
> 
> Clear-Text: http://bugs.gentoo.org/show_bug.cgi?id=322157
> Secure: https://bugs.gentoo.org/show_bug.cgi?id=322157
> 
> 
> ssuominen@gentoo.org changed:
> 
>            What    |Removed                     |Added
> ----------------------------------------------------------------------------
>              Status|NEW                         |RESOLVED
>          Resolution|                            |WONTFIX
> 
> 
> 
> 
> ------- Comment #2 from ssuominen@gentoo.org  2010-07-07 20:06 0000 -------
> The SRC_URI is pointing to something unofficial, so this is a WONTFIX.
> 

[gentoo-dev] Re: [funtoo] [bugzilla-daemon-aBrp7R+bbdUdnm+yROfE0A@public.gmane.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Ryan Hill]

[-- Attachment #1: Type: text/plain, Size: 523 bytes --]

On Thu, 8 Jul 2010 00:56:52 +0200
Enrico Weigelt <weigelt@metux.de> wrote:

> 
> Hi folks,
> 
> 
> YFYI: yet another of my ebuilds kicked-down.
> 
> It's an improved version of procmail, which automatically creates
> missing maildir directories.

Upstream first (TM).

-- 
fonts, gcc-porting,                                   and it's all by design
toolchain, wxwidgets                        to keep us from losing our minds
@ gentoo.org                EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Robin H. Johnson]

On Thu, Jul 08, 2010 at 12:56:52AM +0200, Enrico Weigelt wrote:
> YFYI: yet another of my ebuilds kicked-down.
> 
> It's an improved version of procmail, which automatically creates
> missing maildir directories.
Stock procmail does this already.

From procmailrc:
If the mailbox is specified to be an MH folder or maildir folder,
procmail will create the necessary directories if they don't exist,
rather than treat the mail‐ box as a non-existent filename.

And I know said functionality works, because I deliver my list mail to
.ML.${LISTNAME}.${YYYY}-${MM}/

P.S. That level of crossposting is not nice. Everybody who is only on
one of those lists would not be able to cross-post like you've done.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[gentoo-dev] Re: [funtoo] [bugzilla-daemon-aBrp7R+bbdUdnm+yROfE0A@public.gmane.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Paul Arthur]

On 2010-07-07, Enrico Weigelt <weigelt@metux.de> wrote:
>
> Hi folks,
>
>
> YFYI: yet another of my ebuilds kicked-down.
>
> It's an improved version of procmail, which automatically creates
> missing maildir directories.

procmail already does that.  And if it didn't, you should work with the
maintainers of procmail to get it to, rather than forking it uselessly.

-- 
Talking loudly, resisting suppression and control, and defending
yourself are all signs of vigorous aliveness. 
    --danth on RPGnet

Re: [gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Enrico Weigelt]

* Robin H. Johnson <robbat2@gentoo.org> schrieb:

Hi,

> > It's an improved version of procmail, which automatically creates
> > missing maildir directories.
> Stock procmail does this already.
> 
> From procmailrc:
> If the mailbox is specified to be an MH folder or maildir folder,
> procmail will create the necessary directories if they don't exist,
> rather than treat the mail??? box as a non-existent filename.

Only applies to maildir mailboxes, not mbox. The problem is: if
you have mbox'es in subdirs which dont exist yet, procmail fails.


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Enrico Weigelt]

* Samuli Suominen <ssuominen@gentoo.org> schrieb:
> On 07/08/2010 01:56 AM, Enrico Weigelt wrote:
> > 
> > Hi folks,
> > 
> > 
> > YFYI: yet another of my ebuilds kicked-down.
> > 
> > It's an improved version of procmail, which automatically creates
> > missing maildir directories.
> 
> Provide your patches as plain/text attachments like everyone else does,

I've already explained the strategy behind the git repo (and not 
doing plaintext patches). Please refer to my paper, and my other
mails posted recently on this list.


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Jacob Godserv]

On Sat, Jul 10, 2010 at 12:13, Enrico Weigelt <weigelt@metux.de> wrote:
> I've already explained the strategy behind the git repo (and not
> doing plaintext patches). Please refer to my paper, and my other
> mails posted recently on this list.

I'm not quite sure I understand your response here. He didn't ask for
you to explain the strategy. He asked for you to provide plain-text
patches.

I think a point has been made pretty clear at this point: they need
plain-text patches. Requiring developers to learn an entirely new
work-flow just to get your patches into the tree is time-consuming and
bothersome, at the least. This isn't a discussion about whether or not
your system is better. It's about how best to integrate your work into
Gentoo.

The developers have decided the best way is through plain-text
patches. That's really all there is to it. Take it or leave it.

The discussion about how best to join multiple branches of development
across the open-source world together really does not belong in this
thread, as far as I can tell.

-- 
    Jacob

    "For then there will be great distress, unequaled
    from the beginning of the world until now — and never
    to be equaled again. If those days had not been cut
    short, no one would survive, but for the sake of the
    elect those days will be shortened."

    Are you ready?

Re: [gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Hans de Graaff]

[-- Attachment #1: Type: text/plain, Size: 1041 bytes --]

On Sun, 2010-07-11 at 01:28 -0400, Jacob Godserv wrote:
> On Sat, Jul 10, 2010 at 12:13, Enrico Weigelt <weigelt@metux.de> wrote:
> > I've already explained the strategy behind the git repo (and not
> > doing plaintext patches). Please refer to my paper, and my other
> > mails posted recently on this list.
> 
> I'm not quite sure I understand your response here. He didn't ask for
> you to explain the strategy. He asked for you to provide plain-text
> patches.

I do understand the response, because part of the strategy mentioned
*is* not to provide plain-text patches, but instead manage them,
possibly jointly with other distributions, in a midstream repository.

From the comments it looks like most developers (including me) won't be
happy to switch to such an intermediate point at the moment. Perhaps the
best approach would be to focus on packages with unmaintained or
abandoned upstreams. For those packages there is a much more clear
benefit of pooling together distribution patches.

Kind regards,

Hans

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: [gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Enrico Weigelt]

* Hans de Graaff <graaff@gentoo.org> schrieb:

> I do understand the response, because part of the strategy mentioned
> *is* not to provide plain-text patches, but instead manage them,
> possibly jointly with other distributions, in a midstream repository.

Exactly.

The point is: I have to maintain lots of packages for different distros, 
as well as for my own build system. I cannot do this manually for each 
single distro, so I prefer doing _generic_ fixes and let the distro 
buildfiles only do the plain standard build process 
(eg. ./autogen.sh && ./configure && make && make install). 


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Nirbheek Chauhan]

On Sun, Jul 11, 2010 at 12:39 PM, Enrico Weigelt <weigelt@metux.de> wrote:
> The point is: I have to maintain lots of packages for different distros,
> as well as for my own build system. I cannot do this manually for each
> single distro, so I prefer doing _generic_ fixes and let the distro
> buildfiles only do the plain standard build process
> (eg. ./autogen.sh && ./configure && make && make install).
>
>

I know the problem you are trying to solve is quite important.
Various distros apply their own patches ontop of products, which often
leads to duplicated work, regressions and even security bugs, all
because there aren't enough eyeballs. We need a solution for this.

Now, in a perfect world, we wouldn't need this, since upstream would
quickly merge bugfixes, and patches would only have to be kept till
the next release. However, we have seen that upstreams are often
stubborn, or don't agree with us. Some distros (like Gentoo,
Slackware, ArchLinux) prefer to keep patching to a *minimum*. A lot of
teams have a policy of "upstream first" before adding a patch to the
tree (unless it is small/hackish/temporary, very distro specific, and
has no place upstream).

On the other hand, Debian/Ubuntu like to carry a thousand lines of
patches on top of packages like Firefox, and Ubuntu likes to diverge
/majorly/ from upstream releases. Fedora also suffers from this
problem of not agreeing with upstream quite often[1]; but w.r.t.
GNOME/Xorg/udev they *are* upstream, so the effect is lessened. IIRC,
OpenSuse has had a good policy of upstreaming often.


If I understand your system correctly, you essentially maintain clones
of upstream repos, with all the various distro patches applied on top,
and release tarballs as well. I don't see how these various distros
can be made to agree with each other and I certainly can't see them
using a common tarball source. On a technical level, it's got serious
security, trust, and redundancy problems. It is extremely important
that distros collaborate in some form when it comes to patches that
*can* be shared, but the solution you have devised is fundamentally
flawed.

I can say today with extreme confidence that no major distros will use
your repositories and your tarballs, or work with you to collaborate
with each other. There is simply very little incentive to change the
way we work.

A practical solution to the problem of patch sharing is to have a
website with a search interface for upstream source tarballs, which
can display all the patches that various distros apply, as well as a
download link for the patchsets (hotlinked to the distro files where
possible). Currently, it is extremely painful to find out what patches
each distro has applied on top of a give source package, and as a
result, collaboration is minimal.

As the popularity of such a website increases (you'll need to
advertise it!), you'll find that at least the divergent/community
supported distros (debian/slackware/arch/gentoo/opensolaris) will
start using it heavily and sharing of patches (wherever feasible) will
occur automatically.

Distro packagers are much more comfortable with downloading patchsets
from a foreign source than complete tarballs. They already have a
trust-relationship with upstream, and they can verify the integrity of
a patchset much easier than an arbitrary tarball.

I know you have spent a lot of time on this already, but please
understand it from where we stand. We're short on manpower, and
there's no real benefits of shifting our tarball source; OTOH there
are major disadvantages too unless we pitch in with manpower
ourselves. And honestly speaking, that manpower is better spent making
stuff work locally.

Please consider the "patch-website" idea above. We definitely need
someone to code it up, gather the source-package to distro patches
mappings,
and advertise it. And this is a solution that can easily be created
and maintained by one man alone, and will solve the problem of distro
collaboration as well!

Best,

1. https://bugs.gentoo.org/291328

-- 
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

[gentoo-dev] Re: [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Duncan]

Nirbheek Chauhan posted on Sun, 11 Jul 2010 14:28:35 +0530 as excerpted:

> ...

> Please consider the "patch-website" idea above. We definitely need
> someone to code it up, gather the source-package to distro patches
> mappings,

> ...

Thanks for writing this up.  Well said. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Enrico Weigelt]

* Nirbheek Chauhan <nirbheek@gentoo.org> schrieb:

> If I understand your system correctly, you essentially maintain clones
> of upstream repos, with all the various distro patches applied on top,
> and release tarballs as well. 

Yes. And if some upstream does not provide suitable vcs access
(or doesnt even have one), I make an pseudo-upstream branch by
committing the release source trees.

> I don't see how these various distros can be made to agree with 
> each other and I certainly can't see them using a common tarball 
> source. 

Thats not even necessary. They just should use the infrastructure,
as described in my paper. So everyone can easily set up automatic
notifications, cherry-pick, etc, etc.

> On a technical level, it's got serious security, trust, and 
> redundancy problems.

Git makes that very easy ;-p

> It is extremely important that distros collaborate in some form 
> when it comes to patches that *can* be shared, 

If we're doing a good job (my generic fixes instead of distro-
specfic dirty hacks) about 99% can be shared ;-p

> but the solution you have devised is fundamentally flawed.

If it's really flawed, then just for pure "social" reasons, no
serious technical ones.

> A practical solution to the problem of patch sharing is to 
> have a website with a search interface for upstream source 
> tarballs, which can display all the patches that various 
> distros apply, as well as a download link for the patchsets 
> (hotlinked to the distro files where possible). 

Too complicated, and actually would not help me a single bit.
What I could offer is an (semi-)automatic import mechanism 
(assuming certain package managers dont do such insane things 
like directly sed'ing sources etc) - there's still a bunch of
work to do for that, but its possible.

> Distro packagers are much more comfortable with downloading 
> patchsets from a foreign source than complete tarballs. 

man git-format-patch ;-p

Maybe I could set up an git webfrontend (or automatically push
to some public service). 

> I know you have spent a lot of time on this already, but please
> understand it from where we stand. We're short on manpower, and
> there's no real benefits of shifting our tarball source; OTOH there
> are major disadvantages too unless we pitch in with manpower
> ourselves. And honestly speaking, that manpower is better spent making
> stuff work locally.

Well, Gentoo is short of manpower ? hmm, perhaps some should think
about why so many folks are resigning and so few fresh coming in
(at least according to this lists traffic) ;-O

> Please consider the "patch-website" idea above. We definitely need
> someone to code it up, gather the source-package to distro patches
> mappings, and advertise it.

Actually, I once had somehing in that area, called "comprehensive
source database", but unfurtinately it got lost in an disk array
crash a few years ago, and I didnt find the time to rewrite it yet.

Meanwhile I dont need it anymore, since I gave up maintaining
plaintext patches in favour of git. And that makes my daily works
_much_ easier.

Oh, btw: I'm announcing my oss-qm releases via twitter:

    http://twitter.com/oss_qm


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Nirbheek Chauhan]

On Sun, Jul 11, 2010 at 3:23 PM, Enrico Weigelt <weigelt@metux.de> wrote:
> * Nirbheek Chauhan <nirbheek@gentoo.org> schrieb:
>> I don't see how these various distros can be made to agree with
>> each other and I certainly can't see them using a common tarball
>> source.
>
> Thats not even necessary. They just should use the infrastructure,
> as described in my paper. So everyone can easily set up automatic
> notifications, cherry-pick, etc, etc.
>

Why should we? I am *yet* to see a single reason for us to change how
we work other than "please use this since I've been putting a lot of
effort into it".

>> On a technical level, it's got serious security, trust, and
>> redundancy problems.
>
> Git makes that very easy ;-p
>

No, it does not. The security problems come because you are the single
point of failure. The trust problems come because we have no reason to
trust you. The redundancy problems come because if your hosting goes
down or you lose interest, we're left high and dry. Git has nothing to
do with any of this.

>> It is extremely important that distros collaborate in some form
>> when it comes to patches that *can* be shared,
>
> If we're doing a good job (my generic fixes instead of distro-
> specfic dirty hacks) about 99% can be shared ;-p
>

I'd advise you to take a look at the sort of patching Ubuntu/Debian
does, and then revisit that figure. You'll find it more along the
lines of 30%.


>> A practical solution to the problem of patch sharing is to
>> have a website with a search interface for upstream source
>> tarballs, which can display all the patches that various
>> distros apply, as well as a download link for the patchsets
>> (hotlinked to the distro files where possible).
>
> Too complicated, and actually would not help me a single bit.

Help *you*? I thought this was about helping the distros. If your
proposal is not about making our work easier, please don't waste our
time.

>> Distro packagers are much more comfortable with downloading
>> patchsets from a foreign source than complete tarballs.
>
> man git-format-patch ;-p
>

So why don't you submit that to bugzilla?

>> I know you have spent a lot of time on this already, but please
>> understand it from where we stand. We're short on manpower, and
>> there's no real benefits of shifting our tarball source; OTOH there
>> are major disadvantages too unless we pitch in with manpower
>> ourselves. And honestly speaking, that manpower is better spent making
>> stuff work locally.
>
> Well, Gentoo is short of manpower ? hmm, perhaps some should think
> about why so many folks are resigning and so few fresh coming in
> (at least according to this lists traffic) ;-O
>

I'm beginning to think that you're not taking my honest advice very seriously.

>> Please consider the "patch-website" idea above. We definitely need
>> someone to code it up, gather the source-package to distro patches
>> mappings, and advertise it.
>
> Actually, I once had somehing in that area, called "comprehensive
> source database", but unfurtinately it got lost in an disk array
> crash a few years ago, and I didnt find the time to rewrite it yet.
>
> Meanwhile I dont need it anymore, since I gave up maintaining
> plaintext patches in favour of git. And that makes my daily works
> _much_ easier.
>

You don't need to maintain **anything** manually if you code the
website properly. That's the whole point. You get major benefits with
minimal long-term work which can be done by a single person in their
free time.

This job is easily automated to simply aggregate links to patches
which all the distros manually publish themselves. For Gentoo, it's
the ebuilds; for Debian/Ubuntu, they actually publish the diffs[1];
Fedora keeps its patches in a common CVS repo[2], etc etc. Once the
website is up and running, maintenance is minimal, and can be done by
a single person looking at it in their free time.

1. See packages.(debian|ubuntu).(org|com)
2. cvs.fedoraproject.org/viewvc/

-- 
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

Re: [gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Jorge Manuel B. S. Vicetto]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nirbheek,

thanks for writing such a well thought-out and comprehensive reply to
Enrico. I agree with all the points you raised.

On 11-07-2010 10:28, Nirbheek Chauhan wrote:
> On Sun, Jul 11, 2010 at 3:23 PM, Enrico Weigelt <weigelt@metux.de> wrote:
>> * Nirbheek Chauhan <nirbheek@gentoo.org> schrieb:
>>> I don't see how these various distros can be made to agree with
>>> each other and I certainly can't see them using a common tarball
>>> source.
>>
>> Thats not even necessary. They just should use the infrastructure,
>> as described in my paper. So everyone can easily set up automatic
>> notifications, cherry-pick, etc, etc.
>>
> 
> Why should we? I am *yet* to see a single reason for us to change how
> we work other than "please use this since I've been putting a lot of
> effort into it".

Enrico,

just because you decided to "offer" some service, it in no way "forces
us" to accept it.

>>> On a technical level, it's got serious security, trust, and
>>> redundancy problems.
>>
>> Git makes that very easy ;-p
>>
> 
> No, it does not. The security problems come because you are the single
> point of failure. The trust problems come because we have no reason to
> trust you. The redundancy problems come because if your hosting goes
> down or you lose interest, we're left high and dry. Git has nothing to
> do with any of this.

These 3 issues are in my view the most important issues regarding your
proposal and I agree with Nirbheek's reply.
With your proposal, you're expecting us (distro maintainers) to put our
trust and our users safety in you and your service - what made you think
we would or should do it? Besides, what significant gain would we have
to justify trusting your service?

>>> It is extremely important that distros collaborate in some form
>>> when it comes to patches that *can* be shared,
>>
>> If we're doing a good job (my generic fixes instead of distro-
>> specfic dirty hacks) about 99% can be shared ;-p
>>
> 
> I'd advise you to take a look at the sort of patching Ubuntu/Debian
> does, and then revisit that figure. You'll find it more along the
> lines of 30%.
> 
> 
>>> A practical solution to the problem of patch sharing is to
>>> have a website with a search interface for upstream source
>>> tarballs, which can display all the patches that various
>>> distros apply, as well as a download link for the patchsets
>>> (hotlinked to the distro files where possible).
>>
>> Too complicated, and actually would not help me a single bit.
> 
> Help *you*? I thought this was about helping the distros. If your
> proposal is not about making our work easier, please don't waste our
> time.
> 
>>> Distro packagers are much more comfortable with downloading
>>> patchsets from a foreign source than complete tarballs.
>>
>> man git-format-patch ;-p
>>
> 
> So why don't you submit that to bugzilla?

Please don't assume replies are based solely on people not knowing the
tools (git in this case).

>>> I know you have spent a lot of time on this already, but please
>>> understand it from where we stand. We're short on manpower, and
>>> there's no real benefits of shifting our tarball source; OTOH there
>>> are major disadvantages too unless we pitch in with manpower
>>> ourselves. And honestly speaking, that manpower is better spent making
>>> stuff work locally.
>>
>> Well, Gentoo is short of manpower ? hmm, perhaps some should think
>> about why so many folks are resigning and so few fresh coming in
>> (at least according to this lists traffic) ;-O

This type of argument when you're trying to convince others to use your
service is in the least a disservice to your purpose.

> I'm beginning to think that you're not taking my honest advice very seriously.
> 
>>> Please consider the "patch-website" idea above. We definitely need
>>> someone to code it up, gather the source-package to distro patches
>>> mappings, and advertise it.
>>
>> Actually, I once had somehing in that area, called "comprehensive
>> source database", but unfurtinately it got lost in an disk array
>> crash a few years ago, and I didnt find the time to rewrite it yet.
>>
>> Meanwhile I dont need it anymore, since I gave up maintaining
>> plaintext patches in favour of git. And that makes my daily works
>> _much_ easier.
>>
> 
> You don't need to maintain **anything** manually if you code the
> website properly. That's the whole point. You get major benefits with
> minimal long-term work which can be done by a single person in their
> free time.
> 
> This job is easily automated to simply aggregate links to patches
> which all the distros manually publish themselves. For Gentoo, it's
> the ebuilds; for Debian/Ubuntu, they actually publish the diffs[1];
> Fedora keeps its patches in a common CVS repo[2], etc etc. Once the
> website is up and running, maintenance is minimal, and can be done by
> a single person looking at it in their free time.
> 
> 1. See packages.(debian|ubuntu).(org|com)
> 2. cvs.fedoraproject.org/viewvc/
> 

- -- 
Regards,

Jorge Vicetto (jmbsvicetto) - jmbsvicetto at gentoo dot org
Gentoo- forums / Userrel / Devrel / KDE / Elections
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.15 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJMOfchAAoJEC8ZTXQF1qEPoCcQAKlDi9d7GgZPK00kq+6lRZRF
+JW5kTvOoOyu75X9nglzxKuJCWqOFMV6T1X5CvKOr/5XOJbdmAdrd7flSHxixlsI
DZi+62u3at7rq26pPOpt5wCNdR+PVSvGp0J8Y+X1AtB8UpYk6P/3Zjqk1ZyS9HSp
zqX4TCnEXOoQd6uNueiPh3qF7hr5F1dZSsqEaO99A9CVPwkGzLZE2/yyYDPB2ZU9
KqFdd+MyBpdgCOW02QjZAymKfn1C9sDjifWflj48mAhJEgRSMrAmf3/m8dzEy5qm
T/qNGTgehN1cNgIREm4BBiwIwgYoYDUAr9plurgTDuJ6S8uSOUnk++8EF07uUnjo
qTKR6Xxf86YN0zjwEgQhzfY8FfVvD26DJVg3woJSzrDM+ZGXfiCBjzX+tOpcRq54
vNE8+egMrce3V0ypI6iLc3fvY7pyOyRUuuwMT4vzau9fb7EL3YcGJvtXT+3ozm+n
bXpp+wK8BJy7QRa3O+A6d9GbuLM8u+rsRw85Oc/j59rJ/5e+QWqOvg1su7p6bG4D
rL26CxnRUMOAM9VtL2zfVa4rZTqsD1KVaT5RQ+u989P4FHJc+NcmLtE1cvV42iR7
GEl3fHinq37XiSug2YwTIO8pnc8O0jYibQw3IDmHlNdxFCZ3lgsC/Ir7mIUjkzgh
3Gcc1L+1LNeYh+fF4wNN
=Lpc6
-----END PGP SIGNATURE-----

Re: [gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Enrico Weigelt]

* Nirbheek Chauhan <nirbheek@gentoo.org> schrieb:

> > Thats not even necessary. They just should use the infrastructure,
> > as described in my paper. So everyone can easily set up automatic
> > notifications, cherry-pick, etc, etc.
> 
> Why should we? 

To make tracking and applying other's changes much easier.
Currently it's quite work intensive to do so.

If - hypothetically - everyone work within an git repo, using a 
normalized naming/versioning scheme, it is trivial to set some
little tracking system, informing package maintainers if something
happens (new upstream, new patches from distro XYZ, etc). And
we can inspect that easily, take patches, etc, etc, using 
standard git tools.

> I am *yet* to see a single reason for us to change how we work 
> other than "please use this since I've been putting a lot of
> effort into it".

I have no intention to urge you to use my approach, I'm just
offering you my works, nothing more.

> >> On a technical level, it's got serious security, trust, and
> >> redundancy problems.
> >
> > Git makes that very easy ;-p
> >
> 
> No, it does not. The security problems come because you are the single
> point of failure. 

Which SPOF ? 

man 1 git-cone ;-p

> The trust problems come because we have no reason to trust you.

You dont have to trust me. Just let me point out some possible 
leak points (if I missed some, feel free to add them ...):

a) repository manipulation (directly changing objects): will 
   be found out by git-gc.

b) injecting changed upstream: you can easily compare my UPSTREAM.*
   tags against the real upstream's tarballs or vcs tags. 
   
   BTW: as long as not all upstreams sign their releases, our trust 
   relies just relies on their server's integrity (and the connection
   to them).
   
c) manipulated tags: someone, perhaps myself overwrites other's tags 
   use signed tags, and check the signatures (easily doable by a 
   little shell script
 
d) some vendor (possibly myself) adds crappy changes: you'll most
   likely have a look at the changes before cherry-picking them.

> The redundancy problems come because if your hosting goes
> down or you lose interest, we're left high and dry. 

That wouldn't affect your clones. You simply won't get anything
new from my site anymore.

> > If we're doing a good job (my generic fixes instead of distro-
> > specfic dirty hacks) about 99% can be shared ;-p
> >
> 
> I'd advise you to take a look at the sort of patching Ubuntu/Debian
> does, and then revisit that figure. You'll find it more along the
> lines of 30%.

I never claimed that Ubuntu does clean and generic fixes. 

BTW: Gentoo tends to have similar problems, just look at the zlib ebuild:


>> # trust exit status of the compiler rather than stderr #55434
>> # -if test "`(...) 2>&1`" = ""; then
>> # +if (...) 2>/dev/null; then
>> sed -i 's|\<test "`\([^"]*\) 2>&1`" = ""|\1 2>/dev/null|' configure || die
>> sed -i -e '/ldconfig/d' Makefile* || die
					
> >> A practical solution to the problem of patch sharing is to
> >> have a website with a search interface for upstream source
> >> tarballs, which can display all the patches that various
> >> distros apply, as well as a download link for the patchsets
> >> (hotlinked to the distro files where possible).
> >
> > Too complicated, and actually would not help me a single bit.
> 
> Help *you*? I thought this was about helping the distros. 

Yes, I first want to solve my daily business problems, but in a way
that other folks can benefit from my works.

> >> Distro packagers are much more comfortable with downloading
> >> patchsets from a foreign source than complete tarballs.
> >
> > man git-format-patch ;-p
> >
> 
> So why don't you submit that to bugzilla?

Yet too complicated/work intensive to do this for each individual
distro. It would be a completely different issue, if there was
some robot interface for that.

On the other hand, if you pull from my repo, you can easily hack
up a little bot, which tells you when something happens (or 
even send you patches).

> > Meanwhile I dont need it anymore, since I gave up maintaining
> > plaintext patches in favour of git. And that makes my daily works
> > _much_ easier.
> >
> 
> You don't need to maintain **anything** manually if you code the
> website properly. That's the whole point. You get major benefits with
> minimal long-term work which can be done by a single person in their
> free time.

First, I have to build that website and maintain it over a long time.
Then I'll have to do a lot of advertisement work to get people to
actually push their patches there.

On the other hand, the git-based infrastructure is already there,
people can use it right now. And it gives my exactly what I need.
So I prefer spending my time in advocating this one.

> This job is easily automated to simply aggregate links to patches
> which all the distros manually publish themselves. 

It's not that simple. Many distros don't even do proper patches, 
instead wildly copy over or directly sed certain sourcesfiles,
or even (like Debian) use their own broken tarballs.
(the worst srcpkg I've ever seen is Debian's mysql-5.0.32 ...)

And even *if* we assume, that everyone's just doing patches, we
still need to transform the everbody's strange (often not even
linear) versioning schemes. One of OSS-QMs primary goals is to
use a strictly normalized/canonical naming/versioning scheme in 
the primay source repository.


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Enrico Weigelt]

* Jorge Manuel B. S. Vicetto <jmbsvicetto@gentoo.org> schrieb:

> just because you decided to "offer" some service, it in no way "forces
> us" to accept it.

No, it does not. I never claimed that.

And so really I dont understand that fundamentalistic kind of
argumentation (which is really near to ranting).

> With your proposal, you're expecting us (distro maintainers) to put our
> trust and our users safety in you and your service - what made you think
> we would or should do it? 

Not at all, please refer to my other answer (some minutes earlier).

> >> Well, Gentoo is short of manpower ? hmm, perhaps some should think
> >> about why so many folks are resigning and so few fresh coming in
> >> (at least according to this lists traffic) ;-O
> 
> This type of argument when you're trying to convince others to use your
> service is in the least a disservice to your purpose.

That particular argument is not about the OSS-QM project, but the 
general social climate @Gentoo. Actually, when talking to collegues,
most of them share this. The most commonly used term I've heared 
from them about Gentoo devs is "total lack of social competence".
(to be fair: this also applies to lots of other OSS projects).


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [gentoo-dev] Re: [funtoo] [bugzilla-daemon-aBrp7R+bbdUdnm+yROfE0A@public.gmane.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Nathan Phillip Brink]

[-- Attachment #1: Type: text/plain, Size: 1274 bytes --]

On Wed, Jul 07, 2010 at 06:03:49PM -0600, Ryan Hill wrote:
> On Thu, 8 Jul 2010 00:56:52 +0200
> Enrico Weigelt <weigelt@metux.de> wrote:
> 
> > 
> > Hi folks,
> > 
> > 
> > YFYI: yet another of my ebuilds kicked-down.
> > 
> > It's an improved version of procmail, which automatically creates
> > missing maildir directories.
> 
> Upstream first (TM).

I'm pretty sure that the main problem with procmail is that its upstream is quite dead. The proper solution might be simply a fork of procmail by someone who is willing to accept and apply distros' patches. This would be completely independent of distributions... or could be similar to Fedora (or a Fedora maintainer) starting up tigervnc. The problem with this approach would be getting enough momentum behind it and having distributions recognize such a fork.

I was myself interested in pushing a patch (unfortunately it's still unfinished :-/) to procmail's upstream, but had trouble finding the upstream. If anyone has had better luck or knows of a fork like I describe above, I'd be interested. Of course, knowing that there is an upstream to submit such a patch to would be a great encouragement for me to finish writing it. ;-)

-- 
binki

Look out for missing or extraneous apostrophes!

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Nirbheek Chauhan]

On Mon, Jul 12, 2010 at 8:14 AM, Enrico Weigelt <weigelt@metux.de> wrote:
> * Nirbheek Chauhan <nirbheek@gentoo.org> schrieb:
>> > Thats not even necessary. They just should use the infrastructure,
>> > as described in my paper. So everyone can easily set up automatic
>> > notifications, cherry-pick, etc, etc.
>>
>> Why should we?
>
> To make tracking and applying other's changes much easier.
> Currently it's quite work intensive to do so.
>
> If - hypothetically - everyone work within an git repo, using a
> normalized naming/versioning scheme, it is trivial to set some
> little tracking system, informing package maintainers if something
> happens (new upstream, new patches from distro XYZ, etc). And
> we can inspect that easily, take patches, etc, etc, using
> standard git tools.
>

Ah, so you want us to use your git repos as patch managers? That
clears up a few things.

>> No, it does not. The security problems come because you are the single
>> point of failure.
>
> Which SPOF ?
>
> man 1 git-cone ;-p
>

So you don't want us to use your tarballs? That's a relief. If you'd
tell us on the bugs you opened that you want us to start using your
git repos, and add another layer of abstraction to our patch process,
say so! Then we can clearly say why we don't want to use them.

>> The trust problems come because we have no reason to trust you.
>
> You dont have to trust me. Just let me point out some possible
> leak points (if I missed some, feel free to add them ...):
>

[snip info about basic git working]
>   BTW: as long as not all upstreams sign their releases, our trust
>   relies just relies on their server's integrity (and the connection
>   to them).
>

Difference is that there are multiple upstreams while you are one, and
the larger upstreams (such as GNOME/KDE/FDO) have a professional
admins devoted to the security of their machines, while you're using a
free service on a public git website.

[snip proposal about us changing our workflow]

> d) some vendor (possibly myself) adds crappy changes: you'll most
>   likely have a look at the changes before cherry-picking them.
>

Makes sense. If we use your git repos for pulling patches we'll verify
them before applying them locally.

>> The redundancy problems come because if your hosting goes
>> down or you lose interest, we're left high and dry.
>
> That wouldn't affect your clones. You simply won't get anything
> new from my site anymore.
>

Of course, since now I understand that you don't want us to use your
tarballs, the hosting problem is a non-issue.

Oh wait, I just remembered. All the ebuilds that you submitted use
*your* tarballs.  And since you want us to use snapshot tarballs, the
same old problems of trust, security, redundancy come into play. If
you went away, all of them would break and we would need to go through
each and every ebuild, fix the SRC_URI to point back to upstream, and
apply the patches the way we do now.

Please decide if you want us to use your git repos as patch
aggregators or as snapshot tarball sources. Although, the question is
quite futile. The latter is completely unacceptable, and the former is
a major change in workflow and a dependency on you that few (if any)
will accept.

>> > Meanwhile I dont need it anymore, since I gave up maintaining
>> > plaintext patches in favour of git. And that makes my daily works
>> > _much_ easier.
>> >
>>
>> You don't need to maintain **anything** manually if you code the
>> website properly. That's the whole point. You get major benefits with
>> minimal long-term work which can be done by a single person in their
>> free time.
>
> First, I have to build that website and maintain it over a long time.
> Then I'll have to do a lot of advertisement work to get people to
> actually push their patches there.
>

Why would they push their patches? You'd be an RSS reader. An
aggregator, You wouldn't need anyone to push anything.

> On the other hand, the git-based infrastructure is already there,
> people can use it right now. And it gives my exactly what I need.
> So I prefer spending my time in advocating this one.
>

Yes, in a sense you've already made the patch aggregation website.
Except you use git to store whatever patches you get manually from the
various sources. In essence, you've traded lots of short-term work for
extended work of manual patch aggregation and integration. Sure,
that's your choice, but that ensures that we will never make our
workflow dependent on your continued interest in the project.

>> This job is easily automated to simply aggregate links to patches
>> which all the distros manually publish themselves.
>
> It's not that simple. Many distros don't even do proper patches,
> instead wildly copy over or directly sed certain sourcesfiles,
> or even (like Debian) use their own broken tarballs.
> (the worst srcpkg I've ever seen is Debian's mysql-5.0.32 ...)
>

*Shrug* you aggregate whatever is there in patch form. Refinements can
happen later. The same problem is also present in your current
approach, but that didn't stop you.

> And even *if* we assume, that everyone's just doing patches, we
> still need to transform the everbody's strange (often not even
> linear) versioning schemes. One of OSS-QMs primary goals is to
> use a strictly normalized/canonical naming/versioning scheme in
> the primay source repository.
>

I'm reading this as: "I want everyone to change to my way of doing
things because it supposedly makes patch management easier". But since
you say that you're only offering a service, and we aren't compelled
to use it, I won't feel bad about firmly saying "No thanks. Most of us
see no net benefit in relation to the proposed change in workflow".

On the other hand, if you propose benefits w/o a change of workflow,
I'm sure many more people will be interested. Otherwise, all your
current efforts *will* go to waste. Infact, I've explained this twice
now[1], and you should really understand that people will not change
their workflow unless you give them a very good reason to do so.
"Possibly better patch management as long as I'm interested in it" is
not a valid reason (to say the least).


1. This means I won't try to explain it again.

-- 
~Nirbheek Chauhan

Gentoo GNOME+Mozilla Team

Re: [gentoo-dev] [bugzilla-daemon@gentoo.org: [Bug 322157] [mail-filter/procmail] new ebuild + autocreate maildirs]

[Enrico Weigelt]

* Nirbheek Chauhan <nirbheek@gentoo.org> schrieb:

> Ah, so you want us to use your git repos as patch managers? That
> clears up a few things.

I dont want you to use *my* repos. But I'd like to advocate git-based
workflows (eg. downstream branches w/ rebase, etc) instead of loose
patches. And I'm offering you an proven model for that.

> >> No, it does not. The security problems come because you are the single
> >> point of failure.
> >
> > Which SPOF ?
> >
> > man 1 git-cone ;-p
> >
> 
> So you don't want us to use your tarballs? 

Why still using tarballs at all (if the UPSTREAM.* branches actually
come from tarballs is a different topic) ? Even on projects w/ heavy
change rates, eg. the Linux kernel, the git repo isn't much larger
than a release tarball. (and here, the vast majority is shared between
several kernel types, eg. vanilla <-> ovz <-> xen).

Tarballs provide only work on the granularity of one big tree at once,
so support only one operation: fetch a whole tree at once. 
No differential transmissions or storage, no changesets, etc, etc.

> >   BTW: as long as not all upstreams sign their releases, our trust
> >   relies just relies on their server's integrity (and the connection
> >   to them).
> 
> Difference is that there are multiple upstreams while you are one, and
> the larger upstreams (such as GNOME/KDE/FDO) have a professional
> admins devoted to the security of their machines, while you're using a
> free service on a public git website.

I never said, that I want to be your (only) upstream. Again: all I'm 
offering is a generic model (and a fist reference implementation).

> > d) some vendor (possibly myself) adds crappy changes: you'll most
> >   likely have a look at the changes before cherry-picking them.
> 
> Makes sense. If we use your git repos for pulling patches we'll verify
> them before applying them locally.

Right. And you would put your changes in your repos and automatically
pushing it back to the concentrated repository, and other distros would
do the same. So in the end, everybody still has his own repo, but also
everything's collected in a central place.

> > That wouldn't affect your clones. You simply won't get anything
> > new from my site anymore.
> 
> Of course, since now I understand that you don't want us to use your
> tarballs, the hosting problem is a non-issue.

Exactly.

> Oh wait, I just remembered. All the ebuilds that you submitted use
> *your* tarballs.  And since you want us to use snapshot tarballs, the
> same old problems of trust, security, redundancy come into play. 

Just make your own repo, maybe put into into an simple dns-based 
cluster (multiple a-records), and send me the pointer - so I'll give
you a fixed ebuild. Trivial.

> Please decide if you want us to use your git repos as patch
> aggregators or as snapshot tarball sources. 

Please differentiate between the model and the concrete implementation.
The model just specifies the basic logic structure, eg. naming etc.
But the infrastructure is an different issue. You'd most likely have
your own repos, but I'd like to have your refs automatically pushed
(within your namespace) into my aggregator repos.

> > First, I have to build that website and maintain it over a long time.
> > Then I'll have to do a lot of advertisement work to get people to
> > actually push their patches there.
> >
> 
> Why would they push their patches? You'd be an RSS reader. An
> aggregator, You wouldn't need anyone to push anything.

Assuming they all publish their patches via RSS-feeds.

And still this would only operate on single patches, not changesets
and branches w/ histories.
 
> > On the other hand, the git-based infrastructure is already there,
> > people can use it right now. And it gives my exactly what I need.
> > So I prefer spending my time in advocating this one.
> 
> Yes, in a sense you've already made the patch aggregation website.
> Except you use git to store whatever patches you get manually from 
> the various sources. 

No, I've made a branch aggregator. Git does not store patches, it 
stores histories of trees. Thats completely different, see above. 

> > It's not that simple. Many distros don't even do proper patches,
> > instead wildly copy over or directly sed certain sourcesfiles,
> > or even (like Debian) use their own broken tarballs.
> > (the worst srcpkg I've ever seen is Debian's mysql-5.0.32 ...)
> 
> *Shrug* you aggregate whatever is there in patch form. Refinements 
> can happen later. 

I could only aggregate patches, not other changes (eg. those done
by those ugly sed hacks), and also missing the right apply order.
So I'd loose a lot of important information.

> The same problem is also present in your current approach, but
> that didn't stop you.

Right, that's why I'd people to use a proper vcs from start up,
and I'll step by step tweak certain packaging systems to create
git commits. (eg. a tweaked portage could import epatches directly
into git and also commits between all commands that might change
the tree, within the src_unpack stage).

> On the other hand, if you propose benefits w/o a change of workflow,
> I'm sure many more people will be interested. 

The major benefit is that changesets can be easily shared between
various distros and upstreams, up to fully notifications, etc.


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------