[gentoo-dev] questions about small fixes/cleanups

[gentoo-dev] questions about small fixes/cleanups

[Michael Mair-Keimberger]

[-- Attachment #1: Type: text/plain, Size: 4037 bytes --]

Hello Gentoo-Team,

For some time now I'm trying to help improve the gentoo tree. I've
started with fixing some HOMEPAGE variables in ebuilds and now removing
obsolete patches from packages.
Usually (especially with obsolete patches) it's quite simple to fix
things, however every now and then i find some "special" cases were i
don't know whats correct way to fix it.
Even though i could ask on irc, I'm usually to lazy and now there are a
few question were i would like to have an answer for. :)

Homepages:
* Dead homepages: I know there is this general rule of pointing to the
	No_homepage wiki page if there is no upstream homepage. However, 
	when I fixed some homepage's i usually used an archived [1] homepage
	of the original homepage. While for some dev's this isn't a
	problem, others do not like the idea.
	Personally i would prefer having some homepage rather than no
	homepage at all. It would be nice to have a clear statement 
	about that.
* Waiting time: I couldn't find any references of how long to wait until
	a homepage can be changed because it's offline. Usually i would
	check for bugs, because if the bug is already quite old and the
	homepage is still offline it's safe to say we can change them.
	However, i also have an script which checks for offline
	homepages. But since a homepage can be temporally offline or just
	not available from my side of the internet i would like to know
	how long i should wait until i can fix it. I think 1 month
	would be a good start?
* Redirection: There are quite a few pages which aren't exactly offline,
	but only forward the request to the current homepage. (like most
	of the gentoo project pages). I haven't touch them yet, but i
	would like to know if fixing them would be appreciated?
* old ebuilds: It's often the case that if a homepage get's changed,
	older versions of the ebuild still use the old homepage. While
	it looks like portage/eix always show's the homepage of the
	newest ebuild i would like to know if older versions should be
	updated as well. Especially if only the homepage gets changed
	which usually doesn't require a reversion bump.

Patches:
* Wildcard patching: Usually i use my "patchtest" script for finding
	obsolete patches and while it still finds lots of false
	positives i also saw a rather odd patching style: Patching via
	wildcards. For example: 
	epatch "${FILESDIR}"/${P}*.patch
	I looks a bit unsafe to use wildcards for patching, but I
	couldn't find anything which forbids it. Would be nice to know
	if that is ok.
* scripts in FILESDIR: In some packages i found scripts which doesn't
	get used by the ebuild, but is probably used for generating
	patches/tarballs. Should i file a bugs about them as i though the
	FILESDIR should be only used for patches. Wouldn't be
	/etc/portage/scripts the perfect place for such scripts?

Metadaa:
* metadata.xml: In order to find the correct maintainer for PR's I've
	opened on github i also wrote a simply python scripts which 
	extract's the metadata.xml files.
	Regarding GLEP 68 a name isn't required for a maintainer. However,
	if i see only a e-mail address in one package, the same e-mail in
	another package but also with a name, is it appreciated to fix
	the first package?

I hope those questions aren't too stupid and i hope i can get some
answers as it would make some decisions a bit easier..
Also please don't judge my English, it's not the best :)

Furthermore i also suggest creating a wiki page for people who look for
simple tasks were they can help improve gentoo.
After all, removing obsolete patches or update HOMEPAGE variables isn't
really difficult, but it's a nice start to get involved. - and those are
just two tasks, which btw are already quite heavy. (it's about 2 months now
since i look for obsolete patches, and I'm now at dev-python...).
I know i could start that by myself, but I'm really bad at wikis :D


[1] http://libarchive.org/

-- 
greetings
Michael Mair-Keimberger

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [gentoo-dev] questions about small fixes/cleanups

[M. J. Everitt]

[-- Attachment #1: Type: text/plain, Size: 5167 bytes --]

On 13/09/16 19:57, Michael Mair-Keimberger wrote:
> Hello Gentoo-Team,
>
> For some time now I'm trying to help improve the gentoo tree. I've
> started with fixing some HOMEPAGE variables in ebuilds and now removing
> obsolete patches from packages.
> Usually (especially with obsolete patches) it's quite simple to fix
> things, however every now and then i find some "special" cases were i
> don't know whats correct way to fix it.
> Even though i could ask on irc, I'm usually to lazy and now there are a
> few question were i would like to have an answer for. :)
>
> Homepages:
> * Dead homepages: I know there is this general rule of pointing to the
> 	No_homepage wiki page if there is no upstream homepage. However, 
> 	when I fixed some homepage's i usually used an archived [1] homepage
> 	of the original homepage. While for some dev's this isn't a
> 	problem, others do not like the idea.
> 	Personally i would prefer having some homepage rather than no
> 	homepage at all. It would be nice to have a clear statement 
> 	about that.
> * Waiting time: I couldn't find any references of how long to wait until
> 	a homepage can be changed because it's offline. Usually i would
> 	check for bugs, because if the bug is already quite old and the
> 	homepage is still offline it's safe to say we can change them.
> 	However, i also have an script which checks for offline
> 	homepages. But since a homepage can be temporally offline or just
> 	not available from my side of the internet i would like to know
> 	how long i should wait until i can fix it. I think 1 month
> 	would be a good start?
> * Redirection: There are quite a few pages which aren't exactly offline,
> 	but only forward the request to the current homepage. (like most
> 	of the gentoo project pages). I haven't touch them yet, but i
> 	would like to know if fixing them would be appreciated?
> * old ebuilds: It's often the case that if a homepage get's changed,
> 	older versions of the ebuild still use the old homepage. While
> 	it looks like portage/eix always show's the homepage of the
> 	newest ebuild i would like to know if older versions should be
> 	updated as well. Especially if only the homepage gets changed
> 	which usually doesn't require a reversion bump.
>
> Patches:
> * Wildcard patching: Usually i use my "patchtest" script for finding
> 	obsolete patches and while it still finds lots of false
> 	positives i also saw a rather odd patching style: Patching via
> 	wildcards. For example: 
> 	epatch "${FILESDIR}"/${P}*.patch
> 	I looks a bit unsafe to use wildcards for patching, but I
> 	couldn't find anything which forbids it. Would be nice to know
> 	if that is ok.
> * scripts in FILESDIR: In some packages i found scripts which doesn't
> 	get used by the ebuild, but is probably used for generating
> 	patches/tarballs. Should i file a bugs about them as i though the
> 	FILESDIR should be only used for patches. Wouldn't be
> 	/etc/portage/scripts the perfect place for such scripts?
>
> Metadaa:
> * metadata.xml: In order to find the correct maintainer for PR's I've
> 	opened on github i also wrote a simply python scripts which 
> 	extract's the metadata.xml files.
> 	Regarding GLEP 68 a name isn't required for a maintainer. However,
> 	if i see only a e-mail address in one package, the same e-mail in
> 	another package but also with a name, is it appreciated to fix
> 	the first package?
>
> I hope those questions aren't too stupid and i hope i can get some
> answers as it would make some decisions a bit easier..
> Also please don't judge my English, it's not the best :)
>
> Furthermore i also suggest creating a wiki page for people who look for
> simple tasks were they can help improve gentoo.
> After all, removing obsolete patches or update HOMEPAGE variables isn't
> really difficult, but it's a nice start to get involved. - and those are
> just two tasks, which btw are already quite heavy. (it's about 2 months now
> since i look for obsolete patches, and I'm now at dev-python...).
> I know i could start that by myself, but I'm really bad at wikis :D
>
>
> [1] http://libarchive.org/
>
I strongly recommend getting in touch with the Gentoo QA team either via
their qa@g.o alias, or come to #gentoo-qa on Freenode IRC. This kind of
thing could be really helpful for keeping the tree 'in-shape' and the
more people who can contribute effective scripts to help identify and
rectify 'bad' ebuilds the better. Feel free to idle in the #gentoo-*
channels .. everyone else does, and you can always scroll back to see
what's going on, and ask for more immediate assistance, leave notes for
devs, etc quite easily.

Dev's should be committing their new files using a tool called 'repoman'
which is maintained by the Gentoo Portage team (dev-portage@g.o) who may
also benefit from some bad examples to trap in this utility. It's
recently been split from the Portage package into its own
dev-portage/repoman which you may also find useful. Portage's git repo
is at https://gitweb.gentoo.org/proj/portage.git/ if you wish to peruse!
Cheers,
Michael.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]

Re: [gentoo-dev] questions about small fixes/cleanups

[Daniel Campbell]

[-- Attachment #1.1: Type: text/plain, Size: 5219 bytes --]

On 09/13/2016 11:57 AM, Michael Mair-Keimberger wrote:
> Hello Gentoo-Team,
> 
> For some time now I'm trying to help improve the gentoo tree. I've
> started with fixing some HOMEPAGE variables in ebuilds and now removing
> obsolete patches from packages.
> Usually (especially with obsolete patches) it's quite simple to fix
> things, however every now and then i find some "special" cases were i
> don't know whats correct way to fix it.
> Even though i could ask on irc, I'm usually to lazy and now there are a
> few question were i would like to have an answer for. :)
> 
> Homepages:
> * Dead homepages: I know there is this general rule of pointing to the
> 	No_homepage wiki page if there is no upstream homepage. However, 
> 	when I fixed some homepage's i usually used an archived [1] homepage
> 	of the original homepage. While for some dev's this isn't a
> 	problem, others do not like the idea.
> 	Personally i would prefer having some homepage rather than no
> 	homepage at all. It would be nice to have a clear statement 
> 	about that.
> * Waiting time: I couldn't find any references of how long to wait until
> 	a homepage can be changed because it's offline. Usually i would
> 	check for bugs, because if the bug is already quite old and the
> 	homepage is still offline it's safe to say we can change them.
> 	However, i also have an script which checks for offline
> 	homepages. But since a homepage can be temporally offline or just
> 	not available from my side of the internet i would like to know
> 	how long i should wait until i can fix it. I think 1 month
> 	would be a good start?
> * Redirection: There are quite a few pages which aren't exactly offline,
> 	but only forward the request to the current homepage. (like most
> 	of the gentoo project pages). I haven't touch them yet, but i
> 	would like to know if fixing them would be appreciated?
> * old ebuilds: It's often the case that if a homepage get's changed,
> 	older versions of the ebuild still use the old homepage. While
> 	it looks like portage/eix always show's the homepage of the
> 	newest ebuild i would like to know if older versions should be
> 	updated as well. Especially if only the homepage gets changed
> 	which usually doesn't require a reversion bump.
> 
> Patches:
> * Wildcard patching: Usually i use my "patchtest" script for finding
> 	obsolete patches and while it still finds lots of false
> 	positives i also saw a rather odd patching style: Patching via
> 	wildcards. For example: 
> 	epatch "${FILESDIR}"/${P}*.patch
> 	I looks a bit unsafe to use wildcards for patching, but I
> 	couldn't find anything which forbids it. Would be nice to know
> 	if that is ok.
> * scripts in FILESDIR: In some packages i found scripts which doesn't
> 	get used by the ebuild, but is probably used for generating
> 	patches/tarballs. Should i file a bugs about them as i though the
> 	FILESDIR should be only used for patches. Wouldn't be
> 	/etc/portage/scripts the perfect place for such scripts?
> 
> Metadaa:
> * metadata.xml: In order to find the correct maintainer for PR's I've
> 	opened on github i also wrote a simply python scripts which 
> 	extract's the metadata.xml files.
> 	Regarding GLEP 68 a name isn't required for a maintainer. However,
> 	if i see only a e-mail address in one package, the same e-mail in
> 	another package but also with a name, is it appreciated to fix
> 	the first package?
> 
> I hope those questions aren't too stupid and i hope i can get some
> answers as it would make some decisions a bit easier..
> Also please don't judge my English, it's not the best :)
> 
> Furthermore i also suggest creating a wiki page for people who look for
> simple tasks were they can help improve gentoo.
> After all, removing obsolete patches or update HOMEPAGE variables isn't
> really difficult, but it's a nice start to get involved. - and those are
> just two tasks, which btw are already quite heavy. (it's about 2 months now
> since i look for obsolete patches, and I'm now at dev-python...).
> I know i could start that by myself, but I'm really bad at wikis :D
> 
> 
> [1] http://libarchive.org/
> 
I think a lot of your suggestions are sane and could improve the quality
of the tree quite a bit. I'm not part of the QA team so I don't know
their opinions on those particular things (in fact they've not really
updated the wiki much on their policies so it's hard to check things at
a glance...). If I could suggest something, it'd be encouraging
conversation with maintainers, as QA policy and making changes won't
produce any lasting improvement unless maintainers understand what
they're screwing up and/or have written guidelines they can check
periodically.

Integration with repoman would be the single most effective way to make
that happen, but I have to wonder if repoman should be taking on that
much responsibility. It's not unreasonable to expect maintainers to
actually visit the package's URL to make sure upstream is doing well.

-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [gentoo-dev] questions about small fixes/cleanups

[Michael Orlitzky]

On 09/13/2016 02:57 PM, Michael Mair-Keimberger wrote:
> 
> * Redirection: There are quite a few pages which aren't exactly offline,
> 	but only forward the request to the current homepage. (like most
> 	of the gentoo project pages). I haven't touch them yet, but i
> 	would like to know if fixing them would be appreciated?

Yes please.


> * old ebuilds: It's often the case that if a homepage get's changed,
> 	older versions of the ebuild still use the old homepage. While
> 	it looks like portage/eix always show's the homepage of the
> 	newest ebuild i would like to know if older versions should be
> 	updated as well. Especially if only the homepage gets changed
> 	which usually doesn't require a reversion bump.

I would personally leave these alone, but if you want to fix them, go
ahead. The HOMEPAGE is a weird variable, and belongs in metadata.xml
instead. Users actually want to view it, and not on a per-ebuild basis,
but on a per-package one. As a result, all existing package managers
already do the obvious thing and show the newest HOMEPAGE, making the
value in older ebuilds irrelevant.

If the fix were important, you should do a revision bump, but due to the
unusual nature of HOMEPAGE, the change isn't that important.


> Patches:
> * Wildcard patching: Usually i use my "patchtest" script for finding
> 	obsolete patches and while it still finds lots of false
> 	positives i also saw a rather odd patching style: Patching via
> 	wildcards. For example: 
> 	epatch "${FILESDIR}"/${P}*.patch
> 	I looks a bit unsafe to use wildcards for patching, but I
> 	couldn't find anything which forbids it. Would be nice to know
> 	if that is ok.

Please file a bug and tell people not to do that. If they ever need to
add a new patch, it would affect old stable ebuilds and violate our
policy on the matter.

It would be nice if we could add this check to repoman; however, some
wildcard constructs are safe. For example, if we're pulling in a bunch
of Debian patches in SRC_URI, it makes sense to apply them all. The
SRC_URI is guaranteed not to change out from under us, unlike the
contents of FILESDIR.


> * scripts in FILESDIR: In some packages i found scripts which doesn't
> 	get used by the ebuild, but is probably used for generating
> 	patches/tarballs. Should i file a bugs about them as i though the
> 	FILESDIR should be only used for patches. Wouldn't be
> 	/etc/portage/scripts the perfect place for such scripts?


If they're not used by the tree, they probably don't belong in the tree,
but maybe I lack imagination. Can you give a few examples?

Re: [gentoo-dev] questions about small fixes/cleanups

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 3768 bytes --]

On Tue, 13 Sep 2016 19:49:22 -0400
Michael Orlitzky <mjo@gentoo.org> wrote:

> > Patches:
> > * Wildcard patching: Usually i use my "patchtest" script for finding
> > 	obsolete patches and while it still finds lots of false
> > 	positives i also saw a rather odd patching style: Patching via
> > 	wildcards. For example: 
> > 	epatch "${FILESDIR}"/${P}*.patch
> > 	I looks a bit unsafe to use wildcards for patching, but I
> > 	couldn't find anything which forbids it. Would be nice to know
> > 	if that is ok.  
> 
> Please file a bug and tell people not to do that. If they ever need to
> add a new patch, it would affect old stable ebuilds and violate our
> policy on the matter.
> 
> It would be nice if we could add this check to repoman; however, some
> wildcard constructs are safe. For example, if we're pulling in a bunch
> of Debian patches in SRC_URI, it makes sense to apply them all. The
> SRC_URI is guaranteed not to change out from under us, unlike the
> contents of FILESDIR.


Clearly the problem is not the use of wildcards in "epatch", but the use of wildcards
anywhere in conjunction with "${FILESDIR}"

For instance, if they were not patches, but blobs of shell script ala eblits, you might
see the following happen:

   for i in "${FILESDIR}"/${P}*.sh"; do 
      source $i
   done

This would also be a QA Violation because it would be just as likely to be broken by bumps
as patches.

> 
> > * scripts in FILESDIR: In some packages i found scripts which doesn't
> > 	get used by the ebuild, but is probably used for generating
> > 	patches/tarballs. Should i file a bugs about them as i though the
> > 	FILESDIR should be only used for patches. Wouldn't be
> > 	/etc/portage/scripts the perfect place for such scripts?  
> 
> 
> If they're not used by the tree, they probably don't belong in the tree,
> but maybe I lack imagination. Can you give a few examples?

I agree if that if the scripts in question aren't used by the ebuilds, then 
grounds to keep them included are weakened.

However, It rather depends on what those scripts are doing and how important
it is to keep those scripts synchronised between developers working on a given thing.

If for instance a given specific ebuild needs some scripts to generate parts of ebuilds
in a consistent way, but the result of it is static code that is placed in the ebuild,
then under the grounds of "ebuild doesn't use it", it would be removed.

But that script has no other place to be stored that makes sense, because putting it
in an isolated repo just hides the existence of that script, and discourages its
use when its use should be encouraged.

And putting it somewhere other than gentoo.git means there's another step required
by maintainers to keep the tree "updated" and coherent, and reduces another place
failures can leak in if they forget to sync the independent repository.

Thus, it introduces a conflict of purpose: 

1. We need certain things to be centralised and "in the repo" for the point of 
maintainership and coherence.

2. We desire things that users don't strictly need don't get replicated where
they're not wanted.

A dogma of keeping the tree clean thus penalises how developers work, and makes
it harder to produce the quality we want to deliver to users, but having them
in tree reduces the quality we deliver to users.

Though, to be pragmatic, the burden of having those scripts present in tree
seems more a cosmetic problem from the users perspective, while removing them
produces a functional problem.

So I'd reason that as long as these scripts follow policy of not being
too large, and they're not abused everywhere, the pragmatic view is to permit
them staying.



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [gentoo-dev] questions about small fixes/cleanups

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 841 bytes --]

>>>>> On Tue, 13 Sep 2016, Michael Orlitzky wrote:

>> * scripts in FILESDIR: In some packages i found scripts which
>> doesn't get used by the ebuild, but is probably used for generating
>> patches/tarballs. Should i file a bugs about them as i though the
>> FILESDIR should be only used for patches. Wouldn't be
>> /etc/portage/scripts the perfect place for such scripts?

> If they're not used by the tree, they probably don't belong in the
> tree, but maybe I lack imagination. Can you give a few examples?

We had a review of such files before the git conversion:
https://bugs.gentoo.org/550434

Especially, there's a list of "maintainer scripts" in comment #13.
At the time, we didn't do anything about them. There are very few of
such files (namely, 12 when that comment was written) so the space
taken by them is negligible.

Ulrich

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-dev] questions about small fixes/cleanups

[Michael Orlitzky]

On 09/14/2016 01:54 AM, Ulrich Mueller wrote:
> 
> We had a review of such files before the git conversion:
> https://bugs.gentoo.org/550434
> 
> Especially, there's a list of "maintainer scripts" in comment #13.
> At the time, we didn't do anything about them. There are very few of
> such files (namely, 12 when that comment was written) so the space
> taken by them is negligible.
> 

I'll take the low hanging fruit:

  * dev-libs/collada-dom/files/take_snapshot.sh
  * sys-libs/libcxxrt/files/prepare_snapshot.sh

Those two are completely trivial; clone a repo, rm -rf .git, tar it up.

  * sys-libs/libcxx/files/prepare_snapshot.sh

This one's only marginally more complicated because it uses find/xargs
to remove all ".svn" directories (why not use svn export?). It also
appears unused, now that there are regular releases being made upstream.

  * sys-apps/superiotool/files/make-tarball.sh

Another trivial three-liner; svn export and tar it up.

  * net-misc/ntp/files/man-pages/genmans.sh

Generating man pages from HTML can be done in src_compile().

Re: [gentoo-dev] questions about small fixes/cleanups

[Alexis Ballier]

On Wed, 14 Sep 2016 08:45:58 -0400
Michael Orlitzky <mjo@gentoo.org> wrote:

> On 09/14/2016 01:54 AM, Ulrich Mueller wrote:
> > 
> > We had a review of such files before the git conversion:
> > https://bugs.gentoo.org/550434
> > 
> > Especially, there's a list of "maintainer scripts" in comment #13.
> > At the time, we didn't do anything about them. There are very few of
> > such files (namely, 12 when that comment was written) so the space
> > taken by them is negligible.
> >   
> 
> I'll take the low hanging fruit:
> 
>   * dev-libs/collada-dom/files/take_snapshot.sh
>   * sys-libs/libcxxrt/files/prepare_snapshot.sh
> 
> Those two are completely trivial; clone a repo, rm -rf .git, tar it
> up.


Guess you never had to maintain packages whose releases are only
snapshots. They're trivial but it's a waste of time to re-assemble all
the bits every couple of months when you want to make a snapshot.

Questions one needs to answer:

- What SCM is used ?
- What is the URI ?
- How to fetch it ?
- What should be cleaned up after fetching ?
- What dirname to use so that it matches ebuilds' $S ?
- What compression to use so that it matches SRC_URI ?




Those scripts are made to automate this, and as you can see are closely
related to ebuilds, and thus fit perfectly next to them.

Re: [gentoo-dev] questions about small fixes/cleanups

[Michael Orlitzky]

On 09/14/2016 09:21 AM, Alexis Ballier wrote:
> 
> Guess you never had to maintain packages whose releases are only
> snapshots. They're trivial but it's a waste of time to re-assemble all
> the bits every couple of months when you want to make a snapshot.
> 
> Questions one needs to answer:
> 
> - What SCM is used ?
> - What is the URI ?
> - How to fetch it ?
> - What should be cleaned up after fetching ?

These are all the same question, which does take a second to google, but
isn't all that time consuming.


> - What dirname to use so that it matches ebuilds' $S ?
> - What compression to use so that it matches SRC_URI ?

cat Manifest

But seriously, once you've googled the repo, you might notice that it's
hosted at github, and that you don't need to create your own snapshot
tarballs any more. Want a snapshot from 20150423? Try,

https://github.com/pathscale/libcxxrt/archive/665b0223ee27547175dc327a386fcd88243c7677.tar.gz

Put COMMIT=665b0223ee27547175dc327a386fcd88243c7677 in the ebuild, stick
that in $SRC_URI and $S, and then future snapshots will work without
modification.

Re: [gentoo-dev] questions about small fixes/cleanups

[Alexis Ballier]

On Wed, 14 Sep 2016 09:38:45 -0400
Michael Orlitzky <mjo@gentoo.org> wrote:

> On 09/14/2016 09:21 AM, Alexis Ballier wrote:
> > 
> > Guess you never had to maintain packages whose releases are only
> > snapshots. They're trivial but it's a waste of time to re-assemble
> > all the bits every couple of months when you want to make a
> > snapshot.
> > 
> > Questions one needs to answer:
> > 
> > - What SCM is used ?
> > - What is the URI ?
> > - How to fetch it ?
> > - What should be cleaned up after fetching ?  
> 
> These are all the same question, which does take a second to google,
> but isn't all that time consuming.

Manual stuff isnt that time consuming until you realize it can be
automated :)

> > - What dirname to use so that it matches ebuilds' $S ?
> > - What compression to use so that it matches SRC_URI ?  
> 
> cat Manifest

not really

> 
> But seriously, once you've googled the repo, you might notice that
> it's hosted at github, and that you don't need to create your own
> snapshot tarballs any more. Want a snapshot from 20150423? Try,
> 
> https://github.com/pathscale/libcxxrt/archive/665b0223ee27547175dc327a386fcd88243c7677.tar.gz
> 
> Put COMMIT=665b0223ee27547175dc327a386fcd88243c7677 in the ebuild,
> stick that in $SRC_URI and $S, and then future snapshots will work
> without modification.

that might be better, but how do you map date / $PV to commit ?

remember, we want ebuilds that are as much as possible
version-agnostic, otherwise it breaks simple copy of ebuilds to bump a
version...

Re: [gentoo-dev] questions about small fixes/cleanups

[James Le Cuirot]

On Wed, 14 Sep 2016 15:50:28 +0200
Alexis Ballier <aballier@gentoo.org> wrote:

> > But seriously, once you've googled the repo, you might notice that
> > it's hosted at github, and that you don't need to create your own
> > snapshot tarballs any more. Want a snapshot from 20150423? Try,
> > 
> > https://github.com/pathscale/libcxxrt/archive/665b0223ee27547175dc327a386fcd88243c7677.tar.gz
> > 
> > Put COMMIT=665b0223ee27547175dc327a386fcd88243c7677 in the ebuild,
> > stick that in $SRC_URI and $S, and then future snapshots will work
> > without modification.  
> 
> that might be better, but how do you map date / $PV to commit ?

You can't.

> remember, we want ebuilds that are as much as possible
> version-agnostic, otherwise it breaks simple copy of ebuilds to bump a
> version...

Ideally yes but this still beats manually preparing a tarball. I use
this approach a lot.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

Re: [gentoo-dev] questions about small fixes/cleanups

[James Le Cuirot]

On Wed, 14 Sep 2016 14:53:18 +0100
James Le Cuirot <chewi@gentoo.org> wrote:

> On Wed, 14 Sep 2016 15:50:28 +0200
> Alexis Ballier <aballier@gentoo.org> wrote:
> 
> > > But seriously, once you've googled the repo, you might notice that
> > > it's hosted at github, and that you don't need to create your own
> > > snapshot tarballs any more. Want a snapshot from 20150423? Try,
> > > 
> > > https://github.com/pathscale/libcxxrt/archive/665b0223ee27547175dc327a386fcd88243c7677.tar.gz
> > > 
> > > Put COMMIT=665b0223ee27547175dc327a386fcd88243c7677 in the ebuild,
> > > stick that in $SRC_URI and $S, and then future snapshots will work
> > > without modification.    
> > 
> > that might be better, but how do you map date / $PV to commit ?  
> 
> You can't.

Sorry, let me correct that, you can't by date but you can by $PV if
they have created versioned git tags. Just swap the commit reference
for the tag name.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

Re: [gentoo-dev] questions about small fixes/cleanups

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 1073 bytes --]

On Wed, 14 Sep 2016 14:55:06 +0100
James Le Cuirot <chewi@gentoo.org> wrote:

> On Wed, 14 Sep 2016 14:53:18 +0100
> James Le Cuirot <chewi@gentoo.org> wrote:
> 
> > On Wed, 14 Sep 2016 15:50:28 +0200
> > Alexis Ballier <aballier@gentoo.org> wrote:
> >   
>  [...]  
>  [...]  
> > 
> > You can't.  
> 
> Sorry, let me correct that, you can't by date but you can by $PV if
> they have created versioned git tags. Just swap the commit reference
> for the tag name.
> 

Also, you'll probably want one of

    SRC_URI="<WHATEVER>/archive/${SHA1}.tar.gz -> ${PN}-${SHA1}.tar.gz" 
    SRC_URI="<WHATEVER>/archive/${TAG}.tar.gz -> ${PN}-${TAG}.tar.gz" 

Because otherwise your tars are just going to be distfiles/dcd4e6df145853de19af532b0f581a8f0ddfe5ae.tar.gz
or distfiles/0.1.0.tar.gz ( and the latter is destined to collide )

and you'll then also need one of

  S="${WORKDIR}/${PN}-${SHA1}"
  S="${WORKDIR}/${PN}-${TAG}"

But if $TAG contains a "v" prefix, the first will work, but the second will
 need the "v" stripped.

Because github.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [gentoo-dev] questions about small fixes/cleanups

[Alexis Ballier]

On Wed, 14 Sep 2016 14:55:06 +0100
James Le Cuirot <chewi@gentoo.org> wrote:

> On Wed, 14 Sep 2016 14:53:18 +0100
> James Le Cuirot <chewi@gentoo.org> wrote:
> 
> > On Wed, 14 Sep 2016 15:50:28 +0200
> > Alexis Ballier <aballier@gentoo.org> wrote:
> >   
> > > > But seriously, once you've googled the repo, you might notice
> > > > that it's hosted at github, and that you don't need to create
> > > > your own snapshot tarballs any more. Want a snapshot from
> > > > 20150423? Try,
> > > > 
> > > > https://github.com/pathscale/libcxxrt/archive/665b0223ee27547175dc327a386fcd88243c7677.tar.gz
> > > > 
> > > > Put COMMIT=665b0223ee27547175dc327a386fcd88243c7677 in the
> > > > ebuild, stick that in $SRC_URI and $S, and then future
> > > > snapshots will work without modification.      
> > > 
> > > that might be better, but how do you map date / $PV to
> > > commit ?    
> > 
> > You can't.  
> 
> Sorry, let me correct that, you can't by date but you can by $PV if
> they have created versioned git tags. Just swap the commit reference
> for the tag name.

We're talking of repos without tags :)

Re: [gentoo-dev] questions about small fixes/cleanups

[Michael Orlitzky]

On 09/14/2016 09:50 AM, Alexis Ballier wrote:
> 
> that might be better, but how do you map date / $PV to commit ?
> 

Well, for that last one, I just looked down the list of commits and
found the last one that happened before the date of the snapshot.

But, if you're creating a new snapshot, it's easy. What day is it?
That's the date of the snapshot, and you want the newest commit.


> remember, we want ebuilds that are as much as possible
> version-agnostic, otherwise it breaks simple copy of ebuilds to bump a
> version...
> 

With the COMMIT=... approach, all you have to do is copy the ebuild to a
new one with today's date, and update the COMMIT variable to the latest
commit listed on github.

There is the possibility that more than one commit can happen on the
same day, but that's a problem with snapshots in general, and not the
method of generating them.

Re: [gentoo-dev] questions about small fixes/cleanups

[Alexis Ballier]

On Wed, 14 Sep 2016 10:04:20 -0400
Michael Orlitzky <mjo@gentoo.org> wrote:

> On 09/14/2016 09:50 AM, Alexis Ballier wrote:
> > 
> > that might be better, but how do you map date / $PV to commit ?
> >   
> 
> Well, for that last one, I just looked down the list of commits and
> found the last one that happened before the date of the snapshot.
> 
> But, if you're creating a new snapshot, it's easy. What day is it?
> That's the date of the snapshot, and you want the newest commit.

Seems I missed some questions in the original reply:
- What day is it ?
- How to write that so that it is PMS-compliant and properly ordered
  and following ebuild's conventions ?

Nothing is hard, it's just a complete waste of time to redo that
every time.


> > remember, we want ebuilds that are as much as possible
> > version-agnostic, otherwise it breaks simple copy of ebuilds to
> > bump a version...
> >   
> 
> With the COMMIT=... approach, all you have to do is copy the ebuild
> to a new one with today's date, and update the COMMIT variable to the
> latest commit listed on github.

So, to sum it up, I have to:
- Open a browser, go to github (*)
- Find out latest commit hash, copy it
- (*) Copy the ebuild, setting a 8 digit version representing the date
- Open an editor
- Edit COMMIT='...' variable by pasting what was found on github. (*)


Thanks, but I prefer by far:
- Run a script
- Copy ebuild, setting version printed by the script.


(*) represents everytime i have to switch my hands between keyboard and
mouse

Re: [gentoo-dev] questions about small fixes/cleanups

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 768 bytes --]

On Wed, 14 Sep 2016 16:41:54 +0200
Alexis Ballier <aballier@gentoo.org> wrote:

> 
> So, to sum it up, I have to:
> - Open a browser, go to github (*)
> - Find out latest commit hash, copy it
> - (*) Copy the ebuild, setting a 8 digit version representing the date
> - Open an editor
> - Edit COMMIT='...' variable by pasting what was found on github. (*)
> 
> 
> Thanks, but I prefer by far:
> - Run a script
> - Copy ebuild, setting version printed by the script.
> 
> 
> (*) represents everytime i have to switch my hands between keyboard and
> mouse
> 


You can also abuse `git ls-remote Path/To/Repo` -> list of SHA1s -> ebuild -> sed

But you're back to writing a script =)

Or you can do that manually, it reduces the number of *'s 

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [gentoo-dev] questions about small fixes/cleanups

[Ian Stakenvicius]

[-- Attachment #1.1: Type: text/plain, Size: 1102 bytes --]

On 14/09/16 12:20 PM, Kent Fredric wrote:
> On Wed, 14 Sep 2016 16:41:54 +0200
> Alexis Ballier <aballier@gentoo.org> wrote:
> 
>>
>> So, to sum it up, I have to:
>> - Open a browser, go to github (*)
>> - Find out latest commit hash, copy it
>> - (*) Copy the ebuild, setting a 8 digit version representing the date
>> - Open an editor
>> - Edit COMMIT='...' variable by pasting what was found on github. (*)
>>
>>
>> Thanks, but I prefer by far:
>> - Run a script
>> - Copy ebuild, setting version printed by the script.
>>
>>
>> (*) represents everytime i have to switch my hands between keyboard and
>> mouse
>>
> 
> 
> You can also abuse `git ls-remote Path/To/Repo` -> list of SHA1s -> ebuild -> sed
> 
> But you're back to writing a script =)
> 
> Or you can do that manually, it reduces the number of *'s 
> 


More to the point of the original query, such scripts don't
necessarily need to exist in the gentoo repo; they can be in local
repos or project overlays instead.

Note also that I'm perfectly fine with scripts like this staying in
${FILESDIR}.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 213 bytes --]

Re: [gentoo-dev] questions about small fixes/cleanups

[Daniel Campbell]

[-- Attachment #1.1: Type: text/plain, Size: 1951 bytes --]

On 09/14/2016 06:21 AM, Alexis Ballier wrote:
> On Wed, 14 Sep 2016 08:45:58 -0400
> Michael Orlitzky <mjo@gentoo.org> wrote:
> 
>> On 09/14/2016 01:54 AM, Ulrich Mueller wrote:
>>>
>>> We had a review of such files before the git conversion:
>>> https://bugs.gentoo.org/550434
>>>
>>> Especially, there's a list of "maintainer scripts" in comment #13.
>>> At the time, we didn't do anything about them. There are very few of
>>> such files (namely, 12 when that comment was written) so the space
>>> taken by them is negligible.
>>>   
>>
>> I'll take the low hanging fruit:
>>
>>   * dev-libs/collada-dom/files/take_snapshot.sh
>>   * sys-libs/libcxxrt/files/prepare_snapshot.sh
>>
>> Those two are completely trivial; clone a repo, rm -rf .git, tar it
>> up.
> 
> 
> Guess you never had to maintain packages whose releases are only
> snapshots. They're trivial but it's a waste of time to re-assemble all
> the bits every couple of months when you want to make a snapshot.
> 
> Questions one needs to answer:
> 
> - What SCM is used ?
> - What is the URI ?
> - How to fetch it ?
> - What should be cleaned up after fetching ?
> - What dirname to use so that it matches ebuilds' $S ?
> - What compression to use so that it matches SRC_URI ?
> 
> 
> 
> 
> Those scripts are made to automate this, and as you can see are closely
> related to ebuilds, and thus fit perfectly next to them.
> 
Why are those ebuilds not live? If upstream doesn't do real releases and
can't even be bothered to tag the commit that marks a release, then why
are you (or someone else) doing all this work? The snapshot scripts
could be replaced by a live ebuild and upstream can handle bugs as they
come since they can't be bothered to version their software.

Just my 2¢.

-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [gentoo-dev] questions about small fixes/cleanups

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 956 bytes --]

On 09/19/2016 09:51 AM, Daniel Campbell wrote:
> On 09/14/2016 06:21 AM, Alexis Ballier wrote:
>> On Wed, 14 Sep 2016 08:45:58 -0400
>> Michael Orlitzky <mjo@gentoo.org> wrote:

..

>>
> Why are those ebuilds not live? If upstream doesn't do real releases and
> can't even be bothered to tag the commit that marks a release, then why
> are you (or someone else) doing all this work? The snapshot scripts
> could be replaced by a live ebuild and upstream can handle bugs as they
> come since they can't be bothered to version their software.

I would expect because live ebuilds can't be keyworded, so snapshot is
the correct way to do it.

That said; I'm not sure if it makes sense to include this sort of
scripts in the Gentoo tree instead of a separate git repo e.g for the
project.

-- 
Kristian Fiskerstrand
OpenPGP certificate reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [gentoo-dev] questions about small fixes/cleanups

[Chí-Thanh Christopher Nguyễn]

[-- Attachment #1: Type: text/plain, Size: 875 bytes --]

Kristian Fiskerstrand schrieb:
>> Why are those ebuilds not live? If upstream doesn't do real releases and
>> can't even be bothered to tag the commit that marks a release, then why
>> are you (or someone else) doing all this work? The snapshot scripts
>> could be replaced by a live ebuild and upstream can handle bugs as they
>> come since they can't be bothered to version their software.
>
> I would expect because live ebuilds can't be keyworded, so snapshot is
> the correct way to do it.

I think the rule of not keywording live ebuilds originally applied only to 
those who fetch git/svn/cvs head.
If a live ebuild fetches a particular revision, then the disadvantage is 
still that it can't be mirrored (hence a snapshot is preferable from user 
POV), but other than that it can be tested fine.


Best regards,
Chí-Thanh Christopher Nguyễn



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] questions about small fixes/cleanups

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 1162 bytes --]

On 09/19/2016 10:00 AM, Chí-Thanh Christopher Nguyễn wrote:
> Kristian Fiskerstrand schrieb:
>>> Why are those ebuilds not live? If upstream doesn't do real releases and
>>> can't even be bothered to tag the commit that marks a release, then why
>>> are you (or someone else) doing all this work? The snapshot scripts
>>> could be replaced by a live ebuild and upstream can handle bugs as they
>>> come since they can't be bothered to version their software.
>>
>> I would expect because live ebuilds can't be keyworded, so snapshot is
>> the correct way to do it.
> 
> I think the rule of not keywording live ebuilds originally applied only
> to those who fetch git/svn/cvs head.
> If a live ebuild fetches a particular revision, then the disadvantage is
> still that it can't be mirrored (hence a snapshot is preferable from
> user POV), but other than that it can be tested fine.

Or a release CD / any offline system, or verifying OpenPGP signature of
the distributed tarball, or...


-- 
Kristian Fiskerstrand
OpenPGP certificate reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [gentoo-dev] questions about small fixes/cleanups

[Alexis Ballier]

On Mon, 19 Sep 2016 00:51:37 -0700
Daniel Campbell <zlg@gentoo.org> wrote:

> On 09/14/2016 06:21 AM, Alexis Ballier wrote:
> > On Wed, 14 Sep 2016 08:45:58 -0400
> > Michael Orlitzky <mjo@gentoo.org> wrote:
> >   
> >> On 09/14/2016 01:54 AM, Ulrich Mueller wrote:  
> >>>
> >>> We had a review of such files before the git conversion:
> >>> https://bugs.gentoo.org/550434
> >>>
> >>> Especially, there's a list of "maintainer scripts" in comment #13.
> >>> At the time, we didn't do anything about them. There are very few
> >>> of such files (namely, 12 when that comment was written) so the
> >>> space taken by them is negligible.
> >>>     
> >>
> >> I'll take the low hanging fruit:
> >>
> >>   * dev-libs/collada-dom/files/take_snapshot.sh
> >>   * sys-libs/libcxxrt/files/prepare_snapshot.sh
> >>
> >> Those two are completely trivial; clone a repo, rm -rf .git, tar it
> >> up.  
> > 
> > 
> > Guess you never had to maintain packages whose releases are only
> > snapshots. They're trivial but it's a waste of time to re-assemble
> > all the bits every couple of months when you want to make a
> > snapshot.
> > 
> > Questions one needs to answer:
> > 
> > - What SCM is used ?
> > - What is the URI ?
> > - How to fetch it ?
> > - What should be cleaned up after fetching ?
> > - What dirname to use so that it matches ebuilds' $S ?
> > - What compression to use so that it matches SRC_URI ?
> > 
> > 
> > 
> > 
> > Those scripts are made to automate this, and as you can see are
> > closely related to ebuilds, and thus fit perfectly next to them.
> >   
> Why are those ebuilds not live?

reproducible builds :)