[gentoo-dev] Deprecating repoman

[gentoo-dev] Deprecating repoman

[Matt Turner]

I'd like to deprecate and ultimately remove repoman. I believe that
dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
are far superior replacements, and it makes sense to have people using
the same tool and seeing the same warnings as in the CI.

Are there any useful checks or behaviors of repoman that are missing
from pkgcheck and pkgcommit?

Thanks,
Matt

Re: [gentoo-dev] Deprecating repoman

[Rich Freeman]

On Wed, Mar 9, 2022 at 4:00 PM Matt Turner <mattst88@gentoo.org> wrote:
>
> Are there any useful checks or behaviors of repoman that are missing
> from pkgcheck and pkgcommit?

Would it make sense to package pkgcommit or otherwise stick it
somewhere official?  I know there is a copy on mgorny's github
repo/blog, but if this is going to become the recommended way to do
commits we might want to stick it someplace a bit more canonical, and
having it in a package would ensure that if there are any changes they
get distributed.

Obviously there are other ways to set it, but it lacks obtaining the
gpg key to use from make.conf.  I'm not sure that is really the best
place to put it in any case.  Just figured I'd point it out.

-- 
Rich

Re: [gentoo-dev] Deprecating repoman

[Matt Turner]

On Wed, Mar 9, 2022 at 1:15 PM Rich Freeman <rich0@gentoo.org> wrote:
>
> On Wed, Mar 9, 2022 at 4:00 PM Matt Turner <mattst88@gentoo.org> wrote:
> >
> > Are there any useful checks or behaviors of repoman that are missing
> > from pkgcheck and pkgcommit?
>
> Would it make sense to package pkgcommit or otherwise stick it
> somewhere official?  I know there is a copy on mgorny's github
> repo/blog, but if this is going to become the recommended way to do
> commits we might want to stick it someplace a bit more canonical, and
> having it in a package would ensure that if there are any changes they
> get distributed.

I agree, this would be nice. (And it's already in a package,
app-portage/mgorny-dev-scripts as I mentioned)

> Obviously there are other ways to set it, but it lacks obtaining the
> gpg key to use from make.conf.  I'm not sure that is really the best
> place to put it in any case.  Just figured I'd point it out.

I think some bit of this message was lost? Looks like you're asking
about where we should set the GPG key used for signing commits. IMO
.git/config is the right place. Configuring it in make.conf was not a
good idea, and as a follow-on I'd like to remove that.

Re: [gentoo-dev] Deprecating repoman

[Brian Evans]

[-- Attachment #1.1: Type: text/plain, Size: 587 bytes --]

On 3/9/2022 4:00 PM, Matt Turner wrote:
> I'd like to deprecate and ultimately remove repoman. I believe that
> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> are far superior replacements, and it makes sense to have people using
> the same tool and seeing the same warnings as in the CI.
> 
> Are there any useful checks or behaviors of repoman that are missing
> from pkgcheck and pkgcommit?

I certainly would miss 'repoman manifest'.  I bet other scripts would 
too including at least one in the mentioned app-portage/mgonry-dev-scripts.

Brian

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Ionen Wolkens]

[-- Attachment #1: Type: text/plain, Size: 1062 bytes --]

On Wed, Mar 09, 2022 at 01:00:37PM -0800, Matt Turner wrote:
> I'd like to deprecate and ultimately remove repoman. I believe that
> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)

There's also dev-util/pkgdev as an alternative to pkgcommit with more
features for those that want them that will feel similar to repoman
- pkgdev commit -c/-b bugnum (repoman commit -c/-b bugnum)
- pkgdev manifest (repoman manifest)
- pkgdev push (runs pkgcheck scan --commits before pushing)
- updates copyright year / manifest on commit
- basic commit message templates, like auto "cat/pkg: add <version>" on
  bumps without manual editing -- prepend cat/pkg: like other tools
  if nothing specific

And other handy features like pkgdev mask --rites days

> are far superior replacements, and it makes sense to have people using
> the same tool and seeing the same warnings as in the CI.
> 
> Are there any useful checks or behaviors of repoman that are missing
> from pkgcheck and pkgcommit?
> 
> Thanks,
> Matt
> 

-- 
ionen

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Anna Vyalkova]

On 2022-03-09 13:00, Matt Turner wrote:
> I'd like to deprecate and ultimately remove repoman. I believe that
> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> are far superior replacements, and it makes sense to have people using
> the same tool and seeing the same warnings as in the CI.

"pkgdev commit" (from dev-util/pkgdev) is another great replacement for
"repoman commit".

Re: [gentoo-dev] Deprecating repoman

[Maciej Barć]

[-- Attachment #1.1.1: Type: text/plain, Size: 679 bytes --]

> Are there any useful checks or behaviors of repoman that are missing
> from pkgcheck and pkgcommit?

Fixing ebuild copyright date is the first one that comes to mind.


On 3/9/22 10:00 PM, Matt Turner wrote:
> I'd like to deprecate and ultimately remove repoman. I believe that
> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> are far superior replacements, and it makes sense to have people using
> the same tool and seeing the same warnings as in the CI.
> 
> Are there any useful checks or behaviors of repoman that are missing
> from pkgcheck and pkgcommit?
> 
> Thanks,
> Matt
> 

-- 
Have a great day!

~ Maciej XGQT Barć

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 6297 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Matt Turner]

On Wed, Mar 9, 2022 at 1:27 PM Maciej Barć <xgqt@gentoo.org> wrote:
>
> > Are there any useful checks or behaviors of repoman that are missing
> > from pkgcheck and pkgcommit?
>
> Fixing ebuild copyright date is the first one that comes to mind.

pkgcommit does this.

Re: [gentoo-dev] Deprecating repoman

[Brian Evans]

[-- Attachment #1.1: Type: text/plain, Size: 556 bytes --]

On 3/9/2022 4:28 PM, Matt Turner wrote:
> On Wed, Mar 9, 2022 at 1:27 PM Maciej Barć <xgqt@gentoo.org> wrote:
>>
>>> Are there any useful checks or behaviors of repoman that are missing
>>> from pkgcheck and pkgcommit?
>>
>> Fixing ebuild copyright date is the first one that comes to mind.
> 
> pkgcommit does this.
> 
I highly doubt it as it sits in the github repo
https://github.com/mgorny/mgorny-dev-scripts/blob/master/pkgcommit 
basically sets up $EDITOR with "cat/pkg:" with an optional message in 
front then passes to 'git commit'

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Matthias Maier]

Just a quick though:

Looking at the man page of repoman it doesn't look to difficult to
replicate the behavior with pkgcheck. Meaning, we could think of
creating a drop-in replacement for "repoman [full]" (which would just
call pkgcheck) and "repoman commit" which actually does much more than
just prepending the git commit summary line: repoman commit does

   - update the manifest
   - bail out if files are not correctly "git add"ed
   - add the output of [pkgcheck] as a comment to the git commit
     description

Best,
Matthias

Re: [gentoo-dev] Deprecating repoman

[Matt Turner]

On Wed, Mar 9, 2022 at 1:32 PM Brian Evans <grknight@gentoo.org> wrote:
>
> On 3/9/2022 4:28 PM, Matt Turner wrote:
> > On Wed, Mar 9, 2022 at 1:27 PM Maciej Barć <xgqt@gentoo.org> wrote:
> >>
> >>> Are there any useful checks or behaviors of repoman that are missing
> >>> from pkgcheck and pkgcommit?
> >>
> >> Fixing ebuild copyright date is the first one that comes to mind.
> >
> > pkgcommit does this.
> >
> I highly doubt it as it sits in the github repo
> https://github.com/mgorny/mgorny-dev-scripts/blob/master/pkgcommit
> basically sets up $EDITOR with "cat/pkg:" with an optional message in
> front then passes to 'git commit'

Ah, sorry. copybump (called by pkgbump) is actually what updates the copyright:

https://github.com/mgorny/mgorny-dev-scripts/blob/master/copybump

Re: [gentoo-dev] Deprecating repoman

[Matt Turner]

On Wed, Mar 9, 2022 at 1:37 PM Matthias Maier <tamiko@gentoo.org> wrote:
>
>
> Just a quick though:
>
> Looking at the man page of repoman it doesn't look to difficult to
> replicate the behavior with pkgcheck. Meaning, we could think of
> creating a drop-in replacement for "repoman [full]" (which would just
> call pkgcheck) and "repoman commit" which actually does much more than
> just prepending the git commit summary line: repoman commit does
>
>    - update the manifest
>    - bail out if files are not correctly "git add"ed
>    - add the output of [pkgcheck] as a comment to the git commit
>      description

I wouldn't block anyone from doing this, but it's not something I'm
personally interested in pursuing. I see very little value here.

Re: [gentoo-dev] Deprecating repoman

[Matthias Maier]

On Wed, Mar  9, 2022, at 15:47 CST, Matt Turner <mattst88@gentoo.org> wrote:

> [...]
>
> I wouldn't block anyone from doing this, but it's not something I'm
> personally interested in pursuing. I see very little value here.

I did not intend to imply that you should do anything.

I just want to point out that we had been educating developers for using
"repoman" to commit for a long time and looking at the "git log" history
shows that repoman is still used by a sizable number of developers
(around 50%-ish).

So, it would be very nice to have a somewhat official "blessed"
alternative in place.


For example, dev-util/pkgdev was suggested as an alternative, but I
cannot assess whether this is indeed a viable alternative.


Best,
Matthias

Re: [gentoo-dev] Deprecating repoman

[Matt Turner]

On Wed, Mar 9, 2022 at 2:34 PM Matthias Maier <tamiko@gentoo.org> wrote:
>
> On Wed, Mar  9, 2022, at 15:47 CST, Matt Turner <mattst88@gentoo.org> wrote:
>
> > [...]
> >
> > I wouldn't block anyone from doing this, but it's not something I'm
> > personally interested in pursuing. I see very little value here.
>
> I did not intend to imply that you should do anything.
>
> I just want to point out that we had been educating developers for using
> "repoman" to commit for a long time and looking at the "git log" history
> shows that repoman is still used by a sizable number of developers
> (around 50%-ish).

I think you just made that number up :)

> So, it would be very nice to have a somewhat official "blessed"
> alternative in place.
>
>
> For example, dev-util/pkgdev was suggested as an alternative, but I
> cannot assess whether this is indeed a viable alternative.

Why not? Give it a try.

Re: [gentoo-dev] Deprecating repoman

[Matthias Maier]

On Wed, Mar  9, 2022, at 17:32 CST, Matt Turner <mattst88@gentoo.org> wrote:

> I think you just made that number up :)

Nah. My sample was just bad (ago just made a sizable number of commits.

I would estimate the current usage of repoman to be about 20-25%, down
from well over 80-90% back when we just switched to git:

  https://tamiko.43-1.org/temp/repoman-usage.pdf

>> For example, dev-util/pkgdev was suggested as an alternative, but I
>> cannot assess whether this is indeed a viable alternative.
>
> Why not?

Because I haven't used it so far.

> Give it a try.

Absolutely, the faster I can ditch repoman, the better.

Best,
Matthias

Re: [gentoo-dev] Deprecating repoman

[Matt Turner]

On Wed, Mar 9, 2022 at 4:18 PM Matthias Maier <tamiko@gentoo.org> wrote:
>
>
> On Wed, Mar  9, 2022, at 17:32 CST, Matt Turner <mattst88@gentoo.org> wrote:
>
> > I think you just made that number up :)
>
> Nah. My sample was just bad (ago just made a sizable number of commits.
>
> I would estimate the current usage of repoman to be about 20-25%, down
> from well over 80-90% back when we just switched to git:
>
>   https://tamiko.43-1.org/temp/repoman-usage.pdf

Very interesting. Thanks!

Re: [gentoo-dev] Deprecating repoman

[Joonas Niilola]

[-- Attachment #1.1: Type: text/plain, Size: 1572 bytes --]

On 9.3.2022 23.00, Matt Turner wrote:
> I'd like to deprecate and ultimately remove repoman. I believe that
> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> are far superior replacements, and it makes sense to have people using
> the same tool and seeing the same warnings as in the CI.
> 
> Are there any useful checks or behaviors of repoman that are missing
> from pkgcheck and pkgcommit?
> 
> Thanks,
> Matt
> 

I still fail to see the "why" in here. Repoman is better than pure 'git
commit' that I know some people still like to use, and as long as it's
kept maintained.
We should be teaching people about the alternatives, and let them choose
whatever they like more.
https://wiki.gentoo.org/wiki/Package_maintainer%27s_responsibilities#Ebuilds_and_git_workflow

Repoman is a very lightweight tool. All that being said, I can't think
of a single feature that pkgdev+pkgcheck don't cover when switching from
repoman.

The quick global CI checks after each commit saves us from a lot of
trouble. If you do bad commits, you get immediately noticed about it and
you can fix it rather quickly usually. When you "get hit", you also
learn about pkgcheck and we've seen that this is the point when people
usually integrate it to their workflows. I'd also like to point out
whenever doing more complicated pushs, one can trigger the automatic CI
check in our Github mirror via a pull request before pushing.

My only worry is: are pkgcheck and pkgdev _really_ maintained anymore?
More than repoman is?

-- juippis

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Matt Turner]

On Wed, Mar 9, 2022 at 11:09 PM Joonas Niilola <juippis@gentoo.org> wrote:
>
> On 9.3.2022 23.00, Matt Turner wrote:
> > I'd like to deprecate and ultimately remove repoman. I believe that
> > dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> > are far superior replacements, and it makes sense to have people using
> > the same tool and seeing the same warnings as in the CI.
> >
> > Are there any useful checks or behaviors of repoman that are missing
> > from pkgcheck and pkgcommit?
> >
> > Thanks,
> > Matt
> >
>
> I still fail to see the "why" in here. Repoman is better than pure 'git
> commit' that I know some people still like to use, and as long as it's
> kept maintained.

repoman is inferior to other tooling mentioned. The other tooling is
actually run in CI. Developers should get the same warnings locally as
in CI. Restated another way: I'm tired of telling people to stop using
repoman or "pkgcheck would have caught that".

> My only worry is: are pkgcheck and pkgdev _really_ maintained anymore?
> More than repoman is?

More than repoman is, definitely.

Re: [gentoo-dev] Deprecating repoman

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1609 bytes --]

On Thu, Mar 10, 2022 at 09:29:59AM -0800, Matt Turner wrote:
> On Wed, Mar 9, 2022 at 11:09 PM Joonas Niilola <juippis@gentoo.org> wrote:
> >
> > On 9.3.2022 23.00, Matt Turner wrote:
> > > I'd like to deprecate and ultimately remove repoman. I believe that
> > > dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> > > are far superior replacements, and it makes sense to have people using
> > > the same tool and seeing the same warnings as in the CI.
> > >
> > > Are there any useful checks or behaviors of repoman that are missing
> > > from pkgcheck and pkgcommit?
> > >
> > > Thanks,
> > > Matt
> > >
> >
> > I still fail to see the "why" in here. Repoman is better than pure 'git
> > commit' that I know some people still like to use, and as long as it's
> > kept maintained.
> 
> repoman is inferior to other tooling mentioned. The other tooling is
> actually run in CI. Developers should get the same warnings locally as
> in CI. Restated another way: I'm tired of telling people to stop using
> repoman or "pkgcheck would have caught that".

I am going to nit-pick here, but pkgcheck pulls in pkgcore still. As far
asI know, pkgcore was meant to be a portage-like package manager, but it
isn't maintained. So, can we break that dependency before we make
pkgcheck the official tool for qa checks? I would rather not have
pkgcore landing on everyone's systems unless it is usable. If I am wrong
about pkgcore, please correct me and I'll be quiet, but if not let's
make pkgcheck independent from it before we deprecate repoman.

Thanks,

William

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] Deprecating repoman

[John Helmert III]

[-- Attachment #1: Type: text/plain, Size: 2094 bytes --]

On Thu, Mar 10, 2022 at 12:07:40PM -0600, William Hubbs wrote:
> On Thu, Mar 10, 2022 at 09:29:59AM -0800, Matt Turner wrote:
> > On Wed, Mar 9, 2022 at 11:09 PM Joonas Niilola <juippis@gentoo.org> wrote:
> > >
> > > On 9.3.2022 23.00, Matt Turner wrote:
> > > > I'd like to deprecate and ultimately remove repoman. I believe that
> > > > dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> > > > are far superior replacements, and it makes sense to have people using
> > > > the same tool and seeing the same warnings as in the CI.
> > > >
> > > > Are there any useful checks or behaviors of repoman that are missing
> > > > from pkgcheck and pkgcommit?
> > > >
> > > > Thanks,
> > > > Matt
> > > >
> > >
> > > I still fail to see the "why" in here. Repoman is better than pure 'git
> > > commit' that I know some people still like to use, and as long as it's
> > > kept maintained.
> > 
> > repoman is inferior to other tooling mentioned. The other tooling is
> > actually run in CI. Developers should get the same warnings locally as
> > in CI. Restated another way: I'm tired of telling people to stop using
> > repoman or "pkgcheck would have caught that".
> 
> I am going to nit-pick here, but pkgcheck pulls in pkgcore still. As far
> asI know, pkgcore was meant to be a portage-like package manager, but it
> isn't maintained. So, can we break that dependency before we make
> pkgcheck the official tool for qa checks? I would rather not have
> pkgcore landing on everyone's systems unless it is usable. If I am wrong
> about pkgcore, please correct me and I'll be quiet, but if not let's
> make pkgcheck independent from it before we deprecate repoman.

Yes, pkgcheck pulls in pkgcore, and yes, pkgcore wants to function as
a package manager, but it doesn't conflict with Portage, so there's no
concern in pulling it in. As long as you don't call the executables it
installs (notably pmerge, maybe others), it won't cause any
problems. pkgcheck can also already be considered our official CI
tool, since it's what does our CI.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Joshua Kinard]

On 3/9/2022 16:00, Matt Turner wrote:
> I'd like to deprecate and ultimately remove repoman. I believe that
> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> are far superior replacements, and it makes sense to have people using
> the same tool and seeing the same warnings as in the CI.
> 
> Are there any useful checks or behaviors of repoman that are missing
> from pkgcheck and pkgcommit?
> 
> Thanks,
> Matt

repoman has been //the// goto tool for checking in a change since before I
was a developer in 2003.  It does everything we need, in one simple tool.
Your proposal looks to replace repoman's functionality (and snark) with two
or more packages, including tools from a package named after a fellow developer.

Additionally, "I believe that <foo> are far superior replacements" is an
entirely subjective opinion and, frankly, completely invalid as
justification to replace a tool that has worked fine for the last 20+ years.
 What is so fundamentally broken about repoman that cannot be fixed such
that it needs total replacement by multiple independent tools?  Please
document. the pros and cons here so that we can all make an informed decision.

I'm not opposed to making our tools better, but I think before anything can
replace the RepoMan, several more boxes need to be ticked:

- functionality from multiple tools should be packaged into a single tool.
  * caveat: at least provide a wrapper that, using args, can invoke the
    individual tools if needed.

- app-portage/mgorny-dev-scripts needs a new name.  It's fine if it's
intended to only be a collection of useful scripts for individual developers
on an as-needed basis, but if it's to be the Official Tool(TM), then it
needs to have a proper name.  If not all of the scripts contained within it
are applicable to the sole function of checking a change into the tree, then
only the scripts that deal with change validation and committing should be
broken out into a separate package, and ostensibly combined with any other
tools/scripts into a single package, and preferably a single tool, to get
the job done.

- all of our developer documentation needs to be updated to reference the
new tool and the new way of doing things, as well as a warning not to use
repoman any further after a set date.  Additionally, a news item is probably
advisable so that developers and proxy maintainers alike can get the message.

- we need the snark.  Gentoo is all about personality, and RepoMan has been
scouring our neighborhoods for two decades and change, and while some may
think this is utterly frivolous, I will actually miss seeing those messages
on the console every time I commit a change.  They give the RepoMan
personality and a soul, and thus, contribute to the uniqueness that is Gentoo.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] Deprecating repoman

[Joshua Kinard]

On 3/9/2022 16:47, Matt Turner wrote:
> On Wed, Mar 9, 2022 at 1:37 PM Matthias Maier <tamiko@gentoo.org> wrote:
>>
>>
>> Just a quick though:
>>
>> Looking at the man page of repoman it doesn't look to difficult to
>> replicate the behavior with pkgcheck. Meaning, we could think of
>> creating a drop-in replacement for "repoman [full]" (which would just
>> call pkgcheck) and "repoman commit" which actually does much more than
>> just prepending the git commit summary line: repoman commit does
>>
>>    - update the manifest
>>    - bail out if files are not correctly "git add"ed
>>    - add the output of [pkgcheck] as a comment to the git commit
>>      description
> 
> I wouldn't block anyone from doing this, but it's not something I'm
> personally interested in pursuing. I see very little value here.

First, you're trying to justify replacing repoman on an entirely subjective
opinion of "I think <foo> is superior", then when you get feedback on the
idea, you dismiss that feedback with more opinion.

Why do you not see value here?  The actions described are actually quite a
few useful steps in the process of checking a change into the tree.  If you
expect developers to do those steps on their own, that increases, not
decreases, the chances of making a mistake.  Or are these steps already
handled by one of the other scripts in the replacement packages you propose?
 If so, please say so and identify which one(s).

My opinion is that any tool that replaces repoman should, at a minimum,
replace like functionality with like functionality, plus benefits or
enhancements.  This looks more like a step backwards, not a step forwards.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] Deprecating repoman

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 614 bytes --]

> > 
> > I wouldn't block anyone from doing this, but it's not something I'm
> > personally interested in pursuing. I see very little value here.
> 
> First, you're trying to justify replacing repoman on an entirely subjective
> opinion of "I think <foo> is superior" ...

Well, if you've ever tried it you'll notice that <foo> for <foo> != repoman
actually finishes the checks within a finite amount of time. Kind of, the 
most blatant argument for ditching repoman, actually.


-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Alec Warner]

On Thu, Mar 10, 2022 at 10:27 AM Joshua Kinard <kumba@gentoo.org> wrote:
>
> On 3/9/2022 16:00, Matt Turner wrote:
> > I'd like to deprecate and ultimately remove repoman. I believe that
> > dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> > are far superior replacements, and it makes sense to have people using
> > the same tool and seeing the same warnings as in the CI.
> >
> > Are there any useful checks or behaviors of repoman that are missing
> > from pkgcheck and pkgcommit?
> >
> > Thanks,
> > Matt
>
> repoman has been //the// goto tool for checking in a change since before I
> was a developer in 2003.  It does everything we need, in one simple tool.
> Your proposal looks to replace repoman's functionality (and snark) with two
> or more packages, including tools from a package named after a fellow developer.
>
> Additionally, "I believe that <foo> are far superior replacements" is an
> entirely subjective opinion and, frankly, completely invalid as
> justification to replace a tool that has worked fine for the last 20+ years.
>  What is so fundamentally broken about repoman that cannot be fixed such
> that it needs total replacement by multiple independent tools?  Please
> document. the pros and cons here so that we can all make an informed decision.

So here is the more basic argument, you can agree or disagree.

*The goal I want is for people to commit to the tree and not break things.*

If we could accomplish this with no tooling at all, that would be
great. Sadly humans are fallible and so we have tools to check their
work. Currently this tooling has two parts:
 - pre-push tooling that you run prior to pushing.
 - post-push CI tooling that informs you when you break the tree.

So holding to our goal of not breaking the tree, it's better for
developers to run the same QA tool in pre-push that CI is using,
because our metric for 'breaking the tree' is the CI tool and if the
CI tool passes locally, there is a strong likelihood it will not break
in CI either. Note this argument is generic, I'm not even saying which
tools are in use, or which tools are better, or anything like that.

Here we see Matt discussing a workflow he finds frustrating.
 - A developer does a push.
 - Their push breaks CI.
 - He inquires about their workflow.
 - He learns they did not run the CI tool in their pre-push workflow.
 - He tells them they should run the CI tool in their pre-push workflow.
 - This happens many times, causing this thread.

The point of the thread then is to convince people to run the CI tool
in pre-push, as a matter of policy, to reduce tree breakage and reduce
the occurrence of the above conversation.

So the generic argument above, we now get into the specifics.

>
> I'm not opposed to making our tools better, but I think before anything can
> replace the RepoMan, several more boxes need to be ticked:
>
> - functionality from multiple tools should be packaged into a single tool.
>   * caveat: at least provide a wrapper that, using args, can invoke the
>     individual tools if needed.

I do not believe it's reasonable to provide a 'drop-in' backwards
compatibility tool guarantee.
I believe we should provide a table of common repoman actions, with a
mapping to their replacements; so that users can understand how to do
similar actions.

>
> - app-portage/mgorny-dev-scripts needs a new name.  It's fine if it's
> intended to only be a collection of useful scripts for individual developers
> on an as-needed basis, but if it's to be the Official Tool(TM), then it
> needs to have a proper name.  If not all of the scripts contained within it
> are applicable to the sole function of checking a change into the tree, then
> only the scripts that deal with change validation and committing should be
> broken out into a separate package, and ostensibly combined with any other
> tools/scripts into a single package, and preferably a single tool, to get
> the job done.

I don't consider this a blocker, but I think it's mostly a discussion
between mattst88 and mgorny.

>
> - all of our developer documentation needs to be updated to reference the
> new tool and the new way of doing things, as well as a warning not to use
> repoman any further after a set date.  Additionally, a news item is probably
> advisable so that developers and proxy maintainers alike can get the message.

We should hold Matt accountable for updating any relevant development
documentation, including the table I mentioned above.
I'm not sure a news item is strictly necessary, we might just p.mask
repoman with a link to the guide that Matt will need to write about
how repoman is being replaced.

>
> - we need the snark.  Gentoo is all about personality, and RepoMan has been
> scouring our neighborhoods for two decades and change, and while some may
> think this is utterly frivolous, I will actually miss seeing those messages
> on the console every time I commit a change.  They give the RepoMan
> personality and a soul, and thus, contribute to the uniqueness that is Gentoo.

Cool.

>
> --
> Joshua Kinard
> Gentoo/MIPS
> kumba@gentoo.org
> rsa6144/5C63F4E3F5C6C943 2015-04-27
> 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943
>
> "The past tempts us, the present confuses us, the future frightens us.  And
> our lives slip away, moment by moment, lost in that vast, terrible in-between."
>
> --Emperor Turhan, Centauri Republic
>

Re: [gentoo-dev] Deprecating repoman

[Alec Warner]

On Thu, Mar 10, 2022 at 10:28 AM Joshua Kinard <kumba@gentoo.org> wrote:
>
> On 3/9/2022 16:47, Matt Turner wrote:
> > On Wed, Mar 9, 2022 at 1:37 PM Matthias Maier <tamiko@gentoo.org> wrote:
> >>
> >>
> >> Just a quick though:
> >>
> >> Looking at the man page of repoman it doesn't look to difficult to
> >> replicate the behavior with pkgcheck. Meaning, we could think of
> >> creating a drop-in replacement for "repoman [full]" (which would just
> >> call pkgcheck) and "repoman commit" which actually does much more than
> >> just prepending the git commit summary line: repoman commit does
> >>
> >>    - update the manifest
> >>    - bail out if files are not correctly "git add"ed
> >>    - add the output of [pkgcheck] as a comment to the git commit
> >>      description
> >
> > I wouldn't block anyone from doing this, but it's not something I'm
> > personally interested in pursuing. I see very little value here.
>
> First, you're trying to justify replacing repoman on an entirely subjective
> opinion of "I think <foo> is superior", then when you get feedback on the
> idea, you dismiss that feedback with more opinion.
>
> Why do you not see value here?  The actions described are actually quite a
> few useful steps in the process of checking a change into the tree.  If you
> expect developers to do those steps on their own, that increases, not
> decreases, the chances of making a mistake.  Or are these steps already
> handled by one of the other scripts in the replacement packages you propose?
>  If so, please say so and identify which one(s).
>
> My opinion is that any tool that replaces repoman should, at a minimum,
> replace like functionality with like functionality, plus benefits or
> enhancements.  This looks more like a step backwards, not a step forwards.

I'd be interested in hearing your workflow, so we can capture it in
the table (mentioned earlier) so its clear how your existing workflow
will work with the new tools (or perhaps there is a gap, or we need to
craft / add additional tools?) I agree on the face it may not be
obvious what workflows look like.

-A

>
> --
> Joshua Kinard
> Gentoo/MIPS
> kumba@gentoo.org
> rsa6144/5C63F4E3F5C6C943 2015-04-27
> 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943
>
> "The past tempts us, the present confuses us, the future frightens us.  And
> our lives slip away, moment by moment, lost in that vast, terrible in-between."
>
> --Emperor Turhan, Centauri Republic
>

Re: [gentoo-dev] Deprecating repoman

[Joshua Kinard]

On 3/10/2022 14:44, Andreas K. Huettel wrote:
>>>
>>> I wouldn't block anyone from doing this, but it's not something I'm
>>> personally interested in pursuing. I see very little value here.
>>
>> First, you're trying to justify replacing repoman on an entirely subjective
>> opinion of "I think <foo> is superior" ...
> 
> Well, if you've ever tried it you'll notice that <foo> for <foo> != repoman
> actually finishes the checks within a finite amount of time. Kind of, the 
> most blatant argument for ditching repoman, actually.

If this is a concern for some, has anyone looked into whether repoman can be
fixed to be more efficient?  If so, how was the determination made that it
cannot be fixed and instead, needs to be replaced?  It's been around for 20+
years.  Surely someone has gotten annoyed enough to look at any issues it
has and attempt to fix them?

That said, I'm not terribly bothered by it.  It is slow, don't get me wrong,
but it's not slow enough that my workflow is significantly impacted.  It
catches most of the mistakes I've ever made before I make them so that I can
fix them.  For me, that's job well done.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] Deprecating repoman

[Joshua Kinard]

On 3/10/2022 14:59, Alec Warner wrote:
> On Thu, Mar 10, 2022 at 10:28 AM Joshua Kinard <kumba@gentoo.org> wrote:
>>
>> On 3/9/2022 16:47, Matt Turner wrote:
>>> On Wed, Mar 9, 2022 at 1:37 PM Matthias Maier <tamiko@gentoo.org> wrote:
>>>>
>>>>
>>>> Just a quick though:
>>>>
>>>> Looking at the man page of repoman it doesn't look to difficult to
>>>> replicate the behavior with pkgcheck. Meaning, we could think of
>>>> creating a drop-in replacement for "repoman [full]" (which would just
>>>> call pkgcheck) and "repoman commit" which actually does much more than
>>>> just prepending the git commit summary line: repoman commit does
>>>>
>>>>    - update the manifest
>>>>    - bail out if files are not correctly "git add"ed
>>>>    - add the output of [pkgcheck] as a comment to the git commit
>>>>      description
>>>
>>> I wouldn't block anyone from doing this, but it's not something I'm
>>> personally interested in pursuing. I see very little value here.
>>
>> First, you're trying to justify replacing repoman on an entirely subjective
>> opinion of "I think <foo> is superior", then when you get feedback on the
>> idea, you dismiss that feedback with more opinion.
>>
>> Why do you not see value here?  The actions described are actually quite a
>> few useful steps in the process of checking a change into the tree.  If you
>> expect developers to do those steps on their own, that increases, not
>> decreases, the chances of making a mistake.  Or are these steps already
>> handled by one of the other scripts in the replacement packages you propose?
>>  If so, please say so and identify which one(s).
>>
>> My opinion is that any tool that replaces repoman should, at a minimum,
>> replace like functionality with like functionality, plus benefits or
>> enhancements.  This looks more like a step backwards, not a step forwards.
> 
> I'd be interested in hearing your workflow, so we can capture it in
> the table (mentioned earlier) so its clear how your existing workflow
> will work with the new tools (or perhaps there is a gap, or we need to
> craft / add additional tools?) I agree on the face it may not be
> obvious what workflows look like.

My workflow is really rather standard when working in the tree itself.  I
work one package directory at a time, apply changes that I've tested outside
of the tree in my local repo, eyeball everything a second time to make sure
I didn't miss something, regenerate the manifest, git add, run 'repoman full
-d -x', fix any issues it finds (if any) and manifest/git add again, then
'repoman commit' and supply a commit message with sign-off.  Lather, rinse,
repeat for other packages.

This is more or less how the dev manual and git for developers guide says we
should be doing it.  Since git is atomic, limit commits to one package at a
time, then, when ready, push the changes in a single batch after rebasing
(to pickup other dev's changes).

If I'm doing something wrong, yell at me in e-mail, open a bug, or call me
out on the mailing list and serve me some crow pie.  That's how I learn not
to repeat it.  So far, though, this process has worked well for me for a
very long time and I've only had to make minor adjustments at key points,
like switching from CVS to git.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] Deprecating repoman

[John Helmert III]

[-- Attachment #1: Type: text/plain, Size: 1774 bytes --]

On Thu, Mar 10, 2022 at 04:53:10PM -0500, Joshua Kinard wrote:
> On 3/10/2022 14:44, Andreas K. Huettel wrote:
> >>>
> >>> I wouldn't block anyone from doing this, but it's not something I'm
> >>> personally interested in pursuing. I see very little value here.
> >>
> >> First, you're trying to justify replacing repoman on an entirely subjective
> >> opinion of "I think <foo> is superior" ...
> > 
> > Well, if you've ever tried it you'll notice that <foo> for <foo> != repoman
> > actually finishes the checks within a finite amount of time. Kind of, the 
> > most blatant argument for ditching repoman, actually.
> 
> If this is a concern for some, has anyone looked into whether repoman can be
> fixed to be more efficient?  If so, how was the determination made that it
> cannot be fixed and instead, needs to be replaced?  It's been around for 20+
> years.  Surely someone has gotten annoyed enough to look at any issues it
> has and attempt to fix them?

It's slow enough that Gentoo CI [1] uses pkgcheck to be remotely
useful.

[1] https://qa-reports.gentoo.org/output/gentoo-ci/output.html

> That said, I'm not terribly bothered by it.  It is slow, don't get me wrong,
> but it's not slow enough that my workflow is significantly impacted.  It
> catches most of the mistakes I've ever made before I make them so that I can
> fix them.  For me, that's job well done.
> 
> -- 
> Joshua Kinard
> Gentoo/MIPS
> kumba@gentoo.org
> rsa6144/5C63F4E3F5C6C943 2015-04-27
> 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943
> 
> "The past tempts us, the present confuses us, the future frightens us.  And
> our lives slip away, moment by moment, lost in that vast, terrible in-between."
> 
> --Emperor Turhan, Centauri Republic
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1071 bytes --]



> On 10 Mar 2022, at 21:57, Joshua Kinard <kumba@gentoo.org> wrote:
>> 
>> I'd be interested in hearing your workflow, so we can capture it in
>> the table (mentioned earlier) so its clear how your existing workflow
>> will work with the new tools (or perhaps there is a gap, or we need to
>> craft / add additional tools?) I agree on the face it may not be
>> obvious what workflows look like.
> 
> My workflow is really rather standard when working in the tree itself.  I
> work one package directory at a time, apply changes that I've tested outside
> of the tree in my local repo, eyeball everything a second time to make sure
> I didn't miss something, regenerate the manifest, git add, run 'repoman full
> -d -x', fix any issues it finds (if any) and manifest/git add again, then
> 'repoman commit' and supply a commit message with sign-off.  Lather, rinse,
> repeat for other packages.
> 

Having the same checks applied as in CI (which affects whether changes
are deployed to users too) is important. pkgcheck has more checks than repoman.

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Joshua Kinard]

On 3/10/2022 14:58, Alec Warner wrote:
> On Thu, Mar 10, 2022 at 10:27 AM Joshua Kinard <kumba@gentoo.org> wrote:
>>
>> On 3/9/2022 16:00, Matt Turner wrote:
>>> I'd like to deprecate and ultimately remove repoman. I believe that
>>> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
>>> are far superior replacements, and it makes sense to have people using
>>> the same tool and seeing the same warnings as in the CI.
>>>
>>> Are there any useful checks or behaviors of repoman that are missing
>>> from pkgcheck and pkgcommit?
>>>
>>> Thanks,
>>> Matt
>>
>> repoman has been //the// goto tool for checking in a change since before I
>> was a developer in 2003.  It does everything we need, in one simple tool.
>> Your proposal looks to replace repoman's functionality (and snark) with two
>> or more packages, including tools from a package named after a fellow developer.
>>
>> Additionally, "I believe that <foo> are far superior replacements" is an
>> entirely subjective opinion and, frankly, completely invalid as
>> justification to replace a tool that has worked fine for the last 20+ years.
>>  What is so fundamentally broken about repoman that cannot be fixed such
>> that it needs total replacement by multiple independent tools?  Please
>> document. the pros and cons here so that we can all make an informed decision.
> 
> So here is the more basic argument, you can agree or disagree.
> 
> *The goal I want is for people to commit to the tree and not break things.*

This goal I agree with, and I am rather pedantic about.  If not mildly
fearful of.  We've all been there at least once in our dev-lives.  It's
almost a rite of passage, if you will, to break the tree with a dumb commit
at least once.  Goal is to never repeat that mistake again.


> If we could accomplish this with no tooling at all, that would be
> great. Sadly humans are fallible and so we have tools to check their
> work. Currently this tooling has two parts:
>  - pre-push tooling that you run prior to pushing.
>  - post-push CI tooling that informs you when you break the tree.
> 
> So holding to our goal of not breaking the tree, it's better for
> developers to run the same QA tool in pre-push that CI is using,
> because our metric for 'breaking the tree' is the CI tool and if the
> CI tool passes locally, there is a strong likelihood it will not break
> in CI either. Note this argument is generic, I'm not even saying which
> tools are in use, or which tools are better, or anything like that.
> 
> Here we see Matt discussing a workflow he finds frustrating.
>  - A developer does a push.
>  - Their push breaks CI.
>  - He inquires about their workflow.
>  - He learns they did not run the CI tool in their pre-push workflow.
>  - He tells them they should run the CI tool in their pre-push workflow.
>  - This happens many times, causing this thread.
> 
> The point of the thread then is to convince people to run the CI tool
> in pre-push, as a matter of policy, to reduce tree breakage and reduce
> the occurrence of the above conversation.

I stick to the officially-published method of checking and committing changes:
https://devmanual.gentoo.org/ebuild-maintenance/git/index.html

The two tools highlighted there for the bulk of the work is repoman and
pkgdev.  repoman is cited twelve times, pkgdev is cited six times.  pkgcheck
is mentioned once.  pkgcommit has no mentions.

From that, one should not be faulted for assuming that repoman is the more
important tool, if not preferred tool, with pkgdev coming in second place.
pkgcheck comes across as entirely optional and even seems equivalent to
'repoman full', and how would one know that pkgcommit even exists?

Not all of us devs are involved in the automated CI stuff that goes on
behind the scenes.  Matt and Michał are, so they know how bad it can be and
the likely countless hours spent fixing things or yelling at others when
these mistakes are made.  But if this isn't documented somewhere, how are
those of us not involved in the CI-side able to keep up?  When I am in doubt
over something, I go to the documentation to address things.  Right now,
that documentation is going to more or less tell me to stick with repoman.

That all said, most of my checks are done pre-commit, not pre-push.  It's
annoying trying to unwind a commit with a mistake in git, so I aim to not
have any in the commit itself.  By the time I get to the push stage,
everything MUST be perfect before I hit 'git push'.  Cause once something is
public, there's no going back.  In a handful of cases, I've gone as far as
nuking my local copy of the tree, re-cloning it, and re-doing everything
again just to fix a mistake that I should have caught pre-commit.


> So the generic argument above, we now get into the specifics.
> 
>>
>> I'm not opposed to making our tools better, but I think before anything can
>> replace the RepoMan, several more boxes need to be ticked:
>>
>> - functionality from multiple tools should be packaged into a single tool.
>>   * caveat: at least provide a wrapper that, using args, can invoke the
>>     individual tools if needed.
> 
> I do not believe it's reasonable to provide a 'drop-in' backwards
> compatibility tool guarantee.
> I believe we should provide a table of common repoman actions, with a
> mapping to their replacements; so that users can understand how to do
> similar actions.

Not opposed to this.  It's a reasonable alternative to crafting a wrapper
script or kludging in deprecated argument parsing to achieve likely limited
backwards compatibility.


>> - app-portage/mgorny-dev-scripts needs a new name.  It's fine if it's
>> intended to only be a collection of useful scripts for individual developers
>> on an as-needed basis, but if it's to be the Official Tool(TM), then it
>> needs to have a proper name.  If not all of the scripts contained within it
>> are applicable to the sole function of checking a change into the tree, then
>> only the scripts that deal with change validation and committing should be
>> broken out into a separate package, and ostensibly combined with any other
>> tools/scripts into a single package, and preferably a single tool, to get
>> the job done.
> 
> I don't consider this a blocker, but I think it's mostly a discussion
> between mattst88 and mgorny.

My fault that I should have stated that I don't think it should block, but I
do think this it looks....hacky?  I mean, say a developer from another
distro gets curious about how Gentoo developers do their magic, and while
reading the devmanual, sees that the officially-sanctioned way to do
something developer-related is to install
app-portage/kumbas-super-awesome-devscripts.  It looks wildly out of place
in official documentation.  We used to have a package called
app-portage/gentoolkit-dev that contained a collection of utilities intended
for Gentoo developers.  I'd argue that maybe something like that would be a
good starting point for a new name.


>>
>> - all of our developer documentation needs to be updated to reference the
>> new tool and the new way of doing things, as well as a warning not to use
>> repoman any further after a set date.  Additionally, a news item is probably
>> advisable so that developers and proxy maintainers alike can get the message.
> 
> We should hold Matt accountable for updating any relevant development
> documentation, including the table I mentioned above.
> I'm not sure a news item is strictly necessary, we might just p.mask
> repoman with a link to the guide that Matt will need to write about
> how repoman is being replaced.

How about making an edit to repoman to have it throw a deprecation warning
for itself and to install whatever we all agree should replace it?


>> - we need the snark.  Gentoo is all about personality, and RepoMan has been
>> scouring our neighborhoods for two decades and change, and while some may
>> think this is utterly frivolous, I will actually miss seeing those messages
>> on the console every time I commit a change.  They give the RepoMan
>> personality and a soul, and thus, contribute to the uniqueness that is Gentoo.
> 
> Cool.

Back in mid-2002, it was exactly Gentoo's snarkness that tipped the scales
for me.  I'd just given up on FreeBSD-4.x on a sparc64 machine (old Sun
Blade 100) and LFS turned out to not be a lot of fun, but Gentoo worked fine
on it.  All of the colors on the terminal gave it zing and pop, and made it
rather fun to work with.  rpm and apt-get were dull;  emerge was cheeky and
playful!  Still is to this day.

That sparc64 machine eventually became my very first dev machine when
Seemant was looking for someone to help out with the MIPS port.  I got a
laugh when I ran repoman for the first time and it tells me it is stalking
my neighborhood, looking for issues.  The developer tools were just as
cheeky and playful as the user-facing tools!

And we're still the only distro with some rad mascots, like Larry the Cow
and Znurt the alien (at least I think it's an alien -- bottom of the main
page, right of the Questions/Comments).  And whatever the thing at the very
bottom-right looking up is.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] Deprecating repoman

[Anna Vyalkova]

On 2022-03-10 12:07, William Hubbs wrote:
> I am going to nit-pick here, but pkgcheck pulls in pkgcore still. As far
> asI know, pkgcore was meant to be a portage-like package manager, but it
> isn't maintained. So, can we break that dependency before we make
> pkgcheck the official tool for qa checks? I would rather not have
> pkgcore landing on everyone's systems unless it is usable. If I am wrong
> about pkgcore, please correct me and I'll be quiet, but if not let's
> make pkgcheck independent from it before we deprecate repoman.

Original pkgcore developer stepped down, now it's maintained not as a
package manager but as a library for pkgcheck.

Re: [gentoo-dev] Deprecating repoman

[Mart Raudsepp]

Ühel kenal päeval, N, 10.03.2022 kell 18:18, kirjutas Joshua Kinard:
> I stick to the officially-published method of checking and committing
> changes:
> https://devmanual.gentoo.org/ebuild-maintenance/git/index.html
> 
> The two tools highlighted there for the bulk of the work is repoman
> and pkgdev.  repoman is cited twelve times, pkgdev is cited six
> times. pkgcheck is mentioned once.  pkgcommit has no mentions.
> 
> From that, one should not be faulted for assuming that repoman is the
> more important tool, if not preferred tool, with pkgdev coming in
> second place.
> pkgcheck comes across as entirely optional and even seems equivalent
> to 'repoman full', and how would one know that pkgcommit even exists?

I believe the very purpose of this thread is to have a consensus/pre-
announcement before actually editing the official documentation as part
of the process of deprecating repoman.

> That all said, most of my checks are done pre-commit, not pre-push. 
> It's annoying trying to unwind a commit with a mistake in git, so I
> aim to not have any in the commit itself.  By the time I get to the
> push stage, everything MUST be perfect before I hit 'git push'. 
> Cause once something is public, there's no going back.  In a handful
> of cases, I've gone as far as nuking my local copy of the tree, re-
> cloning it, and re-doing everything again just to fix a mistake that
> I should have caught pre-commit.

git commit --amend, interactive rebase, etc (all before push).
I suggest to learn them and embrace them. Re-cloning is not fun, just
for something that a quick interactive rebase or even a git reset --
hard destructive operation instead of re-clone would solve.

Also the benefit of using pkgcheck is to actually be able to make the
same checks that CI would do before you push, so you can amend your
commits to fix issues before they hit the server and CI and break the
tree. pkgcheck is so fast that it can do full tree checks in a
reasonable time (repoman would take days on a radiator mips when you go
outside single package), and I believe has features to have it check
all your commits that haven't been pushed yet at once, checking only
what it can to not be too slow to not use (so you don't need to run the
check with each commit but for all of them once you commit - and if
issues, again, git interactive rebase).

repoman does not catch many many QA issues that pkgcheck (and therefore
CI) does.

> How about making an edit to repoman to have it throw a deprecation
> warning for itself and to install whatever we all agree should
> replace it?

I would think that's a natural next step after this thread, and why
this thread exists.

> > 

Mart

Re: [gentoo-dev] Deprecating repoman

[Brian Dolbec]

On Thu, 10 Mar 2022 12:07:40 -0600
William Hubbs <williamh@gentoo.org> wrote:

> On Thu, Mar 10, 2022 at 09:29:59AM -0800, Matt Turner wrote:
> > On Wed, Mar 9, 2022 at 11:09 PM Joonas Niilola <juippis@gentoo.org>
> > wrote:  
> > >
> > > On 9.3.2022 23.00, Matt Turner wrote:  
> > > > I'd like to deprecate and ultimately remove repoman. I believe
> > > > that dev-util/pkgcheck and pkgcommit (from
> > > > app-portage/mgorny-dev-scripts) are far superior replacements,
> > > > and it makes sense to have people using the same tool and
> > > > seeing the same warnings as in the CI.
> > > >
> > > > Are there any useful checks or behaviors of repoman that are
> > > > missing from pkgcheck and pkgcommit?
> > > >
> > > > Thanks,
> > > > Matt
> > > >  
> > >
> > > I still fail to see the "why" in here. Repoman is better than
> > > pure 'git commit' that I know some people still like to use, and
> > > as long as it's kept maintained.  
> > 
> > repoman is inferior to other tooling mentioned. The other tooling is
> > actually run in CI. Developers should get the same warnings locally
> > as in CI. Restated another way: I'm tired of telling people to stop
> > using repoman or "pkgcheck would have caught that".  
> 
> I am going to nit-pick here, but pkgcheck pulls in pkgcore still. As
> far asI know, pkgcore was meant to be a portage-like package manager,
> but it isn't maintained. So, can we break that dependency before we
> make pkgcheck the official tool for qa checks? I would rather not have
> pkgcore landing on everyone's systems unless it is usable. If I am
> wrong about pkgcore, please correct me and I'll be quiet, but if not
> let's make pkgcheck independent from it before we deprecate repoman.
> 
> Thanks,
> 
> William

William.  pkgcheck uses pkgcore base code as it's library for accessing
and evaluating repositories and ebuilds just like repoman uses portage
api's to do the same.   This is why pkgcheck is primarily faster than
repoman.  pkgcore was a re-design from the ground up of how to handle
repositories and ebuilds within them, with efficiency and speed in mind.

Unfortunately, pkgcore never gained enough developer support to to keep
up with EAPI changes and the miriad of portage options that kept
expanding.   So pkgcore never ended up replacing portage as our main
package manager.   This is one reason why Brian eventually gave up on it
and subsequently left gentoo.


While I put a lot of effort into the re-write of repoman so that the
code is maintainable, more easily extendable. It will never be as fast
as pkgcheck due to the legacy portage code it is based on.

Yes, repoman could be updated to do more checks and catch everything
that pkgcheck does.   It can be made to do many of the checks in
parrallel, but it will never be as efficient as pkgcheck due to the
underlying portage code it uses.   I largely stopped working on
repoman/gentoo due to the near constant bitching about nearly everything
I did from another (ahem) gentoo developer which shall remain un-named.


My only concern about the possible deprecation of repoman, is the fact
that is is based on the current official and primary package manager
code used in gentoo.  While pkgcheck is not.   For that reason,
changes made to the core code is mostly automatically carried
over to repoman without need to change repoman as well.   Ideally,
development should be moved to pkgcore and portage be put into
maintenance mode only. With pkgcore taking over as the primary package
manager for gentoo. But Zac is just too damn efficient at
updating/fixing portage bugs!  As well as adding features and
options... (too damn many to remember half of them now)

Re: [gentoo-dev] Deprecating repoman

[Peter Stuge]

Matt Turner wrote:
> repoman is inferior to other tooling mentioned. The other tooling is
> actually run in CI.

The problem seems to be that CI is running something other than
developers run, not the other way around.


> Developers should get the same warnings locally as in CI.

I agree. And developers and their tools should not have to bend to
whatever CI does, I think the other way around makes more sense.


CI doesn't use repoman because of performance issues.

What if pkgcore evolves to provide a portage-compatible API?

Then CI could use repoman without performance problems, and
developers would also enjoy improved performance, without spending
time on replacing tooling which already works for them.

Looking into the future then maybe portage could even come to use
pkgcore for the low-level things that pkgcore does, then even users
could enjoy improved performance.


Right?

//Peter

Re: [gentoo-dev] Deprecating repoman

[Alec Warner]

On Fri, Mar 11, 2022 at 9:14 AM Peter Stuge <peter@stuge.se> wrote:
>
> Matt Turner wrote:
> > repoman is inferior to other tooling mentioned. The other tooling is
> > actually run in CI.
>
> The problem seems to be that CI is running something other than
> developers run, not the other way around.
>
>
> > Developers should get the same warnings locally as in CI.
>
> I agree. And developers and their tools should not have to bend to
> whatever CI does, I think the other way around makes more sense.
>
>
> CI doesn't use repoman because of performance issues.

I think the problem is that making committers use the CI tool is
technically a fairly straightforward change; while everything you
discuss further in the thread is not; because it implies people will
work on improving tools that have shown little or no progress on
improving in the 15 years I've been in Gentoo.

>
> What if pkgcore evolves to provide a portage-compatible API?

No API is planned and no one is working on it.

>
> Then CI could use repoman without performance problems, and
> developers would also enjoy improved performance, without spending
> time on replacing tooling which already works for them.

The benefit of getting people to change their behavior is that the
benefits (less breakage, better tooling) are achieved now; as opposed
to in some hypothetical future where repoman is improved.
Note that we have been waiting for 'improved repoman' for about 8
years; so..I'm not willing to bet on it when we have better tooling
that works today and the primary complaint is that...what exactly?

The new workflow with pkgcheck was announced at the end of 2019:
https://blogs.gentoo.org/mgorny/2019/12/12/a-better-ebuild-workflow-with-pure-git-and-pkgcheck

It's been 2 years, I think we can bring everyone into the fold here.

>
> Looking into the future then maybe portage could even come to use
> pkgcore for the low-level things that pkgcore does, then even users
> could enjoy improved performance.

Many things could happen but again, if you talk to people who work on
pkgcore; there is no concrete plan for this. So I don't see why we are
betting on it happening.

If you read radhermit's blog:
https://pkgcraft.github.io/posts/timesink-chronicles/ you can get one
take on the project and why it struggled.

-A

>
>
> Right?
>
> //Peter
>

Re: [gentoo-dev] Deprecating repoman

[Brian Dolbec]

On Fri, 11 Mar 2022 10:25:19 -0800
Alec Warner <antarus@gentoo.org> wrote:

> On Fri, Mar 11, 2022 at 9:14 AM Peter Stuge <peter@stuge.se> wrote:
> >
> > Matt Turner wrote:  
> > > repoman is inferior to other tooling mentioned. The other tooling
> > > is actually run in CI.  
> >
> > The problem seems to be that CI is running something other than
> > developers run, not the other way around.
> >
> >  
> > > Developers should get the same warnings locally as in CI.  
> >
> > I agree. And developers and their tools should not have to bend to
> > whatever CI does, I think the other way around makes more sense.
> >
> >
> > CI doesn't use repoman because of performance issues.  
> 
> I think the problem is that making committers use the CI tool is
> technically a fairly straightforward change; while everything you
> discuss further in the thread is not; because it implies people will
> work on improving tools that have shown little or no progress on
> improving in the 15 years I've been in Gentoo.
> 

Somewhat incorrect here.  I did the majority of the re-write to make
the code maintainable, extensible not that long ago.  It even has the
ability to be repository configurable and include the ability for
custom repository plugin checks to be added and run.   The CI system was
being developed at the same time using pkgcheck.

See my other reply to WilliamH for more detail about that and
fundamental speed differences.



> >
> > What if pkgcore evolves to provide a portage-compatible API?  
> 
> No API is planned and no one is working on it.
> 

I and a few others did some work to ensure pkgcore had some usable
api's from early in it's development.  But pkgcore is so different from
portage code, it was difficult for early to mediocre python devs to
wrap their head around.   There was even work to make porthole be able
to use pkgcore as it's backend package manager, but it was never fully
utilized due to more work needed on pkgcore for some functionality.


> >
> > Then CI could use repoman without performance problems, and
> > developers would also enjoy improved performance, without spending
> > time on replacing tooling which already works for them.  
> 

No, pkgcore and it's QA tool pkgcheck are designed to work together.
Repoman is designed to work with portage base code.   Checks can be
designed to work on either, but not both.   The missing checks that
the CI does can be added to repoman, but the primary dev(s) creating
them on the CI don't recreate them for repoman.  They want to work on
pkgcheck.

Repoman code will NEVER be as fast as pkgcheck due to the legacy
portage code structure it relies on.   Pkgcore was a re-design from the
ground up to to improve upon the shortcomings of the original portage
structure. This resulted in vast speed improvements and reduction in
lines of code to do the same functionality.   Portage (due to Zac) has
made vast improvements in structure, but still requires a ton more
changes to get to where it should be.


> The benefit of getting people to change their behavior is that the
> benefits (less breakage, better tooling) are achieved now; as opposed
> to in some hypothetical future where repoman is improved.
> Note that we have been waiting for 'improved repoman' for about 8
> years; so..I'm not willing to bet on it when we have better tooling
> that works today and the primary complaint is that...what exactly?
> 
> The new workflow with pkgcheck was announced at the end of 2019:
> https://blogs.gentoo.org/mgorny/2019/12/12/a-better-ebuild-workflow-with-pure-git-and-pkgcheck
> 
> It's been 2 years, I think we can bring everyone into the fold here.
> 
> >
> > Looking into the future then maybe portage could even come to use
> > pkgcore for the low-level things that pkgcore does, then even users
> > could enjoy improved performance.  
> 
> Many things could happen but again, if you talk to people who work on
> pkgcore; there is no concrete plan for this. So I don't see why we are
> betting on it happening.
> 
> If you read radhermit's blog:
> https://pkgcraft.github.io/posts/timesink-chronicles/ you can get one
> take on the project and why it struggled.
> 
> -A
> 
> >
> >
> > Right?
> >
> > //Peter
> >  
> 

Re: [gentoo-dev] Deprecating repoman

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 652 bytes --]

>>>>> On Thu, 10 Mar 2022, Alec Warner wrote:

> I'm not sure a news item is strictly necessary, we might just p.mask
> repoman with a link to the guide that Matt will need to write about
> how repoman is being replaced.

We should distinguish between deprecating the repoman(-only) workflow
and deprecating the repoman tool. Doing the former doesn't imply masking
and removing the repoman package, at least not while it's working and
being maintained as part of Portage.

Also, what would be the harm in having another QA tool available in
addition to pkgcheck? (Long time ago, we used to have three of them.
Does anyone remember qualudis? :)

Ulrich

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 507 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Joshua Kinard]

On 3/11/2022 03:54, Mart Raudsepp wrote:> Ühel kenal päeval, N, 10.03.2022
kell 18:18, kirjutas Joshua Kinard:
>> I stick to the officially-published method of checking and committing
>> changes:
>> https://devmanual.gentoo.org/ebuild-maintenance/git/index.html
>>
>> The two tools highlighted there for the bulk of the work is repoman
>> and pkgdev.  repoman is cited twelve times, pkgdev is cited six times.
>> pkgcheck is mentioned once.  pkgcommit has no mentions.
>>
>> From that, one should not be faulted for assuming that repoman is the
>> more important tool, if not preferred tool, with pkgdev coming in
>> second place. pkgcheck comes across as entirely optional and even
>> seems equivalent to 'repoman full', and how would one know that
>> pkgcommit even exists?
>
> I believe the very purpose of this thread is to have a consensus/pre-
> announcement before actually editing the official documentation as part
> of the process of deprecating repoman.

I feel that the documentation should have had more mentions of these newer
tools as their adoption by other developers accelerated.  Documentation
doesn't have to have a fixed point in time when it fully changes over.  It
can change organically, like almost everything else in the project.


>
>> That all said, most of my checks are done pre-commit, not pre-push.
>> It's annoying trying to unwind a commit with a mistake in git, so I
>> aim to not have any in the commit itself.  By the time I get to the
>> push stage, everything MUST be perfect before I hit 'git push'. Cause
>> once something is public, there's no going back.  In a handful of
>> cases, I've gone as far as nuking my local copy of the tree, re-
>> cloning it, and re-doing everything again just to fix a mistake that I
>> should have caught pre-commit.
>
> git commit --amend, interactive rebase, etc (all before push). I suggest
> to learn them and embrace them. Re-cloning is not fun, just for
> something that a quick interactive rebase or even a git reset -- hard
> destructive operation instead of re-clone would solve.

The few times I resorted to a re-clone, the damage done was beyond
recoverability of --amend or even rebase.  I don't really remember the
specifics, as the last time I did a full re-clone was maybe 14-17 months
ago.  My developer tree lives on my NAS now with ZFS snapshotting, so
recovery is a lot easier, even if I have a really old snapshot.


> Also the benefit of using pkgcheck is to actually be able to make the
> same checks that CI would do before you push, so you can amend your
> commits to fix issues before they hit the server and CI and break the
> tree. pkgcheck is so fast that it can do full tree checks in a
> reasonable time (repoman would take days on a radiator mips when you go
> outside single package), and I believe has features to have it check all
> your commits that haven't been pushed yet at once, checking only what it
> can to not be too slow to not use (so you don't need to run the check
> with each commit but for all of them once you commit - and if issues,
> again, git interactive rebase).

Speed is really not a big issue for me.  I run repoman from my amd64 dev
box, and it's like, maybe 10-13 seconds at most during 'repoman full'?  And
my MIPS systems, while not the slowest of slow of that arch, they do teach
you patience over the years.

The other bits you mention about pkgcheck do sound useful, though.  But I am
a stickler for official documentation, because my risk aversion level when
committing to a public repo that can affect hundreds of thousands of users
is *extremely* high.  When I first signed up as a dev and we had the
gentoo-x86 CVS repo, there were no CI checks.  It was the responsibility of
the dev committing to get it right the first time, or else.  The fact I
haven't blown the tree up in years, even using archaic workflows, should be
proof enough that what I do does work, even if it is sub-optimal in the eyes
of others.

If we do deprecate repoman, that's fine.  I'll learn the new tooling.  My
initial beef was the use of subjective opinion in proposing the initial
change in the first place.  Even if it is blatantly obvious to many that A >
B, these mailing lists serve as a public archive of our work, so when
proposing key changes, regardless of how many people know/don't know about
something, full justification needs to be stated clearly and openly.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] Deprecating repoman

[Joshua Kinard]

On 3/11/2022 13:25, Alec Warner wrote:

[snip]

> 
> The new workflow with pkgcheck was announced at the end of 2019:
> https://blogs.gentoo.org/mgorny/2019/12/12/a-better-ebuild-workflow-with-pure-git-and-pkgcheck
> 
> It's been 2 years, I think we can bring everyone into the fold here.

I've searched my -dev archives for part of that URL, and the only hits I am
getting is this e-mail thread.  Was this URL previously shared on this
mailing list or another?  I do not track the Gentoo Blogs, so unless
something is shared to the mailing lists, I will likely miss it.

That said, I will admit I am uncomfortable with post-commit, pre-push
validation.   I get that git is vastly different, and vastly superior, to
CVS.  Get it right the first time, and you don't have to worry about fixing
it later -- CVS teaches you that very well, and it still works well for git
workflows.  Going back into git post-commit to fix things is still something
I need to learn more about, as my git-fu is still pretty amateurish beyond
the common basics.  Especially when dealing with kernel patch maintenance
and maintaining lots of small, discrete changes that kernel upstream prefers.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] Deprecating repoman

[Arthur Zamarin]

[-- Attachment #1.1: Type: text/plain, Size: 1723 bytes --]

On 11/03/2022 21.51, Joshua Kinard wrote:
> On 3/11/2022 13:25, Alec Warner wrote:
> 
> [snip]
> 
>>
>> The new workflow with pkgcheck was announced at the end of 2019:
>> https://blogs.gentoo.org/mgorny/2019/12/12/a-better-ebuild-workflow-with-pure-git-and-pkgcheck
>>
>> It's been 2 years, I think we can bring everyone into the fold here.
> 
> I've searched my -dev archives for part of that URL, and the only hits I am
> getting is this e-mail thread.  Was this URL previously shared on this
> mailing list or another?  I do not track the Gentoo Blogs, so unless
> something is shared to the mailing lists, I will likely miss it.
> 
> That said, I will admit I am uncomfortable with post-commit, pre-push
> validation.   I get that git is vastly different, and vastly superior, to
> CVS.  Get it right the first time, and you don't have to worry about fixing
> it later -- CVS teaches you that very well, and it still works well for git
> workflows.  Going back into git post-commit to fix things is still something
> I need to learn more about, as my git-fu is still pretty amateurish beyond
> the common basics.  Especially when dealing with kernel patch maintenance
> and maintaining lots of small, discrete changes that kernel upstream prefers.
> 

Just to note, when using pkgdev, I use mainly the command `pkgdev commit
-as`, which stands for auto add files in current working pkg and run
scan *before commit* - which is identical to `repoman full -dx`.

I think I'm going to add configuration mode to pkgdev, so you can set
scan mode to on by default (you can pass `--scan false` to disable it).

-- 
Arthur Zamarin
arthurzam@gentoo.org
Gentoo Linux developer (Python, GURU)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[gentoo-dev] Re: Deprecating repoman

[Matt Turner]

I've filed a PR against devmanual.git to remove references to repoman
and replace them with references to pkgdev where appropriate. Most
everywhere already had pkgdev/pkgcheck text in place so there wasn't
much to do. See: https://github.com/gentoo/devmanual/pull/274

Re: [gentoo-dev] Deprecating repoman

[Francesco Riosa]

[-- Attachment #1: Type: text/plain, Size: 326 bytes --]

Il giorno mer 9 mar 2022 alle ore 22:01 Matt Turner <mattst88@gentoo.org>
ha scritto:

> I'd like to deprecate and ultimately remove repoman. I believe that
> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
>

Hi using  `repoman manifest` in scripts here, what would be the correct
replacement for that?

[-- Attachment #2: Type: text/html, Size: 689 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Matt Turner]

On Fri, Mar 11, 2022 at 4:43 PM Francesco Riosa <vivo75@gmail.com> wrote:
>
> Il giorno mer 9 mar 2022 alle ore 22:01 Matt Turner <mattst88@gentoo.org> ha scritto:
>>
>> I'd like to deprecate and ultimately remove repoman. I believe that
>> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
>
>
> Hi using  `repoman manifest` in scripts here, what would be the correct replacement for that?

pkgdev manifest

Re: [gentoo-dev] Deprecating repoman

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1624 bytes --]



> On 11 Mar 2022, at 19:51, Joshua Kinard <kumba@gentoo.org> wrote:
> 
> On 3/11/2022 13:25, Alec Warner wrote:
> 
> [snip]
> 
>> 
>> The new workflow with pkgcheck was announced at the end of 2019:
>> https://blogs.gentoo.org/mgorny/2019/12/12/a-better-ebuild-workflow-with-pure-git-and-pkgcheck
>> 
>> It's been 2 years, I think we can bring everyone into the fold here.
> 
> I've searched my -dev archives for part of that URL, and the only hits I am
> getting is this e-mail thread.  Was this URL previously shared on this
> mailing list or another?  I do not track the Gentoo Blogs, so unless
> something is shared to the mailing lists, I will likely miss it.

I think you may be latching on a bit to the pkgcommit thing. Nobody is making
you use mgorny's scripts. pkgdev has been advertised on this very ML:

https://marc.info/?l=gentoo-dev&m=161443741531808&w=2

> 
> That said, I will admit I am uncomfortable with post-commit, pre-push
> validation.   I get that git is vastly different, and vastly superior, to
> CVS.  Get it right the first time, and you don't have to worry about fixing
> it later -- CVS teaches you that very well, and it still works well for git
> workflows.  Going back into git post-commit to fix things is still something
> I need to learn more about, as my git-fu is still pretty amateurish beyond
> the common basics.  Especially when dealing with kernel patch maintenance
> and maintaining lots of small, discrete changes that kernel upstream prefers.

You can always do 'pkgcheck scan' before committing, or, I think
'pkgdev commit -A' might work.

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 2898 bytes --]



> On 11 Mar 2022, at 19:39, Joshua Kinard <kumba@gentoo.org> wrote:
> 
> On 3/11/2022 03:54, Mart Raudsepp wrote:> Ühel kenal päeval, N, 10.03.2022
> kell 18:18, kirjutas Joshua Kinard:
>>> I stick to the officially-published method of checking and committing
>>> changes:
>>> https://devmanual.gentoo.org/ebuild-maintenance/git/index.html
>>> 
>>> The two tools highlighted there for the bulk of the work is repoman
>>> and pkgdev.  repoman is cited twelve times, pkgdev is cited six times.
>>> pkgcheck is mentioned once.  pkgcommit has no mentions.
>>> 
>>> From that, one should not be faulted for assuming that repoman is the
>>> more important tool, if not preferred tool, with pkgdev coming in
>>> second place. pkgcheck comes across as entirely optional and even
>>> seems equivalent to 'repoman full', and how would one know that
>>> pkgcommit even exists?
>> 
>> I believe the very purpose of this thread is to have a consensus/pre-
>> announcement before actually editing the official documentation as part
>> of the process of deprecating repoman.
> 
> I feel that the documentation should have had more mentions of these newer
> tools as their adoption by other developers accelerated.  Documentation
> doesn't have to have a fixed point in time when it fully changes over.  It
> can change organically, like almost everything else in the project.

Well, I've done that. I've been adding pkgcheck and pkgdev to the devmanual
over time, and to the wiki.

> [snip]

>> Also the benefit of using pkgcheck is to actually be able to make the
>> same checks that CI would do before you push, so you can amend your
>> commits to fix issues before they hit the server and CI and break the
>> tree. pkgcheck is so fast that it can do full tree checks in a
>> reasonable time (repoman would take days on a radiator mips when you go
>> outside single package), and I believe has features to have it check all
>> your commits that haven't been pushed yet at once, checking only what it
>> can to not be too slow to not use (so you don't need to run the check
>> with each commit but for all of them once you commit - and if issues,
>> again, git interactive rebase).
> 
> Speed is really not a big issue for me.  I run repoman from my amd64 dev
> box, and it's like, maybe 10-13 seconds at most during 'repoman full'?  And
> my MIPS systems, while not the slowest of slow of that arch, they do teach
> you patience over the years.
> 
> The other bits you mention about pkgcheck do sound useful, though.  But I am
> a stickler for official documentation, because my risk aversion level when
> committing to a public repo that can affect hundreds of thousands of users
> is *extremely* high.  When I first signed up as a dev and we had the

It is already mentioned in the devmanual, but we can add it in more places
if you specify which.

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 309 bytes --]



> On 11 Mar 2022, at 17:14, Peter Stuge <peter@stuge.se> wrote:
> [snip]
> Looking into the future then maybe portage could even come to use
> pkgcore for the low-level things that pkgcore does, then even users
> could enjoy improved performance.
> 

Are you volunteering to work on this substantial task?


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 4893 bytes --]



> On 10 Mar 2022, at 23:18, Joshua Kinard <kumba@gentoo.org> wrote:
> 
> On 3/10/2022 14:58, Alec Warner wrote:
>> On Thu, Mar 10, 2022 at 10:27 AM Joshua Kinard <kumba@gentoo.org> wrote:
>>> 
>>> On 3/9/2022 16:00, Matt Turner wrote:
>>>> I'd like to deprecate and ultimately remove repoman. I believe that
>>>> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
>>>> are far superior replacements, and it makes sense to have people using
>>>> the same tool and seeing the same warnings as in the CI.
>>>> 
>>>> Are there any useful checks or behaviors of repoman that are missing
>>>> from pkgcheck and pkgcommit?
>>>> 
>>>> Thanks,
>>>> Matt
>>> 
>>> repoman has been //the// goto tool for checking in a change since before I
>>> was a developer in 2003.  It does everything we need, in one simple tool.
>>> Your proposal looks to replace repoman's functionality (and snark) with two
>>> or more packages, including tools from a package named after a fellow developer.
>>> 
>>> Additionally, "I believe that <foo> are far superior replacements" is an
>>> entirely subjective opinion and, frankly, completely invalid as
>>> justification to replace a tool that has worked fine for the last 20+ years.
>>> What is so fundamentally broken about repoman that cannot be fixed such
>>> that it needs total replacement by multiple independent tools?  Please
>>> document. the pros and cons here so that we can all make an informed decision.

Matt could've given more details about why pkgcheck is superior but I think
this is actually showing/exposing the problem again: we've assumed that
everybody should really know repoman lacks a significant number of the checks
pkgcheck has, as well as being much slower.

Those are the two reasons why it's superior.

>> 
>> So here is the more basic argument, you can agree or disagree.
>> 
>> *The goal I want is for people to commit to the tree and not break things.*
> 
> This goal I agree with, and I am rather pedantic about.  If not mildly
> fearful of.  We've all been there at least once in our dev-lives.  It's
> almost a rite of passage, if you will, to break the tree with a dumb commit
> at least once.  Goal is to never repeat that mistake again.
> 

Right. I spend a fair amount of time fixing issues with repoman doesn't
find but pkgcheck / CI does, or filing bugs for them.

> 
>> If we could accomplish this with no tooling at all, that would be
>> great. Sadly humans are fallible and so we have tools to check their
>> work. Currently this tooling has two parts:
>> - pre-push tooling that you run prior to pushing.
>> - post-push CI tooling that informs you when you break the tree.
>> 
>> So holding to our goal of not breaking the tree, it's better for
>> developers to run the same QA tool in pre-push that CI is using,
>> because our metric for 'breaking the tree' is the CI tool and if the
>> CI tool passes locally, there is a strong likelihood it will not break
>> in CI either. Note this argument is generic, I'm not even saying which
>> tools are in use, or which tools are better, or anything like that.
>> 
>> Here we see Matt discussing a workflow he finds frustrating.
>> - A developer does a push.
>> - Their push breaks CI.
>> - He inquires about their workflow.
>> - He learns they did not run the CI tool in their pre-push workflow.
>> - He tells them they should run the CI tool in their pre-push workflow.
>> - This happens many times, causing this thread.
>> 
>> The point of the thread then is to convince people to run the CI tool
>> in pre-push, as a matter of policy, to reduce tree breakage and reduce
>> the occurrence of the above conversation.
> 
> I stick to the officially-published method of checking and committing changes:
> https://devmanual.gentoo.org/ebuild-maintenance/git/index.html
> 
> The two tools highlighted there for the bulk of the work is repoman and
> pkgdev.  repoman is cited twelve times, pkgdev is cited six times.  pkgcheck
> is mentioned once.  pkgcommit has no mentions.

pkgcommit is just a wrapper around git and pkgcheck, so it is there.

It's not like you aren't allowed to make your own wrappers or aliases or scripts, right?


>> 
> 
> Back in mid-2002, it was exactly Gentoo's snarkness that tipped the scales
> for me.  I'd just given up on FreeBSD-4.x on a sparc64 machine (old Sun
> Blade 100) and LFS turned out to not be a lot of fun, but Gentoo worked fine
> on it.  All of the colors on the terminal gave it zing and pop, and made it
> rather fun to work with.  rpm and apt-get were dull;  emerge was cheeky and
> playful!  Still is to this day.
> 

I have no objection to (and in fact would rather welcome) contributions
to make other tools more "Gentoo-like". Could you make a PR or provide
some patch to add some more personality to them?

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Joshua Kinard]

On 3/11/2022 20:54, Sam James wrote:
> 
> 
>> On 11 Mar 2022, at 19:39, Joshua Kinard <kumba@gentoo.org> wrote:
>>
>> On 3/11/2022 03:54, Mart Raudsepp wrote:> Ühel kenal päeval, N, 10.03.2022
>> kell 18:18, kirjutas Joshua Kinard:
>>>> I stick to the officially-published method of checking and committing
>>>> changes:
>>>> https://devmanual.gentoo.org/ebuild-maintenance/git/index.html
>>>>
>>>> The two tools highlighted there for the bulk of the work is repoman
>>>> and pkgdev.  repoman is cited twelve times, pkgdev is cited six times.
>>>> pkgcheck is mentioned once.  pkgcommit has no mentions.
>>>>
>>>> From that, one should not be faulted for assuming that repoman is the
>>>> more important tool, if not preferred tool, with pkgdev coming in
>>>> second place. pkgcheck comes across as entirely optional and even
>>>> seems equivalent to 'repoman full', and how would one know that
>>>> pkgcommit even exists?
>>>
>>> I believe the very purpose of this thread is to have a consensus/pre-
>>> announcement before actually editing the official documentation as part
>>> of the process of deprecating repoman.
>>
>> I feel that the documentation should have had more mentions of these newer
>> tools as their adoption by other developers accelerated.  Documentation
>> doesn't have to have a fixed point in time when it fully changes over.  It
>> can change organically, like almost everything else in the project.
> 
> Well, I've done that. I've been adding pkgcheck and pkgdev to the devmanual
> over time, and to the wiki.

Yes, I'd seen that, though at least in the ebuild maintenance section on
git, pkgdev's mentions were added in such a way that it was implied to be
comparable to repoman.  Not better than repoman, as many people here are
stating.

People like me, when we see that, are going to conduct an internal risk
assessment.  Use what I know works and get results I know I will get, or use
a completely new tool that I know nothing about, and maybe get the same
result with no perceived benefit or gain.


>> [snip]
> 
>>> Also the benefit of using pkgcheck is to actually be able to make the
>>> same checks that CI would do before you push, so you can amend your
>>> commits to fix issues before they hit the server and CI and break the
>>> tree. pkgcheck is so fast that it can do full tree checks in a
>>> reasonable time (repoman would take days on a radiator mips when you go
>>> outside single package), and I believe has features to have it check all
>>> your commits that haven't been pushed yet at once, checking only what it
>>> can to not be too slow to not use (so you don't need to run the check
>>> with each commit but for all of them once you commit - and if issues,
>>> again, git interactive rebase).
>>
>> Speed is really not a big issue for me.  I run repoman from my amd64 dev
>> box, and it's like, maybe 10-13 seconds at most during 'repoman full'?  And
>> my MIPS systems, while not the slowest of slow of that arch, they do teach
>> you patience over the years.
>>
>> The other bits you mention about pkgcheck do sound useful, though.  But I am
>> a stickler for official documentation, because my risk aversion level when
>> committing to a public repo that can affect hundreds of thousands of users
>> is *extremely* high.  When I first signed up as a dev and we had the
> 
> It is already mentioned in the devmanual, but we can add it in more places
> if you specify which.

Matt's posted a pull request in GH that looks to eliminate all remaining
references of repoman and replace them with pkgdev.  So I think we're done
with this point now once that request is merged in.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] Deprecating repoman

[Joshua Kinard]

On 3/11/2022 20:57, Sam James wrote:
> 
> 
>> On 10 Mar 2022, at 23:18, Joshua Kinard <kumba@gentoo.org> wrote:
>>
>> On 3/10/2022 14:58, Alec Warner wrote:
>>> On Thu, Mar 10, 2022 at 10:27 AM Joshua Kinard <kumba@gentoo.org> wrote:
>>>>
>>>> On 3/9/2022 16:00, Matt Turner wrote:
>>>>> I'd like to deprecate and ultimately remove repoman. I believe that
>>>>> dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
>>>>> are far superior replacements, and it makes sense to have people using
>>>>> the same tool and seeing the same warnings as in the CI.
>>>>>
>>>>> Are there any useful checks or behaviors of repoman that are missing
>>>>> from pkgcheck and pkgcommit?
>>>>>
>>>>> Thanks,
>>>>> Matt
>>>>
>>>> repoman has been //the// goto tool for checking in a change since before I
>>>> was a developer in 2003.  It does everything we need, in one simple tool.
>>>> Your proposal looks to replace repoman's functionality (and snark) with two
>>>> or more packages, including tools from a package named after a fellow developer.
>>>>
>>>> Additionally, "I believe that <foo> are far superior replacements" is an
>>>> entirely subjective opinion and, frankly, completely invalid as
>>>> justification to replace a tool that has worked fine for the last 20+ years.
>>>> What is so fundamentally broken about repoman that cannot be fixed such
>>>> that it needs total replacement by multiple independent tools?  Please
>>>> document. the pros and cons here so that we can all make an informed decision.
> 
> Matt could've given more details about why pkgcheck is superior but I think
> this is actually showing/exposing the problem again: we've assumed that
> everybody should really know repoman lacks a significant number of the checks
> pkgcheck has, as well as being much slower.
> 
> Those are the two reasons why it's superior.

But again, those are subjective observations.  Maybe it's my longer
experience with the project, or maybe it's because I usually refrain from
poking the more complex bits in the tree, or maybe it's the particular niche
I've stuck to, the extra checks that pkgcheck runs haven't really applied to
me.  If I do too many significant changes to an ebuild, I'll re-read its
source a half-dozen times or more to make ***sure*** that I didn't miss
something.  I will grep the tree for similar bits of code to make sure I'm
doing something reasonably sane, I will test that ebuild on at least two
different arches (amd64 and mips), etc.  Maybe I over think it, or maybe I
have some form of hyperpedanticism.  Or maybe I've just been really lucky.
:: shrugs ::

And speed, again, is really quite far down on my list of concerns most of
the time.

That said, yes, I agree with you wholeheartedly that there was a failure at
the Project/Distro level to explain the benefits of using pkgcheck over
repoman scan/full.  That's a communications failure, and it is really
symbolic of a larger issue that projects like ours often struggle with.
Each of us tends to operate off in their own little fiefdom, something I am
just as guilty of as anyone else, and we don't always know what other devs
are doing or how they're doing it.  I wish I had suggestions to offer up at
the moment on fixing this, but, alas...


>>>
>>> So here is the more basic argument, you can agree or disagree.
>>>
>>> *The goal I want is for people to commit to the tree and not break things.*
>>
>> This goal I agree with, and I am rather pedantic about.  If not mildly
>> fearful of.  We've all been there at least once in our dev-lives.  It's
>> almost a rite of passage, if you will, to break the tree with a dumb commit
>> at least once.  Goal is to never repeat that mistake again.
>>
> 
> Right. I spend a fair amount of time fixing issues with repoman doesn't
> find but pkgcheck / CI does, or filing bugs for them.
> 
>>
>>> If we could accomplish this with no tooling at all, that would be
>>> great. Sadly humans are fallible and so we have tools to check their
>>> work. Currently this tooling has two parts:
>>> - pre-push tooling that you run prior to pushing.
>>> - post-push CI tooling that informs you when you break the tree.
>>>
>>> So holding to our goal of not breaking the tree, it's better for
>>> developers to run the same QA tool in pre-push that CI is using,
>>> because our metric for 'breaking the tree' is the CI tool and if the
>>> CI tool passes locally, there is a strong likelihood it will not break
>>> in CI either. Note this argument is generic, I'm not even saying which
>>> tools are in use, or which tools are better, or anything like that.
>>>
>>> Here we see Matt discussing a workflow he finds frustrating.
>>> - A developer does a push.
>>> - Their push breaks CI.
>>> - He inquires about their workflow.
>>> - He learns they did not run the CI tool in their pre-push workflow.
>>> - He tells them they should run the CI tool in their pre-push workflow.
>>> - This happens many times, causing this thread.
>>>
>>> The point of the thread then is to convince people to run the CI tool
>>> in pre-push, as a matter of policy, to reduce tree breakage and reduce
>>> the occurrence of the above conversation.
>>
>> I stick to the officially-published method of checking and committing changes:
>> https://devmanual.gentoo.org/ebuild-maintenance/git/index.html
>>
>> The two tools highlighted there for the bulk of the work is repoman and
>> pkgdev.  repoman is cited twelve times, pkgdev is cited six times.  pkgcheck
>> is mentioned once.  pkgcommit has no mentions.
> 
> pkgcommit is just a wrapper around git and pkgcheck, so it is there.
> 
> It's not like you aren't allowed to make your own wrappers or aliases or scripts, right?

Yes, I could.  but that's not the point I was making.  My point was Matt
recommended pkgcommit as one of the two tools deemed superior to repoman,
but without providing any context.  TBH, I didn't even know that the
containing package was even in the tree, and I certainly didn't even know
pkgcommit existed.  I made the point I did about there being no mentions of
pkgcommit in that part of the devmanual to underscore that fact.

I've got my own small (but growing) collection of trashy little hackscripts
(and a collection of aliases) for maintaining my Gentoo and BSD systems.
Many of them are so specific to things I do locally on my machines, they're
never going to wind up published anywhere, because they likely won't work
for anyone but me.


>>>
>>
>> Back in mid-2002, it was exactly Gentoo's snarkness that tipped the scales
>> for me.  I'd just given up on FreeBSD-4.x on a sparc64 machine (old Sun
>> Blade 100) and LFS turned out to not be a lot of fun, but Gentoo worked fine
>> on it.  All of the colors on the terminal gave it zing and pop, and made it
>> rather fun to work with.  rpm and apt-get were dull;  emerge was cheeky and
>> playful!  Still is to this day.
>>
> 
> I have no objection to (and in fact would rather welcome) contributions
> to make other tools more "Gentoo-like". Could you make a PR or provide
> some patch to add some more personality to them?

Unfortunately, I have been advised by many of my friends and associates to
stay well away from anything remotely resembling comedy.  Like many people,
I get the jokes and laugh along with them, but like a Vogon reading poetry,
I should never attempt to create the jokes.  I get away with the small pun
here and there, but it's only a matter of time before the gallows will find
me for one of those.

And really, much of Portage's built-in humor is largely a function of
carpaski being one of its original authors.  He's even got his own Urban
Dictionary entry, which should tell you a lot about things:

https://www.urbandictionary.com/define.php?term=carpaski

Honestly, see that entry brings back a lot of memories...

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

[gentoo-dev] Re: Deprecating repoman

[Fabian Groffen]

[-- Attachment #1: Type: text/plain, Size: 746 bytes --]

Just wondering, is there a "migration guide" or something?  I've never
used pkg* since joining in 2005.  I can derive some things from the
first look at the below commit, but an "expert opinion" to just map the
standard things from repoman to appropriate commands would be nice.

Are those pkg* keyworded *everywhere*, by the way?

Thanks,
Fabian


On 11-03-2022 13:19:34 -0800, Matt Turner wrote:
> I've filed a PR against devmanual.git to remove references to repoman
> and replace them with references to pkgdev where appropriate. Most
> everywhere already had pkgdev/pkgcheck text in place so there wasn't
> much to do. See: https://github.com/gentoo/devmanual/pull/274
> 

-- 
Fabian Groffen
Gentoo on a different level

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Mart Raudsepp]

Ühel kenal päeval, R, 11.03.2022 kell 21:53, kirjutas Joshua Kinard:
> the extra checks that pkgcheck runs haven't really applied to me.

Looks to be mostly true, as you maintain only a few packages, but you
might find these links useful to filter the CI output per maintainer
regardless:

https://qa-reports.gentoo.org/output/gentoo-ci/output.html;maintainer=kumba
https://qa-reports.gentoo.org/output/gentoo-ci/output.html;maintainer=kumba;include-projects

Or some of these:

https://packages.gentoo.org/maintainer/kumba@gentoo.org/stabilization
https://packages.gentoo.org/maintainer/kumba@gentoo.org/outdated


Mart

Re: [gentoo-dev] Deprecating repoman

[Hank Leininger]

[-- Attachment #1: Type: text/plain, Size: 2068 bytes --]

> On 11-03-2022 13:19:34 -0800, Matt Turner wrote:
> > I've filed a PR against devmanual.git to remove references to repoman
> > and replace them with references to pkgdev where appropriate.

On 2022-03-12, Fabian Groffen wrote:
> 
> Just wondering, is there a "migration guide" or something?  I've never
> used pkg* since joining in 2005.  I can derive some things from the
> first look at the below commit, but an "expert opinion" to just map the
> standard things from repoman to appropriate commands would be nice.

Yes, please this, and not just in devmanual, but also:

  https://www.gentoo.org/glep/glep-0066.html
  https://wiki.gentoo.org/wiki/Gentoo_git_workflow
  https://wiki.gentoo.org/wiki/Gentoo_GitHub

All three of those mention repoman some or a lot, and pkgcheck/pkgdev
little or none.

I'm not a dev, just a forgetful proxy maintainer. When I went to learn
the right workflow to submit better-quality PRs on GitHub instead of
just attaching patches to b.g.o. bugs some years back, I looked at
devmanual but the git workflow specific content seemed to be targeted
only at people with access to git.gentoo.org, so I looked elsewhere.

Those Wiki pages talked specifically about my use case. I think they
only mentioned repoman back then. Even now they are 10x more specific
and detailed about how to use repoman in your workflow, vs how to use
pkgcheck.

And indeed, the devmanual still doesn't line up 1:1 with the apparently
soon-to-be-obsolete content in the wikis. For example, wikis say to use
"repoman -dx full"; devmanual only talks about "repoman full" and
replacing that with "pkgcheck scan --commits". What happened to
--include-dev and --xmlparse? Are they not needed/implied in the
pkgcheck case? (From its manpage, I think maybe the default behavior is
equivalent to enabling both of those, but...?)

This is why you can't trust ignorant fools like me to figure it out for
ourselves :-P

Thanks,

-- 

Hank Leininger <hlein@korelogic.com>
9606 3BF9 B593 4CBC E31A  A384 6200 F6E3 781E 3DD7

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[gentoo-dev] Re: Deprecating repoman

[Matt Turner]

I've filed a Council bug to approve the plan for deprecating repoman.
See https://bugs.gentoo.org/835013

The proposed plan, as copied from the bug is:

> The devmanual has already been updated to contain information about pkgdev, in March 2021. See https://gitweb.gentoo.org/proj/devmanual.git/log/?qt=grep&q=pkgdev
>
> I have opened a devmanual pull request to remove references to repoman that might suggest that it is still an appropriate tool to use. See https://github.com/gentoo/devmanual/pull/274
>
> The next steps once the devmanual change is committed, I think, are
>
> - Give the wiki similar treatment as the devmanual, replacing and removing references to repoman that would suggest it is an appropriate tool to use.
>
> - Modify repoman to emit a warning informing users of its deprecation
>
> - After some period of time, maybe 6 months, give last rites to repoman
>
> - Delete repoman from portage.git
>
> (and of course adding any features/behaviors we find lacking in pkgdev, et al)

[gentoo-dev] Re: Deprecating repoman

[Matt Turner]

On Sat, Mar 12, 2022 at 12:26 PM Matt Turner <mattst88@gentoo.org> wrote:
> I've filed a Council bug to approve the plan for deprecating repoman.
> See https://bugs.gentoo.org/835013

With a vote of 6-0, the Council approved the following motion:

> pkgcheck is now considered the primary Gentoo tool for ebuild QA.  repoman is
> no longer considered sufficient. Council condones the deprecation and removal
> of repoman by the portage team.

There was resistance to the idea of making repoman print a deprecation
warning when executed, so I will drop that idea from the plan.

Re: [gentoo-dev] Deprecating repoman

[Zoltan Puskas]

[-- Attachment #1: Type: text/plain, Size: 3809 bytes --]

I've been using both repoman _and_ pkgcheck becasue I was not sure which
one covers all the checks I need to make. In fact [Pull Requests
wiki](https://wiki.gentoo.org/wiki/Project:GitHub/Pull_requests) has a
big red warning box with the following message, and I quote:

"CI is not an excuse not to run repoman full, at least for the packages
being committed."

[Package maintainer's responsibilities
wiki](https://wiki.gentoo.org/wiki/Package_maintainer%27s_responsibilities)
says "When committing changes, repoman commit **or** pkgdev commit (from the
dev-util/pkgdev package) can be used.", emphasis mine.

I use documentation from time to time to remind myself of details, as I
want to avoid bugging people on IRC or the mailing list, and to provide
up to date and clean PRs. Per said documentation repoman is (was?)
a cornerstone of the workflow, as mentioned by others too. Developers
should not be frowned upon for using Repoman, as they were doing
exactly what was asked of them.

Additionally based on the existing documentation it was not evident
that, pkgcheck covers all checks done by repoman and more, or that it's
faster, or that it has advanced features, like scanning only commits since
HEAD/origin (I myself was only ever using it in the ebuild's directory,
because Repoman is slow when running over more than one ebuild). I've
had to read most of this thread to obtain all these details and to learn
about the proper replacements for all the steps in my workflow.

Today I've unmerged repoman and I've used only pkgdev, pkgcheck, and git
to create a new PR to force myself off Repoman. So far I think the
experience was superior to the Repoman workflow. I like that I can run
pkgcheck at the end before the push only on the commits and I don't have
to worry about Repoman trying to scan the entire tree. I think `pkgdev
commit` is quite sufficient, not sure why would one need to use
pkgcommit.

I found that mgorny's dev scripts package has a lot of irrelevant stuff
I don't need on my system (maybe not even applicable to anyone outside
of mgorny), I'd prefer we don't make it part of the mandated workflow as
is. Maybe if the generally useful scripts are separated out to some
gentoo devtoolkit package or similar, then it's okay.

Summa summarum, I think deprecating repoman is the right direction,
but documentation needs to be updated comprehensively before it's done.

Just a proxy maintainer's 2¢.

Zoltan

P.S.: I'm with Joshua about the snarkiness of the tooling, I liked that
a lot and it's a bit sad that the new tools are a bit sterile.

On Wed, Mar 09, 2022 at 04:25:17PM -0500, Ionen Wolkens wrote:
> On Wed, Mar 09, 2022 at 01:00:37PM -0800, Matt Turner wrote:
> > I'd like to deprecate and ultimately remove repoman. I believe that
> > dev-util/pkgcheck and pkgcommit (from app-portage/mgorny-dev-scripts)
> 
> There's also dev-util/pkgdev as an alternative to pkgcommit with more
> features for those that want them that will feel similar to repoman
> - pkgdev commit -c/-b bugnum (repoman commit -c/-b bugnum)
> - pkgdev manifest (repoman manifest)
> - pkgdev push (runs pkgcheck scan --commits before pushing)
> - updates copyright year / manifest on commit
> - basic commit message templates, like auto "cat/pkg: add <version>" on
>   bumps without manual editing -- prepend cat/pkg: like other tools
>   if nothing specific
> 
> And other handy features like pkgdev mask --rites days
> 
> > are far superior replacements, and it makes sense to have people using
> > the same tool and seeing the same warnings as in the CI.
> > 
> > Are there any useful checks or behaviors of repoman that are missing
> > from pkgcheck and pkgcommit?
> > 
> > Thanks,
> > Matt
> > 
> 
> -- 
> ionen



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 871 bytes --]

>>>>> On Sat, 19 Mar 2022, Zoltan Puskas wrote:

[Please don't top-post.]

> I've been using both repoman _and_ pkgcheck becasue I was not sure which
> one covers all the checks I need to make. In fact [Pull Requests
> wiki](https://wiki.gentoo.org/wiki/Project:GitHub/Pull_requests) has a
> big red warning box with the following message, and I quote:

> "CI is not an excuse not to run repoman full, at least for the packages
> being committed."

That warning box also says: "Furthermore, pkgcheck is imperfect (yet
very fast) and may fail to notice some subtle breakages (especially if
USE flag masking/forcing) is involved."

Looks like the warning was added by mgorny in 2016 [1]. Have these
issues with pkgcheck been fixed in the meantime?

Ulrich

[1] https://wiki.gentoo.org/index.php?title=Project%3AGitHub%2FPull_requests&type=revision&diff=575482&oldid=575480

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 507 bytes --]

Re: [gentoo-dev] Deprecating repoman

[Michał Górny]

On Sat, 2022-03-19 at 08:15 +0100, Ulrich Mueller wrote:
> > > > > > On Sat, 19 Mar 2022, Zoltan Puskas wrote:
> 
> [Please don't top-post.]
> 
> > I've been using both repoman _and_ pkgcheck becasue I was not sure
> > which
> > one covers all the checks I need to make. In fact [Pull Requests
> > wiki](https://wiki.gentoo.org/wiki/Project:GitHub/Pull_requests) has
> > a
> > big red warning box with the following message, and I quote:
> 
> > "CI is not an excuse not to run repoman full, at least for the
> > packages
> > being committed."
> 
> That warning box also says: "Furthermore, pkgcheck is imperfect (yet
> very fast) and may fail to notice some subtle breakages (especially if
> USE flag masking/forcing) is involved."
> 
> Looks like the warning was added by mgorny in 2016 [1]. Have these
> issues with pkgcheck been fixed in the meantime?

Yes, I think all the blockers were fixed since.

-- 
Best regards,
Michał Górny