[gentoo-dev] Continuous integration on GURU

[gentoo-dev] Continuous integration on GURU

[Agostino Sarubbo]

Hello,

I would like to let you know that the CI is now able to work with overlays.

Since Guru is pretty active I took advantage of this situation to implement 
and test the CI

Atm it has been configured to work with master branch.
I don't know how much is useful scan the dev branch since a first revision has 
not yet been done. If you have opinions about, please let me know.

More info at:
https://blogs.gentoo.org/ago/2020/07/04/gentoo-tinderbox/

The bugs will have an internal_ref as guru_ci.

Agostino

Re: [gentoo-dev] Continuous integration on GURU

[Michał Górny]

On Thu, 2021-04-22 at 11:59 +0200, Agostino Sarubbo wrote:
> Hello,
> 
> I would like to let you know that the CI is now able to work with overlays.
> 
> Since Guru is pretty active I took advantage of this situation to implement 
> and test the CI
> 
> Atm it has been configured to work with master branch.
> I don't know how much is useful scan the dev branch since a first revision has 
> not yet been done. If you have opinions about, please let me know.
> 
> More info at:
> https://blogs.gentoo.org/ago/2020/07/04/gentoo-tinderbox/
> 
> The bugs will have an internal_ref as guru_ci.
> 

Well, I suppose scanning the dev branch would be preferable over
the master branch.  In reality, they are usually only a few hours apart
but it might be useful to know of new breakage in dev before it's merged
to master.

It would be ideal if you could do a switch when master and dev are
in sync, and just copy the state from master.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] Continuous integration on GURU

[Agostino Sarubbo]

On giovedì 22 aprile 2021 12:02:20 CEST Michał Górny wrote:
> Well, I suppose scanning the dev branch would be preferable over
> the master branch.  In reality, they are usually only a few hours apart
> but it might be useful to know of new breakage in dev before it's merged
> to master.
> 
> It would be ideal if you could do a switch when master and dev are
> in sync, and just copy the state from master.

Hi,

I think that your approach could be generally valid but for this use case I'm 
against because of the following:

1) The approach is valid in cases like our github PRs and the bot that 
approves the commit. In this case, who moves the commit between branches does 
not know if the scan has been done or not.

2) I don't see the reason to scan against something that we don't know if will 
be the same in master branch

3) We are not doing a similar approach for ::gentoo so I don't see why do this 
for GURU since, after all, it is an overlay

4) Packages in master are supposed to be tested at least from 2 different 
people (who made the commit in dev and who moves the commit to master) so it 
means less bugspam


> One more thing: might be a good idea to consider splitting some
> of the 'big' trackers (like CFLAGS) between Gentoo and GURU.  I think
> solving these bugs in GURU has lower priority than in Gentoo.

I think that trackers like CFLAGS/LDFLAGS are here to track how many packages 
have the problem. I don't see it like (for example) the gcc-porting tracker 
that gives the idea about how much packages need a fix and how much packages 
need to be last-rited


Agostino

Re: [gentoo-dev] Continuous integration on GURU

[Ionen Wolkens]

[-- Attachment #1: Type: text/plain, Size: 1001 bytes --]

On Thu, Apr 22, 2021 at 02:39:02PM +0200, Agostino Sarubbo wrote:
> On giovedì 22 aprile 2021 12:02:20 CEST Michał Górny wrote:
> > One more thing: might be a good idea to consider splitting some
> > of the 'big' trackers (like CFLAGS) between Gentoo and GURU.  I think
> > solving these bugs in GURU has lower priority than in Gentoo.
> 
> I think that trackers like CFLAGS/LDFLAGS are here to track how many packages 
> have the problem. I don't see it like (for example) the gcc-porting tracker 
> that gives the idea about how much packages need a fix and how much packages 
> need to be last-rited

If want to track, can't we just have separate tracker(s) for GURU?

Now when look at this tracker can't tell how much is ::gentoo or
::guru without doing some extra work.

And, even if you don't see it like that, someday someone may want
to go on a fixing spree using those trackers (may it be for ::guru
or ::gentoo) but will keep seeing them mixed up together.
-- 
ionen

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] Continuous integration on GURU

[Agostino Sarubbo]

On giovedì 22 aprile 2021 14:59:00 CEST Ionen Wolkens wrote:
> And, even if you don't see it like that, someday someone may want
> to go on a fixing spree using those trackers (may it be for ::guru
> or ::gentoo) but will keep seeing them mixed up together.

You can do it as wel by doing a separate bugzilla search.

Agostino

Re: [gentoo-dev] Continuous integration on GURU

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 617 bytes --]



> On 22 Apr 2021, at 13:59, Ionen Wolkens <sudinave@gmail.com> wrote:
> [snip]

> And, even if you don't see it like that, someday someone may want
> to go on a fixing spree using those trackers (may it be for ::guru
> or ::gentoo) but will keep seeing them mixed up together.

Yes, this is particularly apt for me. Let’s just have another tracker.

It’s not really helpful to have one for all - there’s no use case other
than looking at the large list. Separate ones have a real purpose -
letting us do sprees or quickly mouseover and see blockers we’ve
been checking on.

> --
> ionen


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Continuous integration on GURU

[Agostino Sarubbo]

Thanks for all feedback.

Please discuss all points internally (branch, email, trackers and so on) and 
send me an email with all changes you would like to have.

Agostino

Re: [gentoo-dev] Continuous integration on GURU

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 512 bytes --]



> On 22 Apr 2021, at 14:31, Agostino Sarubbo <ago@gentoo.org> wrote:
> 
> Thanks for all feedback.
> 
> Please discuss all points internally (branch, email, trackers and so on) and
> send me an email with all changes you would like to have.

As an aside, I do appreciate the CI and it’s turned out to be pretty useful!

It would not surprise me if I’m one of your biggest clients, but I’m going to
choose to believe it’s because I’m super duper productive ;)

> Agostino
> 
> 
> 


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Continuous integration on GURU

[Wolfgang E. Sanyer]

On Thu, Apr 22, 2021 at 8:39 AM Agostino Sarubbo <ago@gentoo.org> wrote:
>
> On giovedì 22 aprile 2021 12:02:20 CEST Michał Górny wrote:
> > Well, I suppose scanning the dev branch would be preferable over
> > the master branch.  In reality, they are usually only a few hours apart
> > but it might be useful to know of new breakage in dev before it's merged
> > to master.
> >
> > It would be ideal if you could do a switch when master and dev are
> > in sync, and just copy the state from master.
>
> Hi,
>
> I think that your approach could be generally valid but for this use case I'm
> against because of the following:
>
> 1) The approach is valid in cases like our github PRs and the bot that
> approves the commit. In this case, who moves the commit between branches does
> not know if the scan has been done or not.
>
> 2) I don't see the reason to scan against something that we don't know if will
> be the same in master branch
>
> 3) We are not doing a similar approach for ::gentoo so I don't see why do this
> for GURU since, after all, it is an overlay
>
> 4) Packages in master are supposed to be tested at least from 2 different
> people (who made the commit in dev and who moves the commit to master) so it
> means less bugspam
>

Perhaps another approach could be to add a third branch, "staging".
Any reviewers could move the appropriate ebuilds to "staging" once all
the reviews and discussions are done but before moving to "master"

In fact, probably a github action could be set up to automatically
move to "master" after the CI stuff passes.

Re: [gentoo-dev] Continuous integration on GURU

[Michał Górny]

On Thu, 2021-04-22 at 14:39 +0200, Agostino Sarubbo wrote:
> On giovedì 22 aprile 2021 12:02:20 CEST Michał Górny wrote:
> > Well, I suppose scanning the dev branch would be preferable over
> > the master branch.  In reality, they are usually only a few hours apart
> > but it might be useful to know of new breakage in dev before it's merged
> > to master.
> > 
> > It would be ideal if you could do a switch when master and dev are
> > in sync, and just copy the state from master.
> 
> Hi,
> 
> I think that your approach could be generally valid but for this use case I'm 
> against because of the following:
> 
> 1) The approach is valid in cases like our github PRs and the bot that 
> approves the commit. In this case, who moves the commit between branches does 
> not know if the scan has been done or not.
> 
> 2) I don't see the reason to scan against something that we don't know if will 
> be the same in master branch
> 
> 3) We are not doing a similar approach for ::gentoo so I don't see why do this 
> for GURU since, after all, it is an overlay
> 
> 4) Packages in master are supposed to be tested at least from 2 different 
> people (who made the commit in dev and who moves the commit to master) so it 
> means less bugspam
> 

This is not how GURU works.  The dev->master merges are always fast-
forward, and are only reviewed to prevent malicious actions.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] Continuous integration on GURU

[Andrew Ammerlaan]

On 22/04/2021 14:39, Agostino Sarubbo wrote:
> On giovedì 22 aprile 2021 12:02:20 CEST Michał Górny wrote:
>> Well, I suppose scanning the dev branch would be preferable over
>> the master branch.  In reality, they are usually only a few hours apart
>> but it might be useful to know of new breakage in dev before it's merged
>> to master.
>>
>> It would be ideal if you could do a switch when master and dev are
>> in sync, and just copy the state from master.
> Hi,
>
> I think that your approach could be generally valid but for this use case I'm
> against because of the following:
>
> 1) The approach is valid in cases like our github PRs and the bot that
> approves the commit. In this case, who moves the commit between branches does
> not know if the scan has been done or not.
>
> 2) I don't see the reason to scan against something that we don't know if will
> be the same in master branch
>
> 3) We are not doing a similar approach for ::gentoo so I don't see why do this
> for GURU since, after all, it is an overlay
>
> 4) Packages in master are supposed to be tested at least from 2 different
> people (who made the commit in dev and who moves the commit to master) so it
> means less bugspam

Well, not quite, packages in master have been manually/visually checked 
for obvious (QA) mistakes and have been automatically checked by 
repoman/pkgcheck. In principle the role of the Trusted Contributor is 
first of all to ensure that no malicious commits make it into the master 
branch, as described on the wiki[1]. In practice this means that we also 
do basic QA checks and look at the repoman/pkgcheck results, since we 
are looking at the commit anyway. We do not actually run emerge on the 
ebuild, back in the early days I used to do that, but the amount of 
commits has grown enormously and it is no longer practically possible to 
do this (and here is where the tinderbox comes in handy).

Furthermore, as per [1] we are not supposed to postpone a merge just 
because a commit has QA issues, fails to build or whatever. The only 
situation in which we do not merge a commit, is if it is malicious. QA 
issues or obvious mistakes are reported back to the author, or fixed 
immediately, but they do not disqualify a commit for merging to the 
master branch.

On the matter of running the tinderbox on the master branch or the dev 
branch, I am undecided. It makes sense to do the checks as soon as 
possible and to fix things before they reach the master branch. On the 
other hand there are sometimes very big issues (missing dependencies and 
such) and I am not sure how the tinderbox will behave in these cases. 
Filing bugs for things which the Trusted Contributors can see in 
repoman/pkgcheck (such as missing dependencies) is a waste of cpu power, 
and only serves to increase the number of emails, so if the tinderbox 
would also file reports for these issues I would advise to run the 
tinderbox only on the master branch. (Then there is also the matter to 
consider that in theory the dev branch could contain malicious code, 
whereas the master branch is guaranteed to be free of this. So if the 
tinderbox runs on the dev branch, then in theory a hypothetical 
malicious commit might break the tinderbox.)

Tl;dr Commits are checked, not actually tested.

[1] 
https://wiki.gentoo.org/wiki/Project:GURU/Information_for_Trusted_Contributor

>> One more thing: might be a good idea to consider splitting some
>> of the 'big' trackers (like CFLAGS) between Gentoo and GURU.  I think
>> solving these bugs in GURU has lower priority than in Gentoo.
> I think that trackers like CFLAGS/LDFLAGS are here to track how many packages
> have the problem. I don't see it like (for example) the gcc-porting tracker
> that gives the idea about how much packages need a fix and how much packages
> need to be last-rited
>
>
> Agostino
>
>
>

Re: [gentoo-dev] Continuous integration on GURU

[Theo Anderson]

On Thu, 22 Apr 2021 11:59:21 +0200
Agostino Sarubbo <ago@gentoo.org> wrote:

> Hello,
> 
> I would like to let you know that the CI is now able to work with
> overlays.
> 
> Since Guru is pretty active I took advantage of this situation to
> implement and test the CI

Thank you for implementing this, it certainly helps raise the standard
of guru packages.

> 
> Atm it has been configured to work with master branch.
> I don't know how much is useful scan the dev branch since a first
> revision has not yet been done. If you have opinions about, please
> let me know.

I'm not sure this would be very beneficial. It's often that feedback is
left on packages in dev to get them in a presentable or
issue free state before being pushed to master. Running
the tinderbox against dev might just make a few too many bugs,
perhaps picking up some which are already in the process of being
fixed. The dev branch doesn't lag too far behind master these days
anyhow.

> 
> More info at:
> https://blogs.gentoo.org/ago/2020/07/04/gentoo-tinderbox/
> 
> The bugs will have an internal_ref as guru_ci.
> 
> Agostino
> 

On the subject of many bugs, there are now almost 150 open bugs
assigned to guru@gentoo.org. With this has come quite an email avalanche
for which I think people must have set up mail filters by now. I
imagine this means that bugs will no longer get the same
amount of attention as they used to. To increase the visibility do you
think it would be a good idea to CC or assign the bugs to the specific
package maintainers rather than guru@gentoo.org?

Cheers
Theo

Re: [gentoo-dev] Continuous integration on GURU

[Agostino Sarubbo]

On giovedì 22 aprile 2021 12:22:00 CEST Theo Anderson wrote:
> On the subject of many bugs, there are now almost 150 open bugs
> assigned to guru@gentoo.org. With this has come quite an email avalanche
> for which I think people must have set up mail filters by now. I
> imagine this means that bugs will no longer get the same
> amount of attention as they used to. To increase the visibility do you
> think it would be a good idea to CC or assign the bugs to the specific
> package maintainers rather than guru@gentoo.org?

Hello Theo,

I'm fine to assign directly and/or cc'ing guru@. For now I'm following the 
guidelines but it takes one second to change it.
Please discuss this internally and let me know if I have to change the 
assignee.

Agostino

Re: [gentoo-dev] Continuous integration on GURU

[Theo Anderson]

On Thu, 22 Apr 2021 13:30:10 +0200
Agostino Sarubbo <ago@gentoo.org> wrote:

> On giovedì 22 aprile 2021 12:22:00 CEST Theo Anderson wrote:
> > On the subject of many bugs, there are now almost 150 open bugs
> > assigned to guru@gentoo.org. With this has come quite an email
> > avalanche for which I think people must have set up mail filters by
> > now. I imagine this means that bugs will no longer get the same
> > amount of attention as they used to. To increase the visibility do
> > you think it would be a good idea to CC or assign the bugs to the
> > specific package maintainers rather than guru@gentoo.org?  
> 
> Hello Theo,
> 
> I'm fine to assign directly and/or cc'ing guru@. For now I'm
> following the guidelines but it takes one second to change it.
> Please discuss this internally and let me know if I have to change
> the assignee.
> 
> Agostino
> 

Hi Agostino,

After some discussion in IRC we've agreed to have the maintainer of
the respective packages be assigned to the bug with guru@g.o be CC'd.
Where no maintainer exists for a package guru@g.o remains as the
assignee. If you could make those changes that would be great (I
suppose doing it retroactively would be a bit of a mission and is out
of the question).

Thanks

Re: [gentoo-dev] Continuous integration on GURU

[Ionen Wolkens]

[-- Attachment #1: Type: text/plain, Size: 635 bytes --]

On Thu, Apr 22, 2021 at 01:45:11PM +0000, Theo Anderson wrote:
> After some discussion in IRC we've agreed to have the maintainer of
> the respective packages be assigned to the bug with guru@g.o be CC'd.
> Where no maintainer exists for a package guru@g.o remains as the
> assignee. If you could make those changes that would be great (I
> suppose doing it retroactively would be a bit of a mission and is out
> of the question).

wrt retroactive, changes like this can be automated with pybugz
and simple scripts assuming all the metadata is right and match
bugzilla accounts, it'd just be a lot of mail spam.
-- 
ionen

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] Continuous integration on GURU

[Agostino Sarubbo]

On giovedì 22 aprile 2021 15:45:11 CEST Theo Anderson wrote:
> Hi Agostino,
> 
> After some discussion in IRC we've agreed to have the maintainer of
> the respective packages be assigned to the bug with guru@g.o be CC'd.
> Where no maintainer exists for a package guru@g.o remains as the
> assignee. If you could make those changes that would be great (I
> suppose doing it retroactively would be a bit of a mission and is out
> of the question).
> 
> Thanks


Existing bugs have been re-assigned.

New bugs will be assigned to maintainer and guru is always CC'ed.
Packages with no maintainer will have guru@ as assignee.

If you notice anything wrong please let me know.

Agostino

Re: [gentoo-dev] Continuous integration on GURU

[Michał Górny]

On Thu, 2021-04-22 at 11:59 +0200, Agostino Sarubbo wrote:
> Hello,
> 
> I would like to let you know that the CI is now able to work with overlays.
> 
> Since Guru is pretty active I took advantage of this situation to implement 
> and test the CI
> 
> Atm it has been configured to work with master branch.
> I don't know how much is useful scan the dev branch since a first revision has 
> not yet been done. If you have opinions about, please let me know.
> 
> More info at:
> https://blogs.gentoo.org/ago/2020/07/04/gentoo-tinderbox/
> 
> The bugs will have an internal_ref as guru_ci.
> 

One more thing: might be a good idea to consider splitting some
of the 'big' trackers (like CFLAGS) between Gentoo and GURU.  I think
solving these bugs in GURU has lower priority than in Gentoo.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] Continuous integration on GURU

[Agostino Sarubbo]

On giovedì 22 aprile 2021 12:25:50 CEST Michał Górny wrote:
> On Thu, 2021-04-22 at 11:59 +0200, Agostino Sarubbo wrote:
> > Hello,
> > 
> > I would like to let you know that the CI is now able to work with
> > overlays.
> > 
> > Since Guru is pretty active I took advantage of this situation to
> > implement
> > and test the CI
> > 
> > Atm it has been configured to work with master branch.
> > I don't know how much is useful scan the dev branch since a first revision
> > has not yet been done. If you have opinions about, please let me know.
> > 
> > More info at:
> > https://blogs.gentoo.org/ago/2020/07/04/gentoo-tinderbox/
> > 
> > The bugs will have an internal_ref as guru_ci.
>