[gentoo-dev] Eselect repository feature request

[gentoo-dev] Eselect repository feature request

[TOMAS FABRIZIO ORSI]

[-- Attachment #1: Type: text/plain, Size: 3862 bytes --]

Hello gentoo devs. The other day I made a feature suggestion to the eselect
repository github page. (Here's the link:
https://github.com/projg2/eselect-repository/issues/20).
Michał Górny suggested that I should contact the gentoo-dev mailing list
with my suggestion, so here it is: My suggestion is to make it possible for
eselect repository to manage overlay dependencies.

As it stands, and as far as I'm concerned, there is no "proper" way of
having an ebuild dependency from another overlay. So overlay writers have
to copy ebuilds from other overlays or rewrite them. This implies
synchronization issues (where the "copied" ebuilds get out of sync from the
original ebuild) and time loss (people writing the same ebuild more than
once).

Michał Górny suggested that I should make an edit to the repositories.xml
file in order to tackle the issue.

This is my general idea:
(I am using the github template as an example, but the idea should apply to
all other templates. I got this file from
https://github.com/gentoo/api-gentoo-org/blob/master/files/overlays/repositories.xml
).

  * GITHUB TEMPLATE
    <repo quality="experimental" status="unofficial">
      <name>XXXXX</name>
      <description lang="en">XXXXXX</description>
      <homepage>https://github.com/XXXX/xxxx</homepage>
      <owner type="person">
        <email>XXXXX</email>
        <name>XXXXX</name>
      </owner>
      <source type="git">https://github.com/XXXX/xxxx.git</source>
      <source type="git">git+ssh://git@github.com/XXXX/xxxx.git</source>
      <feed>https://github.com/XXXX/xxxx/commits/master.atom</feed>
      <dependencies>
          <name>overlayName</name>
      </dependencies>
    </repo>

(the only thing I added is the <dependencies> item)
The <dependencies> item would be a list of overlay dependencies (it could
be empty). It would be represented as a list of repository names that the
overlay depends on. Said dependencies could be stated in the master file
(since this would "extend" its behavior)

The algorithm would use a queue ("dependencies" queue) and a set ("overlays
to be added" set)
It starts by creating the "dependency" queue. After that it adds to the
queue the overlay the user specifically asked for.

The dependency algorithm would look like this:
1. If the queue is empty the algorithm stops
2. Checks the first overlay in the queue's dependencies list
3. If the dependencies list is empty, it does nothing
4. If it says that there is a dependency that the user has already enabled
in their system, it does nothing
5. If the dependency is already in the set of "overlays to be added", it
does nothing*
6. If not 2 or 3 or 4, it adds the dependency to the set of "overlays To be
added" and adds said repository to the queue
7. It "pops" the first item on the queue and repeats the algorithm

*There reason why it is a set is because there is no need to check for
circular dependencies in overlays.

Once the algorithm is finished it would print out something like this

foo overlay (the one the user wants to add) depends on bar overlay.
Would you like to add bar as well? Y/N

If yes:
bar overlay depends on biz overlay.
Would you like to add biz as well? Y/N

And so on

------

PROS: Fairly simple algorithm.
CONS: Unwanted overlays with packages the user will never use.

The main con is that even though the overlay is marked as "dependency", not
*all* packages depend on the other overlays: Only a few.
Having said that, this can be mitigated with only minimal manual
intervention by the user.

I would love to get feedback on this idea and what are some of its
shortcomings/things that I missed/haven't considered yet.
Thanks in advance and thank you for making gentoo so great.

Have a good day

- Tomas Fabrizio Orsi a.k.a Lima Limon

[-- Attachment #2: Type: text/html, Size: 4925 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[Florian Schmaus]

[-- Attachment #1.1.1: Type: text/plain, Size: 2441 bytes --]

On 18.06.23 22:39, TOMAS FABRIZIO ORSI wrote:
> Hello gentoo devs. The other day I made a feature suggestion to the 
> eselect repository github page. (Here's the link: 
> https://github.com/projg2/eselect-repository/issues/20 
> <https://github.com/projg2/eselect-repository/issues/20>).
> Michał Górny suggested that I should contact the gentoo-dev mailing list 
> with my suggestion, so here it is: My suggestion is to make it possible 
> for eselect repository to manage overlay dependencies.
> 
> As it stands, and as far as I'm concerned, there is no "proper" way of 
> having an ebuild dependency from another overlay. So overlay writers 
> have to copy ebuilds from other overlays or rewrite them. This implies 
> synchronization issues (where the "copied" ebuilds get out of sync from 
> the original ebuild) and time loss (people writing the same ebuild more 
> than once).
> 
> Michał Górny suggested that I should make an edit to the 
> repositories.xml file in order to tackle the issue.
> 
> This is my general idea:
> (I am using the github template as an example, but the idea should apply 
> to all other templates. I got this file from 
> https://github.com/gentoo/api-gentoo-org/blob/master/files/overlays/repositories.xml <https://github.com/gentoo/api-gentoo-org/blob/master/files/overlays/repositories.xml>).
> 
>    * GITHUB TEMPLATE
>      <repo quality="experimental" status="unofficial">
>        <name>XXXXX</name>
>        <description lang="en">XXXXXX</description>
>        <homepage>https://github.com/XXXX/xxxx 
> <https://github.com/XXXX/xxxx></homepage>
>        <owner type="person">
>          <email>XXXXX</email>
>          <name>XXXXX</name>
>        </owner>
>        <source type="git">https://github.com/XXXX/xxxx.git 
> <https://github.com/XXXX/xxxx.git></source>
>        <source type="git">git+ssh://git@github.com/XXXX/xxxx.git 
> <http://git@github.com/XXXX/xxxx.git></source>
>        <feed>https://github.com/XXXX/xxxx/commits/master.atom 
> <https://github.com/XXXX/xxxx/commits/master.atom></feed>
>        <dependencies>
>            <name>overlayName</name>
>        </dependencies>
>      </repo>

Isn't that duplicating the information of metadata/layout.conf's 
'master' key-value pair [1]?

- Flow

1: 
https://wiki.gentoo.org/wiki/Repository_format/metadata/layout.conf#masters

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 17843 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[TOMAS FABRIZIO ORSI]

[-- Attachment #1: Type: text/plain, Size: 3496 bytes --]

>
> Isn't that duplicating the information of metadata/layout.conf's
> 'master' key-value pair [1]?
>

Yes, I agree that it would be duplicating that information. As a matter of
fact, Michał Górny pointed the same thing out.
However, Michał also added, quote: "What's really lacking here is support
for specifying dependencies via repositories.xml. Once such a thing is
implemented, We can look into having eselect-repository handle that once
it's specified." (Here's the link to the conversation:
https://github.com/projg2/eselect-repository/issues/20#issuecomment-1579098528
)
That's why I added the <dependencies> line in the repositories.xml template.

Having said that, maybe the information present in the masters key could be
used to add the overlay dependencies in the repositories present in the xml
file as of today.

Thank you for your reply,

--
Best regards,
- Tomas Fabrizio Orsi


El mar, 20 jun 2023 a las 10:44, Florian Schmaus (<flow@gentoo.org>)
escribió:

> On 18.06.23 22:39, TOMAS FABRIZIO ORSI wrote:
> > Hello gentoo devs. The other day I made a feature suggestion to the
> > eselect repository github page. (Here's the link:
> > https://github.com/projg2/eselect-repository/issues/20
> > <https://github.com/projg2/eselect-repository/issues/20>).
> > Michał Górny suggested that I should contact the gentoo-dev mailing list
> > with my suggestion, so here it is: My suggestion is to make it possible
> > for eselect repository to manage overlay dependencies.
> >
> > As it stands, and as far as I'm concerned, there is no "proper" way of
> > having an ebuild dependency from another overlay. So overlay writers
> > have to copy ebuilds from other overlays or rewrite them. This implies
> > synchronization issues (where the "copied" ebuilds get out of sync from
> > the original ebuild) and time loss (people writing the same ebuild more
> > than once).
> >
> > Michał Górny suggested that I should make an edit to the
> > repositories.xml file in order to tackle the issue.
> >
> > This is my general idea:
> > (I am using the github template as an example, but the idea should apply
> > to all other templates. I got this file from
> >
> https://github.com/gentoo/api-gentoo-org/blob/master/files/overlays/repositories.xml
> <
> https://github.com/gentoo/api-gentoo-org/blob/master/files/overlays/repositories.xml
> >).
> >
> >    * GITHUB TEMPLATE
> >      <repo quality="experimental" status="unofficial">
> >        <name>XXXXX</name>
> >        <description lang="en">XXXXXX</description>
> >        <homepage>https://github.com/XXXX/xxxx
> > <https://github.com/XXXX/xxxx></homepage>
> >        <owner type="person">
> >          <email>XXXXX</email>
> >          <name>XXXXX</name>
> >        </owner>
> >        <source type="git">https://github.com/XXXX/xxxx.git
> > <https://github.com/XXXX/xxxx.git></source>
> >        <source type="git">git+ssh://git@github.com/XXXX/xxxx.git
> > <http://git@github.com/XXXX/xxxx.git></source>
> >        <feed>https://github.com/XXXX/xxxx/commits/master.atom
> > <https://github.com/XXXX/xxxx/commits/master.atom></feed>
> >        <dependencies>
> >            <name>overlayName</name>
> >        </dependencies>
> >      </repo>
>
> Isn't that duplicating the information of metadata/layout.conf's
> 'master' key-value pair [1]?
>
> - Flow
>
> 1:
> https://wiki.gentoo.org/wiki/Repository_format/metadata/layout.conf#masters
>

[-- Attachment #2: Type: text/html, Size: 6262 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[Florian Schmaus]

[-- Attachment #1.1.1: Type: text/plain, Size: 690 bytes --]

On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:
>     Isn't that duplicating the information of metadata/layout.conf's
>     'master' key-value pair [1]?
> 
> 
> Yes, I agree that it would be duplicating that information. As a matter 
> of fact, Michał Górny pointed the same thing out.
> However, Michał also added, quote: "What's really lacking here is 
> support for specifying dependencies via |repositories.xml|

Do we need to duplicate the information in repositories.xml, with all 
the drawbacks of duplication?

Can't eselect repository add the new repository, then read the 'masters' 
value from layout.conf, and add the missing repositories recursively?

- Flow

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 17843 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[Mike Gilbert]

On Tue, Jun 20, 2023 at 1:08 PM Florian Schmaus <flow@gentoo.org> wrote:
>
> On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:
> >     Isn't that duplicating the information of metadata/layout.conf's
> >     'master' key-value pair [1]?
> >
> >
> > Yes, I agree that it would be duplicating that information. As a matter
> > of fact, Michał Górny pointed the same thing out.
> > However, Michał also added, quote: "What's really lacking here is
> > support for specifying dependencies via |repositories.xml|
>
> Do we need to duplicate the information in repositories.xml, with all
> the drawbacks of duplication?
>
> Can't eselect repository add the new repository, then read the 'masters'
> value from layout.conf, and add the missing repositories recursively?

That would be a significant change in behavior for eselect repository.
Currently, it does not actually sync any repos; it just manages the
config in /etc/portage/repos.conf.

Re: [gentoo-dev] Eselect repository feature request

[Andrew Ammerlaan]

On 20/06/2023 19:26, Mike Gilbert wrote:
> On Tue, Jun 20, 2023 at 1:08 PM Florian Schmaus <flow@gentoo.org> wrote:
>>
>> On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:
>>>      Isn't that duplicating the information of metadata/layout.conf's
>>>      'master' key-value pair [1]?
>>>
>>>
>>> Yes, I agree that it would be duplicating that information. As a matter
>>> of fact, Michał Górny pointed the same thing out.
>>> However, Michał also added, quote: "What's really lacking here is
>>> support for specifying dependencies via |repositories.xml|
>>
>> Do we need to duplicate the information in repositories.xml, with all
>> the drawbacks of duplication?
>>
>> Can't eselect repository add the new repository, then read the 'masters'
>> value from layout.conf, and add the missing repositories recursively?
> 
> That would be a significant change in behavior for eselect repository.
> Currently, it does not actually sync any repos; it just manages the
> config in /etc/portage/repos.conf.
> 
Or maybe we just output a warning after syncing if a repository listed 
in masters= is not enabled? Less automated, but simpler. I don't think 
this situation happens very often so this is probably good enough IMO.

Best regards,
Andrew

Re: [gentoo-dev] Eselect repository feature request

[TOMAS FABRIZIO ORSI]

[-- Attachment #1: Type: text/plain, Size: 1871 bytes --]

A warning could be a great way of making the user aware of this situation.
Having said that, if eselect repository is able to check and warn the user
of a not synced overlay(ies) dependency, then the hard bit is done (Right?).
Being able to "sync it for them" should only be a tiny extra step. That
step being: "Hey, there's a dependency that you are missing. Would you like
to add it? (Yes/No)"

Best regards,
- Tomas Fabrizio Orsi


El mar, 20 jun 2023 a las 15:07, Andrew Ammerlaan (<
andrewammerlaan@gentoo.org>) escribió:

> On 20/06/2023 19:26, Mike Gilbert wrote:
> > On Tue, Jun 20, 2023 at 1:08 PM Florian Schmaus <flow@gentoo.org> wrote:
> >>
> >> On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:
> >>>      Isn't that duplicating the information of metadata/layout.conf's
> >>>      'master' key-value pair [1]?
> >>>
> >>>
> >>> Yes, I agree that it would be duplicating that information. As a matter
> >>> of fact, Michał Górny pointed the same thing out.
> >>> However, Michał also added, quote: "What's really lacking here is
> >>> support for specifying dependencies via |repositories.xml|
> >>
> >> Do we need to duplicate the information in repositories.xml, with all
> >> the drawbacks of duplication?
> >>
> >> Can't eselect repository add the new repository, then read the 'masters'
> >> value from layout.conf, and add the missing repositories recursively?
> >
> > That would be a significant change in behavior for eselect repository.
> > Currently, it does not actually sync any repos; it just manages the
> > config in /etc/portage/repos.conf.
> >
> Or maybe we just output a warning after syncing if a repository listed
> in masters= is not enabled? Less automated, but simpler. I don't think
> this situation happens very often so this is probably good enough IMO.
>
> Best regards,
> Andrew
>
>

[-- Attachment #2: Type: text/html, Size: 2597 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[Andrew Ammerlaan]

On 21/06/2023 04:17, TOMAS FABRIZIO ORSI wrote:
> A warning could be a great way of making the user aware of this situation.
> Having said that, if eselect repository is able to check and warn the 
> user of a not synced overlay(ies) dependency, then the hard bit is done 

What I meant is that emerge --sync/eix --sync does this check instead of 
eselect repository. To me this makes sense since we can only do this 
check *after* syncing. Plus the user can also make changes to repos.conf 
without using eselect repository, having the check done post-syncing 
covers these cases as well.

You're probably right that once we have some message that says "Warning: 
{repo1} is missing, it is required by {repo2}" it should be trivial to 
call 'eselect repository enable {repo1}'. This however would require 
that profiles/repo_name is always equal to the name entry in 
repositories.xml, I'm not sure if this is currently always the case.

Best regards,
Andrew

Re: [gentoo-dev] Eselect repository feature request

[TOMAS FABRIZIO ORSI]

[-- Attachment #1: Type: text/plain, Size: 2401 bytes --]

>
> What I meant is that emerge --sync/eix --sync does this check instead of
> eselect repository. To me this makes sense since we can only do this
> check *after* syncing.

That is a great point; I had not considered it.
So, you are saying to have emerge --sync/eix --sync check the masters
key and then warn the user of not synced up overlays, correct?

This however would require
> that profiles/repo_name is always equal to the name entry in
> repositories.xml, I'm not sure if this is currently always the case.

I had not considered this possibility either.
If that's the case, then I believe there are to possible behaviors:
If the names do match, then emerge --sync/eix --sync could both:
     1. issue a warning of the missing overlay dependency
     2. run the command to add said overlay (with confirmation).
On the other hand, if the names do not match, a missing dependency warning
could be issued, skipping the second step.
This sounds to me to be fairly reasonable behaviour.

In the cases where the names do not match; what could be the best
solution to "fix" this? Change repositories.xml on a case by case basis?
Ask overlay owners to change their profiles/repo_name and
profiles/masters key? Neither?

Best regards,
- Tomas Fabrizio Orsi

El mié, 21 jun 2023 a las 4:15, Andrew Ammerlaan (<
andrewammerlaan@gentoo.org>) escribió:

> On 21/06/2023 04:17, TOMAS FABRIZIO ORSI wrote:
> > A warning could be a great way of making the user aware of this
> situation.
> > Having said that, if eselect repository is able to check and warn the
> > user of a not synced overlay(ies) dependency, then the hard bit is done
>
> What I meant is that emerge --sync/eix --sync does this check instead of
> eselect repository. To me this makes sense since we can only do this
> check *after* syncing. Plus the user can also make changes to repos.conf
> without using eselect repository, having the check done post-syncing
> covers these cases as well.
>
> You're probably right that once we have some message that says "Warning:
> {repo1} is missing, it is required by {repo2}" it should be trivial to
> call 'eselect repository enable {repo1}'. This however would require
> that profiles/repo_name is always equal to the name entry in
> repositories.xml, I'm not sure if this is currently always the case.
>
> Best regards,
> Andrew
>
>
>

[-- Attachment #2: Type: text/html, Size: 3281 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[Andrew Ammerlaan]

On 21/06/2023 15:40, TOMAS FABRIZIO ORSI wrote:
>     What I meant is that emerge --sync/eix --sync does this check
>     instead of
>     eselect repository. To me this makes sense since we can only do this
>     check *after* syncing. 
> 
> That is a great point; I had not considered it.
> So, you are saying to have emerge --sync/eix --sync check the masters
> key and then warn the user of not synced up overlays, correct?

Yes

>     This however would require
>     that profiles/repo_name is always equal to the name entry in
>     repositories.xml, I'm not sure if this is currently always the case.
> 
> I had not considered this possibility either.
> If that's the case, then I believe there are to possible behaviors:
> If the names do match, then emerge --sync/eix --sync could both:
>       1. issue a warning of the missing overlay dependency
>       2. run the command to add said overlay (with confirmation).
> On the other hand, if the names do not match, a missing dependency warning
> could be issued, skipping the second step.
> This sounds to me to be fairly reasonable behaviour.
> 
> In the cases where the names do not match; what could be the best
> solution to "fix" this? Change repositories.xml on a case by case basis?
> Ask overlay owners to change their profiles/repo_name and
> profiles/masters key? Neither?

I just checked the internal overlays team guide[1] and it does 
explicitly say that repo_name should match the repositories.xml <name>. 
So probably there is no problem. That being said it is possible for the 
owner to change repo_name and (accidentally) not update 
repositories.xml. These cases should be fixed but exactly how would have 
to be determined on a case by case basis.

In any case, this is just something to keep in mind when writing this 
check. It is not fully guaranteed that eselect repository can find the 
repository that is requested in some master= entry.

Best regards,
Andrew

[1] 
https://wiki.gentoo.org/wiki/Project:Overlays/Internal_Overlays_team_guide

Re: [gentoo-dev] Eselect repository feature request

[TOMAS FABRIZIO ORSI]

[-- Attachment #1: Type: text/plain, Size: 3212 bytes --]

>
> I just checked the internal overlays team guide[1] and it does
> explicitly say that repo_name should match the repositories.xml <name>.

Oh, great news then. Thanks for checking. Also thank you overlays team!

That being said it is possible for the
> owner to change repo_name and (accidentally) not update
> repositories.xml. These cases should be fixed but exactly how would have
> to be determined on a case by case basis.
>
That is true, however I think those are "edge" cases. Furthermore, how
often do
people change their overlay's name? I legitimately do not know.

In any case, this is just something to keep in mind when writing this
> check. It is not fully guaranteed that eselect repository can find the
> repository that is requested in some master= entry.
>
Great point.
Btw, in your opinion, do you think having this emerge --sync + eselect
repository feature is doable?

Best regards,
- Tomas Fabrizio Orsi


El mié, 21 jun 2023 a las 10:58, Andrew Ammerlaan (<
andrewammerlaan@gentoo.org>) escribió:

> On 21/06/2023 15:40, TOMAS FABRIZIO ORSI wrote:
> >     What I meant is that emerge --sync/eix --sync does this check
> >     instead of
> >     eselect repository. To me this makes sense since we can only do this
> >     check *after* syncing.
> >
> > That is a great point; I had not considered it.
> > So, you are saying to have emerge --sync/eix --sync check the masters
> > key and then warn the user of not synced up overlays, correct?
>
> Yes
>
> >     This however would require
> >     that profiles/repo_name is always equal to the name entry in
> >     repositories.xml, I'm not sure if this is currently always the case.
> >
> > I had not considered this possibility either.
> > If that's the case, then I believe there are to possible behaviors:
> > If the names do match, then emerge --sync/eix --sync could both:
> >       1. issue a warning of the missing overlay dependency
> >       2. run the command to add said overlay (with confirmation).
> > On the other hand, if the names do not match, a missing dependency
> warning
> > could be issued, skipping the second step.
> > This sounds to me to be fairly reasonable behaviour.
> >
> > In the cases where the names do not match; what could be the best
> > solution to "fix" this? Change repositories.xml on a case by case basis?
> > Ask overlay owners to change their profiles/repo_name and
> > profiles/masters key? Neither?
>
> I just checked the internal overlays team guide[1] and it does
> explicitly say that repo_name should match the repositories.xml <name>.
> So probably there is no problem. That being said it is possible for the
> owner to change repo_name and (accidentally) not update
> repositories.xml. These cases should be fixed but exactly how would have
> to be determined on a case by case basis.
>
> In any case, this is just something to keep in mind when writing this
> check. It is not fully guaranteed that eselect repository can find the
> repository that is requested in some master= entry.
>
> Best regards,
> Andrew
>
> [1]
> https://wiki.gentoo.org/wiki/Project:Overlays/Internal_Overlays_team_guide
>
>
>

[-- Attachment #2: Type: text/html, Size: 4467 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[Andrew Ammerlaan]

On 21/06/2023 16:12, TOMAS FABRIZIO ORSI wrote:
>     In any case, this is just something to keep in mind when writing this
>     check. It is not fully guaranteed that eselect repository can find the
>     repository that is requested in some master= entry.
> 
> Great point.
> Btw, in your opinion, do you think having this emerge --sync + eselect 
> repository feature is doable?

Sure, I think it could work.

Best regards,
Andrew

Re: [gentoo-dev] Eselect repository feature request

[TOMAS FABRIZIO ORSI]

[-- Attachment #1: Type: text/plain, Size: 947 bytes --]

>
> Sure, I think it could work.

Great to hear.
In that case I could try to give it a try and make a pull request to the
emerge --sync with a basic idea.
Any tips?
If anyone else is interested/free (I'm guessing the latter is unlikely
haha.) I'd love to have a helping hand!

Thank you for your kind responses,

Best regards,
- Tomas Fabrizio Orsi


El mié, 21 jun 2023 a las 11:30, Andrew Ammerlaan (<
andrewammerlaan@gentoo.org>) escribió:

> On 21/06/2023 16:12, TOMAS FABRIZIO ORSI wrote:
> >     In any case, this is just something to keep in mind when writing this
> >     check. It is not fully guaranteed that eselect repository can find
> the
> >     repository that is requested in some master= entry.
> >
> > Great point.
> > Btw, in your opinion, do you think having this emerge --sync + eselect
> > repository feature is doable?
>
> Sure, I think it could work.
>
> Best regards,
> Andrew
>
>
>

[-- Attachment #2: Type: text/html, Size: 1588 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[Mike Gilbert]

On Wed, Jun 21, 2023 at 10:43 AM TOMAS FABRIZIO ORSI <torsi@fi.uba.ar> wrote:
>>
>> Sure, I think it could work.
>
> Great to hear.
> In that case I could try to give it a try and make a pull request to the emerge --sync with a basic idea.
> Any tips?

So emerge already emits a warning message when a repo is missing:

https://gitweb.gentoo.org/proj/portage.git/tree/lib/portage/repository/config.py?h=portage-3.0.48.1#n1070

That looks something like this:

$ sudo emerge --sync
Unavailable repository 'foo' referenced by masters entry in
'/var/db/repos/gentoo/metadata/layout.conf'

Re: [gentoo-dev] Eselect repository feature request

[TOMAS FABRIZIO ORSI]

[-- Attachment #1: Type: text/plain, Size: 1116 bytes --]

>
> $ sudo emerge --sync
> Unavailable repository 'foo' referenced by masters entry in
> '/var/db/repos/gentoo/metadata/layout.conf'

Oh! I was not aware of this feature. I apologize.
So, in theory, could '%s' be used to automatically enable the missing
repository
(if available in repositories.xml)?

Thank you for you insight,

Best regards,
- Tomas Fabrizio Orsi

El mié, 21 jun 2023 a las 12:07, Mike Gilbert (<floppym@gentoo.org>)
escribió:

> On Wed, Jun 21, 2023 at 10:43 AM TOMAS FABRIZIO ORSI <torsi@fi.uba.ar>
> wrote:
> >>
> >> Sure, I think it could work.
> >
> > Great to hear.
> > In that case I could try to give it a try and make a pull request to the
> emerge --sync with a basic idea.
> > Any tips?
>
> So emerge already emits a warning message when a repo is missing:
>
>
> https://gitweb.gentoo.org/proj/portage.git/tree/lib/portage/repository/config.py?h=portage-3.0.48.1#n1070
>
> That looks something like this:
>
> $ sudo emerge --sync
> Unavailable repository 'foo' referenced by masters entry in
> '/var/db/repos/gentoo/metadata/layout.conf'
>
>

[-- Attachment #2: Type: text/html, Size: 1994 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[Florian Schmaus]

On 20.06.23 19:26, Mike Gilbert wrote:
> On Tue, Jun 20, 2023 at 1:08 PM Florian Schmaus <flow@gentoo.org> wrote:
>>
>> On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:
>>>      Isn't that duplicating the information of metadata/layout.conf's
>>>      'master' key-value pair [1]?
>>>
>>>
>>> Yes, I agree that it would be duplicating that information. As a matter
>>> of fact, Michał Górny pointed the same thing out.
>>> However, Michał also added, quote: "What's really lacking here is
>>> support for specifying dependencies via |repositories.xml|
>>
>> Do we need to duplicate the information in repositories.xml, with all
>> the drawbacks of duplication?
>>
>> Can't eselect repository add the new repository, then read the 'masters'
>> value from layout.conf, and add the missing repositories recursively?
> 
> That would be a significant change in behavior for eselect repository.

Right, but it seems to be a desirable behaviour. Cases where the user 
wants to add a repo but not immediately sync it are probably rare.

Furthermore, it would avoid duplicating the information, which avoids 
the typical drawbacks of duplication (e.g., the two sets getting out of 
sync).

I've looked at the eselect-repository code, and it seems not hard to 
change the behaviour of "eselect repository add" to add and sync a 
repository and then, recursively, add and sync further required 
repositories.

I may give it a shot, but ideally I'd know if it has a chance to be 
accepted upstream first. Or maybe there is a good reason why 
eselect-repository behaves as it currently does that I am missing?

- Flow

Re: [gentoo-dev] Eselect repository feature request

[Mike Gilbert]

On Wed, Jun 21, 2023 at 11:41 AM Florian Schmaus <flow@gentoo.org> wrote:
>
> On 20.06.23 19:26, Mike Gilbert wrote:
> > On Tue, Jun 20, 2023 at 1:08 PM Florian Schmaus <flow@gentoo.org> wrote:
> >>
> >> On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:
> >>>      Isn't that duplicating the information of metadata/layout.conf's
> >>>      'master' key-value pair [1]?
> >>>
> >>>
> >>> Yes, I agree that it would be duplicating that information. As a matter
> >>> of fact, Michał Górny pointed the same thing out.
> >>> However, Michał also added, quote: "What's really lacking here is
> >>> support for specifying dependencies via |repositories.xml|
> >>
> >> Do we need to duplicate the information in repositories.xml, with all
> >> the drawbacks of duplication?
> >>
> >> Can't eselect repository add the new repository, then read the 'masters'
> >> value from layout.conf, and add the missing repositories recursively?
> >
> > That would be a significant change in behavior for eselect repository.
>
> Right, but it seems to be a desirable behaviour. Cases where the user
> wants to add a repo but not immediately sync it are probably rare.
>
> Furthermore, it would avoid duplicating the information, which avoids
> the typical drawbacks of duplication (e.g., the two sets getting out of
> sync).
>
> I've looked at the eselect-repository code, and it seems not hard to
> change the behaviour of "eselect repository add" to add and sync a
> repository and then, recursively, add and sync further required
> repositories.
>
> I may give it a shot, but ideally I'd know if it has a chance to be
> accepted upstream first. Or maybe there is a good reason why
> eselect-repository behaves as it currently does that I am missing?

I can't speak for "upstream", but here are my concerns:

1. As a developer, I might just want to create the repos.conf config
snippet and sync the repo manually.
2. As a user, I might have any arbitrary reason for not wanting to
sync immediately.
3. eselect-repository does not currently depend on any particular
package manager. It writes config files intended for Portage, but it
does not actually invoke any Portage commands. That feels like a
significant distinction to me.
4. If you start invoking Portage commands, you then have to deal with
the possibility of people using alternate package managers. pkgcore
can also utilize Portage's repos.conf, and the user might prefer to
use pmaint instead of emaint or emerge --sync.

Re: [gentoo-dev] Eselect repository feature request

[TOMAS FABRIZIO ORSI]

[-- Attachment #1: Type: text/plain, Size: 4279 bytes --]

>
> 1. As a developer, I might just want to create the repos.conf config
> snippet and sync the repo manually.
> 2. As a user, I might have any arbitrary reason for not wanting to
> sync immediately.

As a matter of fact, Michał Górny raised the same questions:
"A bit of a problem is how to design UI in eselect-repository. I'm not 100%
sure
that having it automatically install dependent repositories without
confirmation is a good idea"
(
https://github.com/projg2/eselect-repository/issues/20#issuecomment-1579288719)

Ideally, confirmation would be asked before proceeding to sync the other
overlays.
I agree that doing something without the user's knowledge is not ideal.

3. eselect-repository does not currently depend on any particular
> package manager. It writes config files intended for Portage, but it
> does not actually invoke any Portage commands. That feels like a
> significant distinction to me.
> 4. If you start invoking Portage commands, you then have to deal with
> the possibility of people using alternate package managers. pkgcore
> can also utilize Portage's repos.conf, and the user might prefer to
> use pmaint instead of emaint or emerge --sync.


I had not considered that possibility either. In that case, could not the
overlay
dependency resolution be handled as a module?
Said module could be a common interface for different package managers.
Then, the execution of said module would be handled on a per package
manager/sync program basis?

Best regards,
- Tomas Fabrizio Orsi

El mié, 21 jun 2023 a las 12:57, Mike Gilbert (<floppym@gentoo.org>)
escribió:

> On Wed, Jun 21, 2023 at 11:41 AM Florian Schmaus <flow@gentoo.org> wrote:
> >
> > On 20.06.23 19:26, Mike Gilbert wrote:
> > > On Tue, Jun 20, 2023 at 1:08 PM Florian Schmaus <flow@gentoo.org>
> wrote:
> > >>
> > >> On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:
> > >>>      Isn't that duplicating the information of metadata/layout.conf's
> > >>>      'master' key-value pair [1]?
> > >>>
> > >>>
> > >>> Yes, I agree that it would be duplicating that information. As a
> matter
> > >>> of fact, Michał Górny pointed the same thing out.
> > >>> However, Michał also added, quote: "What's really lacking here is
> > >>> support for specifying dependencies via |repositories.xml|
> > >>
> > >> Do we need to duplicate the information in repositories.xml, with all
> > >> the drawbacks of duplication?
> > >>
> > >> Can't eselect repository add the new repository, then read the
> 'masters'
> > >> value from layout.conf, and add the missing repositories recursively?
> > >
> > > That would be a significant change in behavior for eselect repository.
> >
> > Right, but it seems to be a desirable behaviour. Cases where the user
> > wants to add a repo but not immediately sync it are probably rare.
> >
> > Furthermore, it would avoid duplicating the information, which avoids
> > the typical drawbacks of duplication (e.g., the two sets getting out of
> > sync).
> >
> > I've looked at the eselect-repository code, and it seems not hard to
> > change the behaviour of "eselect repository add" to add and sync a
> > repository and then, recursively, add and sync further required
> > repositories.
> >
> > I may give it a shot, but ideally I'd know if it has a chance to be
> > accepted upstream first. Or maybe there is a good reason why
> > eselect-repository behaves as it currently does that I am missing?
>
> I can't speak for "upstream", but here are my concerns:
>
> 1. As a developer, I might just want to create the repos.conf config
> snippet and sync the repo manually.
> 2. As a user, I might have any arbitrary reason for not wanting to
> sync immediately.
> 3. eselect-repository does not currently depend on any particular
> package manager. It writes config files intended for Portage, but it
> does not actually invoke any Portage commands. That feels like a
> significant distinction to me.
> 4. If you start invoking Portage commands, you then have to deal with
> the possibility of people using alternate package managers. pkgcore
> can also utilize Portage's repos.conf, and the user might prefer to
> use pmaint instead of emaint or emerge --sync.
>
>

[-- Attachment #2: Type: text/html, Size: 5880 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[Mike Gilbert]

On Wed, Jun 21, 2023 at 12:47 PM TOMAS FABRIZIO ORSI <torsi@fi.uba.ar> wrote:
> I had not considered that possibility either. In that case, could not the overlay
> dependency resolution be handled as a module?
> Said module could be a common interface for different package managers.
> Then, the execution of said module would be handled on a per package manager/sync program basis?

I'm not quite certain what you mean by "module" here, but that sounds
like unnecessary extra abstraction.

I think flow's idea to make the sync command configurable somehow
would be sufficient, assuming there is demand for it.

Re: [gentoo-dev] Eselect repository feature request

[TOMAS FABRIZIO ORSI]

[-- Attachment #1: Type: text/plain, Size: 1463 bytes --]

>
> I think flow's idea to make the sync command configurable somehow
> would be sufficient, assuming there is demand for it.

I agree.
I'm glad Flow has their ideas straight.

By the way Flow, could you share the links of the repo/PR so that we can
follow the development along? I would love to see it! ^_^

I'm not quite certain what you mean by "module" here, but that sounds
> like unnecessary extra abstraction.
>

I think my line of thought was a bit overkill.
Not to mention that I am not that well versed (if at all)
with portage's implementation detail.

PS: I forgot to mention this in the original email, but I did file a
bug with this feature request. https://bugs.gentoo.org/907959


El mié, 21 jun 2023 a las 14:46, Mike Gilbert (<floppym@gentoo.org>)
escribió:

> On Wed, Jun 21, 2023 at 12:47 PM TOMAS FABRIZIO ORSI <torsi@fi.uba.ar>
> wrote:
> > I had not considered that possibility either. In that case, could not
> the overlay
> > dependency resolution be handled as a module?
> > Said module could be a common interface for different package managers.
> > Then, the execution of said module would be handled on a per package
> manager/sync program basis?
>
> I'm not quite certain what you mean by "module" here, but that sounds
> like unnecessary extra abstraction.
>
> I think flow's idea to make the sync command configurable somehow
> would be sufficient, assuming there is demand for it.
>
>

[-- Attachment #2: Type: text/html, Size: 2606 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[Florian Schmaus]

[-- Attachment #1.1.1: Type: text/plain, Size: 695 bytes --]

On 21/06/2023 19.59, TOMAS FABRIZIO ORSI wrote:
>     I think flow's idea to make the sync command configurable somehow
>     would be sufficient, assuming there is demand for it.
> 
> I agree.
> I'm glad Flow has their ideas straight.
> 
> By the way Flow, could you share the links of the repo/PR so that we can
> follow the development along? I would love to see it! ^_^

https://github.com/projg2/eselect-repository/pull/23

Unlike I previously assumed, this currently shells out twice to portage:

1. to query the list of currently available repositories, via "portageq 
get_repos /"
2. to sync a repository, via "emaint sync -r ${repo}"

Feedback welcomed. :)

- Flow

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 17273 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[TOMAS FABRIZIO ORSI]

[-- Attachment #1: Type: text/plain, Size: 1113 bytes --]

Hello,

Long time no see. Update on the feature request: Michał Górny rejected
Florian's the pull request, raising concerns regarding the use of gentoo
specific tooling;
stating that eselect-repository should work (as I understood it) as a
standalone program.

Originally, Górny suggested updating the repositories.xml file by adding
the masters entry.

That's why I did some data processing of the overlays.
(It turns out only a small set of overlays have additional dependencies
besides gentoo).
With this data now processed, adding the masters entry to the
repositories.xml
should be way easier.
Here's the repo: https://github.com/lima-limon-inc/OverlayStats
(it also contains information regarding package amounts by overlay,
which I found it quite interesting)
And here's a google sheet with the data loaded in:
https://docs.google.com/spreadsheets/d/1Spti8KCFQbi0kszvgBnxMtAlUC3RU1HZU8MZHlYP5xQ/edit?usp=sharing

Having said that, there's still the issue of data duplication that Florian
raised.
That I do not know how could be tackled.

Best regards,
- Tomas Fabrizio Orsi

[-- Attachment #2: Type: text/html, Size: 1608 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[Anna]

[-- Attachment #1: Type: text/plain, Size: 347 bytes --]

> I think flow's idea to make the sync command configurable somehow
> would be sufficient, assuming there is demand for it.

eselect-repository already has a config file at
/etc/eselect/repository.conf, we could just have a SYNC_COMMAND config
option in there

having eselect-repository auto-sync (and recursively so) sounds like a
nice thing imo

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[Florian Schmaus]

[-- Attachment #1.1.1: Type: text/plain, Size: 3090 bytes --]

On 21/06/2023 17.56, Mike Gilbert wrote:
> On Wed, Jun 21, 2023 at 11:41 AM Florian Schmaus <flow@gentoo.org> wrote:
>>
>> On 20.06.23 19:26, Mike Gilbert wrote:
>>> On Tue, Jun 20, 2023 at 1:08 PM Florian Schmaus <flow@gentoo.org> wrote:
>>>>
>>>> On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:
>>>>>       Isn't that duplicating the information of metadata/layout.conf's
>>>>>       'master' key-value pair [1]?
>>>>>
>>>>>
>>>>> Yes, I agree that it would be duplicating that information. As a matter
>>>>> of fact, Michał Górny pointed the same thing out.
>>>>> However, Michał also added, quote: "What's really lacking here is
>>>>> support for specifying dependencies via |repositories.xml|
>>>>
>>>> Do we need to duplicate the information in repositories.xml, with all
>>>> the drawbacks of duplication?
>>>>
>>>> Can't eselect repository add the new repository, then read the 'masters'
>>>> value from layout.conf, and add the missing repositories recursively?
>>>
>>> That would be a significant change in behavior for eselect repository.
>>
>> Right, but it seems to be a desirable behaviour. Cases where the user
>> wants to add a repo but not immediately sync it are probably rare.
>>
>> Furthermore, it would avoid duplicating the information, which avoids
>> the typical drawbacks of duplication (e.g., the two sets getting out of
>> sync).
>>
>> I've looked at the eselect-repository code, and it seems not hard to
>> change the behaviour of "eselect repository add" to add and sync a
>> repository and then, recursively, add and sync further required
>> repositories.
>>
>> I may give it a shot, but ideally I'd know if it has a chance to be
>> accepted upstream first. Or maybe there is a good reason why
>> eselect-repository behaves as it currently does that I am missing?
> 
> I can't speak for "upstream", but here are my concerns:
> 
> 1. As a developer, I might just want to create the repos.conf config
> snippet and sync the repo manually.
> 2. As a user, I might have any arbitrary reason for not wanting to
> sync immediately.

Would an opt-out switch be enough to alleviate those concerns of you?


> 3. eselect-repository does not currently depend on any particular
> package manager. It writes config files intended for Portage, but it
> does not actually invoke any Portage commands. That feels like a
> significant distinction to me.
> 4. If you start invoking Portage commands, you then have to deal with
> the possibility of people using alternate package managers. pkgcore
> can also utilize Portage's repos.conf, and the user might prefer to
> use pmaint instead of emaint or emerge --sync.

Those two points seem to be based on the same fundamental concern.

The only portage specific code would be the call to "emaint sync -r 
$repo" (remember that "emerge --sync" is just a wrapper for "emaint sync 
--auto"). I think it would be easy to add later 1. add support for 
different package managers (if the need arises), and 2. make the "sync 
command" user configurable.

- Flow

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 17273 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[Mike Gilbert]

On Wed, Jun 21, 2023 at 12:49 PM Florian Schmaus <flow@gentoo.org> wrote:
>
> On 21/06/2023 17.56, Mike Gilbert wrote:
> > On Wed, Jun 21, 2023 at 11:41 AM Florian Schmaus <flow@gentoo.org> wrote:
> >>
> >> On 20.06.23 19:26, Mike Gilbert wrote:
> >>> On Tue, Jun 20, 2023 at 1:08 PM Florian Schmaus <flow@gentoo.org> wrote:
> >>>>
> >>>> On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:
> >>>>>       Isn't that duplicating the information of metadata/layout.conf's
> >>>>>       'master' key-value pair [1]?
> >>>>>
> >>>>>
> >>>>> Yes, I agree that it would be duplicating that information. As a matter
> >>>>> of fact, Michał Górny pointed the same thing out.
> >>>>> However, Michał also added, quote: "What's really lacking here is
> >>>>> support for specifying dependencies via |repositories.xml|
> >>>>
> >>>> Do we need to duplicate the information in repositories.xml, with all
> >>>> the drawbacks of duplication?
> >>>>
> >>>> Can't eselect repository add the new repository, then read the 'masters'
> >>>> value from layout.conf, and add the missing repositories recursively?
> >>>
> >>> That would be a significant change in behavior for eselect repository.
> >>
> >> Right, but it seems to be a desirable behaviour. Cases where the user
> >> wants to add a repo but not immediately sync it are probably rare.
> >>
> >> Furthermore, it would avoid duplicating the information, which avoids
> >> the typical drawbacks of duplication (e.g., the two sets getting out of
> >> sync).
> >>
> >> I've looked at the eselect-repository code, and it seems not hard to
> >> change the behaviour of "eselect repository add" to add and sync a
> >> repository and then, recursively, add and sync further required
> >> repositories.
> >>
> >> I may give it a shot, but ideally I'd know if it has a chance to be
> >> accepted upstream first. Or maybe there is a good reason why
> >> eselect-repository behaves as it currently does that I am missing?
> >
> > I can't speak for "upstream", but here are my concerns:
> >
> > 1. As a developer, I might just want to create the repos.conf config
> > snippet and sync the repo manually.
> > 2. As a user, I might have any arbitrary reason for not wanting to
> > sync immediately.
>
> Would an opt-out switch be enough to alleviate those concerns of you?

Yes.

>
> > 3. eselect-repository does not currently depend on any particular
> > package manager. It writes config files intended for Portage, but it
> > does not actually invoke any Portage commands. That feels like a
> > significant distinction to me.
> > 4. If you start invoking Portage commands, you then have to deal with
> > the possibility of people using alternate package managers. pkgcore
> > can also utilize Portage's repos.conf, and the user might prefer to
> > use pmaint instead of emaint or emerge --sync.
>
> Those two points seem to be based on the same fundamental concern.
>
> The only portage specific code would be the call to "emaint sync -r
> $repo" (remember that "emerge --sync" is just a wrapper for "emaint sync
> --auto"). I think it would be easy to add later 1. add support for
> different package managers (if the need arises), and 2. make the "sync
> command" user configurable.

Sure, that seems sensible.

Re: [gentoo-dev] Eselect repository feature request

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 3504 bytes --]


Florian Schmaus <flow@gentoo.org> writes:

> [[PGP Signed Part:Undecided]]
> On 21/06/2023 17.56, Mike Gilbert wrote:
>> On Wed, Jun 21, 2023 at 11:41 AM Florian Schmaus <flow@gentoo.org> wrote:
>>>
>>> On 20.06.23 19:26, Mike Gilbert wrote:
>>>> On Tue, Jun 20, 2023 at 1:08 PM Florian Schmaus <flow@gentoo.org> wrote:
>>>>>
>>>>> On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:
>>>>>>       Isn't that duplicating the information of metadata/layout.conf's
>>>>>>       'master' key-value pair [1]?
>>>>>>
>>>>>>
>>>>>> Yes, I agree that it would be duplicating that information. As a matter
>>>>>> of fact, Michał Górny pointed the same thing out.
>>>>>> However, Michał also added, quote: "What's really lacking here is
>>>>>> support for specifying dependencies via |repositories.xml|
>>>>>
>>>>> Do we need to duplicate the information in repositories.xml, with all
>>>>> the drawbacks of duplication?
>>>>>
>>>>> Can't eselect repository add the new repository, then read the 'masters'
>>>>> value from layout.conf, and add the missing repositories recursively?
>>>>
>>>> That would be a significant change in behavior for eselect repository.
>>>
>>> Right, but it seems to be a desirable behaviour. Cases where the user
>>> wants to add a repo but not immediately sync it are probably rare.
>>>
>>> Furthermore, it would avoid duplicating the information, which avoids
>>> the typical drawbacks of duplication (e.g., the two sets getting out of
>>> sync).
>>>
>>> I've looked at the eselect-repository code, and it seems not hard to
>>> change the behaviour of "eselect repository add" to add and sync a
>>> repository and then, recursively, add and sync further required
>>> repositories.
>>>
>>> I may give it a shot, but ideally I'd know if it has a chance to be
>>> accepted upstream first. Or maybe there is a good reason why
>>> eselect-repository behaves as it currently does that I am missing?
>> I can't speak for "upstream", but here are my concerns:
>> 1. As a developer, I might just want to create the repos.conf config
>> snippet and sync the repo manually.
>> 2. As a user, I might have any arbitrary reason for not wanting to
>> sync immediately.
>
> Would an opt-out switch be enough to alleviate those concerns of you?
>
>
>> 3. eselect-repository does not currently depend on any particular
>> package manager. It writes config files intended for Portage, but it
>> does not actually invoke any Portage commands. That feels like a
>> significant distinction to me.
>> 4. If you start invoking Portage commands, you then have to deal with
>> the possibility of people using alternate package managers. pkgcore
>> can also utilize Portage's repos.conf, and the user might prefer to
>> use pmaint instead of emaint or emerge --sync.
>
> Those two points seem to be based on the same fundamental concern.
>
> The only portage specific code would be the call to "emaint sync -r
> $repo" (remember that "emerge --sync" is just a wrapper for "emaint
> sync --auto"). I think it would be easy to add later 1. add support
> for different package managers (if the need arises), and 2. make the
> "sync command" user configurable.

While looking at this, it might be worth evaluating 2 other things
which users have mentioned during the migration away from layman:
1. Adding a way to fully disable the cache fetching;
2. Add a way to use the "real" upstream sources instead of our mirrored
ones

best,
sam

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 377 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[TOMAS FABRIZIO ORSI]

[-- Attachment #1: Type: text/plain, Size: 4099 bytes --]

>
> 2. Add a way to use the "real" upstream sources instead of our mirrored
> ones


Isn't this eselect repository's default behaviour? Or am I misunderstanding?
When I run "eselect repository list" I get the source repositories, not the
mirrored ones.
Is it using the mirrored one behind the scenes?

Best regards,
- Tomas Fabrizio Orsi
El mié, 21 jun 2023 a las 15:44, Sam James (<sam@gentoo.org>) escribió:

>
> Florian Schmaus <flow@gentoo.org> writes:
>
> > [[PGP Signed Part:Undecided]]
> > On 21/06/2023 17.56, Mike Gilbert wrote:
> >> On Wed, Jun 21, 2023 at 11:41 AM Florian Schmaus <flow@gentoo.org>
> wrote:
> >>>
> >>> On 20.06.23 19:26, Mike Gilbert wrote:
> >>>> On Tue, Jun 20, 2023 at 1:08 PM Florian Schmaus <flow@gentoo.org>
> wrote:
> >>>>>
> >>>>> On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:
> >>>>>>       Isn't that duplicating the information of
> metadata/layout.conf's
> >>>>>>       'master' key-value pair [1]?
> >>>>>>
> >>>>>>
> >>>>>> Yes, I agree that it would be duplicating that information. As a
> matter
> >>>>>> of fact, Michał Górny pointed the same thing out.
> >>>>>> However, Michał also added, quote: "What's really lacking here is
> >>>>>> support for specifying dependencies via |repositories.xml|
> >>>>>
> >>>>> Do we need to duplicate the information in repositories.xml, with all
> >>>>> the drawbacks of duplication?
> >>>>>
> >>>>> Can't eselect repository add the new repository, then read the
> 'masters'
> >>>>> value from layout.conf, and add the missing repositories recursively?
> >>>>
> >>>> That would be a significant change in behavior for eselect repository.
> >>>
> >>> Right, but it seems to be a desirable behaviour. Cases where the user
> >>> wants to add a repo but not immediately sync it are probably rare.
> >>>
> >>> Furthermore, it would avoid duplicating the information, which avoids
> >>> the typical drawbacks of duplication (e.g., the two sets getting out of
> >>> sync).
> >>>
> >>> I've looked at the eselect-repository code, and it seems not hard to
> >>> change the behaviour of "eselect repository add" to add and sync a
> >>> repository and then, recursively, add and sync further required
> >>> repositories.
> >>>
> >>> I may give it a shot, but ideally I'd know if it has a chance to be
> >>> accepted upstream first. Or maybe there is a good reason why
> >>> eselect-repository behaves as it currently does that I am missing?
> >> I can't speak for "upstream", but here are my concerns:
> >> 1. As a developer, I might just want to create the repos.conf config
> >> snippet and sync the repo manually.
> >> 2. As a user, I might have any arbitrary reason for not wanting to
> >> sync immediately.
> >
> > Would an opt-out switch be enough to alleviate those concerns of you?
> >
> >
> >> 3. eselect-repository does not currently depend on any particular
> >> package manager. It writes config files intended for Portage, but it
> >> does not actually invoke any Portage commands. That feels like a
> >> significant distinction to me.
> >> 4. If you start invoking Portage commands, you then have to deal with
> >> the possibility of people using alternate package managers. pkgcore
> >> can also utilize Portage's repos.conf, and the user might prefer to
> >> use pmaint instead of emaint or emerge --sync.
> >
> > Those two points seem to be based on the same fundamental concern.
> >
> > The only portage specific code would be the call to "emaint sync -r
> > $repo" (remember that "emerge --sync" is just a wrapper for "emaint
> > sync --auto"). I think it would be easy to add later 1. add support
> > for different package managers (if the need arises), and 2. make the
> > "sync command" user configurable.
>
> While looking at this, it might be worth evaluating 2 other things
> which users have mentioned during the migration away from layman:
> 1. Adding a way to fully disable the cache fetching;
> 2. Add a way to use the "real" upstream sources instead of our mirrored
> ones
>
> best,
> sam
>

[-- Attachment #2: Type: text/html, Size: 5706 bytes --]

Re: [gentoo-dev] Eselect repository feature request

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 4298 bytes --]


TOMAS FABRIZIO ORSI <torsi@fi.uba.ar> writes:

>  2. Add a way to use the "real" upstream sources instead of our mirrored
>  ones
>
>  
> Isn't this eselect repository's default behaviour? Or am I misunderstanding?
> When I run "eselect repository list" I get the source repositories, not the mirrored ones.
> Is it using the mirrored one behind the scenes?

(Please don't top-post.)

No, it uses the mirrores ones with metadata.

>
> Best regards,
> - Tomas Fabrizio Orsi
> El mié, 21 jun 2023 a las 15:44, Sam James (<sam@gentoo.org>) escribió:
>
>  Florian Schmaus <flow@gentoo.org> writes:
>
>  > [[PGP Signed Part:Undecided]]
>  > On 21/06/2023 17.56, Mike Gilbert wrote:
>  >> On Wed, Jun 21, 2023 at 11:41 AM Florian Schmaus <flow@gentoo.org> wrote:
>  >>>
>  >>> On 20.06.23 19:26, Mike Gilbert wrote:
>  >>>> On Tue, Jun 20, 2023 at 1:08 PM Florian Schmaus <flow@gentoo.org> wrote:
>  >>>>>
>  >>>>> On 20.06.23 16:41, TOMAS FABRIZIO ORSI wrote:
>  >>>>>>       Isn't that duplicating the information of metadata/layout.conf's
>  >>>>>>       'master' key-value pair [1]?
>  >>>>>>
>  >>>>>>
>  >>>>>> Yes, I agree that it would be duplicating that information. As a matter
>  >>>>>> of fact, Michał Górny pointed the same thing out.
>  >>>>>> However, Michał also added, quote: "What's really lacking here is
>  >>>>>> support for specifying dependencies via |repositories.xml|
>  >>>>>
>  >>>>> Do we need to duplicate the information in repositories.xml, with all
>  >>>>> the drawbacks of duplication?
>  >>>>>
>  >>>>> Can't eselect repository add the new repository, then read the 'masters'
>  >>>>> value from layout.conf, and add the missing repositories recursively?
>  >>>>
>  >>>> That would be a significant change in behavior for eselect repository.
>  >>>
>  >>> Right, but it seems to be a desirable behaviour. Cases where the user
>  >>> wants to add a repo but not immediately sync it are probably rare.
>  >>>
>  >>> Furthermore, it would avoid duplicating the information, which avoids
>  >>> the typical drawbacks of duplication (e.g., the two sets getting out of
>  >>> sync).
>  >>>
>  >>> I've looked at the eselect-repository code, and it seems not hard to
>  >>> change the behaviour of "eselect repository add" to add and sync a
>  >>> repository and then, recursively, add and sync further required
>  >>> repositories.
>  >>>
>  >>> I may give it a shot, but ideally I'd know if it has a chance to be
>  >>> accepted upstream first. Or maybe there is a good reason why
>  >>> eselect-repository behaves as it currently does that I am missing?
>  >> I can't speak for "upstream", but here are my concerns:
>  >> 1. As a developer, I might just want to create the repos.conf config
>  >> snippet and sync the repo manually.
>  >> 2. As a user, I might have any arbitrary reason for not wanting to
>  >> sync immediately.
>  >
>  > Would an opt-out switch be enough to alleviate those concerns of you?
>  >
>  >
>  >> 3. eselect-repository does not currently depend on any particular
>  >> package manager. It writes config files intended for Portage, but it
>  >> does not actually invoke any Portage commands. That feels like a
>  >> significant distinction to me.
>  >> 4. If you start invoking Portage commands, you then have to deal with
>  >> the possibility of people using alternate package managers. pkgcore
>  >> can also utilize Portage's repos.conf, and the user might prefer to
>  >> use pmaint instead of emaint or emerge --sync.
>  >
>  > Those two points seem to be based on the same fundamental concern.
>  >
>  > The only portage specific code would be the call to "emaint sync -r
>  > $repo" (remember that "emerge --sync" is just a wrapper for "emaint
>  > sync --auto"). I think it would be easy to add later 1. add support
>  > for different package managers (if the need arises), and 2. make the
>  > "sync command" user configurable.
>
>  While looking at this, it might be worth evaluating 2 other things
>  which users have mentioned during the migration away from layman:
>  1. Adding a way to fully disable the cache fetching;
>  2. Add a way to use the "real" upstream sources instead of our mirrored
>  ones
>
>  best,
>  sam


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 377 bytes --]