[gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Zac Medico]

For those who may not know, chromium-os currently uses a
hard-host-depends ebuild as a workaround for our lack of HDEPEND support
[1]. We could easily add HDEPEND in EAPI 5 if we want, since we already
have a Portage patch attached to bug #317337 [2]. Here is a summary of
what that Portage patch will do:

In EAPI 5 or later, DEPEND has been divided into two parts:
DEPEND for build-time target dependencies, and HDEPEND for
build-time host dependencies. This division is designed
specifically to minimize difficulty in the process of
adapting ebuilds that were written for earlier EAPIs,
and therefore it also minimizes the adjustments that
ebuild developers will have to make to the thought
processes involved when writing ebuilds from scratch. In
an environment that does not involve cross-compilation,
HDEPEND behaves the same as DEPEND. When an ebuild is
converted from EAPI 4 or earlier to EAPI 5 or later,
in order to support cross-compilation environments, some
dependencies may need to be migrated to HDEPEND.

For ebuilds that have EAPI 5 or later, the emerge
--root-deps option has no effect since it is made obsolete
by division between DEPEND and HDEPEND. If EAPI 4 or
earlier ebuilds are used in combination with EAPI 5 or
later ebuilds, the --root-deps behavior will still be
applied to the EAPI 4 or earlier ebuilds (there is no
behavior change for ebuilds having older EAPIs).

[1]
http://www.chromium.org/chromium-os/how-tos-and-troubleshooting/portage-build-faq
[2] https://bugs.gentoo.org/show_bug.cgi?id=317337
-- 
Thanks,
Zac

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Richard Yao]

[-- Attachment #1: Type: text/plain, Size: 1706 bytes --]

On 08/31/2012 04:03 PM, Zac Medico wrote:
> For those who may not know, chromium-os currently uses a
> hard-host-depends ebuild as a workaround for our lack of HDEPEND support
> [1]. We could easily add HDEPEND in EAPI 5 if we want, since we already
> have a Portage patch attached to bug #317337 [2]. Here is a summary of
> what that Portage patch will do:
> 
> In EAPI 5 or later, DEPEND has been divided into two parts:
> DEPEND for build-time target dependencies, and HDEPEND for
> build-time host dependencies. This division is designed
> specifically to minimize difficulty in the process of
> adapting ebuilds that were written for earlier EAPIs,
> and therefore it also minimizes the adjustments that
> ebuild developers will have to make to the thought
> processes involved when writing ebuilds from scratch. In
> an environment that does not involve cross-compilation,
> HDEPEND behaves the same as DEPEND. When an ebuild is
> converted from EAPI 4 or earlier to EAPI 5 or later,
> in order to support cross-compilation environments, some
> dependencies may need to be migrated to HDEPEND.
> 
> For ebuilds that have EAPI 5 or later, the emerge
> --root-deps option has no effect since it is made obsolete
> by division between DEPEND and HDEPEND. If EAPI 4 or
> earlier ebuilds are used in combination with EAPI 5 or
> later ebuilds, the --root-deps behavior will still be
> applied to the EAPI 4 or earlier ebuilds (there is no
> behavior change for ebuilds having older EAPIs).
> 
> [1]
> http://www.chromium.org/chromium-os/how-tos-and-troubleshooting/portage-build-faq
> [2] https://bugs.gentoo.org/show_bug.cgi?id=317337

I like this. It has my support.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 2101 bytes --]

On Fri, 31 Aug 2012 13:03:00 -0700
Zac Medico <zmedico@gentoo.org> wrote:
> For those who may not know, chromium-os currently uses a
> hard-host-depends ebuild as a workaround for our lack of HDEPEND
> support [1]. We could easily add HDEPEND in EAPI 5 if we want, since
> we already have a Portage patch attached to bug #317337 [2]. Here is
> a summary of what that Portage patch will do:
> 
> In EAPI 5 or later, DEPEND has been divided into two parts:
> DEPEND for build-time target dependencies, and HDEPEND for
> build-time host dependencies. This division is designed
> specifically to minimize difficulty in the process of
> adapting ebuilds that were written for earlier EAPIs,
> and therefore it also minimizes the adjustments that
> ebuild developers will have to make to the thought
> processes involved when writing ebuilds from scratch. In
> an environment that does not involve cross-compilation,
> HDEPEND behaves the same as DEPEND. When an ebuild is
> converted from EAPI 4 or earlier to EAPI 5 or later,
> in order to support cross-compilation environments, some
> dependencies may need to be migrated to HDEPEND.
> 
> For ebuilds that have EAPI 5 or later, the emerge
> --root-deps option has no effect since it is made obsolete
> by division between DEPEND and HDEPEND. If EAPI 4 or
> earlier ebuilds are used in combination with EAPI 5 or
> later ebuilds, the --root-deps behavior will still be
> applied to the EAPI 4 or earlier ebuilds (there is no
> behavior change for ebuilds having older EAPIs).

What exactly would the rules be for handling a package that is in both
DEPEND and HDEPEND, when ROOT is in effect? Would the versions be
expected to match? What about use flags?

Also, we're getting rather a lot of *DEPEND variables here... If we're
making people make major changes to their deps, which for HDEPEND we
definitely would be, then the "it's expensive since people would have
to redo their deps" argument against a combined DEPENDENCIES variable
goes out of the window, so we should rethink that too.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Zac Medico]

On 08/31/2012 01:46 PM, Ciaran McCreesh wrote:
> On Fri, 31 Aug 2012 13:03:00 -0700
> What exactly would the rules be for handling a package that is in both
> DEPEND and HDEPEND, when ROOT is in effect? Would the versions be
> expected to match? What about use flags?

For the sake of simplicity, I would treat them as entirely independent.
It should be easy enough for users to apply manual configuration
adjustments in order to resolve any conflicts of this nature that may
arise. If there turns out to be a strong demand for additional
constraints, we can consider adding them in a future EAPI (possibly
using a combined DEPENDENCIES variable).

> Also, we're getting rather a lot of *DEPEND variables here... If we're
> making people make major changes to their deps, which for HDEPEND we
> definitely would be,

Well, I not sure that "major changes" is a really good characterization.
We're just talking about migrating a few things from DEPEND to HDEPEND,
and it's not strictly required. The migration is only needed when
fulfilling a request to support cross-compilation in a particular ebuild.

> then the "it's expensive since people would have
> to redo their deps" argument against a combined DEPENDENCIES variable
> goes out of the window, so we should rethink that too.
-- 
Thanks,
Zac

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Fabio Erculiani]

I like this as well.
However, since we're going to introduce a *DEPEND split, how about
splitting PDEPEND as well?

As far as I've seen, PDEPEND has two (or more?) different meanings:
- advisory (for instance, informing users about plugins)
- cycle-breaking to help the dependency solver

Would it be possible to add support for ODEPEND (as in "optional"
dependencies -- I don't really care about the variable name) as well?
This would be quite beneficial under certain circumstances. One of
these is when ebuilds are shipped with PDEPENDs which are not required
at runtime nor for cycle-breaking...

Another scenario in where ODEPEND would be nice to have is with
systemd init files pulled in by USE=systemd (and generally use? (
sys-apps/systemd ) in *DEPEND). Providing full systemd support for all
the packages without forcing users to have it installed, given that
openrc is the de-facto standard init system in Gentoo (and we don't
have any openrc? ( sys-apps/openrc )), would be a nice features for
binpkg repos. Users could then choose to enable or disable ODEPEND
during dependencies calculation via make.conf or argv.

I don't want to diverge too much from the HDEPEND discussion, but I
think that if we're going to split *DEPEND, it might be a good
opportunity to do it right _once_ and _for all_.

-- 
Fabio Erculiani

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 550 bytes --]

On Fri, 31 Aug 2012 23:40:27 +0200
Fabio Erculiani <lxnay@gentoo.org> wrote:
> Would it be possible to add support for ODEPEND (as in "optional"
> dependencies -- I don't really care about the variable name) as well?
> This would be quite beneficial under certain circumstances. One of
> these is when ebuilds are shipped with PDEPENDs which are not required
> at runtime nor for cycle-breaking...

This is what we call "suggestions" and SDEPEND. There are already two
proposals for dealing with this general issue.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Zac Medico]

On 08/31/2012 02:40 PM, Fabio Erculiani wrote:
> I like this as well.
> However, since we're going to introduce a *DEPEND split, how about
> splitting PDEPEND as well?
> 
> As far as I've seen, PDEPEND has two (or more?) different meanings:
> - advisory (for instance, informing users about plugins)
> - cycle-breaking to help the dependency solver
> 
> Would it be possible to add support for ODEPEND (as in "optional"
> dependencies -- I don't really care about the variable name) as well?
> This would be quite beneficial under certain circumstances. One of
> these is when ebuilds are shipped with PDEPENDs which are not required
> at runtime nor for cycle-breaking...
> 
> Another scenario in where ODEPEND would be nice to have is with
> systemd init files pulled in by USE=systemd (and generally use? (
> sys-apps/systemd ) in *DEPEND). Providing full systemd support for all
> the packages without forcing users to have it installed, given that
> openrc is the de-facto standard init system in Gentoo (and we don't
> have any openrc? ( sys-apps/openrc )), would be a nice features for
> binpkg repos. Users could then choose to enable or disable ODEPEND
> during dependencies calculation via make.conf or argv.
> 
> I don't want to diverge too much from the HDEPEND discussion, but I
> think that if we're going to split *DEPEND, it might be a good
> opportunity to do it right _once_ and _for all_.

For optional dependencies, I'm pretty happy with the "runtime-switchable
USE flags" proposal:

  https://gist.github.com/2945569
-- 
Thanks,
Zac

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 344 bytes --]

On Fri, 31 Aug 2012 14:58:49 -0700
Zac Medico <zmedico@gentoo.org> wrote:
> For optional dependencies, I'm pretty happy with the
> "runtime-switchable USE flags" proposal:
> 
>   https://gist.github.com/2945569

Do we have an implementation of this yet? I have extreme doubts about
the viability of the idea...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Zac Medico]

On 08/31/2012 03:15 PM, Ciaran McCreesh wrote:
> On Fri, 31 Aug 2012 14:58:49 -0700
> Zac Medico <zmedico@gentoo.org> wrote:
>> For optional dependencies, I'm pretty happy with the
>> "runtime-switchable USE flags" proposal:
>>
>>   https://gist.github.com/2945569
> 
> Do we have an implementation of this yet? I have extreme doubts about
> the viability of the idea...

No, but there's a request here:

  https://bugs.gentoo.org/show_bug.cgi?id=424283
-- 
Thanks,
Zac

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Fabio Erculiani]

On Fri, Aug 31, 2012 at 11:58 PM, Zac Medico <zmedico@gentoo.org> wrote:
>
> For optional dependencies, I'm pretty happy with the "runtime-switchable
> USE flags" proposal:
>
>   https://gist.github.com/2945569

runtime-switchable USE flags for optional dependencies o.O? It sounds
like using a spoon to eat spaghetti to me.
I think SDEPEND is a much simpler approach to the issue, why
introducing a new kind of USE flags to address what really belongs to
*DEPEND?

> --
> Thanks,
> Zac
>



-- 
Fabio Erculiani

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 694 bytes --]

On Sat, 1 Sep 2012 00:18:07 +0200
Fabio Erculiani <lxnay@gentoo.org> wrote:

> On Fri, Aug 31, 2012 at 11:58 PM, Zac Medico <zmedico@gentoo.org>
> wrote:
> >
> > For optional dependencies, I'm pretty happy with the
> > "runtime-switchable USE flags" proposal:
> >
> >   https://gist.github.com/2945569
> 
> runtime-switchable USE flags for optional dependencies o.O? It sounds
> like using a spoon to eat spaghetti to me.
> I think SDEPEND is a much simpler approach to the issue, why
> introducing a new kind of USE flags to address what really belongs to
> *DEPEND?

Because otherwise you can't use USE flags. The rationale is there.


-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Zac Medico]

On 08/31/2012 03:18 PM, Fabio Erculiani wrote:
> On Fri, Aug 31, 2012 at 11:58 PM, Zac Medico <zmedico@gentoo.org> wrote:
>>
>> For optional dependencies, I'm pretty happy with the "runtime-switchable
>> USE flags" proposal:
>>
>>   https://gist.github.com/2945569
> 
> runtime-switchable USE flags for optional dependencies o.O? It sounds
> like using a spoon to eat spaghetti to me.

All suggested deps are not equal, so USE flags give you the ability to
pick and choose the ones that you want.

> I think SDEPEND is a much simpler approach to the issue, why
> introducing a new kind of USE flags to address what really belongs to
> *DEPEND?

I guess we could combine the two ideas if we allow USE conditionals
inside SDEPEND.
-- 
Thanks,
Zac

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 569 bytes --]

On Fri, 31 Aug 2012 16:03:25 -0700
Zac Medico <zmedico@gentoo.org> wrote:
> > runtime-switchable USE flags for optional dependencies o.O? It
> > sounds like using a spoon to eat spaghetti to me.
> 
> All suggested deps are not equal, so USE flags give you the ability to
> pick and choose the ones that you want.

So does --take / --ignore with suggested dependencies, with the added
advantage that suggested packages don't end up being brought in without
user request just because a user has a particular use flag enabled
globally.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Zac Medico]

On 08/31/2012 04:07 PM, Ciaran McCreesh wrote:
> On Fri, 31 Aug 2012 16:03:25 -0700
> Zac Medico <zmedico@gentoo.org> wrote:
>>> runtime-switchable USE flags for optional dependencies o.O? It
>>> sounds like using a spoon to eat spaghetti to me.
>>
>> All suggested deps are not equal, so USE flags give you the ability to
>> pick and choose the ones that you want.
> 
> So does --take / --ignore with suggested dependencies, with the added
> advantage that suggested packages don't end up being brought in without
> user request just because a user has a particular use flag enabled
> globally.

If the USE flags have ambiguous meanings doesn't that mean that they've
been poorly named?
-- 
Thanks,
Zac

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1202 bytes --]

On Fri, 31 Aug 2012 18:45:59 -0700
Zac Medico <zmedico@gentoo.org> wrote:
> On 08/31/2012 04:07 PM, Ciaran McCreesh wrote:
> > On Fri, 31 Aug 2012 16:03:25 -0700
> > Zac Medico <zmedico@gentoo.org> wrote:
> >>> runtime-switchable USE flags for optional dependencies o.O? It
> >>> sounds like using a spoon to eat spaghetti to me.
> >>
> >> All suggested deps are not equal, so USE flags give you the
> >> ability to pick and choose the ones that you want.
> > 
> > So does --take / --ignore with suggested dependencies, with the
> > added advantage that suggested packages don't end up being brought
> > in without user request just because a user has a particular use
> > flag enabled globally.
> 
> If the USE flags have ambiguous meanings doesn't that mean that
> they've been poorly named?

It's not like that. It's that in practice, suggestions are mostly for a
particular specific feature (such as git-send-email support), not for a
general concept (such as email in general).

It also defeats the point of suggestions, if they're not made visible.
For users, suggestions should look like suggestions, and they should
be able to see them easily.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Zac Medico]

On 09/01/2012 09:00 AM, Ciaran McCreesh wrote:
> On Fri, 31 Aug 2012 18:45:59 -0700
> Zac Medico <zmedico@gentoo.org> wrote:
>> On 08/31/2012 04:07 PM, Ciaran McCreesh wrote:
>>> On Fri, 31 Aug 2012 16:03:25 -0700
>>> Zac Medico <zmedico@gentoo.org> wrote:
>>>>> runtime-switchable USE flags for optional dependencies o.O? It
>>>>> sounds like using a spoon to eat spaghetti to me.
>>>>
>>>> All suggested deps are not equal, so USE flags give you the
>>>> ability to pick and choose the ones that you want.
>>>
>>> So does --take / --ignore with suggested dependencies, with the
>>> added advantage that suggested packages don't end up being brought
>>> in without user request just because a user has a particular use
>>> flag enabled globally.
>>
>> If the USE flags have ambiguous meanings doesn't that mean that
>> they've been poorly named?
> 
> It's not like that. It's that in practice, suggestions are mostly for a
> particular specific feature (such as git-send-email support), not for a
> general concept (such as email in general).
> 
> It also defeats the point of suggestions, if they're not made visible.
> For users, suggestions should look like suggestions, and they should
> be able to see them easily.

This sounds more like a user-interface issue than a problem with
runtime-switchable USE flags (GLEP 62). The nice thing about
runtime-switchable USE flags is that makes it possible to allow users to
unify all of their optional dependency choices in their USE flag settings.

You can still implement a --take / --ignore mechanism while allowing the
use of runtime-switchable USE conditionals in SDEPEND. It's simply a
matter of ignoring the USE conditionals and instead using your --take /
--ignore mechanism to select atoms.
-- 
Thanks,
Zac

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1072 bytes --]

On Sat, 01 Sep 2012 11:15:16 -0700
Zac Medico <zmedico@gentoo.org> wrote:
> This sounds more like a user-interface issue than a problem with
> runtime-switchable USE flags (GLEP 62). The nice thing about
> runtime-switchable USE flags is that makes it possible to allow users
> to unify all of their optional dependency choices in their USE flag
> settings.

The nice thing about GLEP 62 is that no-one has implemented it and tried
it with lots of packages and a bunch of users, thus figuring out just
how much of a pain in the ass getting this right is... Right now we're
debating the merits of a tried and tested solution versus an entirely
hypothetical idea.

If you really think unification is an advantage, you could treat
exheres-style suggestion names as a special USE_EXPAND group. But
practical experience suggests that suggestions *shouldn't* be unified,
and that the way to make the feature useful to users is to get the user
to explicitly accept or reject suggestions, and to make suggestions look
like suggestions.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 781 bytes --]

On Sat, 1 Sep 2012 00:07:38 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Fri, 31 Aug 2012 16:03:25 -0700
> Zac Medico <zmedico@gentoo.org> wrote:
> > > runtime-switchable USE flags for optional dependencies o.O? It
> > > sounds like using a spoon to eat spaghetti to me.
> > 
> > All suggested deps are not equal, so USE flags give you the ability
> > to pick and choose the ones that you want.
> 
> So does --take / --ignore with suggested dependencies, with the added
> advantage that suggested packages don't end up being brought in
> without user request just because a user has a particular use flag
> enabled globally.

Yes because runtime SSL support is *so much* different than build-time
one.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 463 bytes --]

On Fri, 31 Aug 2012 16:03:25 -0700
Zac Medico <zmedico@gentoo.org> wrote:
> > I think SDEPEND is a much simpler approach to the issue, why
> > introducing a new kind of USE flags to address what really belongs
> > to *DEPEND?
> 
> I guess we could combine the two ideas if we allow USE conditionals
> inside SDEPEND.

But you don't want to do that... The point of suggestions is that they
can be considered on their own merits.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1830 bytes --]

On Fri, 31 Aug 2012 14:58:49 -0700
Zac Medico <zmedico@gentoo.org> wrote:

> On 08/31/2012 02:40 PM, Fabio Erculiani wrote:
> > I like this as well.
> > However, since we're going to introduce a *DEPEND split, how about
> > splitting PDEPEND as well?
> > 
> > As far as I've seen, PDEPEND has two (or more?) different meanings:
> > - advisory (for instance, informing users about plugins)
> > - cycle-breaking to help the dependency solver
> > 
> > Would it be possible to add support for ODEPEND (as in "optional"
> > dependencies -- I don't really care about the variable name) as
> > well? This would be quite beneficial under certain circumstances.
> > One of these is when ebuilds are shipped with PDEPENDs which are
> > not required at runtime nor for cycle-breaking...
> > 
> > Another scenario in where ODEPEND would be nice to have is with
> > systemd init files pulled in by USE=systemd (and generally use? (
> > sys-apps/systemd ) in *DEPEND). Providing full systemd support for
> > all the packages without forcing users to have it installed, given
> > that openrc is the de-facto standard init system in Gentoo (and we
> > don't have any openrc? ( sys-apps/openrc )), would be a nice
> > features for binpkg repos. Users could then choose to enable or
> > disable ODEPEND during dependencies calculation via make.conf or
> > argv.
> > 
> > I don't want to diverge too much from the HDEPEND discussion, but I
> > think that if we're going to split *DEPEND, it might be a good
> > opportunity to do it right _once_ and _for all_.
> 
> For optional dependencies, I'm pretty happy with the
> "runtime-switchable USE flags" proposal:
> 
>   https://gist.github.com/2945569

The canonical URI is: http://www.gentoo.org/proj/en/glep/glep-0062.html

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1855 bytes --]

On Fri, 31 Aug 2012 14:11:38 -0700
Zac Medico <zmedico@gentoo.org> wrote:
> On 08/31/2012 01:46 PM, Ciaran McCreesh wrote:
> > On Fri, 31 Aug 2012 13:03:00 -0700
> > What exactly would the rules be for handling a package that is in
> > both DEPEND and HDEPEND, when ROOT is in effect? Would the versions
> > be expected to match? What about use flags?
> 
> For the sake of simplicity, I would treat them as entirely
> independent. It should be easy enough for users to apply manual
> configuration adjustments in order to resolve any conflicts of this
> nature that may arise. If there turns out to be a strong demand for
> additional constraints, we can consider adding them in a future EAPI
> (possibly using a combined DEPENDENCIES variable).

The thing is... Without some kind of "the same" constraint, we'd be
adding a feature which would probably work most of the time only by
coincidence.

> > Also, we're getting rather a lot of *DEPEND variables here... If
> > we're making people make major changes to their deps, which for
> > HDEPEND we definitely would be,
> 
> Well, I not sure that "major changes" is a really good
> characterization. We're just talking about migrating a few things
> from DEPEND to HDEPEND, and it's not strictly required. The migration
> is only needed when fulfilling a request to support cross-compilation
> in a particular ebuild.

Where are you getting "a few" from? Is this "a few seems to be enough
to make it work", or "someone carefully analysed lots of packages to
work out exactly what dependencies are HDEPEND, and measured it"? I
strongly suspect we're in "works by coincidence" territory again --
"adding packages to HDEPEND as breakages are encountered" is a long way
from "having an accurate HDEPEND". Are we aiming for the former or the
latter?

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Zac Medico]

On 08/31/2012 02:53 PM, Ciaran McCreesh wrote:
> On Fri, 31 Aug 2012 14:11:38 -0700
> Zac Medico <zmedico@gentoo.org> wrote:
>> On 08/31/2012 01:46 PM, Ciaran McCreesh wrote:
>>> On Fri, 31 Aug 2012 13:03:00 -0700
>>> What exactly would the rules be for handling a package that is in
>>> both DEPEND and HDEPEND, when ROOT is in effect? Would the versions
>>> be expected to match? What about use flags?
>>
>> For the sake of simplicity, I would treat them as entirely
>> independent. It should be easy enough for users to apply manual
>> configuration adjustments in order to resolve any conflicts of this
>> nature that may arise. If there turns out to be a strong demand for
>> additional constraints, we can consider adding them in a future EAPI
>> (possibly using a combined DEPENDENCIES variable).
> 
> The thing is... Without some kind of "the same" constraint, we'd be
> adding a feature which would probably work most of the time only by
> coincidence.

Shrug, I don't do any cross-compilation myself, but maybe someone who
does could comment on the usefulness of this kind of constraint. Mike?
Brian?

>>> Also, we're getting rather a lot of *DEPEND variables here... If
>>> we're making people make major changes to their deps, which for
>>> HDEPEND we definitely would be,
>>
>> Well, I not sure that "major changes" is a really good
>> characterization. We're just talking about migrating a few things
>> from DEPEND to HDEPEND, and it's not strictly required. The migration
>> is only needed when fulfilling a request to support cross-compilation
>> in a particular ebuild.
> 
> Where are you getting "a few" from? Is this "a few seems to be enough
> to make it work", or "someone carefully analysed lots of packages to
> work out exactly what dependencies are HDEPEND, and measured it"? I
> strongly suspect we're in "works by coincidence" territory again --
> "adding packages to HDEPEND as breakages are encountered" is a long way
> from "having an accurate HDEPEND". Are we aiming for the former or the
> latter?

I would think of it like prefix support in EAPI 3. Ebuilds using EAPI 3
were not required to support prefix. Similarly, ebuilds using EAPI 5
will not be required to support cross-compilation.
-- 
Thanks,
Zac

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Jorge Manuel B. S. Vicetto]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 31-08-2012 20:46, Ciaran McCreesh wrote:

<snip>

> Also, we're getting rather a lot of *DEPEND variables here... If
> we're making people make major changes to their deps, which for
> HDEPEND we definitely would be, then the "it's expensive since
> people would have to redo their deps" argument against a combined
> DEPENDENCIES variable goes out of the window, so we should rethink
> that too.

I have to agree with Ciaran, instead of multiplying DEPEND variables,
it's probably time we move to a single DEPENDENCIES variable.

- -- 
Regards,

Jorge Vicetto (jmbsvicetto) - jmbsvicetto at gentoo dot org
Gentoo- forums / Userrel / Devrel / KDE / Elections / RelEng
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJQRpeVAAoJEC8ZTXQF1qEP/UgQALd+7oqAODQA5bXdyqV+Qix+
mDN66c/6UO4VS2dyhfCEyA3osJzFS4u6mIuR7uFpXoKXGGs+MYdl7EG9C0k48zUu
YLCDD56oyk6wACxBk7EHWVql1rvFoFemMUw5YUVq71w3FU9hrpBi/DXKsoAlCRyw
4B2p6t8p6efll3vzbcz7M0LZseiox4GBTFCrtxR5zwgvx3b0gKvgU1Pv+AT3SBQK
J3IOxb09GSLCJKo56+iDHGuS5RwBBmdWP9l3+AdbjR2LoQ05f8o8a7/geg1Qqg/Z
gVVSo4WDN2kIDJOvCBuXuo95a0KKFt/zUgfwjsqe02fRu2mDiWAju4L6vk2WE316
4yfMULI6HrVUk3ra+O4ZW7eoOuRvPVDpr4vyCVetFe4bx+zmlo/CmzOg/2teMyoc
rlMvOigR/4D+wxX7mbw/0fwZ5tVUbZ2pkdEhKetlpDe+xbWY0LhaczKdizwF7BrT
d+BeazPGWBP/muY0s3VDu3KV/3TRS0tME8GRsDevA9nCfA2plU0ZmmZnTB69tLc+
/dgdexHhc3IuA5eMObwOfSK6r9Jozlrv09TDvb6kHXm+0kqhV/W/aaS1qT4Bjlxd
psMjf9lSJHLcXuhtOz9OW4qmhp4BGCA8Rgeoq25Yw8E2eH0abvDbHR5U7u1hEpnQ
j6rJ0fZ27tfbMecd5i8b
=Zv/I
-----END PGP SIGNATURE-----

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Fabio Erculiani]

On Wed, Sep 5, 2012 at 2:06 AM, Jorge Manuel B. S. Vicetto
<jmbsvicetto@gentoo.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 31-08-2012 20:46, Ciaran McCreesh wrote:
>
> <snip>
>
>> Also, we're getting rather a lot of *DEPEND variables here... If
>> we're making people make major changes to their deps, which for
>> HDEPEND we definitely would be, then the "it's expensive since
>> people would have to redo their deps" argument against a combined
>> DEPENDENCIES variable goes out of the window, so we should rethink
>> that too.
>
> I have to agree with Ciaran, instead of multiplying DEPEND variables,
> it's probably time we move to a single DEPENDENCIES variable.

So, let's say that I only want to apply a filter function against the
buildtime dependencies, in that case I'd need to parse *all* the
dependencies anyway?
The complexity would become:
O((b + r + p) * P)
b = amount of buildtime dependencies
r = amount of runtime dependencies
p = amount of post-dependencies
P = amount of packages i need to apply the filter to

Instead of just:
O(b * P)

It sounds like a good "dis-optimization". Some pkgs have really long
list of *DEPEND.

>
> - --
> Regards,
>
> Jorge Vicetto (jmbsvicetto) - jmbsvicetto at gentoo dot org
> Gentoo- forums / Userrel / Devrel / KDE / Elections / RelEng
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.19 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQIcBAEBAgAGBQJQRpeVAAoJEC8ZTXQF1qEP/UgQALd+7oqAODQA5bXdyqV+Qix+
> mDN66c/6UO4VS2dyhfCEyA3osJzFS4u6mIuR7uFpXoKXGGs+MYdl7EG9C0k48zUu
> YLCDD56oyk6wACxBk7EHWVql1rvFoFemMUw5YUVq71w3FU9hrpBi/DXKsoAlCRyw
> 4B2p6t8p6efll3vzbcz7M0LZseiox4GBTFCrtxR5zwgvx3b0gKvgU1Pv+AT3SBQK
> J3IOxb09GSLCJKo56+iDHGuS5RwBBmdWP9l3+AdbjR2LoQ05f8o8a7/geg1Qqg/Z
> gVVSo4WDN2kIDJOvCBuXuo95a0KKFt/zUgfwjsqe02fRu2mDiWAju4L6vk2WE316
> 4yfMULI6HrVUk3ra+O4ZW7eoOuRvPVDpr4vyCVetFe4bx+zmlo/CmzOg/2teMyoc
> rlMvOigR/4D+wxX7mbw/0fwZ5tVUbZ2pkdEhKetlpDe+xbWY0LhaczKdizwF7BrT
> d+BeazPGWBP/muY0s3VDu3KV/3TRS0tME8GRsDevA9nCfA2plU0ZmmZnTB69tLc+
> /dgdexHhc3IuA5eMObwOfSK6r9Jozlrv09TDvb6kHXm+0kqhV/W/aaS1qT4Bjlxd
> psMjf9lSJHLcXuhtOz9OW4qmhp4BGCA8Rgeoq25Yw8E2eH0abvDbHR5U7u1hEpnQ
> j6rJ0fZ27tfbMecd5i8b
> =Zv/I
> -----END PGP SIGNATURE-----
>



-- 
Fabio Erculiani

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 751 bytes --]

On Wed, 5 Sep 2012 09:19:45 +0200
Fabio Erculiani <lxnay@gentoo.org> wrote:
> So, let's say that I only want to apply a filter function against the
> buildtime dependencies, in that case I'd need to parse *all* the
> dependencies anyway?

Yes, and? If your dependency parser's time isn't "so tiny it makes
absolutely no difference compared to the filesystem access time needed
to get the metadata in the first place" then you're doing something
severely wrong.

And if we are going the "parser time" route, then given the heavy
intersection between build and run dependencies, the overall amount of
data to be processed in the common case that all dependencies are
required is smaller with unified DEPENDENCIES.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Rich Freeman]

On Wed, Sep 5, 2012 at 3:19 AM, Fabio Erculiani <lxnay@gentoo.org> wrote:
> The complexity would become:
> O((b + r + p) * P)
> b = amount of buildtime dependencies
> r = amount of runtime dependencies
> p = amount of post-dependencies
> P = amount of packages i need to apply the filter to
>
> Instead of just:
> O(b * P)

Well, actually, in both cases the complexity is O(n), assuming you
only need to make a constant number of passes through the deps per
package.

The whole point of O notation is that it is about how it scales, not
how long it takes.  An O(n) algorithm can actually be slower than an
O(n^n) algorithm even on a large dataset.  However, the key is that at
some point if the dataset gets large enough the O(n) algorithm will
always become faster.

I tend to agree with Cirian though - the time for some code to run
through the dependencies array and do something isn't going to be very
high.  If a piece of code has to do it many times there is nothing
that says the package manager can't index it.

Rich

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Fabio Erculiani]

On Wed, Sep 5, 2012 at 12:44 PM, Rich Freeman <rich0@gentoo.org> wrote:
> On Wed, Sep 5, 2012 at 3:19 AM, Fabio Erculiani <lxnay@gentoo.org> wrote:
>> The complexity would become:
>> O((b + r + p) * P)
>> b = amount of buildtime dependencies
>> r = amount of runtime dependencies
>> p = amount of post-dependencies
>> P = amount of packages i need to apply the filter to
>>
>> Instead of just:
>> O(b * P)
>
> Well, actually, in both cases the complexity is O(n), assuming you
> only need to make a constant number of passes through the deps per
> package.
>
> The whole point of O notation is that it is about how it scales, not
> how long it takes.  An O(n) algorithm can actually be slower than an
> O(n^n) algorithm even on a large dataset.  However, the key is that at
> some point if the dataset gets large enough the O(n) algorithm will
> always become faster.
>
> I tend to agree with Cirian though - the time for some code to run
> through the dependencies array and do something isn't going to be very
> high.  If a piece of code has to do it many times there is nothing
> that says the package manager can't index it.

I don't want to diverge (again) from the main topic, but I think that
you're just oversimplifying it.
If you consider parsing an ebuild something hidden behind a lot of
abstraction layers, O(n) vs O(n/2) is a big difference, even if both,
normalized, are still O(n). And I would never design an API which
assumes that O(n/2) equals to O(n), because you don't know how that is
going to be used in upper layers, today, tomorrow and in 5 years.

>
> Rich
>



-- 
Fabio Erculiani

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 532 bytes --]

On Wed, 5 Sep 2012 13:23:19 +0200
Fabio Erculiani <lxnay@gentoo.org> wrote:
> If you consider parsing an ebuild something hidden behind a lot of
> abstraction layers, O(n) vs O(n/2) is a big difference, even if both,
> normalized, are still O(n). And I would never design an API which
> assumes that O(n/2) equals to O(n), because you don't know how that is
> going to be used in upper layers, today, tomorrow and in 5 years.

Uhm. O(n) == O(n/2). Anything assuming they're different is just wrong.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Rich Freeman]

On Wed, Sep 5, 2012 at 7:27 AM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> Uhm. O(n) == O(n/2). Anything assuming they're different is just wrong.

We're basically debating definitions.  O notation is used to indicate
how algorithms scale and nobody uses O(n/2) and such as a result.

An algorithm that is twice as slow is still twice as slow.  That might
or might not matter.  However,  this isn't the domain where O notation
is used.  You use O notation when your algorithm takes 30 seconds to
run and you want to know what happens with the dataset doubles 3000
times.  It generally doesn't matter if the result is that it will take
1 trillion years to operate or 2 trillion years.  You care more about
whether it will take minutes, hours, weeks, years, or whatever.

I can't really think of any practical examples where multiplying the
time to parse a list of maybe 50 items vs 5 lists of 10 items is going
to make that big of a difference.  They're just lines in a text file -
your CPU can compare a few billions characters per second.  Sure, if
you add 75 layers of abstraction you might be able to find just the
right point where a factor of 5 is going to make it intolerable but a
factor of 1 is almost acceptable, but go ahead and add/remove a few
layers and suddenly it is all fine or all horrible anyway.  That is a
bit contrived.  That's why everybody ignores constant factors in O
notation anyway.

Rich

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Alexis Ballier]

On Wed, 5 Sep 2012 08:46:13 -0400
Rich Freeman <rich0@gentoo.org> wrote:

> On Wed, Sep 5, 2012 at 7:27 AM, Ciaran McCreesh
> <ciaran.mccreesh@googlemail.com> wrote:
> > Uhm. O(n) == O(n/2). Anything assuming they're different is just
> > wrong.
> 
> We're basically debating definitions.  O notation is used to indicate
> how algorithms scale and nobody uses O(n/2) and such as a result.
> 
> An algorithm that is twice as slow is still twice as slow.  That might
> or might not matter.  However,  this isn't the domain where O notation
> is used.  You use O notation when your algorithm takes 30 seconds to
> run and you want to know what happens with the dataset doubles 3000
> times.  It generally doesn't matter if the result is that it will take
> 1 trillion years to operate or 2 trillion years.  You care more about
> whether it will take minutes, hours, weeks, years, or whatever.
> 
> I can't really think of any practical examples where multiplying the
> time to parse a list of maybe 50 items vs 5 lists of 10 items is going
> to make that big of a difference.  They're just lines in a text file -
> your CPU can compare a few billions characters per second.  Sure, if
> you add 75 layers of abstraction you might be able to find just the
> right point where a factor of 5 is going to make it intolerable but a
> factor of 1 is almost acceptable, but go ahead and add/remove a few
> layers and suddenly it is all fine or all horrible anyway.  That is a
> bit contrived.  That's why everybody ignores constant factors in O
> notation anyway.

ewww, t_n=O(n) means 'there exists a constant C such that t_n <= C n';
it is just nonsensical to talk about O(n) vs O(n/2): take 2C instead
of C. If you care about the constant C then dont talk about O(). Measure
it in terms of CPU cycles for example, but then it becomes dependent on
a lot of things (compiler, hardware, etc.). However, it can be proved
that (with a sane implementation) all these differences scale only by a
constant factor between two implementations; this is why, when you talk
about algorithms and not execution time on a given hardware, the O()
asymptotic estimations are the ones that make sense. O() does not say
anything about execution time, but give a good idea of how your
algorithm will scale in the future: a O(n) algorithm will be able to
process twice as much data in the same time when your hardware will be
twice as fast, a O(2^n) one will be able to process only one more.

A.

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Alec Warner]

On Wed, Sep 5, 2012 at 2:46 PM, Rich Freeman <rich0@gentoo.org> wrote:
> On Wed, Sep 5, 2012 at 7:27 AM, Ciaran McCreesh
> <ciaran.mccreesh@googlemail.com> wrote:
>> Uhm. O(n) == O(n/2). Anything assuming they're different is just wrong.
>
> We're basically debating definitions.  O notation is used to indicate
> how algorithms scale and nobody uses O(n/2) and such as a result.
>
> An algorithm that is twice as slow is still twice as slow.  That might
> or might not matter.  However,  this isn't the domain where O notation
> is used.  You use O notation when your algorithm takes 30 seconds to
> run and you want to know what happens with the dataset doubles 3000
> times.  It generally doesn't matter if the result is that it will take
> 1 trillion years to operate or 2 trillion years.  You care more about
> whether it will take minutes, hours, weeks, years, or whatever.
>
> I can't really think of any practical examples where multiplying the
> time to parse a list of maybe 50 items vs 5 lists of 10 items is going
> to make that big of a difference.  They're just lines in a text file -
> your CPU can compare a few billions characters per second.  Sure, if
> you add 75 layers of abstraction you might be able to find just the
> right point where a factor of 5 is going to make it intolerable but a
> factor of 1 is almost acceptable, but go ahead and add/remove a few
> layers and suddenly it is all fine or all horrible anyway.  That is a
> bit contrived.  That's why everybody ignores constant factors in O
> notation anyway.

so tl;dr, this doesn't matter because string comparison is very fast.

-A

>
> Rich
>

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2126 bytes --]

On Wed, 05 Sep 2012 00:06:45 +0000
"Jorge Manuel B. S. Vicetto" <jmbsvicetto@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 31-08-2012 20:46, Ciaran McCreesh wrote:
> 
> <snip>
> 
> > Also, we're getting rather a lot of *DEPEND variables here... If
> > we're making people make major changes to their deps, which for
> > HDEPEND we definitely would be, then the "it's expensive since
> > people would have to redo their deps" argument against a combined
> > DEPENDENCIES variable goes out of the window, so we should rethink
> > that too.
> 
> I have to agree with Ciaran, instead of multiplying DEPEND variables,
> it's probably time we move to a single DEPENDENCIES variable.

Well, if you really insist that Gentoo is the right place to reinvent
the wheel of bash variables...

If we really want to go this route, then please at least require
explicit label at start of DEPENDENCIES. And the same when appending to
DEPENDENCIES -- just so 'unlikely' mistakes will leave us with hours of
debugging.

Furthermore, please think of eclasses. They *all* will have to append
dependencies in two ways, in an EAPI-conditional way. Exherbo doesn't
have that problem because they don't care about backwards
compatibility. We will support old EAPIs for at least few years if not
until the end of Gentoo. Not that appending dependencies in eclasses is
really that good idea.

Remember that this requirement will actually cause migration to EAPI 5
to be even harder than to any previous EAPIs. Migrating a single ebuild
will require rewriting the dependencies, and migrating an eclass will
require adding a lot of dirty code. Especially if it is python.eclass.

And we will have to convert them back to old-style dependencies anyway.
For the sake of compatibility with external tools.

And as a final note, on a request from ulm, I have created a wiki page
where you could list all proposed new dependency types so we can have
a broader look when deciding:

http://wiki.gentoo.org/wiki/Future_EAPI/New_dependency_types

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 1534 bytes --]

On Wed, 5 Sep 2012 18:15:43 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> If we really want to go this route, then please at least require
> explicit label at start of DEPENDENCIES. And the same when appending
> to DEPENDENCIES -- just so 'unlikely' mistakes will leave us with
> hours of debugging.

We should take the exheres-0 rules for labels and eclasses, which limit
labels' scopes to blocks, and which introduce an extra ( ) block around
the outside when doing eclass variable merging.

> Not that appending dependencies in eclasses is really that good idea.

Dependencies aren't appended over eclasses, they're merged.

(And I have a sneaking recollection of PMS not documenting this
properly...)

> Remember that this requirement will actually cause migration to EAPI 5
> to be even harder than to any previous EAPIs. Migrating a single
> ebuild will require rewriting the dependencies, and migrating an
> eclass will require adding a lot of dirty code.

Migrating to EAPI 5 requires rewriting dependencies anyway if we're
adding in HDEPEND. Also, earlier EAPIs have introduced new phase
functions, which is a far ickier change for ebuilds than this.

> Especially if it is python.eclass.

You know what the solution there is...

> And we will have to convert them back to old-style dependencies
> anyway. For the sake of compatibility with external tools.

No, external tools are required to be EAPI aware. If they're not, then
the external tools need fixing.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2194 bytes --]

On Thu, 6 Sep 2012 06:58:51 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Wed, 5 Sep 2012 18:15:43 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > If we really want to go this route, then please at least require
> > explicit label at start of DEPENDENCIES. And the same when appending
> > to DEPENDENCIES -- just so 'unlikely' mistakes will leave us with
> > hours of debugging.
> 
> We should take the exheres-0 rules for labels and eclasses, which
> limit labels' scopes to blocks, and which introduce an extra ( )
> block around the outside when doing eclass variable merging.

Because? I believe we should take 'Gentoo rules', including required
explicit build+run at the start.

> > Not that appending dependencies in eclasses is really that good
> > idea.
> 
> Dependencies aren't appended over eclasses, they're merged.

Thanks for correcting my wording, like the naming was really relevant
to the topic.

> (And I have a sneaking recollection of PMS not documenting this
> properly...)

Yes, I think PMS is pretty silent about this. I think it doesn't even
say that in phase functions the contents are implementation-defined.

> > Remember that this requirement will actually cause migration to
> > EAPI 5 to be even harder than to any previous EAPIs. Migrating a
> > single ebuild will require rewriting the dependencies, and
> > migrating an eclass will require adding a lot of dirty code.
> 
> Migrating to EAPI 5 requires rewriting dependencies anyway if we're
> adding in HDEPEND. Also, earlier EAPIs have introduced new phase
> functions, which is a far ickier change for ebuilds than this.

Do you really believe in HDEPEND in EAPI 5? I've already postponed this
in my mind. Also, not every single ebuild will actually need it.

> > And we will have to convert them back to old-style dependencies
> > anyway. For the sake of compatibility with external tools.
> 
> No, external tools are required to be EAPI aware. If they're not, then
> the external tools need fixing.

Changing package manager API like that between EAPI is just bad. You
know that, don't you?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 3661 bytes --]

On Thu, 6 Sep 2012 09:39:25 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> On Thu, 6 Sep 2012 06:58:51 +0100
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > On Wed, 5 Sep 2012 18:15:43 +0200
> > Michał Górny <mgorny@gentoo.org> wrote:
> > > If we really want to go this route, then please at least require
> > > explicit label at start of DEPENDENCIES. And the same when
> > > appending to DEPENDENCIES -- just so 'unlikely' mistakes will
> > > leave us with hours of debugging.
> > 
> > We should take the exheres-0 rules for labels and eclasses, which
> > limit labels' scopes to blocks, and which introduce an extra ( )
> > block around the outside when doing eclass variable merging.
> 
> Because? I believe we should take 'Gentoo rules', including required
> explicit build+run at the start.

You mean, you want to invent some new rules that don't quite work,
rather than using the ones that do... The whole "initial labels" thing
isn't an issue for exheres-0, since rather than appending, the
resulting metadata variable ends up with extra ( ) blocks like this:

    (
        stuff/from-the-exheres
    )
    (
        stuff/from-exlib-1
    )
    (
        stuff/from-exlib-2
    )

so there's no possibility of labels ending up applied to the wrong
thing.

If you just append, you'd have to have some way of validating that
eclasses all individually specify an initial label. That's not
something that can easily be done.

> > (And I have a sneaking recollection of PMS not documenting this
> > properly...)
> 
> Yes, I think PMS is pretty silent about this. I think it doesn't even
> say that in phase functions the contents are implementation-defined.

That's covered under the general rules for environment variables. The
issue is that PMS doesn't make a good distinction between the metadata
variable's value and the environment variable value. The wording that's
in there currently dates back to how things worked a very long time ago.

> > > Remember that this requirement will actually cause migration to
> > > EAPI 5 to be even harder than to any previous EAPIs. Migrating a
> > > single ebuild will require rewriting the dependencies, and
> > > migrating an eclass will require adding a lot of dirty code.
> > 
> > Migrating to EAPI 5 requires rewriting dependencies anyway if we're
> > adding in HDEPEND. Also, earlier EAPIs have introduced new phase
> > functions, which is a far ickier change for ebuilds than this.
> 
> Do you really believe in HDEPEND in EAPI 5? I've already postponed
> this in my mind. Also, not every single ebuild will actually need it.

*shrug* if all the new *DEPEND stuff ends up in EAPI 6, then there's no
point in DEPENDENCIES for EAPI 5. But we'll have to sort this out
sooner or later...

> > > And we will have to convert them back to old-style dependencies
> > > anyway. For the sake of compatibility with external tools.
> > 
> > No, external tools are required to be EAPI aware. If they're not,
> > then the external tools need fixing.
> 
> Changing package manager API like that between EAPI is just bad. You
> know that, don't you?

Your choices are to make the package manager API abstract out this sort
of thing, or to require client updates for a new EAPI. And as it's
fairly hard to predict what's going to be in a new EAPI, being too
abstract is pretty much a lost cause.

You can't simply convert new concepts to old concepts, since there's no
pre-EAPI-5 concept of what any of these new dependency types are.
Treating an HDEPEND or an IDEPEND as a DEPEND is just wrong.

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2950 bytes --]

On Thu, 6 Sep 2012 09:00:40 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Thu, 6 Sep 2012 09:39:25 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > On Thu, 6 Sep 2012 06:58:51 +0100
> > Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > > On Wed, 5 Sep 2012 18:15:43 +0200
> > > Michał Górny <mgorny@gentoo.org> wrote:
> > > > If we really want to go this route, then please at least require
> > > > explicit label at start of DEPENDENCIES. And the same when
> > > > appending to DEPENDENCIES -- just so 'unlikely' mistakes will
> > > > leave us with hours of debugging.
> > > 
> > > We should take the exheres-0 rules for labels and eclasses, which
> > > limit labels' scopes to blocks, and which introduce an extra ( )
> > > block around the outside when doing eclass variable merging.
> > 
> > Because? I believe we should take 'Gentoo rules', including required
> > explicit build+run at the start.
> 
> You mean, you want to invent some new rules that don't quite work,
> rather than using the ones that do... The whole "initial labels" thing
> isn't an issue for exheres-0, since rather than appending, the
> resulting metadata variable ends up with extra ( ) blocks like this:
> 
>     (
>         stuff/from-the-exheres
>     )
>     (
>         stuff/from-exlib-1
>     )
>     (
>         stuff/from-exlib-2
>     )
> 
> so there's no possibility of labels ending up applied to the wrong
> thing.
> 
> If you just append, you'd have to have some way of validating that
> eclasses all individually specify an initial label. That's not
> something that can easily be done.

In that format there is not a single thing which *can be done easily*.

But what I was saying is that I dislike the implicit 'no label ==
build+run'. It's unclear, very unclear.

Why the heck:

    ( foo/bar )

introduces another label than:

    use? ( foo/bar )

?

> > > > Remember that this requirement will actually cause migration to
> > > > EAPI 5 to be even harder than to any previous EAPIs. Migrating a
> > > > single ebuild will require rewriting the dependencies, and
> > > > migrating an eclass will require adding a lot of dirty code.
> > > 
> > > Migrating to EAPI 5 requires rewriting dependencies anyway if
> > > we're adding in HDEPEND. Also, earlier EAPIs have introduced new
> > > phase functions, which is a far ickier change for ebuilds than
> > > this.
> > 
> > Do you really believe in HDEPEND in EAPI 5? I've already postponed
> > this in my mind. Also, not every single ebuild will actually need
> > it.
> 
> *shrug* if all the new *DEPEND stuff ends up in EAPI 6, then there's
> no point in DEPENDENCIES for EAPI 5. But we'll have to sort this out
> sooner or later...

Yes, there's more time for a meaningful discussion without switching
everything upside-down just because you did it.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 462 bytes --]

On Thu, 6 Sep 2012 10:27:55 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> But what I was saying is that I dislike the implicit 'no label ==
> build+run'. It's unclear, very unclear.
> 
> Why the heck:
> 
>     ( foo/bar )
> 
> introduces another label than:
> 
>     use? ( foo/bar )
> 
> ?

Labels are propagated into child blocks, so it doesn't introduce a new
label. I think you've misunderstood something here...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 732 bytes --]

On Thu, 6 Sep 2012 09:31:29 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Thu, 6 Sep 2012 10:27:55 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > But what I was saying is that I dislike the implicit 'no label ==
> > build+run'. It's unclear, very unclear.
> > 
> > Why the heck:
> > 
> >     ( foo/bar )
> > 
> > introduces another label than:
> > 
> >     use? ( foo/bar )
> > 
> > ?
> 
> Labels are propagated into child blocks, so it doesn't introduce a new
> label. I think you've misunderstood something here...

Right, too many ()s for me.

Then my argument that I want us to require explicit label at start of
global () still holds.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Brian Harring]

On Wed, Sep 05, 2012 at 12:06:45AM +0000, Jorge Manuel B. S. Vicetto wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 31-08-2012 20:46, Ciaran McCreesh wrote:
> 
> <snip>
> 
> > Also, we're getting rather a lot of *DEPEND variables here... If
> > we're making people make major changes to their deps, which for
> > HDEPEND we definitely would be, then the "it's expensive since
> > people would have to redo their deps" argument against a combined
> > DEPENDENCIES variable goes out of the window, so we should rethink
> > that too.
> 
> I have to agree with Ciaran, instead of multiplying DEPEND variables,
> it's probably time we move to a single DEPENDENCIES variable.

Personally, my complaints re: it are that  1) while minor, the 
labels in some cases are overly verbose; recommendations instead of 
recommends, suggestions instead of suggests, etc.  2) An actual 
flaw in their design (imo): it tries to intermix two different 
forms of parsing, without any real justification for *why* beyond 
*hey look kids, I can!*;  The two can intersect in slightly fucked 
up ways, case in point:

DEPENDENCIES="
run+build:
  cat/the
  x? ( cat/cow
test:
  y? ( cat/says
    z? ( cat/moo
)))

Now, there may be some unstated rules that disallow that, but if that 
*is* allowed, that's frankly dumb.  As to if it's disallowed, it's 
kind of a design flaw that the situation can occur requiring an 
explicit suppression.


Rather than invent and try intermixing a secondary form, just using 
the existing strikes me as saner; either we can have a specific 
use_expand group like thus:

DEPENDENCIES="
dep_run? ( cat/monkeys )
dep_run+build? ( cat/foo )"

Or, preferable imo, do away w/ the +, use a more natural ',' for phase 
separation, and use ':';

DEPENDENCIES="
dep:run? ( cat/monkeys )
dep:run,build? ( cat/foo )"

Doing it that way reuses the existing parsing infrastructure (good) 
via just requiring a change to the use validation machinery (easy if 
the PM is implemented sanely).

It also is able to express things that exheres variation can't do as 
cleanly; considering build/fetch/post/run/test as the viable dep 
targets:

DEPENDENCIES="
build+fetch+post+test:
  some-dep"
vs
DEPENDENCIES="!dep:run? ( some-dep )"

I don't much expect that to occur, but the potential exists, thus 
mentioning it.


One unstated fault re: DEPENDENCIES btw, is it will not play nice w/ 
exactly one of blocks.  Treating '^^' as "exactly one of", consider:

DEPENDENCIES="
^^ (
  run:
    cat/blah
  build:
    cat/dar
  cat/foon
)"

Is that a stupid dep?  You bet your ass it is..  But it would have to 
be explicitly suppressed by the parser for any such construct- moreso, 
repoman would have to spot it which is slightly unfun.

Finally, one note; while certain folk have been making lots of noise 
about DEPENDENCIES being the best thing since sliced bread, their 
isn't much comment about how one actually transitions to it without 
making eclass authors who have to support both pre-DEPENDENCIES, and 
post-DEPENDENCIES eapis happy; kind of swiss cheeses the hell out of 
the code frankly.

A compatibility hack that stacks them is strongly advisable; something 
akin to the following:

Literally, we do the following:
inherit() {
  if eapi blah; then
    local DEPEND PDEPEND RDEPEND
    <usual saving/protection of DEPENDENCIES var>
  else
    <usual saving/protection of DEPEND/PDEPEND/RDEPEND vars>
  fi
  <normal sourcing machinery>
  if eapi blah; then
    local _deps=( ) _x
    for _x in DEPENDENCIES DEPEND RDEPEND PDEPEND; do
      [ -n "${!_x}" ] && deps+=( "${!_x}" )
    done
    [ ${#deps} -ne 0 ] && DEPENDENCIES="${deps[*]}"
    unset DEPEND RDEPEND PDEPEND _x _deps
    <normal stacking/restoration of DEPENDENCIES rules>
  else
    <normal stacking/restoration of RDEPEND/PDEPEND/DEPEND>
  fi
}

Via that, we eclasses that are pure DEPENDENCIES eapi wise, just set 
the DEPENDENCIES directly; those that have to support multiple eapi's 
(aka, every fricking eclass that exists right now) can just use the 
old form, shifting into the new form as things progress.


Either way, the dual parsing required for exheres version I'm -1 on; 
I'm generally wary of NIH modifications, but the form I've mentioned 
that reuses our existing machinery is +.5...ish... from my standpoint 
(+1 on the form, just kind of 'meh' on the single var angle despite 
mostly agreeing w/ the reasoning).

Either way, on w/ the flaming/insinuations of 
idiocy/counter-insinuations of being a wee bit too friendly w/ 
sheep...
~harring

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1805 bytes --]

On Thu, 6 Sep 2012 01:11:45 -0700
Brian Harring <ferringb@gmail.com> wrote:

> A compatibility hack that stacks them is strongly advisable;
> something akin to the following:
> 
> Literally, we do the following:
> inherit() {
>   if eapi blah; then
>     local DEPEND PDEPEND RDEPEND
>     <usual saving/protection of DEPENDENCIES var>
>   else
>     <usual saving/protection of DEPEND/PDEPEND/RDEPEND vars>
>   fi
>   <normal sourcing machinery>
>   if eapi blah; then
>     local _deps=( ) _x
>     for _x in DEPENDENCIES DEPEND RDEPEND PDEPEND; do
>       [ -n "${!_x}" ] && deps+=( "${!_x}" )
>     done
>     [ ${#deps} -ne 0 ] && DEPENDENCIES="${deps[*]}"
>     unset DEPEND RDEPEND PDEPEND _x _deps
>     <normal stacking/restoration of DEPENDENCIES rules>
>   else
>     <normal stacking/restoration of RDEPEND/PDEPEND/DEPEND>
>   fi
> }
> 
> Via that, we eclasses that are pure DEPENDENCIES eapi wise, just set 
> the DEPENDENCIES directly; those that have to support multiple eapi's 
> (aka, every fricking eclass that exists right now) can just use the 
> old form, shifting into the new form as things progress.

If we decide to go with a such a hack, then we either have to support
it indefinitely, or to decide to drop the support in some further EAPI.

If we go for the latter, then it's just delaying the ugly conditional
eclasses will have to suffer at some random point in the future. Well,
maybe two eclasses less if we wait with it for an EAPI which will
provide 'killer features' which will render the eclasses unusable with
older EAPIs. And way, it will be a bit confusing to remember two switch
points...

If we go for the former... then some developers will ask: why eclasses
and not ebuilds? Why?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] HDEPEND (host dependencies for cross-compilation) for EAPI 5?

[Brian Harring]

On Tue, Sep 11, 2012 at 7:13 AM, Michał Górny <mgorny@gentoo.org> wrote:
> On Thu, 6 Sep 2012 01:11:45 -0700
> Brian Harring <ferringb@gmail.com> wrote:
>
>> A compatibility hack that stacks them is strongly advisable;
>> something akin to the following:
>>
>> Literally, we do the following:
>> inherit() {
>>   if eapi blah; then
>>     local DEPEND PDEPEND RDEPEND
>>     <usual saving/protection of DEPENDENCIES var>
>>   else
>>     <usual saving/protection of DEPEND/PDEPEND/RDEPEND vars>
>>   fi
>>   <normal sourcing machinery>
>>   if eapi blah; then
>>     local _deps=( ) _x
>>     for _x in DEPENDENCIES DEPEND RDEPEND PDEPEND; do
>>       [ -n "${!_x}" ] && deps+=( "${!_x}" )
>>     done
>>     [ ${#deps} -ne 0 ] && DEPENDENCIES="${deps[*]}"
>>     unset DEPEND RDEPEND PDEPEND _x _deps
>>     <normal stacking/restoration of DEPENDENCIES rules>
>>   else
>>     <normal stacking/restoration of RDEPEND/PDEPEND/DEPEND>
>>   fi
>> }
>>
>> Via that, we eclasses that are pure DEPENDENCIES eapi wise, just set
>> the DEPENDENCIES directly; those that have to support multiple eapi's
>> (aka, every fricking eclass that exists right now) can just use the
>> old form, shifting into the new form as things progress.
>
> If we decide to go with a such a hack, then we either have to support
> it indefinitely, or to decide to drop the support in some further EAPI.

It's a transitional hack; we support it as long as we need the
transition coverage.  Meaning if by EAPI7, everyone is using EAPI5 and
up (or by EAPI7 it's basicaly insane to try and support
0,1,2,3,4,5,6,7), we drop the transition code in EAPI8.  This is no
different to how RDEPEND/DEPEND autosetting was phased out.

> If we go for the latter, then it's just delaying the ugly conditional
> eclasses will have to suffer at some random point in the future.

Hate to say "na uh", but... na uh.  The point is as long as an eclass
has to support older EAPI's, they can use set DEPEND/RDEPEND and it
would work fine.

If the eclass needs the newer depends types (hdep, fdep, whatever) for
something, it levels an EAPI check, and then sets DEPENDENCIES.
Meanwhile, until it moves it's minimal EAPI to one requiring
DEPENDENCIES, they can use the old syntax and have it stacked.

Basically, either we can have git.eclass (for example) doing:

if has $EAPI 0 1 2 3 4; then
  DEPEND=">=dev-util/git-1.6"
else
  DEPENDENCIES="dep:build? ( >=dev-util/git-1.6 )"
fi

Or, as long as it's suppporting EAPI's 0 1 2 3 4 and the transition is
still enabled for the actual EAPI it's being used in...
DEPEND=">=dev-util/git-1.6"

Doing this means no eclass code has to maintain EAPI5 dependencies
code and <EAPI5 depends/rdepends in parallel; they can transition to
pure DEPENDENCIES form at their own pace, dependent on their code, and
the EAPIs they support.  At some point we state "the transition hack
will be removed in the next EAPI"- giving decent forewarning- then
remove it if it's not an undue dev burden.  As for eclasses/ebuilds
that are purely >=EAPI5, they convert to DEPENDENCIES syntax as they
go/have spare cycles.

Bluntly, this approach makes transition pretty painless for the vast
majority of the tree.  In the grand scheme of things, this is actually
one of the simplest/cleanest migration EAPI will see I suspect.

There really isn't a sane argument to be made against this beyond
"screw it, just make the devs convert immediately and damn the costs"
(which I don't view as sane).

Either way, strongly get the feeling you're arguing purely because it
had the term DEPENDENCIES in it...

~harring