[gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[William Hubbs]

[-- Attachment #1.1: Type: text/plain, Size: 606 bytes --]

All,

the dependencies for dev-lang/go need to force a rebuild every time the
compiler is downgraded or upgraded.

To avoid abusing slot dependencies for dev-lang/go since it is not
needed at runtime I need to do the following.

I need to introduce virtual/go-1.7.5, virtual/go-1.8 and
virtual/go-9999. The attachment shows 1.8 but it is the same for all
versions.

Also, the attached patch needs to be applied to golang-base.eclass.

This is based on the following wiki page:

Ahttps://wiki.gentoo.org/wiki/Project:Quality_Assurance/Subslots#Using_virtuals_to_expose_multiple_ABIs

Any thoughts?

William

[-- Attachment #1.2: go-1.8.ebuild --]
[-- Type: text/plain, Size: 295 bytes --]

# Copyright 1999-2017 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2

EAPI=6

DESCRIPTION="virtual for the Go compiler"
SLOT="0/${PV}"
		KEYWORDS="-* ~amd64 ~arm ~arm64 ~ppc64 ~x86 ~amd64-fbsd ~x86-fbsd ~x64-macos ~x64-solaris"

DEPEND="~dev-lang/go-${PV}"

[-- Attachment #1.3: golang-base-virtual.patch --]
[-- Type: text/x-diff, Size: 430 bytes --]

diff --git a/eclass/golang-base.eclass b/eclass/golang-base.eclass
index 0cfd07e..e841763 100644
--- a/eclass/golang-base.eclass
+++ b/eclass/golang-base.eclass
@@ -22,7 +22,8 @@ if [[ -z ${_GOLANG_BASE} ]]; then
 
 _GOLANG_BASE=1
 
-DEPEND=">=dev-lang/go-1.7:="
+DEPEND=">=virtual/go-1.7"
+RDEPEND=">=virtual/go-1.7:="
 
 # Do not complain about CFLAGS etc since go projects do not use them.
 QA_FLAGS_IGNORED='.*'

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 163 bytes --]

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Mike Gilbert]

On Wed, Mar 1, 2017 at 7:18 PM, William Hubbs <williamh@gentoo.org> wrote:
> All,
>
> the dependencies for dev-lang/go need to force a rebuild every time the
> compiler is downgraded or upgraded.

This already happens for me. It is working as expected.

> To avoid abusing slot dependencies for dev-lang/go since it is not
> needed at runtime I need to do the following.
>
> I need to introduce virtual/go-1.7.5, virtual/go-1.8 and
> virtual/go-9999. The attachment shows 1.8 but it is the same for all
> versions.
>
> Also, the attached patch needs to be applied to golang-base.eclass.
>
> This is based on the following wiki page:
>
> Ahttps://wiki.gentoo.org/wiki/Project:Quality_Assurance/Subslots#Using_virtuals_to_expose_multiple_ABIs
>
> Any thoughts?

I don't see how introducing a layer of indirection solves any problem here.

[gentoo-dev] Re: new virtual -- virtual/go to fix go build time dependencies

[Jonathan Callen]

[-- Attachment #1.1: Type: text/plain, Size: 1578 bytes --]

On 03/01/2017 07:18 PM, William Hubbs wrote:
> All,
> 
> the dependencies for dev-lang/go need to force a rebuild every time the
> compiler is downgraded or upgraded.
> 
> To avoid abusing slot dependencies for dev-lang/go since it is not
> needed at runtime I need to do the following.
> 
> I need to introduce virtual/go-1.7.5, virtual/go-1.8 and
> virtual/go-9999. The attachment shows 1.8 but it is the same for all
> versions.
> 
> Also, the attached patch needs to be applied to golang-base.eclass.
> 
> This is based on the following wiki page:
> 
> Ahttps://wiki.gentoo.org/wiki/Project:Quality_Assurance/Subslots#Using_virtuals_to_expose_multiple_ABIs
> 
> Any thoughts?
> 
> William
> 

Unfortunately, this would allow the user to remove all versions of
dev-lang/go before attempting to compile a package using go, as
virtual/go has no RDEPENDs, and only RDEPENDs of DEPENDs are required to
be present (not DEPENDs of DEPENDs).

That is, the following sequence of actions would be perfectly legal,
according to the dependencies:

Merge dev-lang/go
Merge virtual/go
Unmerge dev-lang/go
Merge {package using golang-base.eclass}

While my initial thought was to just add dev-lang/go to the DEPEND in
the eclass (in addition to virtual/go), that doesn't quite work as the
two packages are allowed to get out of sync because virtual/go doesn't
RDEPEND on dev-lang/go at all (so you could install virtual/go for your
current version of dev-lang/go, then upgrade dev-lang/go without
upgrading virtual/go).

-- 
Jonathan Callen


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 862 bytes --]

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Alexis Ballier]

On Wed, 1 Mar 2017 18:18:01 -0600
William Hubbs <williamh@gentoo.org> wrote:

> To avoid abusing slot dependencies for dev-lang/go since it is not
> needed at runtime I need to do the following.

Is it really abusing ?
:= deps in DEPEND only would also make sense for e.g. code generators

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Michael Orlitzky]

On 03/02/2017 04:58 AM, Alexis Ballier wrote:
> 
> Is it really abusing ?
> := deps in DEPEND only would also make sense for e.g. code generators
> 

Slot operator dependencies are ignored in DEPEND:

  Indicates that any slot value is acceptable. In addition, for runtime
  dependencies, indicates that the package will break unless a matching
  package with slot and sub-slot equal to the slot and sub-slot of the
  best installed version at the time the package was built is available.

In other words, the ":=" only does something special in RDEPEND. That
makes sense when you think of it as meaning "the thing will break"
rather than "I want to do a rebuild." The only reason it's not an error
to put them in DEPEND is because it would annoy everyone doing
DEPEND="${RDEPEND}".

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Mike Gilbert]

On Thu, Mar 2, 2017 at 9:03 AM, Michael Orlitzky <mjo@gentoo.org> wrote:
> On 03/02/2017 04:58 AM, Alexis Ballier wrote:
>>
>> Is it really abusing ?
>> := deps in DEPEND only would also make sense for e.g. code generators
>>
>
> Slot operator dependencies are ignored in DEPEND:
>
>   Indicates that any slot value is acceptable. In addition, for runtime
>   dependencies, indicates that the package will break unless a matching
>   package with slot and sub-slot equal to the slot and sub-slot of the
>   best installed version at the time the package was built is available.
>
> In other words, the ":=" only does something special in RDEPEND. That
> makes sense when you think of it as meaning "the thing will break"
> rather than "I want to do a rebuild." The only reason it's not an error
> to put them in DEPEND is because it would annoy everyone doing
> DEPEND="${RDEPEND}".

Portage has interesting behavior for ":=" in DEPEND: it varies
depending on your "with-bdeps" setting.

floppym@naomi ~ % emerge -uDpv --with-bdeps=n @world

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U  ] dev-lang/go-1.8:0/1.8::gentoo [1.7.5:0/1.7.5::gentoo]
USE="-gccgo" 0 KiB

Total: 1 package (1 upgrade), Size of downloads: 0 KiB


floppym@naomi ~ % emerge -uDpv --with-bdeps=y @world

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  r  U  ] dev-lang/go-1.8:0/1.8::gentoo [1.7.5:0/1.7.5::gentoo]
USE="-gccgo" 0 KiB
[ebuild  rR    ] app-admin/cli53-0.8.7::gentoo  0 KiB

Total: 2 packages (1 upgrade, 1 reinstall), Size of downloads: 0 KiB

The following packages are causing rebuilds:

  (dev-lang/go-1.8:0/1.8::gentoo, ebuild scheduled for merge) causes
rebuilds for:
    (app-admin/cli53-0.8.7:0/0::gentoo, ebuild scheduled for merge)

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Michael Orlitzky]

On 03/02/2017 09:24 AM, Mike Gilbert wrote:
>>
>> In other words, the ":=" only does something special in RDEPEND. That
>> makes sense when you think of it as meaning "the thing will break"
>> rather than "I want to do a rebuild." The only reason it's not an error
>> to put them in DEPEND is because it would annoy everyone doing
>> DEPEND="${RDEPEND}".
> 
> Portage has interesting behavior for ":=" in DEPEND: it varies
> depending on your "with-bdeps" setting.
> 

This is why we can't have nice things.

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Ciaran McCreesh]

On Thu, 2 Mar 2017 09:47:35 -0500
Michael Orlitzky <mjo@gentoo.org> wrote:
> On 03/02/2017 09:24 AM, Mike Gilbert wrote:
> >>
> >> In other words, the ":=" only does something special in RDEPEND.
> >> That makes sense when you think of it as meaning "the thing will
> >> break" rather than "I want to do a rebuild." The only reason it's
> >> not an error to put them in DEPEND is because it would annoy
> >> everyone doing DEPEND="${RDEPEND}".  
> > 
> > Portage has interesting behavior for ":=" in DEPEND: it varies
> > depending on your "with-bdeps" setting.
> >   
> 
> This is why we can't have nice things.

Actually you can't have nice things because the labels proposal was
voted down for "being invented by the wrong people".

-- 
Ciaran McCreesh

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Alexis Ballier]

On Thu, 2 Mar 2017 14:56:37 +0000
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Thu, 2 Mar 2017 09:47:35 -0500
> Michael Orlitzky <mjo@gentoo.org> wrote:
> > On 03/02/2017 09:24 AM, Mike Gilbert wrote:  
> > >>
> > >> In other words, the ":=" only does something special in RDEPEND.
> > >> That makes sense when you think of it as meaning "the thing will
> > >> break" rather than "I want to do a rebuild." The only reason it's
> > >> not an error to put them in DEPEND is because it would annoy
> > >> everyone doing DEPEND="${RDEPEND}".    
> > > 
> > > Portage has interesting behavior for ":=" in DEPEND: it varies
> > > depending on your "with-bdeps" setting.
> > >     
> > 
> > This is why we can't have nice things.  
> 
> Actually you can't have nice things because the labels proposal was
> voted down for "being invented by the wrong people".

Don't get too much into conspiracy theories, I think it has been
dismissed because it would require rewriting every dependency and
there has not been any portage implementation afaik.

Instead of that, people seem to prefer having [A-Z]DEPEND :)

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Zac Medico]

On 03/02/2017 06:47 AM, Michael Orlitzky wrote:
> On 03/02/2017 09:24 AM, Mike Gilbert wrote:
>>>
>>> In other words, the ":=" only does something special in RDEPEND. That
>>> makes sense when you think of it as meaning "the thing will break"
>>> rather than "I want to do a rebuild." The only reason it's not an error
>>> to put them in DEPEND is because it would annoy everyone doing
>>> DEPEND="${RDEPEND}".
>>
>> Portage has interesting behavior for ":=" in DEPEND: it varies
>> depending on your "with-bdeps" setting.
>>
> 
> This is why we can't have nice things.

For those that are interested, I'm planning to to make --with-bdeps
automatically enabled when possible:

https://bugs.gentoo.org/show_bug.cgi?id=598444#c4

I hope to implement this very soon, to be included in the next portage
release.
-- 
Thanks,
Zac

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Michael Orlitzky]

On 03/02/2017 02:05 PM, Zac Medico wrote:
>>
>> This is why we can't have nice things.
> 
> For those that are interested, I'm planning to to make --with-bdeps
> automatically enabled when possible:
> 


I agree with this ^ but I don't think portage should rebuild for DEPEND
at all. It creates one more dangerous "it works in portage!" situation
that will plague users of other package managers.

(I'm not saying it couldn't be useful, but it should go in the next EAPI
if we're gonna do it.)

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Zac Medico]

On 03/02/2017 11:24 AM, Michael Orlitzky wrote:
> On 03/02/2017 02:05 PM, Zac Medico wrote:
>>>
>>> This is why we can't have nice things.
>>
>> For those that are interested, I'm planning to to make --with-bdeps
>> automatically enabled when possible:
>>
> 
> 
> I agree with this ^ but I don't think portage should rebuild for DEPEND
> at all. It creates one more dangerous "it works in portage!" situation
> that will plague users of other package managers.
> 
> (I'm not saying it couldn't be useful, but it should go in the next EAPI
> if we're gonna do it.)

PMS doesn't specify when rebuilds are supposed to be triggered. You can
consider the rebuilds as a means to satisfy the dependencies. Saying
that the package manager should not make an effort to satisfy
dependencies would be silly.
-- 
Thanks,
Zac

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Michael Orlitzky]

On 03/02/2017 04:06 PM, Zac Medico wrote:
>>
>> I agree with this ^ but I don't think portage should rebuild for DEPEND
>> at all. It creates one more dangerous "it works in portage!" situation
>> that will plague users of other package managers.
>>
>> (I'm not saying it couldn't be useful, but it should go in the next EAPI
>> if we're gonna do it.)
> 
> PMS doesn't specify when rebuilds are supposed to be triggered. You can
> consider the rebuilds as a means to satisfy the dependencies. Saying
> that the package manager should not make an effort to satisfy
> dependencies would be silly.

It doesn't violate the PMS to do the extra rebuilds, but the PMS also
doesn't say that they should happen.

Hypothetical situation: a developer notices that Go packages need to be
rebuilt when the compiler changes, so he adds subslot operators to
DEPEND and everything looks fine. Until someone with a different package
manager tries to use it, that is; the rebuilds aren't triggered unless
you're using portage.

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Ciaran McCreesh]

On Thu, 2 Mar 2017 16:25:54 -0500
Michael Orlitzky <mjo@gentoo.org> wrote:
> On 03/02/2017 04:06 PM, Zac Medico wrote:
> >>
> >> I agree with this ^ but I don't think portage should rebuild for
> >> DEPEND at all. It creates one more dangerous "it works in
> >> portage!" situation that will plague users of other package
> >> managers.
> >>
> >> (I'm not saying it couldn't be useful, but it should go in the
> >> next EAPI if we're gonna do it.)  
> > 
> > PMS doesn't specify when rebuilds are supposed to be triggered. You
> > can consider the rebuilds as a means to satisfy the dependencies.
> > Saying that the package manager should not make an effort to satisfy
> > dependencies would be silly.  
> 
> It doesn't violate the PMS to do the extra rebuilds, but the PMS also
> doesn't say that they should happen.
> 
> Hypothetical situation: a developer notices that Go packages need to
> be rebuilt when the compiler changes, so he adds subslot operators to
> DEPEND and everything looks fine. Until someone with a different
> package manager tries to use it, that is; the rebuilds aren't
> triggered unless you're using portage.

The point is to specify dependencies declaratively. A dependency
expresses a dependency, not an action. If you can't express the kind of
dependency you need, then we need either labels or another *DEPEND
variable to take care of it, not a bodge.

-- 
Ciaran McCreesh

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Alexis Ballier]

On Thu, 2 Mar 2017 13:06:45 -0800
Zac Medico <zmedico@gentoo.org> wrote:

> On 03/02/2017 11:24 AM, Michael Orlitzky wrote:
> > On 03/02/2017 02:05 PM, Zac Medico wrote:  
> >>>
> >>> This is why we can't have nice things.  
> >>
> >> For those that are interested, I'm planning to to make --with-bdeps
> >> automatically enabled when possible:
> >>  
> > 
> > 
> > I agree with this ^ but I don't think portage should rebuild for
> > DEPEND at all. It creates one more dangerous "it works in portage!"
> > situation that will plague users of other package managers.
> > 
> > (I'm not saying it couldn't be useful, but it should go in the next
> > EAPI if we're gonna do it.)  
> 
> PMS doesn't specify when rebuilds are supposed to be triggered. You
> can consider the rebuilds as a means to satisfy the dependencies.
> Saying that the package manager should not make an effort to satisfy
> dependencies would be silly.


And then have a nice ref. implementation for next EAPI.


Having barely tested (*) features set in stone at each EAPI bump is even
more dangerous than the "it works in portage!" situations IMHO.


(*) I'm not saying features are not tested, but those that have
been thrown at users for years are much more mature than the
brand new ones in comparison.

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Michael Orlitzky]

On 03/02/2017 04:30 PM, Ciaran McCreesh wrote:
> 
> The point is to specify dependencies declaratively. A dependency
> expresses a dependency, not an action. If you can't express the kind of
> dependency you need, then we need either labels or another *DEPEND
> variable to take care of it, not a bodge.
> 

In the meantime, if you give people an official bodge-maker, they'll use it.

Back on topic:

What kind of dependency do we need, anyway? William, are you saying that
if I upgrade dev-lang/go, then things will break, but if I delete
dev-lang/go, everything is fine?

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Alexis Ballier]

On Thu, 2 Mar 2017 16:46:22 -0500
Michael Orlitzky <mjo@gentoo.org> wrote:

> On 03/02/2017 04:30 PM, Ciaran McCreesh wrote:
> > 
> > The point is to specify dependencies declaratively. A dependency
> > expresses a dependency, not an action. If you can't express the
> > kind of dependency you need, then we need either labels or another
> > *DEPEND variable to take care of it, not a bodge.
> >   
> 
> In the meantime, if you give people an official bodge-maker, they'll
> use it.
> 
> Back on topic:
> 
> What kind of dependency do we need, anyway? William, are you saying
> that if I upgrade dev-lang/go, then things will break, but if I delete
> dev-lang/go, everything is fine?

It's likely like ocaml: you link your programs ~ statically but
everything you link needs to be built with the same compiler. So
that'd be some kind of "build against"-RDEPEND.

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Michael Orlitzky]

On 03/02/2017 04:53 PM, Alexis Ballier wrote:
>>
>> Back on topic:
>>
>> What kind of dependency do we need, anyway? William, are you saying
>> that if I upgrade dev-lang/go, then things will break, but if I delete
>> dev-lang/go, everything is fine?
> 
> It's likely like ocaml: you link your programs ~ statically but
> everything you link needs to be built with the same compiler. So
> that'd be some kind of "build against"-RDEPEND.
> 

The tiny practical part of me thinks it's probably better to add
dev-lang/go:= to RDEPEND than it is to create an entirely new class of
dependencies to handle this.

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Alexis Ballier]

On Thu, 2 Mar 2017 17:36:16 -0500
Michael Orlitzky <mjo@gentoo.org> wrote:

> On 03/02/2017 04:53 PM, Alexis Ballier wrote:
> >>
> >> Back on topic:
> >>
> >> What kind of dependency do we need, anyway? William, are you saying
> >> that if I upgrade dev-lang/go, then things will break, but if I
> >> delete dev-lang/go, everything is fine?  
> > 
> > It's likely like ocaml: you link your programs ~ statically but
> > everything you link needs to be built with the same compiler. So
> > that'd be some kind of "build against"-RDEPEND.
> >   
> 
> The tiny practical part of me thinks it's probably better to add
> dev-lang/go:= to RDEPEND than it is to create an entirely new class of
> dependencies to handle this.

That's what I do with ocaml, it works pretty well :)

My rationale is that for size-constrained installs (embedded,
containers, etc.) you'd have to remove everything you don't need anyway
(/usr/include, gcc, etc.) so one more or one less does not make a
difference. And for normal gentoo installs, you'd want to upgrade, so
better not waste your time uninstalling and reinstalling it every time.

Nevertheless, build-against deps are still useful. Think of a lib
#including eigen in its public headers. Is eigen a build or run
depend ? It's in-between: build-against :)


Alexis.

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1034 bytes --]

Hi all,

I was attending SCALE, but now I'm back to answer this.

On Thu, Mar 02, 2017 at 04:46:22PM -0500, Michael Orlitzky wrote:
> What kind of dependency do we need, anyway? William, are you saying that
> if I upgrade dev-lang/go, then things will break, but if I delete
> dev-lang/go, everything is fine?

Go programs will not *break* if you upgrade or downgrade dev-lang/go, but
they will not get the new features or fixes in the new version of
dev-lang/go until they are rebuilt with the new version.
If you remove dev-lang/go, all Go programs on your system will run
because they have everything they need from the Go libraries statically
linked into them.
dev-lang/go is a build time, not a runtime dependency, but dev-lang/go:=
means nothing unless it is in RDEPEND.

What I need is a way to force all go programs on your system to rebuild
when the version of dev-lang/go on your system changes, and this method
with virtuals is the only way I can think of to make that happen and
allow you to remove dev-lang/go.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Michael Orlitzky]

On 03/07/2017 05:40 PM, William Hubbs wrote:
> Hi all,
> 
> I was attending SCALE, but now I'm back to answer this.
> 
> On Thu, Mar 02, 2017 at 04:46:22PM -0500, Michael Orlitzky wrote:
>> What kind of dependency do we need, anyway? William, are you saying that
>> if I upgrade dev-lang/go, then things will break, but if I delete
>> dev-lang/go, everything is fine?
> 
> Go programs will not *break* if you upgrade or downgrade dev-lang/go, but
> they will not get the new features or fixes in the new version of
> dev-lang/go until they are rebuilt with the new version.

Ha, then I went off on a pointless tangent.

How is this situation different from, say, a C program? You need to
recompile to get the new GCC optimizations, stack-mumbo-jumbo
protections, etc.

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Patrick McLean]

On Tue, 7 Mar 2017 18:51:12 -0500
Michael Orlitzky <mjo@gentoo.org> wrote:

> On 03/07/2017 05:40 PM, William Hubbs wrote:
> > Hi all,
> > 
> > I was attending SCALE, but now I'm back to answer this.
> > 
> > On Thu, Mar 02, 2017 at 04:46:22PM -0500, Michael Orlitzky wrote:  
> >> What kind of dependency do we need, anyway? William, are you
> >> saying that if I upgrade dev-lang/go, then things will break, but
> >> if I delete dev-lang/go, everything is fine?  
> > 
> > Go programs will not *break* if you upgrade or downgrade
> > dev-lang/go, but they will not get the new features or fixes in the
> > new version of dev-lang/go until they are rebuilt with the new
> > version.  
> 
> Ha, then I went off on a pointless tangent.
> 
> How is this situation different from, say, a C program? You need to
> recompile to get the new GCC optimizations, stack-mumbo-jumbo
> protections, etc.
> 
> 

You also need to recompile to get security bugs fixed. With go it's not
just compiler options, it's also the standard library updates that need
a recompile to get.

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Michael Orlitzky]

On 03/07/2017 07:02 PM, Patrick McLean wrote:
> 
> You also need to recompile to get security bugs fixed. With go it's not
> just compiler options, it's also the standard library updates that need
> a recompile to get.
> 

If that's the reasoning, then don't you have the same problem with every
other library under dev-go? None of them are really needed at runtime,
but you would need to rebuild when they change to get security updates,
right?

If all dev-go libraries wind up in RDEPEND solely to force rebuilds on
upgrades, why not do the same with the standard library (dev-lang/go)?

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 910 bytes --]

On Tue, Mar 07, 2017 at 07:13:38PM -0500, Michael Orlitzky wrote:
> On 03/07/2017 07:02 PM, Patrick McLean wrote:
> > 
> > You also need to recompile to get security bugs fixed. With go it's not
> > just compiler options, it's also the standard library updates that need
> > a recompile to get.
> > 
> 
> If that's the reasoning, then don't you have the same problem with every
> other library under dev-go? None of them are really needed at runtime,
> but you would need to rebuild when they change to get security updates,
> right?
 
 Correct, but see below.

> If all dev-go libraries wind up in RDEPEND solely to force rebuilds on
> upgrades, why not do the same with the standard library (dev-lang/go)?

They should not end up in rdepend at all since we only need them at
build-time.

I'm sure there will be more cleanup to do if packages have go libraries
in RDEPEND.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Zac Medico]

On Tue, Mar 7, 2017 at 4:38 PM, William Hubbs <williamh@gentoo.org> wrote:
> On Tue, Mar 07, 2017 at 07:13:38PM -0500, Michael Orlitzky wrote:
>> If all dev-go libraries wind up in RDEPEND solely to force rebuilds on
>> upgrades, why not do the same with the standard library (dev-lang/go)?
>
> They should not end up in rdepend at all since we only need them at
> build-time.

Shouldn't we get rebuilds when the dev-go libraries are upgraded too?
-- 
Thanks,
Zac

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Kent Fredric]

[-- Attachment #1: Type: text/plain, Size: 1006 bytes --]

On Tue, 7 Mar 2017 16:40:06 -0600
William Hubbs <williamh@gentoo.org> wrote:

> What I need is a way to force all go programs on your system to rebuild
> when the version of dev-lang/go on your system changes, and this method
> with virtuals is the only way I can think of to make that happen and
> allow you to remove dev-lang/go.

Given the strength of := binding, I'd discourage against this.

Causing portage resolver catastrophes to solve a "it would be nice if ..." problem
is a bad trade-off.

:= Should be restricted to things that it is *necessary* for.

What I think is needed is a weaker version of := which is advisory: that is, portage
ignores the binding in entirety unless portage options dictate "rebuild things even if
strictly not necessary"

And this levity should mean portage should be more amenable to break graphs
to make install possible. ( Whereas with := , the presence of such a spec causes portage
to have tantrums when the underlying dependency changes )

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1314 bytes --]

W dniu 08.03.2017, śro o godzinie 21∶07 +1300, użytkownik Kent Fredric
napisał:
> On Tue, 7 Mar 2017 16:40:06 -0600
> William Hubbs <williamh@gentoo.org> wrote:
> 
> > What I need is a way to force all go programs on your system to rebuild
> > when the version of dev-lang/go on your system changes, and this method
> > with virtuals is the only way I can think of to make that happen and
> > allow you to remove dev-lang/go.
> 
> Given the strength of := binding, I'd discourage against this.
> 
> Causing portage resolver catastrophes to solve a "it would be nice if ..." problem
> is a bad trade-off.
> 
> := Should be restricted to things that it is *necessary* for.
> 
> What I think is needed is a weaker version of := which is advisory: that is, portage
> ignores the binding in entirety unless portage options dictate "rebuild things even if
> strictly not necessary"
> 
> And this levity should mean portage should be more amenable to break graphs
> to make install possible. ( Whereas with := , the presence of such a spec causes portage
> to have tantrums when the underlying dependency changes )

...which boils down to people having no clue what := actually means
and not caring to learn that before proposing awesome solutions.

-- 
Best regards,
Michał Górny

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Michael Orlitzky]

On 03/08/2017 01:27 AM, Zac Medico wrote:
> On Tue, Mar 7, 2017 at 4:38 PM, William Hubbs <williamh@gentoo.org> wrote:
>> On Tue, Mar 07, 2017 at 07:13:38PM -0500, Michael Orlitzky wrote:
>>> If all dev-go libraries wind up in RDEPEND solely to force rebuilds on
>>> upgrades, why not do the same with the standard library (dev-lang/go)?
>>
>> They should not end up in rdepend at all since we only need them at
>> build-time.
> 
> Shouldn't we get rebuilds when the dev-go libraries are upgraded too?
> 

That's what I was getting at.

Another reading of the PMS reminds me (as Kent pointed out) that
slot-operator deps shouldn't be used for would-be-nice stuff. It says
specifically that := "indicates that the package will break..."

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1103 bytes --]

On Wed, Mar 08, 2017 at 07:44:01AM -0500, Michael Orlitzky wrote:
> On 03/08/2017 01:27 AM, Zac Medico wrote:
> > On Tue, Mar 7, 2017 at 4:38 PM, William Hubbs <williamh@gentoo.org> wrote:
> >> On Tue, Mar 07, 2017 at 07:13:38PM -0500, Michael Orlitzky wrote:
> >>> If all dev-go libraries wind up in RDEPEND solely to force rebuilds on
> >>> upgrades, why not do the same with the standard library (dev-lang/go)?
> >>
> >> They should not end up in rdepend at all since we only need them at
> >> build-time.
> > 
> > Shouldn't we get rebuilds when the dev-go libraries are upgraded too?
> > 
> 
> That's what I was getting at.
> 
> Another reading of the PMS reminds me (as Kent pointed out) that
> slot-operator deps shouldn't be used for would-be-nice stuff. It says
> specifically that := "indicates that the package will break..."

Another option is to not force thisand rely on everyone to use
--with-bdeps=y to make the rebuild happen.

I'm not sure whether this is a good idea, but it makes sense in a way
since we are talking about build-time dependencies.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Zac Medico]

On Wed, Mar 8, 2017 at 11:20 AM, William Hubbs <williamh@gentoo.org> wrote:
> On Wed, Mar 08, 2017 at 07:44:01AM -0500, Michael Orlitzky wrote:
>> On 03/08/2017 01:27 AM, Zac Medico wrote:
>> > On Tue, Mar 7, 2017 at 4:38 PM, William Hubbs <williamh@gentoo.org> wrote:
>> >> On Tue, Mar 07, 2017 at 07:13:38PM -0500, Michael Orlitzky wrote:
>> >>> If all dev-go libraries wind up in RDEPEND solely to force rebuilds on
>> >>> upgrades, why not do the same with the standard library (dev-lang/go)?
>> >>
>> >> They should not end up in rdepend at all since we only need them at
>> >> build-time.
>> >
>> > Shouldn't we get rebuilds when the dev-go libraries are upgraded too?
>> >
>>
>> That's what I was getting at.
>>
>> Another reading of the PMS reminds me (as Kent pointed out) that
>> slot-operator deps shouldn't be used for would-be-nice stuff. It says
>> specifically that := "indicates that the package will break..."
>
> Another option is to not force thisand rely on everyone to use
> --with-bdeps=y to make the rebuild happen.
>
> I'm not sure whether this is a good idea, but it makes sense in a way
> since we are talking about build-time dependencies.

We just merged a portage patch the enables --with-bdeps automatically
when --usepkg is not enabled:

https://bugs.gentoo.org/show_bug.cgi?id=598444
-- 
Thanks,
Zac

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Michael Orlitzky]

On 03/08/2017 02:20 PM, William Hubbs wrote:
> 
> Another option is to not force this and rely on everyone to use
> --with-bdeps=y to make the rebuild happen.
> 

That feature is portage-only. Slot operator deps in DEPEND are meaningless.

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 600 bytes --]

On Wed, Mar 08, 2017 at 07:49:08PM -0500, Michael Orlitzky wrote:
> On 03/08/2017 02:20 PM, William Hubbs wrote:
> > 
> > Another option is to not force this and rely on everyone to use
> > --with-bdeps=y to make the rebuild happen.
> > 
> 
> That feature is portage-only. Slot operator deps in DEPEND are meaningless.

Having things in RDEPEND that are only used at build time is also incorrect.
I'm leaning more toward the way Zac thinks about this. If package
managers are not updating build-time dependencies before they build a
package, this could be considered a bug.

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Michael Orlitzky]

On 03/09/2017 10:36 AM, William Hubbs wrote:
> On Wed, Mar 08, 2017 at 07:49:08PM -0500, Michael Orlitzky wrote:
>> On 03/08/2017 02:20 PM, William Hubbs wrote:
>>>
>>> Another option is to not force this and rely on everyone to use
>>> --with-bdeps=y to make the rebuild happen.
>>>
>>
>> That feature is portage-only. Slot operator deps in DEPEND are meaningless.
> 
> Having things in RDEPEND that are only used at build time is also incorrect.

I was not implying that you have to pick one of the two wrong solutions
and implement it =)

The attractive options at this point are,

  1. Do nothing.

  2. Work with the PMS team to come up with a solution for the problem.

The other proposed solutions don't work:

  * Using RDEPEND is semantically incorrect, and runs afoul of the PMS
    because the packages don't break when their dependencies change.

  * Using slot operator deps in DEPEND is meaningless, and only happens
    to do what you want in portage when --with-bdeps is enabled.

Going forward with either one of those only digs us deeper into the hole
we're already in.

"How do we update insecure libraries?" would have been a good question
to ask *before* adding Go to the tree, because the answer is pretty
clearly "we can't." The right way to fix it now is to create a new class
of dependencies for the CADT languages that ignore the last 40 years of
experience and statically link everything.

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Alexis Ballier]

On Thu, 9 Mar 2017 11:06:54 -0500
Michael Orlitzky <mjo@gentoo.org> wrote:

>   2. Work with the PMS team to come up with a solution for the
> problem.


I think dependency labels & parts fill nicely that void.

I have not done proper research about it but I have yet to see a
serious proposal for its inclusion though.

[...]
> "How do we update insecure libraries?" would have been a good question
> to ask *before* adding Go to the tree, because the answer is pretty
> clearly "we can't." The right way to fix it now is to create a new
> class of dependencies for the CADT languages that ignore the last 40
> years of experience and statically link everything.

Keep in mind that some languages do not play that well with the DSO
model. Curryfication and partial function applications come to mind.
Those are also part of these 40 years of experience :)


Alexis.

Re: [gentoo-dev] new virtual -- virtual/go to fix go build time dependencies

[Kristian Fiskerstrand]

[-- Attachment #1.1: Type: text/plain, Size: 615 bytes --]

On 03/09/2017 05:06 PM, Michael Orlitzky wrote:
> "How do we update insecure libraries?" would have been a good question
> to ask *before* adding Go to the tree, because the answer is pretty
> clearly "we can't." 

As it is now, if a go-package is to be in stable tree; the package
maintainer adding a go package will need to keep track of relevant
dependencies that are embedded and do a revdep of the package if a
vulnerability in the chain is discovered.

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]