[gentoo-dev] rfc: go packages vs repositories

[gentoo-dev] rfc: go packages vs repositories

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 554 bytes --]

All,

in looking at some of the Go ebuilds we have in the tree, I see that
some of them, for example go-tools, have multiple Go packages in a
single repository. This means that something like:

go get -d -u -t golang.org/x/tools

will fail. There is an issue opened upstream about this [1].

My question for this list is, should we wait for this to be resolved
before we do anything, or should we change the go packaging we are doing
to package single go packages instead of repositories?

Thanks,

William

[1] https://github.com/golang/go/issues/11090

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] rfc: go packages vs repositories

[Patrick McLean]

On Fri, 12 Jun 2015 12:54:04 -0500
William Hubbs <williamh@gentoo.org> wrote:

> All,
> 
> in looking at some of the Go ebuilds we have in the tree, I see that
> some of them, for example go-tools, have multiple Go packages in a
> single repository. This means that something like:
> 
> go get -d -u -t golang.org/x/tools
> 
> will fail. There is an issue opened upstream about this [1].
> 
> My question for this list is, should we wait for this to be resolved
> before we do anything, or should we change the go packaging we are
> doing to package single go packages instead of repositories?

I would vote for packaging separate packages as we do for other
languages. We could make go ebuilds simply install their sources to
something like /usr/share/go/${PN}-${SLOT}. Then have an eclass build
GOPATH from the installation paths of all the dependencies of that
package. All go libraries should have subslots that match ${PV}, and go
packages should use slot operator deps to make sure they get rebuild
when a library gets updated.

AFAIK this is the only way to get sane and predictable dependency
behaviour with go packages, since upstream seems to be think dynamic
linking is evil.

> Thanks,
> 
> William
> 
> [1] https://github.com/golang/go/issues/11090

Re: [gentoo-dev] rfc: go packages vs repositories

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1919 bytes --]

On Fri, Jun 12, 2015 at 11:18:29AM -0700, Patrick McLean wrote:
> On Fri, 12 Jun 2015 12:54:04 -0500
> William Hubbs <williamh@gentoo.org> wrote:
> 
> > All,
> > 
> > in looking at some of the Go ebuilds we have in the tree, I see that
> > some of them, for example go-tools, have multiple Go packages in a
> > single repository. This means that something like:
> > 
> > go get -d -u -t golang.org/x/tools
> > 
> > will fail. There is an issue opened upstream about this [1].
> > 
> > My question for this list is, should we wait for this to be resolved
> > before we do anything, or should we change the go packaging we are
> > doing to package single go packages instead of repositories?
> 
> I would vote for packaging separate packages as we do for other
> languages.

Ok, I'm with you here.

> We could make go ebuilds simply install their sources to
> something like /usr/share/go/${PN}-${SLOT}. Then have an eclass build
> GOPATH from the installation paths of all the dependencies of that
> package. All go libraries should have subslots that match ${PV}, and go
> packages should use slot operator deps to make sure they get rebuild
> when a library gets updated.
> 
> AFAIK this is the only way to get sane and predictable dependency
> behaviour with go packages, since upstream seems to be think dynamic
> linking is evil.

This brings up a whole new question which is sort of confusing to me.

Ebuilds like go-net, and some others that are in the tree, do not build
executables. They only build *.a files which are used at build time by
other pure go consumers. Also, there are no upstream releases for these
types of packages most of the time.

Since the Go compiler bundles all the necessary packages to compile a go
binary, I can't help but wonder if we really need manual snapshots of
packages that build only *.a files in the tree?

Thoughts?

William


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] rfc: go packages vs repositories

[Kristian Fiskerstrand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 06/12/2015 09:58 PM, William Hubbs wrote:
> On Fri, Jun 12, 2015 at 11:18:29AM -0700, Patrick McLean wrote:
>> On Fri, 12 Jun 2015 12:54:04 -0500 William Hubbs
>> <williamh@gentoo.org> wrote:


..

> Since the Go compiler bundles all the necessary packages to compile
> a go binary, I can't help but wonder if we really need manual
> snapshots of packages that build only *.a files in the tree?
> 
> Thoughts?
> 

It gets even worse if you factor in security. With the static linking
you really need a := dependency on all libraries used as you don't
know whether an update is security related.

That said, I understand the structure. I don't like it, but I
understand it, given that it is primarily only intended to be used
within container/docker environments.

Good luck trying to get it to play nice with a package manager though...

- -- 
Kristian Fiskerstrand
Public PGP key 0xE3EDFAE3 at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
-----BEGIN PGP SIGNATURE-----

iQEcBAEBCgAGBQJVezrYAAoJECULev7WN52FtWYH/i2nPEWAcUHjYP/i9DYqbrzJ
pHFub2zZrdCyIqF0n/aYVtgdcU+uXz/iHD/+j/SaIVaGgWBO5kafqEt93Zyw2i4I
yHWzZp5cUWMt4YfUBq63ZBGWQkaK4YsbP9TmuuUGe5ZhuOHBQhKtenue0VBqQ6Bl
tiYZcByCFJ8HHeshCGdr0unAA8K85vIIaDdz/FkkA2rwlFudIWAfgaWhomc60oAV
9aVKllOpqWsIoWn6GYKGuidSWmXMmN6J7EPyGENJENf01oF3Q/D7H5o3IN2uIB7m
FUXtmdPji3eSS77mpyimh4xXi6fzy1x3kWGhmPHBo1PMDdaY9S/mKjy85NS5z2U=
=hT4l
-----END PGP SIGNATURE-----

Re: [gentoo-dev] rfc: go packages vs repositories

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1471 bytes --]

On Fri, Jun 12, 2015 at 10:02:41PM +0200, Kristian Fiskerstrand wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
> 
> On 06/12/2015 09:58 PM, William Hubbs wrote:
> > On Fri, Jun 12, 2015 at 11:18:29AM -0700, Patrick McLean wrote:
> >> On Fri, 12 Jun 2015 12:54:04 -0500 William Hubbs
> >> <williamh@gentoo.org> wrote:
> 
> 
> ..
> 
> > Since the Go compiler bundles all the necessary packages to compile
> > a go binary, I can't help but wonder if we really need manual
> > snapshots of packages that build only *.a files in the tree?
> > 
> > Thoughts?
> > 
> 
> It gets even worse if you factor in security. With the static linking
> you really need a := dependency on all libraries used as you don't
> know whether an update is security related.
> 
> That said, I understand the structure. I don't like it, but I
> understand it, given that it is primarily only intended to be used
> within container/docker environments.
> 
> Good luck trying to get it to play nice with a package manager though...

The other issue is a lot of upstreams for go don't do releases at all;
you just grab everything live.A

Does anyone know how third party go packages are being handled on other
distros?

The go language itself is easy because they do versioned releases.

What I would want to know from other distros is, do they package third
party go packages at all, and do they package packages that are just
libraries?

William

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-dev] rfc: go packages vs repositories

[Kristian Fiskerstrand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 06/12/2015 10:49 PM, William Hubbs wrote:
> On Fri, Jun 12, 2015 at 10:02:41PM +0200, Kristian Fiskerstrand 
> wrote:
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
>> 


...

>> 
>> That said, I understand the structure. I don't like it, but I 
>> understand it, given that it is primarily only intended to be 
>> used within container/docker environments.
>> 
>> Good luck trying to get it to play nice with a package manager 
>> though...
> 
> The other issue is a lot of upstreams for go don't do releases at 
> all; you just grab everything live.A
> 
> Does anyone know how third party go packages are being handled on 
> other distros?

Well, if the workflow is building a container, then testing the exact
version and if everything works, deploy it, then discard the container
once it is outdated, i.e. no updates are performed on it (data is in
its own separate container, so new versions are distributed by
creating a new app container) this makes perfect sense, and scales
across multiple app-containers working in parallel quite nicely.

Since this is (as I understand it) the intention with Go in the first
place, maybe the effort of making this cooperate with the rest of the
system (PM), should also focus on this? rather than trying to make it
fit with the regular structure we're already familiar with.

- -- 
Kristian Fiskerstrand
Public PGP key 0xE3EDFAE3 at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
-----BEGIN PGP SIGNATURE-----

iQEcBAEBCgAGBQJVe0htAAoJECULev7WN52FhSoIAJh3QTR1IEHPXjxHWJZmG7iA
UbeG41eniUM2bn55fNYBmy31idp4Mb3cr2qWo5eJlq51um1dQJdkFsdqZZSSjGm0
pHwngDprrGLmYE+uRd8ewjPfiZZXMsYmSIKNr4mmJgT+6pARnptt4rSrFiH4rGwE
3T2U+It+Yiyi+oKVH3YqyCReeWicwjXSLn3D4JX80rOAgo7YkiLb9Vxk6ilFWY8z
pcQaGbj/nSXk5wKYCamKRTCFkqnSqbCc7L0pmXXqI7j+BJE/cfFRGxPjqWe4KkZr
bLvRjdSqsN/k9M9GCxz8fKO0T0cyMuTloT311vZ1EiC7cpcyIpUs6D22bFgaehw=
=gZ80
-----END PGP SIGNATURE-----

Re: [gentoo-dev] rfc: go packages vs repositories

[Tianon Gravi]

On 12 June 2015 at 13:49, William Hubbs <williamh@gentoo.org> wrote:
> Does anyone know how third party go packages are being handled on other
> distros?

In Debian at least, each Go package is being packaged individually
(using version numbers where possible, but otherwise using fun
0.0~gitYYYYMMDD.1.GITHASH constructions).  See
https://pkg-go.alioth.debian.org/packaging.html for more details
(although the package naming is currently undergoing a revamp to
include the full import path regardless).

♥,
- Tianon
  4096R / B42F 6819 007F 00F8 8E36  4FD4 036A 9C25 BF35 7DD4

[gentoo-dev] Re: rfc: go packages vs repositories

[Duncan]

Patrick McLean posted on Fri, 12 Jun 2015 11:18:29 -0700 as excerpted:

> We could make go ebuilds simply install their sources to something like
> /usr/share/go/${PN}-${SLOT}.

I'm staying out of the general discussion, but this...

These are sources, please use a location within the PM's sources tree.  
For portage that's $DISTDIR (/usr/portage/distfiles by default).  The 
above sources installation would thus be to ${DISTDIR}/go/${PN}-${SLOT}, 
using a path similar to that used by the various live-build eclasses,
git-r3, etc.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: rfc: go packages vs repositories

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1398 bytes --]

On Fri, Jun 12, 2015 at 10:39:57PM +0000, Duncan wrote:
> Patrick McLean posted on Fri, 12 Jun 2015 11:18:29 -0700 as excerpted:
> 
> > We could make go ebuilds simply install their sources to something like
> > /usr/share/go/${PN}-${SLOT}.
> 
> I'm staying out of the general discussion, but this...
> 
> These are sources, please use a location within the PM's sources tree.  
> For portage that's $DISTDIR (/usr/portage/distfiles by default).  The 
> above sources installation would thus be to ${DISTDIR}/go/${PN}-${SLOT}, 
> using a path similar to that used by the various live-build eclasses,
> git-r3, etc.

I have golang-vcs.eclass (shown already on the list) that will use go
get to pull things and put them in /usr/portage/distfiles/go-src; that
is the same as a $GOPATH directory, so you would have
/usr/portage/distfiles/go-src/$GO_PN. Those would be live sources
depending on which vcs upstream is using.

If we want to have versions of sources for snapshots, I would
suggest another location to store those sources. Debian uses
/usr/share/gocode for that, so there would be /usr/share/gocode which
could be added to gopath as well.
directory our ebuilds would add to GOPATH. I may be able to do that in
my golang.eclass.

Here's the idea from Debian, as referenced earlier on the list [1].

William

[1] http://pkg-go.alioth.debian.org/packaging.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]