From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id A8772158094 for ; Fri, 30 Sep 2022 23:49:08 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 9290CE0A64; Fri, 30 Sep 2022 23:49:05 +0000 (UTC) Received: from smtp.gentoo.org (woodpecker.gentoo.org [140.211.166.183]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id 5CB31E0961 for ; Fri, 30 Sep 2022 23:49:05 +0000 (UTC) Received: (nullmailer pid 27423 invoked by uid 1000); Fri, 30 Sep 2022 23:49:02 -0000 Date: Fri, 30 Sep 2022 18:49:02 -0500 From: William Hubbs To: gentoo-dev@lists.gentoo.org Subject: Re: [gentoo-dev] Proposal to undeprecate EGO_SUM Message-ID: Mail-Followup-To: gentoo-dev@lists.gentoo.org References: <20220613074411.341909-1-flow@gentoo.org> <7064997.EcDN3hRarf@bstg> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="EdH+lYvMbG4UyULx" Content-Disposition: inline In-Reply-To: <7064997.EcDN3hRarf@bstg> X-Archives-Salt: 414a5a47-b03e-46ea-8a9d-d9ff662c6b53 X-Archives-Hash: 80c3aba34ae19e323f35b66fe56bc005 --EdH+lYvMbG4UyULx Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Sep 30, 2022 at 10:07:44PM +0200, Arsen Arsenovi=C4=87 wrote: > Hey, >=20 > On Friday, 30 September 2022 02:36:05 CEST William Hubbs wrote: > > I don't know for certain about a vendor tarball, but I do know there > > are instances where a vendor tarball wouldn't work. > > app-containers/containerd is a good example of this, That is why the > > vendor tarball idea was dropped. > It is indeed not possible to verify vendor tarballs[1]. The proposed=20 > solution Go people had would also require network access. >=20 > > Upstream doesn't need to provide a tarball, just an up-to-date > > "vendor" directory at the top level of the project. Two examples that > > do this are docker and kubernetes. > Upstreams doing this sounds like a mess, because then they'd have to=20 > maintain multiple source trees in their repositories, if I understand=20 > what you mean. Well, there isn't a lot of work involved in this for upstream, they just run: $ go mod vendor at the top level of their project and keep that directory in sync in their vcs. The down side is it can be big and some upstreams do not want to do it. >=20 > An alternative to vendor tarballs is modcache tarballs. These are=20 > absolutely massive (~20 times larger IIRC), though, they are verifiable. The modcache tarballs are what I'm calling dependency tarballs, and yes they are bigger than vendor tarballs and verifiable. Also, the go-module eclass sets the GOMODCACHE environment variable to point to the directory where the contents of the dependency tarball ends up which makes it easy for the go tooling to just use the information in that directory. If we can get bug https://bugs.gentoo.org/833567 to happen in eapi 9, that would solve all of this. The next step after I got that to happen would be to put a shared go module cache in, for example, "${DISTDIR}/go-mod", so that all go modules from packages would be downloaded there, and they would be consumed like all distfiles are. > opinion: I see no way around it. Vendor tarballs are the way to go. For= =20 > trivial cases, this can likely be EGO_SUM, but it scales exceedingly=20 > poorly, to the point of the trivial case being a very small percentage=20 > of Go packages. I proposed authenticated automation on Gentoo=20 > infrastructure as a solution to this, and implemented (a slow and=20 > unreliable) proof of concept (posted previously). The obvious question= =20 > of "how will proxy maintainers deal with this" is also relatively=20 > simple: giving them authorization for a subset of packages that they'd=20 > need to work on. This is an obvious increase in the barrier of entry for= =20 > fresh proxy maintainers, but it's still likely less than needing=20 > maintainers to rework ebuilds to use vendor tarballs on dev.g.o. Vendor tarballs are not complete. The best example of this I see in the tr= ee is app-containers/containerd. If you try to build that with a vendor tarball instead of a dependency tarball, the build will break, but it works with a dependency tarball. William >=20 >=20 > [1]: https://github.com/golang/go/issues/27348 > --=20 > Arsen Arsenovi=C4=87 --EdH+lYvMbG4UyULx Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iF0EABECAB0WIQTVeuxEZo4uUHOkQAluVBb0MMRlOAUCYzeAZgAKCRBuVBb0MMRl OCJ+AKCmPCfbr/uCbNid5Vs6p+Erb4eBSACfb/1K/YyTRg9qlsre/GYig7EVe+Q= =2aDz -----END PGP SIGNATURE----- --EdH+lYvMbG4UyULx--