Re: [gentoo-dev] EGO_SUM

[William Hubbs <williamh@gentoo.org>]

[-- Attachment #1: Type: text/plain, Size: 3947 bytes --]

I know I'm pretty late to this thread, but I'm going to respond to some
of the concerns and suggest another alternative.

On Mon, Apr 17, 2023 at 09:37:32AM +0200, Florian Schmaus wrote:
> I want to continue the discussion to re-instate EGO_SUM, potentially 
> leading to a democratic vote on whether EGO_SUM should be re-instated or 
> deprecated.
> 
> For the past months, I tried to find *technical reasons*, e.g., reasons 
> that affect end-users, that justify the deprecation of EGO_SUM. However, 
> I was unable to find any. The closest thing I could find was portage 
> being unable to process an ebuild due to its large environment (bug 
> 830187). However, as this happens while developing an ebuild, it should 
> never affect users. Obviously this is a situation where EGO_SUM can not 
> be used. Fortunately, it does not affect most Go packages, as seen in my 
> previous analysis of Go packages in ::gentoo and their EGO_SUM size. 
> Furthermore, newer portage versions, with USE=gentoo-dev, will 
> proactively warn you if the environment caused by the ebuild becomes large.
> 
> All further arguments for the deprecation of EGO_SUM where of cosmetic 
> nature.
> 
> However, the deprecation of EGO_SUM is harmful to Gentoo and its users. 
> To briefly re-iterate the reasons:
> 
> The EGO_SUM alternatives
> - do not have the same level of trust and therefore have a negative 
> impact on security (a dubious tarball someone put somewhere, especially 
> when proxy-maint)

For this, I would argue that vetting the tarball falls to the developer
who is proxying. If you don't trust the proxy maintainer you
are pushing for, it is easy to make a dependency tarball yourself and
add it to your dev space.

> - are not easily verifiable

I don't have a response to this other than to say that go does its
own verification of modules with the dependency tarballs that it can't
do with vendor tarballs.

> - require additional effort when developing ebuilds

This "additional effort" is pretty subjective. Making a dependency tarball
isn't a lot of work, especially with the script that I posted in this thread.

> - hinder the packaging and Gentoo's adoption of Go-based projects, which 
> is worrisome as Go is very popular

I don't have a response here. I don't see it as much of a henderance
(this is obviously subjective).

> - prevent Go modules from being shared as DISTFILES on the mirrors 
> across various packages
 
 The issue here is really the duplicate data in the dependency or vendor
 tarballs, and yes, there is a lot of it.

> Last but not least, we have the same situation in the Rust ecosystem, 
> but we allow the EGO_SUM "equivalent" there.

I'm not sure it is quite the same because Rust projects tend to have
much smaller numbers of dependencies.


Another thing to consider is that using EGO_SUM adds a significant
amount of processing to the go-module eclass.
I was advised recently that this isn't a good idea since bash is
slow, so I am considering moving most of that processing into
get-ego-vendor by having it generate the contents of SRC_URI directly
instead of using the eclass code to do that.

My thought is to have get-ego-vendor output the value for a variable,
GO_SRC_URI and add that to SRC_URI in the ebuild like so:

# The output from get-ego-vendor:
GO_SRC_URI="
	# dependency 1
	# dependency 2
	"

SRC_URI="https://main-project-here
	${GO_SRC_URI}"

This should speed things up some since most of the processing we are
doing in the eclass would be removed, so I would rather not see the council
force the use of EGO_SUM. This, however, is still going to hit the
limitation of bug 830187.

I am, however, open to another solution, so I will keep following this
thread.

I think the better question should be around what we can do to get bug 721088 or
bug 833567 to move forward.

Thanks,

William


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]