Re: [gentoo-dev] EGO_SUM

[Florian Schmaus <flow@gentoo.org>]

On 01/06/2023 21.55, William Hubbs wrote:
>> The EGO_SUM alternatives
>> - do not have the same level of trust and therefore have a negative
>> impact on security (a dubious tarball someone put somewhere, especially
>> when proxy-maint)
> 
> For this, I would argue that vetting the tarball falls to the developer
> who is proxying. If you don't trust the proxy maintainer you
> are pushing for, it is easy to make a dependency tarball yourself and
> add it to your dev space.
> 
>> - are not easily verifiable
> 
> I don't have a response to this other than to say that go does its
> own verification of modules with the dependency tarballs that it can't
> do with vendor tarballs.

Yes, go has "go mod verify", which was added to the go-mod eclass after 
I asked on 2022-10-21 in #gentoo-dev if the eclass verifies the 
dependency tarball. robbat2 was so kind to provide a proof of concept of 
the security issue I was pointing out, which is available under 
https://gist.github.com/robbat2/82f4c208b6674e707081eda689096d55. This 
demonstration of the issue triggered 
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=733b4944c1a061269f96219cc96530f89d8f439e, 
which made the go-module.eclass run "go mod verify".

Unfortunately, a malicious contributor can trivially sidestep this 
verification step, rendering it ineffective. First, neither portage [1] 
nor PMS require that a later (source) archive can not override an 
existing file. This looseness allows, for example, the (non-upstream) 
dependency tarball, to override (upstream's) go.sum. Secondly, a 
dependency tarball could create the vendor/ directory, preventing the 
condition under which the go-module.eclass runs "go mod verify". Both 
approaches allow the dependency tarball to inject malicious code. With 
the first approach, "go mod verify" completes successfully; with the 
second, "go mod verify" is simply not invoked.

The verification, as is, is ineffective.


>> Last but not least, we have the same situation in the Rust ecosystem,
>> but we allow the EGO_SUM "equivalent" there.
> 
> I'm not sure it is quite the same because Rust projects tend to have
> much smaller numbers of dependencies.

I am curious to know of any specific reason why Rust projects generally 
get by with fewer dependencies. This impression may be deceiving, caused 
by the fact that the Go-lang ecosystem hosts several projects with a 
more significant number of dependencies. If you look at the analysis 
[2], you find that under the top 10 Go packages by EGO_SUM entry count 
are cri-o, prometheus, k3s, and k3d, among others. If someone rewrites 
any of those in Rust, they would probably end up with the same number of 
dependencies.


> Another thing to consider is that using EGO_SUM adds a significant
> amount of processing to the go-module eclass.
> I was advised recently that this isn't a good idea since bash is
> slow, so I am considering moving most of that processing into
> get-ego-vendor by having it generate the contents of SRC_URI directly
> instead of using the eclass code to do that.

Was this analyzed and quantified? Is this hurting us? The cache 
regeneration of an ebuild tree is an embarrassingly parallel operation, 
so this would need to be exponentially complex [3] to be of any 
significance.

It may be possible to tune the existing EGO_SUM handling. We should keep 
EGO_SUM if viable, as it directly maps Go's go.sum and makes developing 
Go-lang ebuilds as frictionless as possible.

- Flow



1: https://github.com/gentoo/portage/pull/1030
2: 
https://dev.gentoo.org/~flow/gentoo-tree-analysis-results/2023-05-17T100838-gentoo-at-2022-02-16-60dc7a03ff2f/post-processed-ego-sum.txt
3: something similar to what was recently found in the latex ebuilds, 
see 
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6ee282f0645dcfccf1836b9cc7ae55556629eb8b