Re: [gentoo-dev] EGO_SUM (was: [gentoo-project] Gentoo Council Election 202306 ... Nominations Open in Just Over 24 Hours.)

[Rich Freeman <rich0@gentoo.org>]

On Wed, Jul 5, 2023 at 2:40 PM Gerion Entrup <gerion.entrup@flump.de> wrote:
>
> Am Mittwoch, 5. Juli 2023, 01:09:30 CEST schrieb Oskari Pirhonen:
> > On Tue, Jul 04, 2023 at 21:56:26 +0000, Robin H. Johnson wrote:
> > >
> > > Developing it requires PMS work in addition to package manager
> > > development, because it introduces phases.
> > >
> > > - primary fetch of $SRC_URI per ebuild, including indirect Manifest
> > > - primary validation of distfiles
> > > - secondary fetch of $SRC_URI per indirect Manifest
> > > - secondary validation of additional distfiles
> > >
> > > A significantly impacted use case is "emerge -f", it now needs to run
> > > downloads twice.
> >
> > I'm not sure double downloading is required. Consider a flow similar to
> > this:
> >
> > 1. distfiles are fetched as per the ebuild
> > 2. distfiles are hashed into a temporary Manifest
> > 3. temporary Manifest is hashed and compared with the hashes stored in
> >    the in-tree Manifest for the direct Manifest
>
> This is exactly, what I meant. A webstorage is not needed. A second
> download process is also not needed. Just an additional Manifest format
> is needed for ebuilds with more than n distfiles.
>

I suspect that Robin was proposing indirect manfests AND src uris, and
not just indirect manifests.  In any case, if he wasn't, then I'd
suggest it would make sense to have that so that we don't need giant
lists of src_uris or go sums or whatever in ebuilds.  Sure, the
manifests are even larger than the original file references, but those
will still be long.  Plus if a file is used by 5 versions of an ebuild
it will be present in the manifests once per hash function, but in the
ebuilds 5 times.

I agree though that if only the manifests are moved to a fetched file
then you could fetch that on the first pass, though you'd still need
the extra logic to parse it.  I'm not sure it really is much of a
difference to the effort involved.

Aren't go sums already content hashes?  It might make even more sense
to create some kind of modular manifest verification logic in portage
so that the same eclass that handles EGO_SUM could tell the package
manager how to check the integrity of the files that are fetched.
Well, assuming we trust whatever hash function they're using (I'm
afraid to check - maybe this isn't such a great idea...).

-- 
Rich