Re: [gentoo-portage-dev] Re: [RFC] New file layout for PKGDIR and binhosts

[Zac Medico <zmedico@gentoo.org>]

On 12/25/2014 02:03 AM, Duncan wrote:
> Zac Medico posted on Wed, 24 Dec 2014 00:13:57 -0800 as excerpted:
> 
>>> If there were some way to
>>> get all the info for the binpkgs into one file (so it could be run on
>>> cron or something), this could mean that I'd only have to do one file
>>> request for all that metadata and would be much quicker than inspecting
>>> all those files.
>>
>> That's what $PKGDIR/Packages is.
> 
> This is a good excuse to ask a question that has bothered me for some 
> time, plus make a request for the new eclean-pkg replacement...
> 
> Normally I like to keep several old binpkgs around for troubleshooting 
> reference or quick-install, but the combined set of kde packages, for 
> instance, can get pretty big, and with monthly iterations, they build up 
> pretty fast.
> 
> But eclean-pkg doesn't have an easy way to say clean up /just/ kde-base/
> *, leaving the currently installed version and one previous version for 
> reference, cleaning out all others.  (Sure I could put /everything/ else 
> on the exclude list, but what's really needed is an include list, plus a 
> "keep N more" option.)
> 
> So I often end up cleaning packages like that out manually by simply 
> deleting them.  My question is thus, does the remaining index/db/cache 
> entry get cleaned properly (when?) or am I leaving uncleanable garbage 
> behind when I do this?

You can run 'emaint --fix binhost' to sync $PKGDIR/Packages up with the
existing tbz2 files.

If you don't run that emaint command, the packages that you removed will
be pruned from $PKGDIR/Packages the next time that you run an 'emerge
--usepkg' command with sufficient privileges.

> Which of course translates into a couple of feature requests for the new 
> eclean-pkg:  
> 
> 1) Make it possible to clean only selected pkgs or categories.
> 
> 2) Have an option to clean up and/or regenerate the cache/index/db/
> whatever files, re-syncing them with what's actually there.

eclean-pkg calls 'emaint --fix binhost' to fix $PKGDIR/Packages after it
removes some tbz2 files.

> Meanwhile, another possible usage for multiple binpkgs per ebuild:
> 
> I commonly run live-builds (mostly kde, tho I'm not ATM), and would 
> /love/ to be able to keep about three generations (current plus two back) 
> around, ideally IDed by their git-commit or the like.  Running live 
> packages can mean even more risk than usual of something not working out, 
> but currently, if the package builds, by the time you find that out, 
> you've obliterated your previous *9999 binpkg and even figuring out which 
> git commit it was isn't easy, let alone doing a quick binpkg rollback, 

FWIW, you can use 'cp -rl $PKGDIR $PKGDIR.backup' to make a hardlink
snapshot of $PKGDIR. Portage will break hardlinks whenever it needs to
update tbz2 files, so you don't have to worry about updates in $PKGDIR
affecting the files in $PKGDIR.backup.

> like you'd do with an ordinary version upgrade.  Were multiple live-
> binpkg-versions kept around, IDed by the git-commit or at least with that 
> info in the metadata somewhere, it'd make things /so/ much easier! =:^)

That's going to require an EAPI extension [1], in order to establish a
protocol for the ebuild to communicate the commit id to the package manager.

> But of course that does make having an automated cleaner that can keep 
> just the last N package versions around while deleting the others, that 
> much more important.

Yes, and we might integrate some on-the-fly garbage collection directly
into portage, or provide a hook for this purpose.

[1] https://bugs.gentoo.org/show_bug.cgi?id=182028
-- 
Thanks,
Zac