Re: [gentoo-dev] Portage QOS

[Patrick McLean <chutzpah@gentoo.org>]

On Fri, 10 Jan 2014 02:19:03 +0100
Tom Wijsman <TomWij@gentoo.org> wrote:

> On Fri, 10 Jan 2014 08:31:21 +0800
> Patrick Lauer <patrick@gentoo.org> wrote:
> 
> > On 01/10/2014 08:16 AM, heroxbd@gentoo.org wrote:
> > > Igor <lanthruster@gmail.com> writes:
> > > 
> > >> The ebuilds have approximately the same time to install, the
> > >> failure rate is about the same, emerge is getting slower.
> > > 
> > > I am curious about the slowness of emerge.
> > > 
> > > How about profile the portage and rewrite the time-crucial part in
> > > C/C++, or ideally, borrowing the counterpart from paludis? How
> > > feasible is that?
> > 
> > Last I checked paludis wasn't faster - on average portage was a few
> > percents faster.
> 

> > For python things you really want  python or C instead of C++...
> 
> Why is this a Python thing? What's the reason to exclude a language?
> 
> > So, what you wanted to ask was:
> > "Which parts of pkgcore can be migrated into portage?"
> 
> Or rather: "What does it take to migrate parts of pkgcore into
> portage?"

Why not just switch to using pkgcore as the default package manager.
radhermit has been doing a lot of work lately getting EAPI 5 support
added, and generally fixing bugs etc.

Since we no longer have anyone intimately familiar with the
portage code, we should just switch to a more modern and readable
system. Rather than having random people trying to learn the convoluted
portage code, let's concentrate on getting pkgcore to a point where
we can replace portage with it.

> > > I guess the dep-tree calculation is the slowest part.
> > Yes, it's doing lots of silly dynamic things (backtracking),
> 
> Exactly, without backtracking (which I can live without) it takes
> around half a minute as compared to a lot of minutes; when it started
> to take almost half an hour it was time to completely nuke
> backtracking.
> 
> Now I happily live without ever needing to enable backtracking
> again. :)
> 
> > and portage codebase on average is not designed for speed.
> 
> Yes, inspecting the profiler results has me worried; though I find
> half a minute acceptable for the amount of packages that are involved
> on my system, so, I'm less concerned but it is still interesting that
> there is the possibility to do things faster. It's just some work to
> actually get it to do that, which requires much more dedication, time
> and knowledge than doing an occasional commit.
> 
> > As a first step I would recommend profiling it and removing unneeded
> > stuff (do less work!), rewriting parts in C is a lot of work and not
> > needed for the first round of speedups.
> 
> See my other mail <20140110020218.0f6244d5@TOMWIJ-GENTOO> for recent
> profiling images; we should indeed start with dealing with the low
> hanging fruit (there are quite some 0-2% speed improvements possible),
> as that can get us to speed it up faster than trying to deal with some
> algorithmic complexity that needs far more insight to redesign, let
> alone to properly re-implement it.
>