[gentoo-dev] Re: have portage be quiet by default

[Duncan <1i5t5.duncan@cox.net>]

Zac Medico posted on Mon, 14 Nov 2011 07:22:19 -0800 as excerpted:

> On 11/14/2011 12:47 AM, Dirkjan Ochtman wrote:
>> On Mon, Nov 14, 2011 at 02:59, Zac Medico <zmedico@gentoo.org> wrote:
>>> Well, it's much easier to gather interest and get feedback if we
>>> deploy the change and ask questions later.
>> 
>> I like the new output, but find it kind of annoying that there's very
>> little feedback on how far the progress is within a single job. Perhaps
>> we could show the currently executing ebuild phase in order to give a
>> little more feedback? Maybe most packages are completely dominated by
>> src_compile(), but for smaller packages it would be helpful, IMO.
> 
> I think that would be an interesting option. I imagine that it would be
> too much information for many people, so I don't think that we'd want to
> enable it by default.

Thanks for thinking "out of the box". =:^)

This is one thing that I've found frustrating with parallel emerging, 
myself.

The first thing I discovered were that some phases (at least the merge 
phase, and I believe install) aren't counted, so the numbers don't 
entirely add up (complete + running + not-started + failed <> total).  
Thus, some packages appear to be simply sitting there doing nothing.  
When a whole bunch of packages are in that state, it appears emerge is 
doing nothing but consuming cycles and real-time, for no visible reason 
at all.

Since for a number of packages (I seem to notice it most on kde packages, 
but perhaps it's all C++ or most CMAKE or some such, plus of course 
primarily data packages that don't have a significant build phase but 
install many or large files) the install phase can be as intense as the 
build phase if not more so, so if some phases aren't counted in 
"running", then I'd definitely prefer they be counted /somewhere/.

Of course, listing the number of packages in each phase (which in single-
mode would by definition list the phase the single package is in) would 
cure the above issue at the same time.

Another problem, but one I'm not sure how to fix (tho the interactive 
display would offer opportunities here), is that it'd be nice to get a 
list of completed packages, in-process packages, and still not started 
packages.  That doesn't so easily fit in a neat summary, but as noted, 
the interactive proposal introduces quite some opportunity, here.

Alternatively, since often the real info desired is the status of a 
particular package, it'd be useful to have either an interactive command 
or a simple separate querying command, that can be run to query the 
status of a particular package.

*** Which brings up an alternative to the whole interactive emerges idea 
as well -- what about a separate command, say "emerging", that when run, 
would simply output a bunch of detail on any pending emerges?

This would certainly offer a safer initial implementation, as well, since 
it's less likely to break the current setup due to bugs, etc.  Another 
noted benefit is that it leaves the existing defaults alone, so it's not 
going to be threatening anyone's sacred cows (or scripted assumptions) in 
terms of how portage has always behaved. =:^)


So, perhaps improve the current "quiet" display marginally, either 
including all phases in "running" or adding another listing so the 
numbers add up, but more importantly...

*** Add a new "emerging" (name up for bikeshedding) command, that 
displays a nice multiline summary, perhaps something like this (the 
second emerge command was for a failure in the first, with a second 
attempt made before completion of the main emerge @world has completed):

Current emerge status:

2 emerge command(s) running.
7 emerge jobs outstanding.

Commands:

** emerge --upgrade --deep --newuse --keep-going @world

Status:

123 of 257 jobs complete, 1 failed, 6 running, 120 not yet started,
7 dependencies removed due to failures.

Running (6):

1 pkgsetup (cat-egory/package-ver)

1 configure (cat-egory/package-ver, cat-egory/package-ver)

2 build (cat-egory/package-ver, cat-egory/package-ver)

1 install (cat-egory/package-ver)

1 merge (cat-egory/package-ver)

Failed (1):

dev-libs/boost-1.47.0-r1

Complete (123):

<list		in		nice		columns>
...

Not started (120):

<list 		in		nice		columns>
...

Removed dependencies due to failure (7):

<list		in		nice		columns>
...


** emerge -u1 boost

Status:

0 of 1 jobs complete, 1 running.

Running (1):

1 build (dev-libs/boost-1.47.0-r1)

Obviously the first implementation might not have all the information 
above, and there might have been some nice information I missed and other 
information that could be displayed better, but it's a very cool idea 
even if I /do/ say so myself. =:^)

Note that zeroed-out listings aren't displayed, so the the --oneshot 
doesn't list failed, not yet started, etc, and inactive phases are also 
not displayed.

Also note how easy this sort of detailed display would make it to retry 
failed jobs and the resulting removed dependencies, once the failure has 
been resolved.

"emerging" could have a "repeat" mode as well as one-shot, with a polling-
delay parameter set to something sane like 30 or 60 seconds by default.  
That way, I wouldn't have to feed it to "watch" =:^)

Because this is a separate command, worries about TMI/TLI should be 
eliminated.  Additionally, it should go some way toward easing the whole 
quiet/noisy debate as well, since there's now another command available 
with an intermediate level of information.

Something like this:

edisplaylog 
If we then throw in one more tool, I'll call it edisplaylog for lack of a 
better name, that reads make.conf to find the elog dir, and can 
automatically pick out the latest log for a package, thus eliminating the 
trouble of doing it manually, that should help eliminate another 
objection to quiet-by-default.  So...

edisplaylog boost

... would find the last elog for boost and display it.

To make things easy for the user, simply ...

edisplaylog

... could be made to filetime search and display the last active log in 
tail -f mode.  If after say a couple seconds of no activity, it re-polled 
the log dir and switched to displaying a different log (have it list the 
log it's displaying at the top) if it was newer, that could pretty much 
replace "noisy" mode entirely.  =:^)

Obviously running simply "edisplaylog" by itself during a parallel-jobs 
emerge would be nearly as confusing as portage not automatically going 
into quiet mode for parallel emerging would be, tho not quite since it 
would display the logfile name at the top, but "edisplaylog <pkg>" would 
still be very useful, ESPECIALLY for those using --keep-going, so they 
could track down and fix whatever failed a build before the parallel 
emerge spit out the results at the end.

Talking about which... an "edisplaylog failed" mode, which would 
automatically find the last failure and displayed the log for it, could 
be very helpful as well. =:^)

Obviously all these ideas entail a lot of coding, and I realize it's easy 
to sit here and make requests without doing the coding, but implementing 
them as separate commands first and incremental implementation should 
help a lot, and I hadn't seen anything like this proposed yet, so...

I'll close with a thanks to Zac and everyone else who has already made 
portage the great tool it is today, because without them, we'd obviously 
not be having this discussion in the first place. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman