From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1RQ5nH-0006qn-5d for garchives@archives.gentoo.org; Mon, 14 Nov 2011 23:18:27 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 0DCDC21C02C; Mon, 14 Nov 2011 23:18:17 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) by pigeon.gentoo.org (Postfix) with ESMTP id 45FAA21C020 for ; Mon, 14 Nov 2011 23:17:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp.gentoo.org (Postfix) with ESMTP id 337391B4019 for ; Mon, 14 Nov 2011 23:17:46 +0000 (UTC) X-Virus-Scanned: by amavisd-new using ClamAV at gentoo.org X-Spam-Score: -3.784 X-Spam-Level: X-Spam-Status: No, score=-3.784 required=5.5 tests=[AWL=0.920, BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.504] Received: from smtp.gentoo.org ([127.0.0.1]) by localhost (smtp.gentoo.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Dpbt3-+BlPZh for ; Mon, 14 Nov 2011 23:17:39 +0000 (UTC) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by smtp.gentoo.org (Postfix) with ESMTP id 2866A1B400C for ; Mon, 14 Nov 2011 23:17:35 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1RQ5mO-0005zu-HD for gentoo-dev@gentoo.org; Tue, 15 Nov 2011 00:17:32 +0100 Received: from ip68-231-22-224.ph.ph.cox.net ([68.231.22.224]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 15 Nov 2011 00:17:32 +0100 Received: from 1i5t5.duncan by ip68-231-22-224.ph.ph.cox.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 15 Nov 2011 00:17:32 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: gentoo-dev@lists.gentoo.org From: Duncan <1i5t5.duncan@cox.net> Subject: [gentoo-dev] Re: have portage be quiet by default Date: Mon, 14 Nov 2011 23:17:20 +0000 (UTC) Message-ID: References: <4EB4FA98.3080201@gentoo.org> <4EBD42EE.4060104@gentoo.org> <4EBEF208.5060500@gentoo.org> <201111121840.16686.vapier@gentoo.org> <4EBFBA75.3030500@gentoo.org> <1321188262-sup-4513@raeviah> <4EBFCD5D.3080807@gentoo.org> <1321194595-sup-8983@raeviah> <4EBFE727.8000903@gentoo.org> <4EC02CC0.9080907@gentoo.org> <4EC04E11.3060500@gentoo.org> <4EC0559C.4020806@gentoo.org> <4EC06E9D.4060803@gentoo.org> <4EC075F9.4070801@gentoo.org> <4EC1322B.2020602@gentoo.org> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-dev@lists.gentoo.org Reply-to: gentoo-dev@lists.gentoo.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: ip68-231-22-224.ph.ph.cox.net User-Agent: Pan/0.135 (Tomorrow I'll Wake Up and Scald Myself with Tea; GIT bb16cbd /st/portage/src/egit-src/pan2) Content-Transfer-Encoding: quoted-printable X-Archives-Salt: 084c4000-ff71-4e65-a798-b1bba148ce2f X-Archives-Hash: 84d04c408151e51ef13fa8fe6ac974d6 Zac Medico posted on Mon, 14 Nov 2011 07:22:19 -0800 as excerpted: > On 11/14/2011 12:47 AM, Dirkjan Ochtman wrote: >> On Mon, Nov 14, 2011 at 02:59, Zac Medico wrote: >>> Well, it's much easier to gather interest and get feedback if we >>> deploy the change and ask questions later. >>=20 >> I like the new output, but find it kind of annoying that there's very >> little feedback on how far the progress is within a single job. Perhap= s >> we could show the currently executing ebuild phase in order to give a >> little more feedback? Maybe most packages are completely dominated by >> src_compile(), but for smaller packages it would be helpful, IMO. >=20 > I think that would be an interesting option. I imagine that it would be > too much information for many people, so I don't think that we'd want t= o > enable it by default. Thanks for thinking "out of the box". =3D:^) This is one thing that I've found frustrating with parallel emerging,=20 myself. The first thing I discovered were that some phases (at least the merge=20 phase, and I believe install) aren't counted, so the numbers don't=20 entirely add up (complete + running + not-started + failed <> total). =20 Thus, some packages appear to be simply sitting there doing nothing. =20 When a whole bunch of packages are in that state, it appears emerge is=20 doing nothing but consuming cycles and real-time, for no visible reason=20 at all. Since for a number of packages (I seem to notice it most on kde packages,= =20 but perhaps it's all C++ or most CMAKE or some such, plus of course=20 primarily data packages that don't have a significant build phase but=20 install many or large files) the install phase can be as intense as the=20 build phase if not more so, so if some phases aren't counted in=20 "running", then I'd definitely prefer they be counted /somewhere/. Of course, listing the number of packages in each phase (which in single- mode would by definition list the phase the single package is in) would=20 cure the above issue at the same time. Another problem, but one I'm not sure how to fix (tho the interactive=20 display would offer opportunities here), is that it'd be nice to get a=20 list of completed packages, in-process packages, and still not started=20 packages. That doesn't so easily fit in a neat summary, but as noted,=20 the interactive proposal introduces quite some opportunity, here. Alternatively, since often the real info desired is the status of a=20 particular package, it'd be useful to have either an interactive command=20 or a simple separate querying command, that can be run to query the=20 status of a particular package. *** Which brings up an alternative to the whole interactive emerges idea=20 as well -- what about a separate command, say "emerging", that when run,=20 would simply output a bunch of detail on any pending emerges? This would certainly offer a safer initial implementation, as well, since= =20 it's less likely to break the current setup due to bugs, etc. Another=20 noted benefit is that it leaves the existing defaults alone, so it's not=20 going to be threatening anyone's sacred cows (or scripted assumptions) in= =20 terms of how portage has always behaved. =3D:^) So, perhaps improve the current "quiet" display marginally, either=20 including all phases in "running" or adding another listing so the=20 numbers add up, but more importantly... *** Add a new "emerging" (name up for bikeshedding) command, that=20 displays a nice multiline summary, perhaps something like this (the=20 second emerge command was for a failure in the first, with a second=20 attempt made before completion of the main emerge @world has completed): Current emerge status: 2 emerge command(s) running. 7 emerge jobs outstanding. Commands: ** emerge --upgrade --deep --newuse --keep-going @world Status: 123 of 257 jobs complete, 1 failed, 6 running, 120 not yet started, 7 dependencies removed due to failures. Running (6): 1 pkgsetup (cat-egory/package-ver) 1 configure (cat-egory/package-ver, cat-egory/package-ver) 2 build (cat-egory/package-ver, cat-egory/package-ver) 1 install (cat-egory/package-ver) 1 merge (cat-egory/package-ver) Failed (1): dev-libs/boost-1.47.0-r1 Complete (123): ... Not started (120): ... Removed dependencies due to failure (7): ... ** emerge -u1 boost Status: 0 of 1 jobs complete, 1 running. Running (1): 1 build (dev-libs/boost-1.47.0-r1) Obviously the first implementation might not have all the information=20 above, and there might have been some nice information I missed and other= =20 information that could be displayed better, but it's a very cool idea=20 even if I /do/ say so myself. =3D:^) Note that zeroed-out listings aren't displayed, so the the --oneshot=20 doesn't list failed, not yet started, etc, and inactive phases are also=20 not displayed. Also note how easy this sort of detailed display would make it to retry=20 failed jobs and the resulting removed dependencies, once the failure has=20 been resolved. "emerging" could have a "repeat" mode as well as one-shot, with a polling= - delay parameter set to something sane like 30 or 60 seconds by default. =20 That way, I wouldn't have to feed it to "watch" =3D:^) Because this is a separate command, worries about TMI/TLI should be=20 eliminated. Additionally, it should go some way toward easing the whole=20 quiet/noisy debate as well, since there's now another command available=20 with an intermediate level of information. Something like this: edisplaylog=20 If we then throw in one more tool, I'll call it edisplaylog for lack of a= =20 better name, that reads make.conf to find the elog dir, and can=20 automatically pick out the latest log for a package, thus eliminating the= =20 trouble of doing it manually, that should help eliminate another=20 objection to quiet-by-default. So... edisplaylog boost ... would find the last elog for boost and display it. To make things easy for the user, simply ... edisplaylog ... could be made to filetime search and display the last active log in=20 tail -f mode. If after say a couple seconds of no activity, it re-polled= =20 the log dir and switched to displaying a different log (have it list the=20 log it's displaying at the top) if it was newer, that could pretty much=20 replace "noisy" mode entirely. =3D:^) Obviously running simply "edisplaylog" by itself during a parallel-jobs=20 emerge would be nearly as confusing as portage not automatically going=20 into quiet mode for parallel emerging would be, tho not quite since it=20 would display the logfile name at the top, but "edisplaylog " would=20 still be very useful, ESPECIALLY for those using --keep-going, so they=20 could track down and fix whatever failed a build before the parallel=20 emerge spit out the results at the end. Talking about which... an "edisplaylog failed" mode, which would=20 automatically find the last failure and displayed the log for it, could=20 be very helpful as well. =3D:^) Obviously all these ideas entail a lot of coding, and I realize it's easy= =20 to sit here and make requests without doing the coding, but implementing=20 them as separate commands first and incremental implementation should=20 help a lot, and I hadn't seen anything like this proposed yet, so... I'll close with a thanks to Zac and everyone else who has already made=20 portage the great tool it is today, because without them, we'd obviously=20 not be having this discussion in the first place. =3D:^) --=20 Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman