[gentoo-dev] [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[gentoo-dev] [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Joe Peterson]

I recently had a user write to me after banging his head against the
wall for a while, trying to get a package working.  By the time he wrote
me, he had already figured it out, but he wanted to convey to me that
what finally helped was actually the emerge output (which stated exactly
how to get things working - in this case, the need to run emerge
--config).  He had not noticed this before and only saw it upon
re-installing, given the transient nature of the emerge messages.

Bottom line here is that there is extremely valuable and critical info
in our emerge output.  In a way, these messages are like Gentoo-specific
READMEs (or release notes and/or install instructions).  However, it is
not saved for a user to use as a resource later (well, except that it is
partially saved in the master emerge.log, but that's not quite useful
enough).  There is no "official" place to go to look for Gentoo
instructions; /usr/share/doc is one logical place, but it only contains
files actually installed, not the notes output by emerge (and these are
usually upstream-supplied, not Gentoo).

I propose that, upon merging a package, we save the emerge messages in
either: 1) a package-specific file that resides somewhere "official" or
2) in the portage DB, so that the messages can be re-read via a portage
utility.  In the latter case, either a new option to "equery" or a new
"q" command (e.g. "equery readme <pkg>" or "qreadme <pkg>" could
retrieve the text).

In either case, there would then be a place to go that is known and
consistent (and can be documented in the Gentoo doc).  It could, in
essense, serve as a kind of "Gentoo package README" collection.  I could
also imagine later expanding on this by letting a given package also
include more thorough README info from a file if the maintainer so desires.

						-Joe

Re: [gentoo-dev] [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Marius Mauch]

On Sun, 30 Nov 2008 09:25:51 -0700
Joe Peterson <lavajoe@gentoo.org> wrote:

> Bottom line here is that there is extremely valuable and critical info
> in our emerge output.  In a way, these messages are like
> Gentoo-specific READMEs (or release notes and/or install
> instructions).  However, it is not saved for a user to use as a
> resource later (well, except that it is partially saved in the master
> emerge.log, but that's not quite useful enough).  There is no
> "official" place to go to look for Gentoo
> instructions; /usr/share/doc is one logical place, but it only
> contains files actually installed, not the notes output by emerge
> (and these are usually upstream-supplied, not Gentoo).
> 
> I propose that, upon merging a package, we save the emerge messages in
> either: 1) a package-specific file that resides somewhere "official"
> or
> 2) in the portage DB, so that the messages can be re-read via a
> portage utility.  In the latter case, either a new option to "equery"
> or a new "q" command (e.g. "equery readme <pkg>" or "qreadme <pkg>"
> could retrieve the text).

By default, messages generated by elog, ewarn and eerror are recorded
in /var/log/portage/elog/summary.log (emerge.log is just a
transaction log, so best to ignore it here). einfo isn't recorded on
purpose as it isn't intended for important information (that's the
purpose of elog). There are some tools available to simplify reading
these messages, and there several additional/alternative delivery
modules available (by mail, IM or in package specific files),
customizable via POTAGE_ELOG_* variables. Don't know if you just
haven't been aware of this, or if you're asking for something
completely different.

Marius

Re: [gentoo-dev] [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Joe Peterson]

Marius Mauch wrote:
> By default, messages generated by elog, ewarn and eerror are recorded
> in /var/log/portage/elog/summary.log (emerge.log is just a
> transaction log, so best to ignore it here). einfo isn't recorded on
> purpose as it isn't intended for important information (that's the
> purpose of elog). There are some tools available to simplify reading
> these messages, and there several additional/alternative delivery
> modules available (by mail, IM or in package specific files),
> customizable via POTAGE_ELOG_* variables. Don't know if you just
> haven't been aware of this, or if you're asking for something
> completely different.

I'm really proposing something different - in essence, the above is to
obscure to really serve as a good official kind of readme source for
users.  There needs to be something simple and straightforward (and
well-documented) as the official thing to look at if one is having
trouble with a package.  In the case I mentioned all it took was for
that user to see the messages, but it did not occur to him that the info
would be there.  I could even imagine that einfo should be included in
what I am suggesting, since it may not be important for logging, but
might be nice to have, nonetheless.

						-Joe

Re: [gentoo-dev] [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Peter Volkov]

Seems that we already have everything you dreamed about: 

http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=3&chap=1#doc_chap4

Take a look at PORTAGE_ELOG_SYSTEM. It even can send that messages by
mail :)

HTH,
--
Peter.

В Вск, 30/11/2008 в 09:25 -0700, Joe Peterson пишет:
> Bottom line here is that there is extremely valuable and critical info
> in our emerge output.  In a way, these messages are like Gentoo-specific
> READMEs (or release notes and/or install instructions).  However, it is
> not saved for a user to use as a resource later (well, except that it is
> partially saved in the master emerge.log, but that's not quite useful
> enough).  There is no "official" place to go to look for Gentoo
> instructions; /usr/share/doc is one logical place, but it only contains
> files actually installed, not the notes output by emerge (and these are
> usually upstream-supplied, not Gentoo).
> 
> I propose that, upon merging a package, we save the emerge messages in
> either: 1) a package-specific file that resides somewhere "official" or
> 2) in the portage DB, so that the messages can be re-read via a portage
> utility.  In the latter case, either a new option to "equery" or a new
> "q" command (e.g. "equery readme <pkg>" or "qreadme <pkg>" could
> retrieve the text).
> 
> In either case, there would then be a place to go that is known and
> consistent (and can be documented in the Gentoo doc).  It could, in
> essense, serve as a kind of "Gentoo package README" collection.  I could
> also imagine later expanding on this by letting a given package also
> include more thorough README info from a file if the maintainer so desires.

Re: [gentoo-dev] [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Joe Peterson]

Peter Volkov wrote:
> Seems that we already have everything you dreamed about: 
> 
> http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=3&chap=1#doc_chap4
> 
> Take a look at PORTAGE_ELOG_SYSTEM. It even can send that messages by
> mail :)

This is all cool, indeed!  :)

I suspect, however, that most users have never played with these
variables.  I think that saving this info in the portage db or making it
more default/official in some way could be a great help.  The core
problem is, I think, that many users do not know where to look when
having trouble, so they may not even realize that what they need is in
the log info.

The reason I was phrasing it more in "readme" terms is that most people
can identify with that concept, and it could be made clear that there
exists Gentoo-specific readme info that is always available (regardless
of whether a user sets up the portage logging stuff).  The bare log
messages could exist as a minimal default for packages that do nothing
special to provide more readme info.

					-Joe

[gentoo-dev] Re: [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Ryan Hill]

[-- Attachment #1: Type: text/plain, Size: 1457 bytes --]

On Sun, 30 Nov 2008 10:11:49 -0700
Joe Peterson <lavajoe@gentoo.org> wrote:

> Peter Volkov wrote:
> > Seems that we already have everything you dreamed about: 
> > 
> > http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=3&chap=1#doc_chap4
> > 
> > Take a look at PORTAGE_ELOG_SYSTEM. It even can send that messages
> > by mail :)
> 
> This is all cool, indeed!  :)
> 
> I suspect, however, that most users have never played with these
> variables.  I think that saving this info in the portage db or making
> it more default/official in some way could be a great help.  The core
> problem is, I think, that many users do not know where to look when
> having trouble, so they may not even realize that what they need is in
> the log info.

How more official can you get?  :P  By default we do save the logs, and
we provide a complete logging facility that can even log to syslog,
mail them to you, or run arbitrary commands.  We link to the
build log on build failure.  We reprint all log messages at the end of
the emerge by default.  If the user ignores these, and doesn't read the
manual, then...

I think educating the user about systems we already have in place beats
adding new ones.


-- 
gcc-porting,                                      by design, by neglect
treecleaner,                              for a fact or just for effect
wxwidgets @ gentoo     EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-dev] [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Alec Warner]

On Sun, Nov 30, 2008 at 9:11 AM, Joe Peterson <lavajoe@gentoo.org> wrote:
> Peter Volkov wrote:
>> Seems that we already have everything you dreamed about:
>>
>> http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=3&chap=1#doc_chap4
>>
>> Take a look at PORTAGE_ELOG_SYSTEM. It even can send that messages by
>> mail :)
>
> This is all cool, indeed!  :)
>
> I suspect, however, that most users have never played with these
> variables.  I think that saving this info in the portage db or making it
> more default/official in some way could be a great help.  The core
> problem is, I think, that many users do not know where to look when
> having trouble, so they may not even realize that what they need is in
> the log info.

I suspect that no one really disagrees with more communication; but I
imagine many are not willing to put time into it.

So I suggest you come up with better ideas to communicate to users
about the existing logging solutions and then
implement them ;)

The gentoo homepage is one way, the GMN is another, Forums is a third.
 Just write one article about it and publish it everywhere.
Also Gentoo-Wiki ;)

-Alec

>
> The reason I was phrasing it more in "readme" terms is that most people
> can identify with that concept, and it could be made clear that there
> exists Gentoo-specific readme info that is always available (regardless
> of whether a user sets up the portage logging stuff).  The bare log
> messages could exist as a minimal default for packages that do nothing
> special to provide more readme info.
>
>                                        -Joe
>
>

Re: [gentoo-dev] [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Dale]

[-- Attachment #1: Type: text/plain, Size: 1645 bytes --]

Joe Peterson wrote:
> Peter Volkov wrote:
>   
>> Seems that we already have everything you dreamed about: 
>>
>> http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=3&chap=1#doc_chap4
>>
>> Take a look at PORTAGE_ELOG_SYSTEM. It even can send that messages by
>> mail :)
>>     
>
> This is all cool, indeed!  :)
>
> I suspect, however, that most users have never played with these
> variables.  I think that saving this info in the portage db or making it
> more default/official in some way could be a great help.  The core
> problem is, I think, that many users do not know where to look when
> having trouble, so they may not even realize that what they need is in
> the log info.
>
> The reason I was phrasing it more in "readme" terms is that most people
> can identify with that concept, and it could be made clear that there
> exists Gentoo-specific readme info that is always available (regardless
> of whether a user sets up the portage logging stuff).  The bare log
> messages could exist as a minimal default for packages that do nothing
> special to provide more readme info.
>
> 					-Joe
>
>
>   

If you have a GUI on your system, give this a look: 
app-portage/elogviewer  That should help you a lot.  I been using it for
a good while and it works pretty well.  I do wish it had little flags in
the list of packages that have been installed.  Sort of a short and
sweet  notice there is something there without actually have to look. 
Maybe a red flag when there is something really serious to know and
other colors for other things.

Anyway, give that a look and see if that helps, if you have  a GUI.

Dale

:-)  :-) 

[-- Attachment #2: Type: text/html, Size: 2217 bytes --]

Re: [gentoo-dev] [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Ben de Groot]

[-- Attachment #1: Type: text/plain, Size: 1458 bytes --]

Joe Peterson wrote:
> Peter Volkov wrote:
>> Seems that we already have everything you dreamed about: 
>>
>> http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=3&chap=1#doc_chap4
>>
>> Take a look at PORTAGE_ELOG_SYSTEM. It even can send that messages by
>> mail :)
> 
> This is all cool, indeed!  :)
> 
> I suspect, however, that most users have never played with these
> variables.  I think that saving this info in the portage db or making it
> more default/official in some way could be a great help.  The core
> problem is, I think, that many users do not know where to look when
> having trouble, so they may not even realize that what they need is in
> the log info.

The info is there, but most users never read more than part 1 of the
Handbook (that is, the installation part). We could, and should in my
opinion, add a big fat warning towards the end of the installation part,
that there is extremely useful information to be found in the other
parts of the Handbook. Maybe we could especially mention some of the
more useful topics, and the elog system would be one of them.

-- 
Ben de Groot
Gentoo Linux developer (lxde, media, desktop-misc)
Gentoo Linux Release Engineering PR liaison
__________________________________________________

yngwin@gentoo.org
http://ben.liveforge.org/
irc://chat.freenode.net/#gentoo-media
irc://irc.oftc.net/#lxde
__________________________________________________



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

[gentoo-dev] Re: [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Duncan]

Ben de Groot <yngwin@gentoo.org> posted 493355A7.90503@gentoo.org,
excerpted below, on  Mon, 01 Dec 2008 04:10:31 +0100:

> The info is there, but most users never read more than part 1 of the
> Handbook (that is, the installation part). We could, and should in my
> opinion, add a big fat warning towards the end of the installation part,
> that there is extremely useful information to be found in the other
> parts of the Handbook. Maybe we could especially mention some of the
> more useful topics, and the elog system would be one of them.

Well, at the end of the Handbook, Pt 1, Installation, in Chapter 12, 
Where to go from here, it already mentions Pt 2, Working with Gentoo.  It 
really should mention Pts 3 & 4, Working with Portage and Gentoo Network 
Configuration, as well, the chapter of interest here of course being in 
Working with Portage.

So yes, we really could improve the end of the Handbook, pt 1, Where to 
go from here, having it mention Pt 3 & 4 as well as Pt 2.  That's 
something we can and should do, absolutely.

Beyond that, however, Gentoo has never been about hand-holding.  It 
expects you to be big enough to cross the street on your own without 
further hand-holding if it provides the stop light telling you when it's 
safe to do so; to be able to find and read the documentation, which 
Gentoo does have a generally excellent reputation in the community for 
providing, on your own.  There are plenty of other distributions out 
there for those who prefer to let the distribution make the decisions and 
take the responsibility.  Gentoo has always been about giving the user 
the ability to decide and configure that for himself, after reading the 
documentation where necessary.  If the user can't do that after we've 
gone to all the work of providing both the means and the documentation on 
configuring, right there in the official handbook even, with links and 
references to the handbook quite well distributed already, well, maybe 
that user really /should/ be looking at a different distribution.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[gentoo-dev] Re: [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Christian Faulhammer]

[-- Attachment #1: Type: text/plain, Size: 859 bytes --]

Hi,

Dale <rdalek1967@gmail.com>:
> If you have a GUI on your system, give this a look: 
> app-portage/elogviewer  That should help you a lot.  I been using it
> for a good while and it works pretty well.  I do wish it had little
> flags in the list of packages that have been installed.  Sort of a
> short and sweet  notice there is something there without actually
> have to look. Maybe a red flag when there is something really serious
> to know and other colors for other things.

 app-portage/elogv (ncurses) and app-portage/kelogviewer (Qt based) are
really nice, too.  Unfortunately the two GUI variants are homeless, so
improvements won't happen from the original upstream.

V-Li

-- 
Christian Faulhammer, Gentoo Lisp project
<URL:http://www.gentoo.org/proj/en/lisp/>, #gentoo-lisp on FreeNode

<URL:http://www.faulhammer.org/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-dev] [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Gilles Dartiguelongue]

[-- Attachment #1: Type: text/plain, Size: 901 bytes --]

Summarizing from what I've read in this thread it seems you want to find
a way to help user find information s/he doesn't look for. 

If users aren't curious about their system they will sure have a hard
time figuring out how to fix it if needs be. PORTAGE_ELOG_* isn't really
that hard to find in the make.conf.example (even though it's new
location makes it a bit harder to find).

As others have said, there are already proper systems, documentation and
linking through other docs. Not finding this is what I'd call lazyness
or lack of google foo. Don't misunderstand me, some stuff can get ouf of
the radar of everyone, it's ok, real people are still here to point you
in the right direction.

If you find a better way to convey these information to the users, then
please surprise me. For now I think we are in a good shape.

-- 
Gilles Dartiguelongue <eva@gentoo.org>
Gentoo

[-- Attachment #2: Ceci est une partie de message numériquement signée --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-dev] [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Joe Peterson]

Gilles Dartiguelongue wrote:
> As others have said, there are already proper systems, documentation and
> linking through other docs. Not finding this is what I'd call lazyness
> or lack of google foo. Don't misunderstand me, some stuff can get ouf of
> the radar of everyone, it's ok, real people are still here to point you
> in the right direction.

I think that I probably did not express my idea as well as I could have, since
most of the responses I have gotten have echoed your thoughts that Gentoo
does, indeed, have the facilities to achieve flexibility in logging, etc.

I totally agree.  Gentoo's capabilities, although not perfect, of course, are
superlative and are a complement to its superb online doc.  I think that's a
big reason why we're all here - we see this and appreciate this.  In fact,
even when I do not include the word "gentoo" in a Google search, I more often
than not end up at a Gentoo doc page - this is impressive.

However, what I see as perhaps a missing "piece" is more conceptual: the
important connection between the valuable info in the emerge logs (and their
somewhat transient default nature) and what a user looks for when he/she has a
problem with a package.  Yes, users will realize this as they use Gentoo (and
will start paying more attention to logs as a result), so I don't think it's a
huge problem, but what this particular user said to me made me think that
there is, perhaps, an opportunity to improve the situation.

There is no Gentoo-specific "readme" facility, which could be the obvious and
de facto place to go when trouble is had.  I can imagine that a fairly simple
and low-effort way of starting such a resource would be to simple echo the log
output into a package-specific file in a known place (or put it in the portage
db).  The logging facilities allow similar things if configured to do it, but
it is not on by default.  Once users know where to go to see the
"instructions" or "notes" on getting a package up and running after
installation, this would become a good place to have such info or to expand on
how the facility works.  Starting with just the plain emerge log output would
be an easy way to get benefit of such a concept has merit.  And by no means
would such a thing be an attempt to replace the excellent on-line docs or
wiki, either - I see both as having unique strengths.  For example, for
detailed info on packages, the wiki/web stuff is the better resource.  For a
quick check of whether a revdep-rebuild might have been necessary after
installing a new package would typically be in the log/notes.  The notes also
have the key advantage that they would *always* contain what the log output
was, whereas whether a wiki or web page exists on a particular package depends
on whether someone spent the time to author one.

My intention with the RFC was to see if the concept has any worth and to kick
it around a bit.  I do not really see this as a deficiency in Gentoo's
technology (which I have a feeling is how many here have interpreted it), but
simply something that, if done correctly, could be useful.

						-Joe

Re: [gentoo-dev] [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Marius Mauch]

On Mon, 01 Dec 2008 15:35:32 -0700
Joe Peterson <lavajoe@gentoo.org> wrote:

> My intention with the RFC was to see if the concept has any worth and
> to kick it around a bit.  I do not really see this as a deficiency in
> Gentoo's technology (which I have a feeling is how many here have
> interpreted it), but simply something that, if done correctly, could
> be useful.

Maybe provide a real example to demonstrate the difference between the
current solutions and what you're looking for, because I still don't
understand what you're after (using all the different terms, logs,
notes, docs, "plain emerge log", ... without further explanation doesn't
help much to clear things up).

Marius

Re: [gentoo-dev] [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Donnie Berkholz]

[-- Attachment #1: Type: text/plain, Size: 956 bytes --]

On 15:35 Mon 01 Dec     , Joe Peterson wrote:
> However, what I see as perhaps a missing "piece" is more conceptual: the
> important connection between the valuable info in the emerge logs (and their
> somewhat transient default nature) and what a user looks for when he/she has a
> problem with a package.  Yes, users will realize this as they use Gentoo (and
> will start paying more attention to logs as a result), so I don't think it's a
> huge problem, but what this particular user said to me made me think that
> there is, perhaps, an opportunity to improve the situation.

Based on the rarity of me seeing this reported as a problem, I'm 
inclined to think it says more about this user than about our system. I 
don't think it's our responsibility to put documentation everywhere 
someone might conceivably look for information.

-- 
Thanks,
Donnie

Donnie Berkholz
Developer, Gentoo Linux
Blog: http://dberkholz.wordpress.com

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: [gentoo-dev] [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Joe Peterson]

Donnie Berkholz wrote:
> On 15:35 Mon 01 Dec     , Joe Peterson wrote:
>> However, what I see as perhaps a missing "piece" is more conceptual: the
>> important connection between the valuable info in the emerge logs (and their
>> somewhat transient default nature) and what a user looks for when he/she has a
>> problem with a package.  Yes, users will realize this as they use Gentoo (and
>> will start paying more attention to logs as a result), so I don't think it's a
>> huge problem, but what this particular user said to me made me think that
>> there is, perhaps, an opportunity to improve the situation.
> 
> Based on the rarity of me seeing this reported as a problem, I'm 
> inclined to think it says more about this user than about our system.

This could very-well be.  However:

> I don't think it's our responsibility to put documentation everywhere 
> someone might conceivably look for information.

I agree with this statement, but I wasn't implying we should duplicate
information everywhere.  I wanted to explore this as an opportunity to
re-think if having an official "de facto" spot for "gentoo readmes"
would make sense, thereby saving log output in a useful place where
users would learn to look regularly.  I agree this would only be
reasonable if it were the right thing, architecturally, for Gentoo, not
just for this one user's issue.

						-Joe

[gentoo-dev] Re: [RFC] Saving package emerge output (einfo, elog, ewarn, etc.) somewhere official

[Duncan]

Joe Peterson <lavajoe@gentoo.org> posted 4946DD7F.80408@gentoo.org,
excerpted below, on  Mon, 15 Dec 2008 15:43:11 -0700:

>> I don't think it's our responsibility to put documentation everywhere
>> someone might conceivably look for information.
> 
> I agree with this statement, but I wasn't implying we should duplicate
> information everywhere.  I wanted to explore this as an opportunity to
> re-think if having an official "de facto" spot for "gentoo readmes"
> would make sense, thereby saving log output in a useful place where
> users would learn to look regularly.  I agree this would only be
> reasonable if it were the right thing, architecturally, for Gentoo, not
> just for this one user's issue.

That suggests an interesting possibility to me.  Let's see if anyone else 
likes it.

What about some place on the web, maybe on packages, having a single 
location with all the standard messages, listed by package and version.  
I'm picturing a single reference location where someone can look up the 
package and see all the routine postinst messages, etc, that it spits 
out.  Perhaps treat metapackages such that they group together all the 
messages from the collected sub-packages.

Then when we (as users) think about a big upgrade, we can go and research 
just what sort of thing the package maintainers already anticipate, and 
can thus better prepare ourselves.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman