[gentoo-dev] Package notices

[gentoo-dev] Package notices

[Eldad Zack]

[-- Attachment #1: Type: text/plain, Size: 727 bytes --]

Hey Guys,

I've got an idea to implement package notices, as opposed to
outputing these important messages to stdout using e{info,warn,error}.

The idea is VERY simple: write the message to a file (say,
/var/portage/notices/{P}) and have a utility to read and delete these
messages. Maybe even build this functionality into etc-config.
(Maybe mail these messages to a configured address?)

For ease-of-use I propose adding a simple wrapper command ("enotice" or
"notice") that would take care of writing into the correct file.

If this looks like a good idea I'll move on and punch out the matching
code.

Comments? Ideas?

-- 

Eldad Zack <eldad@gentoo.org>
Key/Fingerprint at pgp.mit.edu, ID 0x96EA0A93

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Package notices

[William Hubbs]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, Sep 01, 2004 at 10:42:57PM +0300, Eldad Zack wrote:
> Hey Guys,
> 
> I've got an idea to implement package notices, as opposed to
> outputing these important messages to stdout using e{info,warn,error}.
> 
> The idea is VERY simple: write the message to a file (say,
> /var/portage/notices/{P}) and have a utility to read and delete these
> messages. Maybe even build this functionality into etc-config.
> (Maybe mail these messages to a configured address?)
> 
> For ease-of-use I propose adding a simple wrapper command ("enotice" or
> "notice") that would take care of writing into the correct file.
> 
> If this looks like a good idea I'll move on and punch out the matching
> code.
> 
> Comments? Ideas?

Before you get too invalved in writing code for this, take a look at this bug.

http://bugs.gentoo.org/show_bug.cgi?id=11359

The final patch I posted to ebuild.sh (comment #103) does the first part of this -- collecting the messages in a single file, and I have been told that something similar to this will be implemented after portage 2.0.51 is released.

William

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBNjmjblQW9DDEZTgRAguEAJ4kBtmw7Es0K3dslVrbBQ6RJg3viwCgkLTC
D0hAzTVfYvdRE/gQyfxLWcU=
=YO9G
-----END PGP SIGNATURE-----

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Package notices

[Thomas de Grenier de Latour]

On Wed, 1 Sep 2004 16:05:39 -0500
William Hubbs <williamh@gentoo.org> wrote:

> Before you get too invalved in writing code for this, take a
> look at this bug.
> 
> http://bugs.gentoo.org/show_bug.cgi?id=11359

You can also have a look on this one:
 http://bugs.gentoo.org/show_bug.cgi?id=37491
What you want is part of its features (when setting
PORT_SHORTLOGDIR in make.conf). It was last updated against
portage 2.0.50-r1 though, but if there is some interest for it, i
can try to find time to rework it. Anyway, even for my personal
use, i miss it since I had to stop maintaining it. Emerge is sooo
verbose...

-- 
TGL.

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Package notices

[purslow]

040901 Eldad Zack wrote:
> I've got an idea to implement package notices,
> as opposed to outputing these important messages to stdout
> The idea is VERY simple: write the message to a file
> -- say, /var/portage/notices/{P} --
> and have a utility to read and delete these messages.

as a mere user, i wd find this definitely helpful.
at present, important warnings etc can get lost in the copious Emerge output.

keep it simple: such msgs sb written to appropriately-named files
in some obvious subdirectory under  /usr/portage ;
leave it to the user to delete them whenever s/he decides;
make it an option in  /etc/make.conf .

-- 
========================,,============================================
SUPPORT     ___________//___,  Philip Webb : purslow@chass.utoronto.ca
ELECTRIC   /] [] [] [] [] []|  Centre for Urban & Community Studies
TRANSIT    `-O----------O---'  University of Toronto

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Package notices

[Anton Starikov]

At least they can be collected in some buffer and dumped to the output 
stream after process of emerging. It can be realy helpfull, when you 
emerge more than one package at time.

purslow@sympatico.ca wrote:
> 040901 Eldad Zack wrote:
> 
>>I've got an idea to implement package notices,
>>as opposed to outputing these important messages to stdout
>>The idea is VERY simple: write the message to a file
>>-- say, /var/portage/notices/{P} --
>>and have a utility to read and delete these messages.
> 
> 
> as a mere user, i wd find this definitely helpful.
> at present, important warnings etc can get lost in the copious Emerge output.
> 
> keep it simple: such msgs sb written to appropriately-named files
> in some obvious subdirectory under  /usr/portage ;
> leave it to the user to delete them whenever s/he decides;
> make it an option in  /etc/make.conf .
> 


--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Package notices

[Christian Gut]

[-- Attachment #1: Type: text/plain, Size: 802 bytes --]

On Wed, 2004-09-01 at 21:42, Eldad Zack wrote:
> Hey Guys,
> 
> I've got an idea to implement package notices, as opposed to
> outputing these important messages to stdout using e{info,warn,error}.
> 
> The idea is VERY simple: write the message to a file (say,
> /var/portage/notices/{P}) and have a utility to read and delete these
> messages. Maybe even build this functionality into etc-config.
> (Maybe mail these messages to a configured address?)
> 
> For ease-of-use I propose adding a simple wrapper command ("enotice" or
> "notice") that would take care of writing into the correct file.
>
> Comments? Ideas?

another idea: simply hand them over to syslog. This way they get mailed
to the admin and are there for later reference.

-- 
Christian Gut <cycloon@is-root.org>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Package notices

[Alexander Gretencord]

On Thursday 02 September 2004 20:46, Christian Gut wrote:
> another idea: simply hand them over to syslog. This way they get mailed
> to the admin and are there for later reference.

Just read Bug #11359 (http://bugs.gentoo.org/show_bug.cgi?id=11359). This has 
also been discussed there and is only an option for already installed 
systems.

In short: The basic mechnism has to work without eMail and without syslog or 
any other fancy stuff. Just think of a bootstrap build, where no such thing 
is available.


Alex

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Package notices

[N. Owen Gunden]

On Thu, Sep 02, 2004 at 12:46:37PM -0400, purslow@sympatico.ca wrote:
> as a mere user, i wd find this definitely helpful.
> at present, important warnings etc can get lost in the copious Emerge output.

I concur.  In fact, I was the one who first hacked (emphasis on hack :)
together a functions.sh patch to email warnings to root.

> keep it simple: such msgs sb written to appropriately-named files
> in some obvious subdirectory under  /usr/portage ;

Don't want to put it in /usr/portage, else it'll get syncd away!
Somewhere in /var sounds good to me.

 - O

--
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Package notices

[Nicholas Jones]

[-- Attachment #1: Type: text/plain, Size: 1732 bytes --]

> Basically, it could be implemented using a new eclass or just adding the
> enotice function to eutils - I wouldn't want all the einfos logged,
> anyway. (patching notices? no thanks.)

> What I would like, would be messages from packages like cacti.
> enotice itself will write into the file and emit an einfo.

I mentioned what I would consider most acceptable at some point,
but no one appears to have seen that, so it must have gotten lost.

He's what needs to get done for this to work:

functions.sh (from baselayout) dependence needs to go away.
All used functions need to be logged/rewritten to not use those
functions, and instead maintain its own.

All said functions must be rewritten, for portage, with the
capability to handle colors and terminals and the like as
portage does currently. (People don't want ANSI in their logs.)

The output can be writting in any number of ways to a file in
${T} during the execution of the build. This does not violate
sandbox and is quite easy to find.

PORTAGE can then manipulate the log, perform notifications,
and do whatever else is necessary to maintain that file around
cleanings and other merges.

${T}/notices.${PF}.${TYPE}
would be perfectly fine as long as it was definitive/unique
to the particular package. Notices of all types could also
simply be prefixed. '*' '-' '!' or whatever seems best.

If you are so inclined, you could also write a parser for
these files that can designate priorities using color or
some other visual aid. (Keep in mind that it must be
deterministic through simple ascii as well for synthesisers.)
For multiple reasons, I prefer python for these tools.

There are many, many possibilities, but please follow
the outline provided here.

--NJ


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Package notices

[Jason Stubbs]

On Friday 03 September 2004 23:00, Pablo Villalba wrote:
> We're trying to build the house without having the plans for it yet.
> What do we need to do here? Which kind of output would be useful? Imho
> maintaining regular output (maybe flushing it at the end of all the emerge
> process) AND logging it to some special logfile (errors being notices to
> syslog too) would do it.
> Once this is clear and agreed we could think of how we'd have to implement
> it.

Nick layed out half the plan already, although it may not have seemed like it 
- he's very curt but it's usually enough if you know the context.

On Friday 03 September 2004 14:10, Nicholas Jones wrote:
> He's what needs to get done for this to work:
>
> functions.sh (from baselayout) dependence needs to go away.
> All used functions need to be logged/rewritten to not use those
> functions, and instead maintain its own.

This must happen so that there is absolutely no (chance of) effect on starting 
up messages that happen when starting and stopping daemons. Currently, the 
same functions are used for output.

> All said functions must be rewritten, for portage, with the
> capability to handle colors and terminals and the like as
> portage does currently. (People don't want ANSI in their logs.)

Portage has the ability to enable/disable colour based on the terminal type 
and user's preference. Furthermore, if the output is being logged as well as 
going to screen, coloured and non-coloured versions of the output must be 
providable on demand.

> The output can be writting in any number of ways to a file in
> ${T} during the execution of the build. This does not violate
> sandbox and is quite easy to find.

This is a simple design descision that fits in well with portage's current 
security model for merges.

> PORTAGE can then manipulate the log, perform notifications,
> and do whatever else is necessary to maintain that file around
> cleanings and other merges.

In other words, any processing done on the logs once they're written must be 
completely separate from the logging itself.

> ${T}/notices.${PF}.${TYPE}
> would be perfectly fine as long as it was definitive/unique
> to the particular package. Notices of all types could also
> simply be prefixed. '*' '-' '!' or whatever seems best.
>
> If you are so inclined, you could also write a parser for
> these files that can designate priorities using color or
> some other visual aid. (Keep in mind that it must be
> deterministic through simple ascii as well for synthesisers.)
> For multiple reasons, I prefer python for these tools.
>
> There are many, many possibilities, but please follow
> the outline provided here.

These just show further design possibilities within the above rules but it is 
open to whomever decides to implement it on the exact details.

Regards,
Jason Stubbs

--
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Package notices

[Pablo Villalba]

We're trying to build the house without having the plans for it yet.
What do we need to do here? Which kind of output would be useful? Imho  
maintaining regular output (maybe flushing it at the end of all the emerge  
process) AND logging it to some special logfile (errors being notices to  
syslog too) would do it.
Once this is clear and agreed we could think of how we'd have to implement  
it.
- micho

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Package notices

[Jason Stubbs]

On Saturday 04 September 2004 00:10, Eldad Zack wrote:
> On Fri, 2004-09-03 at 08:10, Nicholas Jones wrote:
> > functions.sh (from baselayout) dependence needs to go away.
> > All used functions need to be logged/rewritten to not use those
> > functions, and instead maintain its own.
>
> I propose overriding einfo/... after sourcing functions.sh in ebuild.sh.

No, they really really need to be new commands. A few examples of why:

eutils.eclass:    einfo "Applying various patches (bugfixes/updates)..."
eutils.eclass:    einfo "${EPATCH_SINGLE_MSG}"
eutils.eclass:    einfo "Applying ${x##*/}..."
virtualx.eclass:  einfo "Scanning for a open DISPLAY to start Xvfb..."

mjpegtools/mjpegtools-1.6.0-r7.ebuild: einfo "Building mjpegtools"
drip/drip-0.9.0_alpha3.ebuild:         einfo "Rerunnig autoconf/automake..."
mplayer/mplayer-1.0_pre3-r5.ebuild:    einfo "Make"
mplayer/mplayer-1.0_pre3-r5.ebuild:    einfo "Make completed"
mplayer/mplayer-1.0_pre3-r5.ebuild:    einfo "Make install"
mplayer/mplayer-1.0_pre3-r5.ebuild:    einfo "Make install completed"

This is all stuff that is meaningless if the build succeeds. As far as I know, 
einfo and friends were not created for use in ebuilds and just began being 
used as it was convenient at the time.

Secondly, overriding is not good either. If you are overriding, it must be 
because the functions do different things. If they are doing different 
things, why call them the same thing? 

> > All said functions must be rewritten, for portage, with the
> > capability to handle colors and terminals and the like as
> > portage does currently. (People don't want ANSI in their logs.)
>
> Logfiles are handled seperately while screen output uses ANSI.
> If portage could relay the color decision onwards to bash, it would
> take only a small change to enforce that.
>
> > ${T}/notices.${PF}.${TYPE}
>
> I'd rather use a single file - notices.${PF} as the code shows:
> (should be inserted in ebuild.sh after functions.sh is sourced)
>
> ### Code start ###
>
> einfo() {
>         enotice info "${*}"
> }
>
> ewarn() {
>         enotice warn "${*}"
> }
>
> eerror()
> {
>         enotice error "${*}"
> }
>
> enotice()
> {
>         local color
>         local type
>
>         type="${1}"
>
>         case "${type}" in
>                 "info")
>                         color="${GOOD}"
>                         ;;
>                 "warn")
>                         color="${WARN}"
>                         ;;
>                 "error")
>                         color="${BAD}"
>                         ;;
>         esac
>
>         shift
>
>         echo "[${type}] ${*}" >> ${T}/notices.${PF}
>         echo " ${color}*${NORMAL} ${*}"
>
> }
>
> ### Code End ###
>
> You can also merge the notice file into the package inself:
> (inside dyn_install, just after calling src_install)
>
>  #enotice
>         if [ -n ${PORT_ENOTICE_DIR} && -r ${T}/notices.${PF} ]
>         then
>                 install -d ${D}${PORT_ENOTICE_DIR}
>                 cp ${T}/notices.${PF} ${D}${PORT_ENOTICE_DIR}
>         fi
>
> or any other way you think is best.
>
> If this is acceptable, I can write a matching notice reader.

The code itself is okay, but what functions are required needs to be 
discussed, IMO. Then explicit definitions of when each function is used needs 
to be given and each ebuild/eclass updated to use them.

If everyone really really disagrees, I guess you can just create one more 
function for the meaningless "pretty" stuff that gets outputted by ebuilds 
and eclasses and just convert those. However, be aware that you are adding 
another burden of backward compatibility that will only slow down portage's 
progress further.

Regards,
Jason Stubbs

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Package notices

[Eldad Zack]

[-- Attachment #1: Type: text/plain, Size: 2065 bytes --]

On Fri, 2004-09-03 at 08:10, Nicholas Jones wrote:

> functions.sh (from baselayout) dependence needs to go away.
> All used functions need to be logged/rewritten to not use those
> functions, and instead maintain its own.

I propose overriding einfo/... after sourcing functions.sh in ebuild.sh.

> All said functions must be rewritten, for portage, with the
> capability to handle colors and terminals and the like as
> portage does currently. (People don't want ANSI in their logs.)

Logfiles are handled seperately while screen output uses ANSI.
If portage could relay the color decision onwards to bash, it would
take only a small change to enforce that.

> ${T}/notices.${PF}.${TYPE}

I'd rather use a single file - notices.${PF} as the code shows:
(should be inserted in ebuild.sh after functions.sh is sourced)

### Code start ###

einfo() {
        enotice info "${*}"
}

ewarn() {
        enotice warn "${*}"
}

eerror()
{
        enotice error "${*}"
}

enotice()
{
        local color
        local type

        type="${1}"

        case "${type}" in
                "info")
                        color="${GOOD}"
                        ;;
                "warn")
                        color="${WARN}"
                        ;;
                "error")
                        color="${BAD}"
                        ;;
        esac

        shift

        echo "[${type}] ${*}" >> ${T}/notices.${PF}
        echo " ${color}*${NORMAL} ${*}"

}

### Code End ###

You can also merge the notice file into the package inself:
(inside dyn_install, just after calling src_install)

	#enotice
        if [ -n ${PORT_ENOTICE_DIR} && -r ${T}/notices.${PF} ]
        then
                install -d ${D}${PORT_ENOTICE_DIR}
                cp ${T}/notices.${PF} ${D}${PORT_ENOTICE_DIR}
        fi

or any other way you think is best.

If this is acceptable, I can write a matching notice reader.
-- 

Eldad Zack <eldad@gentoo.org>
Key/Fingerprint at pgp.mit.edu, ID 0x96EA0A93

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Package notices

[Eldad Zack]

[-- Attachment #1: Type: text/plain, Size: 2948 bytes --]

On Fri, 2004-09-03 at 17:15, Jason Stubbs wrote:
> On Saturday 04 September 2004 00:10, Eldad Zack wrote:
> > On Fri, 2004-09-03 at 08:10, Nicholas Jones wrote:
> > > functions.sh (from baselayout) dependence needs to go away.
> > > All used functions need to be logged/rewritten to not use those
> > > functions, and instead maintain its own.
> >
> > I propose overriding einfo/... after sourcing functions.sh in ebuild.sh.
> No, they really really need to be new commands. A few examples of why:

<snip>

> This is all stuff that is meaningless if the build succeeds. As far as I know, 
> einfo and friends were not created for use in ebuilds and just began being 
> used as it was convenient at the time.

My original thought was just that. I wouldn't give a wetslap about some
of the einfo's but some of them are really important.
I tried per-ebuild logging, but it doesn't cut it (for me) - just too
much cruft. I guess I could grep-hack around it... anyway, notices would
limit the messages to only the ebuild (and not the entire process).

Someone said earlier (Thomas) - "Your solution would require patching
hundreds of ebuilds, just to avoid patching portage? Imho, this is a
very wrong approach."

And it has a good point to it - For users to benefit from this change,
all the ebuilds need to me fixed, changing einfo/ewarn/eerror to their
new equiv's.

> Secondly, overriding is not good either. If you are overriding, it must be 
> because the functions do different things. If they are doing different 
> things, why call them the same thing? 

Not different as such - they do almost the same thing, just that the new
functions has an additional side-effect of outputting to a notice file
(and without color, so it is logfile friendly)

> The code itself is okay, but what functions are required needs to be 
> discussed, IMO. Then explicit definitions of when each function is used needs 
> to be given and each ebuild/eclass updated to use them.

Well, what are the needs beyond logging? The functionality can be added
very simply to this function. Including mailing out and the likes - All
the ebuild writer do is exactly what they do now, pass a message out.

> If everyone really really disagrees, I guess you can just create one more 
> function for the meaningless "pretty" stuff that gets outputted by ebuilds 
> and eclasses and just convert those. However, be aware that you are adding 
> another burden of backward compatibility that will only slow down portage's 
> progress further.

That would be a very ugly hack (== something you shouldn't add to
portage) - It would like wrapping around portage and parsing the
per-ebuild logs... or did you mean something else?

BTW, anyone who want to test it can look for the ebuild.sh patch in my
page (http://dev.gentoo.org/~eldad/).

-- 

Eldad Zack <eldad@gentoo.org>
Key/Fingerprint at pgp.mit.edu, ID 0x96EA0A93

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Package notices

[Eldad Zack]

[-- Attachment #1: Type: text/plain, Size: 667 bytes --]

On Fri, 2004-09-03 at 08:10, Nicholas Jones wrote:

> If you are so inclined, you could also write a parser for
> these files that can designate priorities using color or
> some other visual aid. (Keep in mind that it must be
> deterministic through simple ascii as well for synthesisers.)
> For multiple reasons, I prefer python for these tools.

I've whipped up a tool to do just that.
It can be found along the patch (which was slightly amended), on my
page:

http://dev.gentoo.org/~eldad/ebuild.sh-enotice.patch
http://dev.gentoo.org/~eldad/enotice

Comments?

-- 

Eldad Zack <eldad@gentoo.org>
Key/Fingerprint at pgp.mit.edu, ID 0x96EA0A93

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]