[Fwd: Re: [gentoo-dev] Package notices]

[Fwd: Re: [gentoo-dev] Package notices]

[Eldad Zack]

[-- Attachment #1: Type: text/plain, Size: 2028 bytes --]

On Thu, 2004-09-02 at 21:56, Alexander Gretencord wrote:
> On Thursday 02 September 2004 20:46, Christian Gut wrote:
> > another idea: simply hand them over to syslog. This way they get mailed
> > to the admin and are there for later reference.
> 
> Just read Bug #11359 (http://bugs.gentoo.org/show_bug.cgi?id=11359). This has 
> also been discussed there and is only an option for already installed 
> systems.

> In short: The basic mechnism has to work without eMail and without syslog or 
> any other fancy stuff. Just think of a bootstrap build, where no such thing 
> is available.

What I had in mind could be deployed today, without needing to patch
portage.
Basically, it could be implemented using a new eclass or just adding the
enotice function to eutils - I wouldn't want all the einfos logged,
anyway. (patching notices? no thanks.)

What I would like, would be messages from packages like cacti.
enotice itself will write into the file and emit an einfo.

This is what I had in mind:

# void enotice(char* message)
#
#    write informative message (with a newline) into notice log.
#    also emits einfo.
#
enotice() {
	einfo "${*}"

	if [ -n "${ENOTICE_DIR}" ]
	then
		# Not checking if it ${ENOTICE_DIR} exists since
		# install does stat and stops if it exists.
		install ${ENOTICE_DIR}

		echo "${*}" >> ${ENOTICE_DIR}/${PF}
	fi

        return 0
}

To make it work it only requires that ENOTICE_DIR would be set in
make.conf.

It is also sandbox-friendly, since it is a regular file as far as
portage concerns - and that also makes it binary-friendly. The notices
will be included in the binary tbz2.

The external utility to read the messages is also pretty
straight-forward to write.

I'd like to hear from other developers if they think it is useful and if
they plan to use it...
I would also like to go ahead and use it for any ebuilds I currently
maintain.

-- 

Eldad Zack <eldad@gentoo.org>
Key/Fingerprint at pgp.mit.edu, ID 0x96EA0A93

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [Fwd: Re: [gentoo-dev] Package notices]

[Chris Gianelloni]

[-- Attachment #1: Type: text/plain, Size: 2329 bytes --]

On Thu, 2004-09-02 at 16:12, Eldad Zack wrote:
> On Thu, 2004-09-02 at 21:56, Alexander Gretencord wrote:
> > On Thursday 02 September 2004 20:46, Christian Gut wrote:
> > > another idea: simply hand them over to syslog. This way they get mailed
> > > to the admin and are there for later reference.
> > 
> > Just read Bug #11359 (http://bugs.gentoo.org/show_bug.cgi?id=11359). This has 
> > also been discussed there and is only an option for already installed 
> > systems.
> 
> > In short: The basic mechnism has to work without eMail and without syslog or 
> > any other fancy stuff. Just think of a bootstrap build, where no such thing 
> > is available.
> 
> What I had in mind could be deployed today, without needing to patch
> portage.
> Basically, it could be implemented using a new eclass or just adding the
> enotice function to eutils - I wouldn't want all the einfos logged,
> anyway. (patching notices? no thanks.)
> 
> What I would like, would be messages from packages like cacti.
> enotice itself will write into the file and emit an einfo.
> 
> This is what I had in mind:
> 
> # void enotice(char* message)
> #
> #    write informative message (with a newline) into notice log.
> #    also emits einfo.
> #
> enotice() {
> 	einfo "${*}"
> 
> 	if [ -n "${ENOTICE_DIR}" ]
> 	then
> 		# Not checking if it ${ENOTICE_DIR} exists since
> 		# install does stat and stops if it exists.
> 		install ${ENOTICE_DIR}
> 
> 		echo "${*}" >> ${ENOTICE_DIR}/${PF}
> 	fi
> 
>         return 0
> }
> 
> To make it work it only requires that ENOTICE_DIR would be set in
> make.conf.
> 
> It is also sandbox-friendly, since it is a regular file as far as
> portage concerns - and that also makes it binary-friendly. The notices
> will be included in the binary tbz2.
> 
> The external utility to read the messages is also pretty
> straight-forward to write.
> 
> I'd like to hear from other developers if they think it is useful and if
> they plan to use it...
> I would also like to go ahead and use it for any ebuilds I currently
> maintain.

It needs to be able to support the 3 levels of enotices: einfo, ewarn,
and ecrit.

-- 
Chris Gianelloni
Release Engineering - Operations/QA Manager
Games - Developer
Gentoo Linux

Is your power animal a penguin?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [Fwd: Re: [gentoo-dev] Package notices]

[Thomas de Grenier de Latour]

On Thu, 02 Sep 2004 23:12:15 +0300
Eldad Zack <eldad@gentoo.org> wrote:

> Basically, it could be implemented using a new eclass or just
> adding the enotice function to eutils

I don't get your point. Your solution would require patching
hundreds of ebuilds, just to avoid patching portage? Imho, this is
a very wrong approach.

> I wouldn't want all the einfos logged, anyway. (patching
> notices? no thanks.) 

Sure, but this messages are typically the ones from a few
eclasses. That's the ones that should be changed (by using some
variants of e{info,warn,etc.} that don't write to the log files),
it would be much less work. At the contrary, I think almost all
e{info,warn,etc.} from ebuilds are there to be read and should be
in the logs.

-- 
TGL.

--
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Package notices

[Nicholas Jones]

[-- Attachment #1: Type: text/plain, Size: 1732 bytes --]

> Basically, it could be implemented using a new eclass or just adding the
> enotice function to eutils - I wouldn't want all the einfos logged,
> anyway. (patching notices? no thanks.)

> What I would like, would be messages from packages like cacti.
> enotice itself will write into the file and emit an einfo.

I mentioned what I would consider most acceptable at some point,
but no one appears to have seen that, so it must have gotten lost.

He's what needs to get done for this to work:

functions.sh (from baselayout) dependence needs to go away.
All used functions need to be logged/rewritten to not use those
functions, and instead maintain its own.

All said functions must be rewritten, for portage, with the
capability to handle colors and terminals and the like as
portage does currently. (People don't want ANSI in their logs.)

The output can be writting in any number of ways to a file in
${T} during the execution of the build. This does not violate
sandbox and is quite easy to find.

PORTAGE can then manipulate the log, perform notifications,
and do whatever else is necessary to maintain that file around
cleanings and other merges.

${T}/notices.${PF}.${TYPE}
would be perfectly fine as long as it was definitive/unique
to the particular package. Notices of all types could also
simply be prefixed. '*' '-' '!' or whatever seems best.

If you are so inclined, you could also write a parser for
these files that can designate priorities using color or
some other visual aid. (Keep in mind that it must be
deterministic through simple ascii as well for synthesisers.)
For multiple reasons, I prefer python for these tools.

There are many, many possibilities, but please follow
the outline provided here.

--NJ


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

[gentoo-dev] Package notices

[Pablo Villalba]

We're trying to build the house without having the plans for it yet.
What do we need to do here? Which kind of output would be useful? Imho  
maintaining regular output (maybe flushing it at the end of all the emerge  
process) AND logging it to some special logfile (errors being notices to  
syslog too) would do it.
Once this is clear and agreed we could think of how we'd have to implement  
it.
- micho

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Package notices

[Jason Stubbs]

On Friday 03 September 2004 23:00, Pablo Villalba wrote:
> We're trying to build the house without having the plans for it yet.
> What do we need to do here? Which kind of output would be useful? Imho
> maintaining regular output (maybe flushing it at the end of all the emerge
> process) AND logging it to some special logfile (errors being notices to
> syslog too) would do it.
> Once this is clear and agreed we could think of how we'd have to implement
> it.

Nick layed out half the plan already, although it may not have seemed like it 
- he's very curt but it's usually enough if you know the context.

On Friday 03 September 2004 14:10, Nicholas Jones wrote:
> He's what needs to get done for this to work:
>
> functions.sh (from baselayout) dependence needs to go away.
> All used functions need to be logged/rewritten to not use those
> functions, and instead maintain its own.

This must happen so that there is absolutely no (chance of) effect on starting 
up messages that happen when starting and stopping daemons. Currently, the 
same functions are used for output.

> All said functions must be rewritten, for portage, with the
> capability to handle colors and terminals and the like as
> portage does currently. (People don't want ANSI in their logs.)

Portage has the ability to enable/disable colour based on the terminal type 
and user's preference. Furthermore, if the output is being logged as well as 
going to screen, coloured and non-coloured versions of the output must be 
providable on demand.

> The output can be writting in any number of ways to a file in
> ${T} during the execution of the build. This does not violate
> sandbox and is quite easy to find.

This is a simple design descision that fits in well with portage's current 
security model for merges.

> PORTAGE can then manipulate the log, perform notifications,
> and do whatever else is necessary to maintain that file around
> cleanings and other merges.

In other words, any processing done on the logs once they're written must be 
completely separate from the logging itself.

> ${T}/notices.${PF}.${TYPE}
> would be perfectly fine as long as it was definitive/unique
> to the particular package. Notices of all types could also
> simply be prefixed. '*' '-' '!' or whatever seems best.
>
> If you are so inclined, you could also write a parser for
> these files that can designate priorities using color or
> some other visual aid. (Keep in mind that it must be
> deterministic through simple ascii as well for synthesisers.)
> For multiple reasons, I prefer python for these tools.
>
> There are many, many possibilities, but please follow
> the outline provided here.

These just show further design possibilities within the above rules but it is 
open to whomever decides to implement it on the exact details.

Regards,
Jason Stubbs

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Package notices

[Eldad Zack]

[-- Attachment #1: Type: text/plain, Size: 2065 bytes --]

On Fri, 2004-09-03 at 08:10, Nicholas Jones wrote:

> functions.sh (from baselayout) dependence needs to go away.
> All used functions need to be logged/rewritten to not use those
> functions, and instead maintain its own.

I propose overriding einfo/... after sourcing functions.sh in ebuild.sh.

> All said functions must be rewritten, for portage, with the
> capability to handle colors and terminals and the like as
> portage does currently. (People don't want ANSI in their logs.)

Logfiles are handled seperately while screen output uses ANSI.
If portage could relay the color decision onwards to bash, it would
take only a small change to enforce that.

> ${T}/notices.${PF}.${TYPE}

I'd rather use a single file - notices.${PF} as the code shows:
(should be inserted in ebuild.sh after functions.sh is sourced)

### Code start ###

einfo() {
        enotice info "${*}"
}

ewarn() {
        enotice warn "${*}"
}

eerror()
{
        enotice error "${*}"
}

enotice()
{
        local color
        local type

        type="${1}"

        case "${type}" in
                "info")
                        color="${GOOD}"
                        ;;
                "warn")
                        color="${WARN}"
                        ;;
                "error")
                        color="${BAD}"
                        ;;
        esac

        shift

        echo "[${type}] ${*}" >> ${T}/notices.${PF}
        echo " ${color}*${NORMAL} ${*}"

}

### Code End ###

You can also merge the notice file into the package inself:
(inside dyn_install, just after calling src_install)

	#enotice
        if [ -n ${PORT_ENOTICE_DIR} && -r ${T}/notices.${PF} ]
        then
                install -d ${D}${PORT_ENOTICE_DIR}
                cp ${T}/notices.${PF} ${D}${PORT_ENOTICE_DIR}
        fi

or any other way you think is best.

If this is acceptable, I can write a matching notice reader.
-- 

Eldad Zack <eldad@gentoo.org>
Key/Fingerprint at pgp.mit.edu, ID 0x96EA0A93

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Package notices

[Jason Stubbs]

On Saturday 04 September 2004 00:10, Eldad Zack wrote:
> On Fri, 2004-09-03 at 08:10, Nicholas Jones wrote:
> > functions.sh (from baselayout) dependence needs to go away.
> > All used functions need to be logged/rewritten to not use those
> > functions, and instead maintain its own.
>
> I propose overriding einfo/... after sourcing functions.sh in ebuild.sh.

No, they really really need to be new commands. A few examples of why:

eutils.eclass:    einfo "Applying various patches (bugfixes/updates)..."
eutils.eclass:    einfo "${EPATCH_SINGLE_MSG}"
eutils.eclass:    einfo "Applying ${x##*/}..."
virtualx.eclass:  einfo "Scanning for a open DISPLAY to start Xvfb..."

mjpegtools/mjpegtools-1.6.0-r7.ebuild: einfo "Building mjpegtools"
drip/drip-0.9.0_alpha3.ebuild:         einfo "Rerunnig autoconf/automake..."
mplayer/mplayer-1.0_pre3-r5.ebuild:    einfo "Make"
mplayer/mplayer-1.0_pre3-r5.ebuild:    einfo "Make completed"
mplayer/mplayer-1.0_pre3-r5.ebuild:    einfo "Make install"
mplayer/mplayer-1.0_pre3-r5.ebuild:    einfo "Make install completed"

This is all stuff that is meaningless if the build succeeds. As far as I know, 
einfo and friends were not created for use in ebuilds and just began being 
used as it was convenient at the time.

Secondly, overriding is not good either. If you are overriding, it must be 
because the functions do different things. If they are doing different 
things, why call them the same thing? 

> > All said functions must be rewritten, for portage, with the
> > capability to handle colors and terminals and the like as
> > portage does currently. (People don't want ANSI in their logs.)
>
> Logfiles are handled seperately while screen output uses ANSI.
> If portage could relay the color decision onwards to bash, it would
> take only a small change to enforce that.
>
> > ${T}/notices.${PF}.${TYPE}
>
> I'd rather use a single file - notices.${PF} as the code shows:
> (should be inserted in ebuild.sh after functions.sh is sourced)
>
> ### Code start ###
>
> einfo() {
>         enotice info "${*}"
> }
>
> ewarn() {
>         enotice warn "${*}"
> }
>
> eerror()
> {
>         enotice error "${*}"
> }
>
> enotice()
> {
>         local color
>         local type
>
>         type="${1}"
>
>         case "${type}" in
>                 "info")
>                         color="${GOOD}"
>                         ;;
>                 "warn")
>                         color="${WARN}"
>                         ;;
>                 "error")
>                         color="${BAD}"
>                         ;;
>         esac
>
>         shift
>
>         echo "[${type}] ${*}" >> ${T}/notices.${PF}
>         echo " ${color}*${NORMAL} ${*}"
>
> }
>
> ### Code End ###
>
> You can also merge the notice file into the package inself:
> (inside dyn_install, just after calling src_install)
>
>  #enotice
>         if [ -n ${PORT_ENOTICE_DIR} && -r ${T}/notices.${PF} ]
>         then
>                 install -d ${D}${PORT_ENOTICE_DIR}
>                 cp ${T}/notices.${PF} ${D}${PORT_ENOTICE_DIR}
>         fi
>
> or any other way you think is best.
>
> If this is acceptable, I can write a matching notice reader.

The code itself is okay, but what functions are required needs to be 
discussed, IMO. Then explicit definitions of when each function is used needs 
to be given and each ebuild/eclass updated to use them.

If everyone really really disagrees, I guess you can just create one more 
function for the meaningless "pretty" stuff that gets outputted by ebuilds 
and eclasses and just convert those. However, be aware that you are adding 
another burden of backward compatibility that will only slow down portage's 
progress further.

Regards,
Jason Stubbs

--
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Package notices

[Eldad Zack]

[-- Attachment #1: Type: text/plain, Size: 2948 bytes --]

On Fri, 2004-09-03 at 17:15, Jason Stubbs wrote:
> On Saturday 04 September 2004 00:10, Eldad Zack wrote:
> > On Fri, 2004-09-03 at 08:10, Nicholas Jones wrote:
> > > functions.sh (from baselayout) dependence needs to go away.
> > > All used functions need to be logged/rewritten to not use those
> > > functions, and instead maintain its own.
> >
> > I propose overriding einfo/... after sourcing functions.sh in ebuild.sh.
> No, they really really need to be new commands. A few examples of why:

<snip>

> This is all stuff that is meaningless if the build succeeds. As far as I know, 
> einfo and friends were not created for use in ebuilds and just began being 
> used as it was convenient at the time.

My original thought was just that. I wouldn't give a wetslap about some
of the einfo's but some of them are really important.
I tried per-ebuild logging, but it doesn't cut it (for me) - just too
much cruft. I guess I could grep-hack around it... anyway, notices would
limit the messages to only the ebuild (and not the entire process).

Someone said earlier (Thomas) - "Your solution would require patching
hundreds of ebuilds, just to avoid patching portage? Imho, this is a
very wrong approach."

And it has a good point to it - For users to benefit from this change,
all the ebuilds need to me fixed, changing einfo/ewarn/eerror to their
new equiv's.

> Secondly, overriding is not good either. If you are overriding, it must be 
> because the functions do different things. If they are doing different 
> things, why call them the same thing? 

Not different as such - they do almost the same thing, just that the new
functions has an additional side-effect of outputting to a notice file
(and without color, so it is logfile friendly)

> The code itself is okay, but what functions are required needs to be 
> discussed, IMO. Then explicit definitions of when each function is used needs 
> to be given and each ebuild/eclass updated to use them.

Well, what are the needs beyond logging? The functionality can be added
very simply to this function. Including mailing out and the likes - All
the ebuild writer do is exactly what they do now, pass a message out.

> If everyone really really disagrees, I guess you can just create one more 
> function for the meaningless "pretty" stuff that gets outputted by ebuilds 
> and eclasses and just convert those. However, be aware that you are adding 
> another burden of backward compatibility that will only slow down portage's 
> progress further.

That would be a very ugly hack (== something you shouldn't add to
portage) - It would like wrapping around portage and parsing the
per-ebuild logs... or did you mean something else?

BTW, anyone who want to test it can look for the ebuild.sh patch in my
page (http://dev.gentoo.org/~eldad/).

-- 

Eldad Zack <eldad@gentoo.org>
Key/Fingerprint at pgp.mit.edu, ID 0x96EA0A93

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Package notices

[Eldad Zack]

[-- Attachment #1: Type: text/plain, Size: 667 bytes --]

On Fri, 2004-09-03 at 08:10, Nicholas Jones wrote:

> If you are so inclined, you could also write a parser for
> these files that can designate priorities using color or
> some other visual aid. (Keep in mind that it must be
> deterministic through simple ascii as well for synthesisers.)
> For multiple reasons, I prefer python for these tools.

I've whipped up a tool to do just that.
It can be found along the patch (which was slightly amended), on my
page:

http://dev.gentoo.org/~eldad/ebuild.sh-enotice.patch
http://dev.gentoo.org/~eldad/enotice

Comments?

-- 

Eldad Zack <eldad@gentoo.org>
Key/Fingerprint at pgp.mit.edu, ID 0x96EA0A93

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]