[gentoo-user] Automated "emerge -e world"

[gentoo-user] Automated "emerge -e world"

[Daniel Iliev]

Hi, list

I'd like to automate a full re-emerging and to get a record of all
packages that failed. Something like:

##

  emerge -e world || {

  echo "$CATEGORY/$PN" >>failed.txt

  while ! emerge --resume --skipfirst
    do
      echo "$CATEGORY/$PN" >> failed.txt
    done
  }

##


Any ideas how to achieve this?


-- 
Best regards,
Daniel
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Automated "emerge -e world"

[Albert Hopkins]

On Wed, 2007-09-12 at 03:04 +0300, Daniel Iliev wrote:
> Hi, list
> 
> I'd like to automate a full re-emerging and to get a record of all
> packages that failed. Something like:
> 
> ##
> 
>   emerge -e world || {
> 
>   echo "$CATEGORY/$PN" >>failed.txt
> 
>   while ! emerge --resume --skipfirst
>     do
>       echo "$CATEGORY/$PN" >> failed.txt
>     done
>   }
> 
> ##
> 
> 
> Any ideas how to achieve this?
> 
> 
> -- 
> Best regards,
> Daniel

Here's a script that I use.  I use it to build all my packages at the
end of the month.  Works ok for me.  YMMV:

---- CUT HERE 8<-----
#!/usr/bin/python

"""
Like my emptytree bash script, but more reliable and written in Python.
"""


import datetime
import os
import sys
import time

import portage


LOGDIR = '/var/log/emptytree'
PKGLIST = LOGDIR + '/packages'
FAILED = LOGDIR + '/failed'
EMERGE_FLAGS = '--oneshot --nodeps'
TCOLS = 80


def get_list_of_packages(args=None):
    """Get list of packages to emerge, in order"""

    packages = []

    if args is None:
        command = "emerge --nospinner --pretend --emptytree world"
    else:
        command = "emerge --nospinner --pretend " + args
    pipe = os.popen(command, 'r')

    for line in pipe:
        if line.startswith('[ebuild'):
            try:
                right_bracket = line.index(']')
                ebuild_start = right_bracket + 2
                space = line.index(' ', ebuild_start)
            except ValueError:
                continue

            ebuild = line[ebuild_start:space]
            packages.append(ebuild)
    return packages


class PackageList(list):
    """List of packages read/written to a flat file"""

    def __init__(self, filename):
        list.__init__(self)
        self.filename = filename


    def write_to_file(self):
        """Write package list file"""

        package_file = open(self.filename, 'w')
        for package in self:
            package_file.write(package + '\n')

        package_file.close()


    def read_from_file(self):
        """Load packages from file"""

        del self[0:-1] # clear
        try:
            package_file = open(self.filename, 'r')
        except OSError:
            return

        for line in package_file:
            package = line.strip()
            if package:
                self.append(package)


def set_links(current):
    """Create current.out current.err symlinks"""

    current_out = LOGDIR + '/current.stdout'
    current_err = LOGDIR + '/current.stderr'

    if os.path.exists(current_out):
        os.remove(current_out)
    if os.path.exists(current_err):
        os.remove(current_err)

    os.symlink('%s.stdout' % current, current_out)
    os.symlink('%s.stderr' % current, current_err)


def main(argv):
    """Main program entry point"""

    if len(argv) > 1:
        args = ' '.join(argv[1:])
    else:
        args = None
    packages = PackageList(PKGLIST)
    packages.extend(get_list_of_packages(args))
    failed = PackageList(FAILED)

    total = len(packages)
    packages.write_to_file()
    failed.write_to_file()

    start_time = time.gmtime()
    pkg_num = 0
    while len(packages) > 0:
        package = packages.pop(0)
        pkg_num = pkg_num + 1
        print '[%03d/%03d] %s' % (pkg_num, total, package),
        sys.stdout.flush()
        split = portage.pkgsplit(package)
        pname = split[0].split('/')[1] + '-' + split[1] + '-' + split[1]
        command = ('emerge %s =%s '
                '> %s/%s.stdout 2> %s/%s.stderr'
                % (EMERGE_FLAGS, package, LOGDIR, pname, LOGDIR, pname))
        #command = 'echo %s' % package
        set_links(pname)
        status = os.system(command)
        plen = len(package)
        if status == 0:
            print '[SUCCESS]'.rjust(TCOLS - 11 - plen)
        else:
            # emerge failed
            print '[FAILURE]'.rjust(TCOLS - 11 - plen)
            failed.append(package)
            failed.write_to_file()

        packages.write_to_file()
    end_time = time.gmtime()
    time_diff = datetime.datetime(*end_time[:6]) - \
            datetime.datetime(*start_time[:6])
    print 'Finished!'
    print 'Total: %d packages. %d failed' % (total, len(failed))
    print 'Completed in %s' % time_diff


if __name__ == '__main__':

    sys.exit(main(sys.argv))


# 2007-07-10 marduk Took out "resume" and "failed" command-line
#   arguments, since "resume" can be accomplished by
#   "emptytree `cat /var/log/emptytree/packages`" and failed by
#   "emptytree `cat /var/log/emptytree/failed`".
#   Removed "empty" contructor parameter from PackageList since it
#   was only used by "resume" and "failed"
------------------------------
-- 
raise SystemExit


-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Automated "emerge -e world"

[Daniel Iliev]

On Tue, 11 Sep 2007 19:52:55 -0500
Albert Hopkins <marduk@gentoo.org> wrote:

> On Wed, 2007-09-12 at 03:04 +0300, Daniel Iliev wrote:
> > Hi, list
> > 
> > I'd like to automate a full re-emerging and to get a record of all
> > packages that failed. Something like:
> > 
> > ##
> > 
> >   emerge -e world || {
> > 
> >   echo "$CATEGORY/$PN" >>failed.txt
> > 
> >   while ! emerge --resume --skipfirst
> >     do
> >       echo "$CATEGORY/$PN" >> failed.txt
> >     done
> >   }
> > 
> > ##
> > 
> > 
> > Any ideas how to achieve this?
> > 
> > 
> > -- 
> > Best regards,
> > Daniel
> 
> Here's a script that I use.  I use it to build all my packages at the
> end of the month.  Works ok for me.  YMMV:
> 


Wow! Thank you very much!
I was actually expecting some kind of a simple bash script. Anyway I'll
give it a try // even though I'm so afraid of snakes :) //



-- 
Best regards,
Daniel
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Automated "emerge -e world"

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 372 bytes --]

On Wed, 12 Sep 2007 13:36:27 +0300, Daniel Iliev wrote:

> I was actually expecting some kind of a simple bash script. Anyway I'll
> give it a try // even though I'm so afraid of snakes :) //

Python has nothing to do with snakes, although it does share a common
history with spam :)


-- 
Neil Bothwick

How do you know when it's time to tune your bagpipes?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Automated "emerge -e world"

[Mick]

[-- Attachment #1: Type: text/plain, Size: 501 bytes --]

On Wednesday 12 September 2007, Neil Bothwick wrote:
> On Wed, 12 Sep 2007 13:36:27 +0300, Daniel Iliev wrote:
> > I was actually expecting some kind of a simple bash script. Anyway I'll
> > give it a try // even though I'm so afraid of snakes :) //
>
> Python has nothing to do with snakes, although it does share a common
> history with spam :)

There's also the update script mentioned in this GWN:

http://www.gentoo.org/news/en/gwn/20070723-newsletter.xml

HTH.
-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Automated "emerge -e world"

[Daniel Iliev]

On Wed, 12 Sep 2007 23:01:01 +0100
Mick <michaelkintzios@gmail.com> wrote:

> 
> There's also the update script mentioned in this GWN:
> 
> http://www.gentoo.org/news/en/gwn/20070723-newsletter.xml
> 
> HTH.


Thanks! I'll take a look at this one too.

P.S.
For some reason GWN can't make its way to my Inbox for several months
now and obviously I've missed some publications.

...which reminds me to register for the rss feeds. :)


-- 
Best regards,
Daniel
-- 
gentoo-user@gentoo.org mailing list