[gentoo-dev] Portage, Jobs, and Niceness

[gentoo-dev] Portage, Jobs, and Niceness

[Chris White]

[-- Attachment #1: Type: text/plain, Size: 6989 bytes --]

Hi all,

-- Niceness --

  While waiting for kde to compile (read as hours), I decided to take a chance and talk about niceness and jobs this time.  Ok so, we'll start with niceness first:

if portage.settings.has_key("PORTAGE_NICENESS"):
        try:
                os.nice(int(portage.settings["PORTAGE_NICENESS"]))
        except SystemExit, e:
                raise # Needed else can't exit
        except Exception,e:
                print "!!! Failed to change nice value to '"+str(portage.setting
s["PORTAGE_NICENESS"])+"'"
                print "!!!",e

the setting PORTAGE_NICENESS in make.conf sets up the portage niceness.  This code gives you an idea of how it is done.  if it sees PORTAGE_NICENESS, it simply uses the os module's nice function to set the niceness.  Now.. niceness is kind of weird.. so let's demonstrate using American Idol(tm)!  Alright.. so you have your two wannabe singers here, ProcessPest and ProcessPrincess.  Now.. Simon is the cpu here, having to judge which singer he wants more (it is safe to note here that one doesn't suck).  Now, instead of showing who he likes more, he shows who he hates less!  So, we have something like this:

nice -n -19 ProcessPrincess
nice -n 10 ProcessPest

So Simon says to ProcessPrincess "You suck -19, which means I like you, you get more cpu"
So Simon says to ProcessPest "You suck 10, which means you suck, you get screwed"

Now back to our regularly scheduled technical form... Ok, so now that we know what's going on, niceness means how "nice" you are to other programs.  The more "nice" you are, the more willing you are to share your cpu to other programs.  The less "nice" you are, the more you want to hog it to yourself.  Now, when a process is set niceness, it does it by INCREMENT, not setting.  This basically means that if you do this:

PORTAGE_NICENESS=10

portage will increase your niceness from its original (normally 0 unless nice was already applied to it) by 10, giving you 10 niceness.  Now if you do it negative:

PORTAGE_NICENESS=-3

it increases by a negative number.. so it DECREASES portage niceness.  In this case, it would be -3, which is generally an ok value.  Note that the values go from -20 to 20, and that while you can increase niceness no problem, if you want to increases, you're probably going to need to be root.  Note to BE CAREFUL ABOUT NICENESS.  Remember, the same niceness that portage gets will be thusly applied to its child processes.  That means if you're running -j5 with PORTAGE_NICENESS=-19, subprocesses will all recieve -19 niceness too.  That's a lot of programs trying to grab for the cpu (I can tell you from experience that it can happen..)!  Only try this if you have some spectacular multi processor elite setup.  So that's what you get with niceness.  Now let's look at jobs:

-- Jobs --

Then there's jobs.  Now, this is extremly usefull for people that realized "Oh crap, I'm in the middle of emerge-ing such and such package and I just realized I need all my cpu for something more important (homework maybe?)".  No need to fear!  Bash has this nice little deal called jobs.  Let's take a look:

Scenario

<ChrisWhite> Ahh crud!  My xorg-x11 compile is halfway through and I just noticed this security patch I need to get through asap!  What to do! (or I'm compiling kde and realize I missed an assignment at school!).  Nice little handy bash function called jobs can take care of this.  To work with jobs, simply press CTRL+Z during the compile.  This stops the process as so:

(please note I was lazy here and removed my japanese error messages.. now you're not dreaming)

making imake with BOOTSTRAPCFLAGS= and CROSSCOMPILEFLAGS=-DCROSSCOMPILEDIR="" in config/imake
make[2]: Leaving directory `/var/tmp/portage/xorg-x11-6.8.99.8/work/xc/config/imake'
rm -f ./config/makedepend/Makefile.proto
./config/imake/imake -I./config/cf  -s ./config/makedepend/Makefile.proto -f ./config/makedepend/Imakefile -DTOPDIR=../.. -DCURDIR=./config/makedepend
./config/cf/Imake.tmpl:109  include 
                 Imakefile.c:35 :
./config/cf/linux.cf:390: Warning: "BuildLibGlxWithoutPIC" redefined
./config/cf/site.def:44  include ,
                 ./config/cf/Imake.tmpl:46 
                 Imakefile.c:35 :
./config/cf/host.def:63: Warning: this is the location of the previous definition
cd ./config/makedepend && rm -f -r Makefile Makefile.dep makedepend *.o bootstrap
cd ./config/makedepend && make -f Makefile.proto bootstrap
make[2]: Entering directory `/var/tmp/portage/xorg-x11-6.8.99.8/work/xc/config/makedepend'
make[2]: `bootstrap'
make[2]: Leaving directory `/var/tmp/portage/xorg-x11-6.8.99.8/work/xc/config/makedepend'
./config/imake/imake -I./config/cf  -s ./config/imake/Makefile.proto -f ./config/imake/Imakefile -DTOPDIR=../.. -DCURDIR=./config/imake -DBootStrap

[1]+  Stopped                 emerge xorg-x11

And it stopped!  Now that you've done your stuff, you'd like it to run again.  First off, we do a jobs listing like such:

secures chris # jobs
[1]+  Stopped                 emerge xorg-x11

Job 1 is listed here as our stopped emerge xorg-x11.  Please note that jobs are PER SESSION.  This means you can't just pull another console up and expect the current console's jobs to show up in it.  One session, one job list.  So, you have two options:

1) fg 1
2) bg 1

the first puts the process in the foreground
the second puts the process in the background

Let's pull it to the foreground in this case:

secures chris # fg 1
emerge xorg-x11
./config/cf/Imake.tmpl:109  include 
                 Imakefile.c:35 :
./config/cf/linux.cf:390: : "BuildLibGlxWithoutPIC" redefined
./config/cf/site.def:44  include 
                 ./config/cf/Imake.tmpl:46 ,
                 Imakefile.c:35 :
./config/cf/host.def:63: : this is the location of the previous definition
cd ./config/imake && make -f Makefile.proto bootstrapdepend
make[2]: Entering directory `/var/tmp/portage/xorg-x11-6.8.99.8/work/xc/config/imake'
make[2]: `bootstrapdepend' 
make[2]: Leaving directory `/var/tmp/portage/xorg-x11-6.8.99.8/work/xc/config/imake'
cd ./config/imake && make -w -f Makefile.ini BOOTSTRAPCFLAGS="" CC="cc" bootstrap
make[2]: Entering directory `/var/tmp/portage/xorg-x11-6.8.99.8/work/xc/config/imake'
make[2]: `bootstrap' 
make[2]: Leaving directory `/var/tmp/portage/xorg-x11-6.8.99.8/work/xc/config/imake'
cd ./config/imake && make -f Makefile.proto imakeonly
make[2]: Entering directory `/var/tmp/portage/xorg-x11-6.8.99.8/work/xc/config/imake'
make[2]: `imakeonly' 
make[2]: Leaving directory `/var/tmp/portage/xorg-x11-6.8.99.8/work/xc/config/imake'
+ rm -f xmakefile.bak
+ mv xmakefile xmakefile.bak

and viola!  Our compile moves right on!  Please note that jobs is NOT A PROGRAM, it's a part of the shell.  I know bash does it.. I'm not sure if others do.  That's it for now, hope this becomes usefull to some people.

Chris White

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Portage, Jobs, and Niceness

[Georgi Georgiev]

[-- Attachment #1: Type: text/plain, Size: 328 bytes --]

maillog: 31/05/2005-17:48:21(+0900): Chris White types
> Hi all,
...

Shouldn't this go to gentoo-user?

-- 
\    Georgi Georgiev   \  Ferguson's Precept: A crisis is when you     \
/     chutz@gg3.net    /  can't say "let's forget the whole thing."    /
\   +81(90)2877-8845   \                                               \

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Portage, Jobs, and Niceness

[Mike Frysinger]

On Tuesday 31 May 2005 05:05 am, Georgi Georgiev wrote:
> maillog: 31/05/2005-17:48:21(+0900): Chris White types
>
> > Hi all,
>
> ...
>
> Shouldn't this go to gentoo-user?

sounds like it ...
-mike
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Portage, Jobs, and Niceness

[Alec Warner]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Georgi Georgiev wrote:
> maillog: 31/05/2005-17:48:21(+0900): Chris White types
> 
>>Hi all,
> 
> ...
> 
> Shouldn't this go to gentoo-user?
> 
Maybe Ciaranm's guide; I doubt all devs are shell wizards ;)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQpyC1GzglR5RwbyYAQIVhA//fya8SU7iBsocgqrgMe2Z/VtIZejmaFp8
OCkkKdk1Q5rJaclDaHqD9+KLYWUFaxITVKM1cAtVch9QVWaqYSlNdYXUHLBOeYpj
7IwRFIsYejkPofUl4c9NeHXSqP5a9hDtLUvZjHLNVMme6r26pEvr0clcPz18m6YP
lNvFYrjcUlGvKi9rODXdqYuCPU6SfdJLBwIzfme8j5ukt9dgcF0JzBlP7r3UgwIe
xFx4bVyv2OaQ47Dd4lctPbTliTkHIECnduEPkNMTi+FoT9rYeRL0MK4axF5nSIbb
JfHIFypKQaouJmjMgTM9R0KhX9zbNf1DoLY4i/LwReqz6PSk/YcZXGbnb+G2mLqH
Muv8E6uIvc4e95fjWw3IjzlnchPiWOAS8KamoW7cE1wms8PXq7eiqDaxgkRm9j94
X2YKEpDJuCjmQf6CRVA1YFtpo+eyCdHOgEgPZXusTEeLjirKXRbdiicAWBw4S5lF
8v7V2iSDGgakSxIa/+KEEhkE6iMn3mQQx4gJv5D4IHbtCy3xLN1HnSP/b1q9kfUJ
RIABsxGOPpMjNEpomXsfx5th2z+AB+iX/kIor4XhgVsMmzLQDhRcZbf08e9LBpiG
Iq3dmD4JaDR05jYt2eYfRlKE08uakZ5gvbeoKasB4iScbfRVSVfOkhkLVRadFSZT
K9lTumoqIVE=
=AfQB
-----END PGP SIGNATURE-----
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Portage, Jobs, and Niceness

[Ciaran McCreesh]

[-- Attachment #1: Type: text/plain, Size: 438 bytes --]

On Tue, 31 May 2005 11:29:24 -0400 Alec Warner <warnera6@egr.msu.edu>
wrote:
| > Shouldn't this go to gentoo-user?
| > 
| Maybe Ciaranm's guide; I doubt all devs are shell wizards ;)

I don't think really basic 'Unix 101' job control stuff belongs in The
Doc.

-- 
Ciaran McCreesh : Gentoo Developer (Vim, Shell tools, Fluxbox, Cron)
Mail            : ciaranm at gentoo.org
Web             : http://dev.gentoo.org/~ciaranm


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-dev] Portage, Jobs, and Niceness

[Aron Griffis]

[-- Attachment #1: Type: text/plain, Size: 246 bytes --]

Chris,

This is useful information but not appropriate for gentoo-dev.  Some
people have suggested -user, which might be a good place.  Other
places I can think of are the forums or a blog.

Regards,
Aron

--
Aron Griffis
Gentoo Linux Developer


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]