[gentoo-amd64] Unexpected side effect of GCC 4

[gentoo-amd64] Unexpected side effect of GCC 4

[Peter Humphrey]

I have a firewall between my ADSL modem and my local network (running 
Gentoo, naturally), and I use it as an rsync server for portage on the 
network. Specifically, the firewall runs emerge --sync in a cron job every 
morning before I get up, then I synchronise my laptop and desktop systems 
from that at my convenience.

The rsync phase of update over the local network has been extremely slow; I 
couldn't get traffic flowing faster than about 30 - 40 kBps on average, and 
the hub showed tremendous numbers of collisions. I did manage to speed it 
up a bit by changing the /usr/portage partition on the server to reiserfs, 
but that improvement seemed to go away again. Meanwhile, other traffic over 
the network went at full speed, about 270 kBps, with only the occasional 
collision.

Now that I've finally managed to build a full system using GCC 4.1.1 on the 
workstation, I find its rsync traffic has leapt to full speed with no 
collisions to speak of. I'm not complaining, mind, but this is a puzzle. As 
far as I know I haven't changed anything else - certainly the firewall box 
has been running continuously throughout.

The speeds I mention are as shown by gkrellm (a very useful utility). The 
network drivers are a module on the workstation (tg3) and built-in on the 
firewall (8139too). The CFLAGS are "-O2 -march=k8 -pipe" on the firewall 
and the set offered by Duncan recently on the workstation.

Any ideas, anyone?

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Unexpected side effect of GCC 4

[Duncan]

Peter Humphrey <prh@gotadsl.co.uk> posted
200611050845.42865.prh@gotadsl.co.uk, excerpted below, on  Sun, 05 Nov
2006 08:45:42 +0000:

> Now that I've finally managed to build a full system using GCC 4.1.1 on the 
> workstation, I find its rsync traffic has leapt to full speed with no 
> collisions to speak of. I'm not complaining, mind, but this is a puzzle. As 
> far as I know I haven't changed anything else - certainly the firewall box 
> has been running continuously throughout.
> 
> The speeds I mention are as shown by gkrellm (a very useful utility). The 
> network drivers are a module on the workstation (tg3) and built-in on the 
> firewall (8139too). The CFLAGS are "-O2 -march=k8 -pipe" on the firewall 
> and the set offered by Duncan recently on the workstation.

What about linux-headers and glibc?  The new glibc (2.5) has had some very
strange interactions here.  See the earlier thread where I mentioned
dropping back to 2.4 for awhile.  I'll very readily admit whatever is
going on is /way/ beyond my comprehension, but 2.5 does something with
memory transfers that 2.4 apparently never did, and there's something on
my system that simply doesn't fully like it.  xmms (right before the mask,
but I removed it rather than worry about it since I knew it was being
masked anyway), kaffeine, and amarok, all three, aren't fully stable,
while kmplayer seems perfectly fine.  xmms was using a different library
set, but kaffeine, amarok, and kmplayer are using xinelib.  kmplayer
hasn't yet been rebuild and with it the only one remaining stable, I don't
want to mess with it, but I've tried rebuilding all sorts of other stuff
for the others, xinelib, qt, kdelibs, glibc, even gcc (to rebuild
libstdc++), most with several sets of cflags including plain old
-march=amd64 -pipe -O2, with nothing getting the unstable things stable or
upsetting the stability of kmplayer.  The /only/ thing that seems to get
the others stable again is reverting to glibc 2.4.  If kmplayer using the
same xinelib was also unstable, I could understand it.  If xmms using a
different decoder and /not/ using qt and kdelibs was stable, I could
understand that.  However, the particular mix and match that I have simply
doesn't make a lot of sense, and the only common thing to the set is that
reverting to glibc-2.4 magically stabilizes everything once again, even
tho in theory that's a VERY bad thing to do (and requires a bit of fancy
footwork to get portage to do at all).  Yet others report no problems at
all with the same programs and glibc 2.5.  Maybe it's a CFLAGS (or
CXXFLAGS) thing, but if it is, I've certainly not hit the elusive
dependency that's causing the issue.  glibc itself filters nearly all
cflags but -march, -pipe, and -O2, and anyway, I tried rebuilding it with
/just/ those CFLAGS/CXXFLAGS specified, and it didn't change anything.

One of these days I'll do a full emerge --emptytree, and see if that fixes
it, but meanwhile, there's a mystery that doesn't seem to match up with
anything expect for glibc 2.5, only others report no problems with the
exact same apps, so the mystery remains!  I don't like such untraceable
mysteries in my computer system!  Quite apart from any instabilities or
other issues they might trigger, I'm used to being able to ultimately
being able to point at something or some combination of somethings and say
definitively, "that is/was the problem."  It bothers me when I cannot do
so, and all is not right in my world until I /can/ do so (again, quite
apart from actually fixing the issue, just being able to point to it).

Anyway, if you are running glibc 2.5 on any of the machines, how does
the sudden return to full speed rsyncing match up with the machines with
2.5, if at all.  What about the machine with my CFLAGS?  If they are
related in any way, "enquiring minds want to know!"  =8^)  If your mystery
is glibc 2.5 related as well, well then, I'll not feel so strange as at
least there'll be two folks with a similar glibc 2.5 related mystery,
which makes it less of one already! =8^)

Of course, if you still have glibc 2.4 on everything, this won't help at
all, but at least it'll be one thing both you and I can scratch off as
relating to both of us.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Peter Humphrey]

On Sunday 05 November 2006 10:12, Duncan wrote:

> What about linux-headers and glibc?  The new glibc (2.5) has had some
> very strange interactions here.

Good point. On the firewall I have:
$ qpkg -v -I glibc
sys-libs/glibc-2.4-r3
$ qpkg -v -I linux-headers
sys-kernel/linux-headers-2.6.17-r1
$ uname -r
2.6.17-gentoo-r8
$ gcc-config -l
 [1] i686-pc-linux-gnu-3.4.6 *
 [2] i686-pc-linux-gnu-3.4.6-hardened
 [3] i686-pc-linux-gnu-3.4.6-hardenednopie
 [4] i686-pc-linux-gnu-3.4.6-hardenednopiessp
 [5] i686-pc-linux-gnu-3.4.6-hardenednossp
 [6] i686-pc-linux-gnu-4.1.1

and on the workstation I have:
$ qpkg -v -I glibc
/glibc-2.5
$ qpkg -v -I linux-headers
/linux-headers-2.6.17-r1
$ uname -r
2.6.18-gentoo-r1
$ gcc-config -l
 [1] x86_64-pc-linux-gnu-4.1.1 *
(I don't know what's causing qpkg to drop the first part of the package name 
on the workstation. Is this another pointer to glibc?)

<snip description of Duncan's problem>

> One of these days I'll do a full emerge --emptytree, and see if that
> fixes it,

I couldn't get GCC upgraded to 4.1 on this workstation - I got lots of 
unexplained compilation failures, especially in KDE packages, even though I 
was following the instructions carefully and (I hope) intelligently - so in 
the end I created another partition and installed a new 2006.1 system into 
it. (I'd forgotten how many days would be needed for that, even on this 
dual-Opteron-246 box with 4 GB. KDE takes an age. Perhaps I ought to look 
into putting /tmp and /var/tmp onto tmpfs.) I still had to jump through a 
few hoops to get the system installed at all: I remember having to 
emerge --sync on first mounting the already-populated /usr/portage 
partition during the early stages, and revdep-rebuild and fixpackages among 
other things. I couldn't even create a user until after rebooting. I know 
that's how the docs say to do it, but I've always created the user just 
after setting the root password - it seems more logical to me and I don't 
have to think so much. I've probably forgotten lots of other difficulties, 
but perseverance paid off.

> Anyway, if you are running glibc 2.5 on any of the machines, how does
> the sudden return to full speed rsyncing match up with the machines with
> 2.5, if at all.  What about the machine with my CFLAGS?  If they are
> related in any way, "enquiring minds want to know!"  =8^)  If your
> mystery is glibc 2.5 related as well, well then, I'll not feel so strange
> as at least there'll be two folks with a similar glibc 2.5 related
> mystery, which makes it less of one already! =8^)

The firewall box has been untouched and continues to run with glibc-2.4-r3 
and linux-headers-2.16.7-r1; that's the box with the rsync server. The 
workstation is the one with all the changes, being in fact a new 
installation from scratch. If I boot the old system, which I think has the 
same versions of glibc and linux-headers as the firewall box (I'll try that 
in a minute and let you know if it's otherwise), that's when I get ugly, 
crippled rsync; if I boot this present system, with the versions I listed 
at the top, rsync works as advertised. In other words, if this is a 
consequence of glibc, the old version is broken and the new one isn't - the 
converse of your experience, Duncan.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Peter Humphrey]

On Sunday 05 November 2006 11:50, Peter Humphrey wrote:
> On Sunday 05 November 2006 10:12, Duncan wrote:
> > What about linux-headers and glibc?  The new glibc (2.5) has had some
> > very strange interactions here.
>
> Good point. On the firewall I have:
> $ qpkg -v -I glibc
> sys-libs/glibc-2.4-r3
> $ qpkg -v -I linux-headers
> sys-kernel/linux-headers-2.6.17-r1
> $ uname -r
> 2.6.17-gentoo-r8
> $ gcc-config -l
>  [1] i686-pc-linux-gnu-3.4.6 *
>  [2] i686-pc-linux-gnu-3.4.6-hardened
>  [3] i686-pc-linux-gnu-3.4.6-hardenednopie
>  [4] i686-pc-linux-gnu-3.4.6-hardenednopiessp
>  [5] i686-pc-linux-gnu-3.4.6-hardenednossp
>  [6] i686-pc-linux-gnu-4.1.1
>
> and on the workstation I have:
> $ qpkg -v -I glibc
> /glibc-2.5
> $ qpkg -v -I linux-headers
> /linux-headers-2.6.17-r1
> $ uname -r
> 2.6.18-gentoo-r1
> $ gcc-config -l
>  [1] x86_64-pc-linux-gnu-4.1.1 *
> (I don't know what's causing qpkg to drop the first part of the package
> name on the workstation. Is this another pointer to glibc?)

[...]

> The firewall box has been untouched and continues to run with
> glibc-2.4-r3 and linux-headers-2.16.7-r1; that's the box with the rsync
> server. The workstation is the one with all the changes, being in fact a
> new installation from scratch. If I boot the old system, which I think
> has the same versions of glibc and linux-headers as the firewall box
> (I'll try that in a minute and let you know if it's otherwise),

In fact the old workstation system is like this (I'm running it now):
$ qpkg -v -I glibc
/glibc-2.5
$ qpkg -v -I linux-headers
/linux-headers-2.6.17-r1
prh@wstn ~ $ uname -r
2.6.18-gentoo-r1
prh@wstn ~ $ gcc-config -l
 [1] x86_64-pc-linux-gnu-3.4.6 *
 [2] x86_64-pc-linux-gnu-3.4.6-hardened
 [3] x86_64-pc-linux-gnu-3.4.6-hardenednopie
 [4] x86_64-pc-linux-gnu-3.4.6-hardenednopiessp
 [5] x86_64-pc-linux-gnu-3.4.6-hardenednossp
 [6] x86_64-pc-linux-gnu-4.1.1
$ emerge --info | grep FLAGS
CFLAGS="-march=k8 -Os -pipe -frename-registers -fweb -freorder-blocks -freorder-blocks-and-partition -funit-at-a-time -fgcse-sm -fgcse-las -fgcse-after-reload -fmerge-all-constants"
CXXFLAGS="-march=k8 -Os -pipe -frename-registers -fweb -freorder-blocks -funit-at-a-time -fgcse-sm -fgcse-las -fgcse-after-reload -fmerge-all-constants"
Unset:  CTARGET, INSTALL_MASK, LDFLAGS, PORTAGE_RSYNC_EXTRA_OPTS
(Note the partial package name reported by qpkg.)

So I have a pointer away from glibc and linux-headers. CFLAGS on the new 
system are as on the old but with -combine added.The old and new systems 
have the same versions of kernel, linux-headers and glibc, but different 
versions of gcc. Rsync is crippled on the old system but fine on the new 
one.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Hemmann, Volker Armin]

Hi,

hm, my system is perfectly stable with glibc 2.5.

But there are some known issues with taglib. If you have the wrong version. 
xine-lib and stuff using it, are crashy.
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Unexpected side effect of GCC 4

[Duncan]

"Hemmann, Volker Armin" <volker.armin.hemmann@tu-clausthal.de> posted
200611051424.07010.volker.armin.hemmann@tu-clausthal.de, excerpted below,
on  Sun, 05 Nov 2006 14:24:06 +0100:

> hm, my system is perfectly stable with glibc 2.5.
> 
> But there are some known issues with taglib. If you have the wrong version. 
> xine-lib and stuff using it, are crashy.

Thanks.  Have to look into it.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Unexpected side effect of GCC 4

[Duncan]

Peter Humphrey <prh@gotadsl.co.uk> posted
200611051150.06468.prh@gotadsl.co.uk, excerpted below, on  Sun, 05 Nov
2006 11:50:06 +0000:

> and on the workstation I have:
> $ qpkg -v -I glibc
> /glibc-2.5
> $ qpkg -v -I linux-headers
> /linux-headers-2.6.17-r1
> $ uname -r
> 2.6.18-gentoo-r1
> $ gcc-config -l
>  [1] x86_64-pc-linux-gnu-4.1.1 *
> (I don't know what's causing qpkg to drop the first part of the package name 
> on the workstation. Is this another pointer to glibc?)

What version of gentoolkit do you have?  qpkg is deprecated and has been
since at at least gentoolkit 0.2.2-rc1, merged here back on May 27,
according to my logs.  Even if you are using an older version, consider
switching to equery instead, as qpkg hasn't been under development for some
time and may therefore have strange bugs.  I'd say this is probably one
example, tho it's still working sort of correctly for you.

That's my only point for this reply; I'll make another for other points.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Unexpected side effect of GCC 4

[Duncan]

Peter Humphrey <prh@gotadsl.co.uk> posted
200611051210.25023.prh@gotadsl.co.uk, excerpted below, on  Sun, 05 Nov
2006 12:10:24 +0000:

[rewrapped]

> $ emerge --info | grep FLAGS

> CFLAGS="-march=k8 -Os -pipe -frename-registers -fweb -freorder-blocks
> -freorder-blocks-and-partition -funit-at-a-time -fgcse-sm -fgcse-las
> -fgcse-after-reload -fmerge-all-constants"

> CXXFLAGS="-march=k8 -Os -pipe -frename-registers -fweb -freorder-blocks
> -funit-at-a-time -fgcse-sm -fgcse-las -fgcse-after-reload
> -fmerge-all-constants"

> Unset:  CTARGET, INSTALL_MASK, LDFLAGS, PORTAGE_RSYNC_EXTRA_OPTS
 
> So I have a pointer away from glibc and linux-headers. CFLAGS on the new
> system are as on the old but with -combine added.The old and new systems
> have the same versions of kernel, linux-headers and glibc, but different
> versions of gcc. Rsync is crippled on the old system but fine on the new
> one.

Well, I /did/ say gcc 4.1.1 seemed way (as in, vastly) better than 3.4.x,
for me, so at least that's being borne out.

As for CFLAGS, the only change I've made recently is adding
-ftree-vectorize, experimentally, given the discussion here.  I'm not sure
it's the problem, particularly as where I /am/ experiencing issues the
first thing I tried was compiling stuff without it, so if it's the
problem, it's in some obscure dependency somewhere, but I certainly had no
issues before trying it and now I do, so for anyone else thinking of
trying it, I'd recommend staying well away from -ftree-vectorize for the
time being.  I'd really like to be able to say for sure it is the problem,
and find it very frustrating not to be able to nail it down, but suffice
it to say /something's/ causing me problems right now and that's one of my
recent changes, so I'd recommend staying well away from it.

I mentioned that I need to do an emerge -emptytree again, as it has been
awhile.  I'm still debating whether to try it with or without
-ftree-vectorize.  If that's the problem, it may well go away if
everything is compiled with it.  OTOH, it may not.  That's an awful lot of
compiling to do and still have a problem if -ftree-vectorize is causing it
and it doesn't go away with everything compiled with it, but OTOH, if
that's /not/ the problem, and I decide not to try it, I'll never know
whether that was the problem or not.

I'm about to decide to simply play it safe and pretend nobody here ever
called -ftree-vectorize to my attention, at least until the flag has had a
bit more time to mature (say gcc 4.2 or 4.3).  Only just because I'm the
type of person I am, that would bother me as I will have never nailed it
for sure.  In any case, my best guess right now are that the issues I am
having are related to a nasty interplay between -ftree-vectorize and
glibc-2.5.  In looking around at Gentoo glibc bugs, I found at least one
equally strange one, on x86.  The bug required glibc 2.5, and it required
that a certain package be compiled with a certain cflag.  With that cflag
on glibc 2.4, it worked fine, as it did without the cflag on 2.5, but put
them together and things blew up.  It wasn't -ftree-vectorize, and it was
on x86, but I've a rather strong suspicion my case is similar, only on
amd64 and with a different cflag, -ftree-vectorize most likely.  I've just
not figured out which particular package is doing it, and not being able
to figure it out as I said is VERY annoying to me, WAY more so than the
bit of inconvenience the actual bug is causing.  (Yes, I know I said all
that already, but it's still annoying and I'm still griping about it! =8^(

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Hemmann, Volker Armin]

On Monday 06 November 2006 09:17, Duncan wrote:


>
> I mentioned that I need to do an emerge -emptytree again, as it has been
> awhile. 

where is the logic with that?

You don't need to do regularly --emptytree emerges. If you don't change gcc 
you never need it. So why?
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Unexpected side effect of GCC 4

[Duncan]

Peter Humphrey <prh@gotadsl.co.uk> posted
200611051150.06468.prh@gotadsl.co.uk, excerpted below, on  Sun, 05 Nov
2006 11:50:06 +0000:

> (I'd forgotten how many days would be needed for that, even on this 
> dual-Opteron-246 box with 4 GB. KDE takes an age. Perhaps I ought to look 
> into putting /tmp and /var/tmp onto tmpfs.)

Do so.  It makes a /big/ difference!  I don't have all of KDE merged here,
but trying to track down the issue I was having, but between trying to
track down the issue I was having (which I won't repeat here), and the
~arch upgrade to kde 3.5.5, and the several -rX releases of various
kde packages subsequent to 3.5.5, I've done a lot of kde remerging lately.

As best I can tell and from memory of how long it took to emerge kde
before I upgraded memory and put $PORTAGE_TMPDIR on tmpfs, it used to take
me about 14-16 hours to merge what parts of KDE I have back with a single
disk and a gig of memory, shortened to about 10-12 after I upgraded to a
4-disk RAID ($PORTDIR and $PORTAGE_TMPDIR on 4-way striped RAID-0, main
system on RAID-6, so two-data stripes, two parity stripes), to only a bit
over 8 hours, say 9, with $PORTAGE_TMPDIR on tmpfs after upgrading to 8
gig memory. If you aren't running RAID, you'll get the entire benefit of
going from $PORTAGE_TMPDIR on the same drive as you are installing to and
running from to having it on tmpfs, in one shot.  Note that the RAID-6
main system is going to be a bit faster than a single drive, but that I
only have dual Opteron 242s, not the 246s you said you have.  I'm guessing
you'll see a 2-hour minimum drop in compile times, if not 4-6, on KDE
alone, by putting $PORTAGE_TMPDIR on tmpfs.  For an entire emerge
--emptytree world, you are looking probably a half-day of savings at least.

I am /so/ looking forward to that upgrade to dual Opteron 285s!  Even
discounting the upgrade from dual to quad-cores, the clock-speed increase
alone will rock!  I'm guessing perhaps 6 hours (maybe as little as 4) for
my KDE upgrades, and say a day (more or less) for a full system rebuild! 
That'll be SWEET INDEED!  =8^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Peter Humphrey]

On Monday 06 November 2006 07:50, Duncan wrote:

> What version of gentoolkit do you have?  qpkg is deprecated and has been
> since at at least gentoolkit 0.2.2-rc1, merged here back on May 27,
> according to my logs. 

It's gentoolkit-0.2.3_pre2. I know it's deprecated, but that was done far 
too early, as at the time equery couldn't do half the things qpkg did. 
Definitely a case of too much haste.

> Even if you are using an older version, consider switching to equery
> instead, as qpkg hasn't been under development for some time and may
> therefore have strange bugs. 

I do use equery more now, and sooner or later will probably drop qpkg 
quietly. I'm certainly not complaining of bugs in it.

> I'd say this is probably one example, tho it's still working sort of
> correctly for you. 

Quite so.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Hemmann, Volker Armin]

Set Portage_NICENESS to 19, MAKEOPTS ot -jX with X maximum number of cores and 
activate ccache with at least 4GB and you don't need tempfs to install KDE 
(all of it), in less than 10h.
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Peter Humphrey]

On Monday 06 November 2006 08:41, Duncan wrote:

> Peter Humphrey <prh@gotadsl.co.uk> posted
> > Perhaps I ought to look into putting /tmp and /var/tmp onto tmpfs.
>
> Do so.  It makes a /big/ difference!

My reading so far suggests that I include these two lines in /etc/fstab:

tmpfs	/tmp		tmpfs	nodev,nosuid,noexec	0 0
tmpfs	/var/tmp	tmpfs	nodev,nosuid,noexec	0 0

Is that all I have to do? I assume I don't need to specify tmpfs sizes; I 
have 4 GB RAM of which 0.5 GB is unavailable (owing to a motherboard 
problem that prompted the system builder to refund some of my money - small 
consolation). Probably I should put a script into /etc/conf.d/start.local 
to create a symlink from /var/tmp/ccache back to a real cache directory on 
disk, as it seems daft to install a compiler cache and then flush it at 
every reboot - at present mine has 1.5 GB of data. /tmp has 750 MB of stuff 
and is a separate disk partition at present, mounted on whichever system I 
boot.

>  I don't have all of KDE merged here

Nor I; I just have 9 meta-packages because I didn't want the education or 
games stuff, among others. I might have preferred to go to finer grain 
still, but it looked like too much work.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Peter Humphrey]

On Monday 06 November 2006 10:47, Hemmann, Volker Armin wrote:
> Set Portage_NICENESS to 19, MAKEOPTS ot -jX with X maximum number of
> cores and activate ccache with at least 4GB and you don't need tempfs to
> install KDE (all of it), in less than 10h.

Are you implying that setting X > number of cores will slow it down? I 
have -j5 with two single CPUs, and KDE (part of it) takes far longer than 
that.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Hemmann, Volker Armin]

On Monday 06 November 2006 12:48, Peter Humphrey wrote:
> On Monday 06 November 2006 10:47, Hemmann, Volker Armin wrote:
> > Set Portage_NICENESS to 19, MAKEOPTS ot -jX with X maximum number of
> > cores and activate ccache with at least 4GB and you don't need tempfs to
> > install KDE (all of it), in less than 10h.
>
> Are you implying that setting X > number of cores will slow it down? I
> have -j5 with two single CPUs, and KDE (part of it) takes far longer than
> that.

lets ask the other way round. Why should it speed up anything to have X>number 
of cores? Instead of a single thread per core, compiling happily, you have 
two or more competing for one core and regularly kick out each others data 
from the cache.
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Jack Lloyd]

On Mon, Nov 06, 2006 at 01:05:31PM +0100, Hemmann, Volker Armin wrote:

> lets ask the other way round. Why should it speed up anything to have X>number 
> of cores? Instead of a single thread per core, compiling happily, you have 
> two or more competing for one core and regularly kick out each others data 
> from the cache.

To account for I/O wait states
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Hemmann, Volker Armin]

On Monday 06 November 2006 13:49, Jack Lloyd wrote:
> On Mon, Nov 06, 2006 at 01:05:31PM +0100, Hemmann, Volker Armin wrote:
> > lets ask the other way round. Why should it speed up anything to have
> > X>number of cores? Instead of a single thread per core, compiling
> > happily, you have two or more competing for one core and regularly kick
> > out each others data from the cache.
>
> To account for I/O wait states

and how often does something wait for io and how often does some data is 
purged from the cache, because the other make instance is activated?

When I switched from j2 to j1, compiling did not take any longer - but the box 
way much more usable.
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Jack Lloyd]

On Mon, Nov 06, 2006 at 01:59:33PM +0100, Hemmann, Volker Armin wrote:
> On Monday 06 November 2006 13:49, Jack Lloyd wrote:
> > On Mon, Nov 06, 2006 at 01:05:31PM +0100, Hemmann, Volker Armin wrote:
> > > lets ask the other way round. Why should it speed up anything to have
> > > X>number of cores? Instead of a single thread per core, compiling
> > > happily, you have two or more competing for one core and regularly kick
> > > out each others data from the cache.
> >
> > To account for I/O wait states
> 
> and how often does something wait for io and how often does some data is 
> purged from the cache, because the other make instance is activated?
>
> When I switched from j2 to j1, compiling did not take any longer - but the box 
> way much more usable.

OK. <shrug> On my dual core machine, -j3 seems to be the sweet
spot. Simply because something does not work for you does not mean it
is going to be universally a bad idea.
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Peter Humphrey]

On Monday 06 November 2006 12:59, Hemmann, Volker Armin wrote:
> On Monday 06 November 2006 13:49, Jack Lloyd wrote:
> > On Mon, Nov 06, 2006 at 01:05:31PM +0100, Hemmann, Volker Armin wrote:
> > > lets ask the other way round. Why should it speed up anything to have
> > > X>number of cores?
> >
> > To account for I/O wait states
>
> and how often does something wait for io and how often does some data is
> purged from the cache, because the other make instance is activated?

The best way to find out is to test in your own circumstances.

> When I switched from j2 to j1, compiling did not take any longer - but
> the box way much more usable.

Whereas when I changed from j3 to j5 on my 2 CPUs, compiling seemed to take 
less time and I didn't notice any effect on responsiveness.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Unexpected side effect of GCC 4

[Duncan]

Peter Humphrey <prh@gotadsl.co.uk> posted
200611061144.01773.prh@gotadsl.co.uk, excerpted below, on  Mon, 06 Nov
2006 11:44:01 +0000:

> On Monday 06 November 2006 08:41, Duncan wrote:
> 
>> Peter Humphrey <prh@gotadsl.co.uk> posted
>> > Perhaps I ought to look into putting /tmp and /var/tmp onto tmpfs.
>>
>> Do so.  It makes a /big/ difference!
> 
> My reading so far suggests that I include these two lines in /etc/fstab:
> 
> tmpfs  /tmp     tmpfs  nodev,nosuid,noexec  0 0
> tmpfs  /var/tmp tmpfs  nodev,nosuid,noexec  0 0
> 
> Is that all I have to do? I assume I don't need to specify tmpfs sizes; I
> have 4 GB RAM of which 0.5 GB is unavailable (owing to a motherboard
> problem that prompted the system builder to refund some of my money -
> small consolation).

I believe I remember your posts talking about that, back when I still had
only a gig of RAM.  Now that I have more than 3.5 gig of memory myself, I
know a bit more about the memory hole and all that.  If you have bios
entries for configuring it and just couldn't get it to work before, maybe
now that I know a bit more about the issue, we could try to work thru it. 
Of course, if you don't have the BIOS entries, it's just not going to
happen...

You don't /need/ to specify tmpfs size, but it might be worthwhile to do
so.  You don't want a runaway file in /tmp to take up all your memory and
swap.  Be aware that if you mount as tmpfs any location normally
having global write permissions (the usual 1777 of /tmp for instance), you
are letting any account have access to that memory.  If you are the only
human user of the machine, that's probably fine, but if not, you may want
to ensure you are running quotas on it or something, and have them set
appropriately.

If you are just concerned about portage, you can of course point
$PORTAGE_TMPDIR at a location other than the default, and set that
location user portage group portage, 740 permissions or similar, so only
portage (and root of course) can write to it.  It's still possible that
anyone in group portage could abuse it, but not simple users not in group
portage. then.

Here, as I'm the only human user, I don't have to be quite so strict on
security.  To keep things simple, /var/tmp is a symlink to /tmp, so I don't
have to worry about a tmpfs for both dirs.  You'll want to set the
following in make.conf:

PKG_TMPDIR
PORTAGE_TMPDIR
PORTAGE_TMPFS

Note that setting the latter to a small, say 50 meg, tmpfs, is useful
even if you aren't setting PORTAGE_TMPDIR on tmpfs.  It's used for
quick/small stuff like lockfiles and the like.  The portage docs suggest
setting it to /dev/shm, an LSB standard location for such things.  I have
a separate (max 50 meg as I said) tmpfs mounted at /dev/shm and followed
the recommendation to point PORTAGE_TMPFS at it.

Of course, you'll only need PKG_TMPDIR if you have FEATURES=buildpkg set
or otherwise deal with binary packages.

I have /tmp (which is where both the package and portage TMPDIRs point) set
size=5g here, with 8 gig memory. With 3.5 gig of memory, you may wish to
set something a bit smaller, say 3 gig. That should be fine for most
emerges and will keep it from going too much into swap if something should
start hogging your tmpfs.  Of course, you'll have to make it a bit bigger
for merging say OOo (not the bin-version), as it is said to require 5 gigs
of space (I don't use it so wouldn't personally know), but you could use
remount for special cases like that, keeping it to 3 gig or so normally.

> Probably I should put a script into /etc/conf.d/start.local to create a
> symlink from /var/tmp/ccache back to a real cache directory on disk, as
> it seems daft to install a compiler cache and then flush it at every
> reboot - at present mine has 1.5 GB of data. /tmp has 750 MB of stuff
> and is a separate disk partition at present, mounted on whichever system
> I boot.

Don't worry about that symlink.  Set CCACHE_DIR in make.conf directly,
instead. Here, I have it pointed at a subdir on my fast raid-0/striped
array.  That works quite well.  Since all the stuff portage normally uses
(gcc, grep, sed, libraries commonly linked, etc) read into the regular
kernel filesystem cache memory during the first emerge, and with all the
work being done on tmpfs, there's little i/o but the ccache write updates
going on during all but the qmerge phase of subsequent emerges.  The disk
spends most of its time idle, and you can watch the disk activity light
and tell when the kernel's cache flush writes kick in, as it blinks red a
couple times every few seconds.  That's it.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Unexpected side effect of GCC 4

[Duncan]

"Hemmann, Volker Armin" <volker.armin.hemmann@tu-clausthal.de> posted
200611060929.52020.volker.armin.hemmann@tu-clausthal.de, excerpted below,
on  Mon, 06 Nov 2006 09:29:51 +0100:

> where is the logic with that?
> 
> You don't need to do regularly --emptytree emerges. If you don't change
> gcc you never need it. So why?

That's the thing.  I haven't done a full emerge --emptytree since at least
gcc-4.1.0, which was never unmasked (it's 4.1.1 that's unmasked).  I did
one sometime after 4.0, I think during the 4.1.0 release candidates, but
not since.

As for doing it every gcc upgrade, that's a bit ridiculous when you are
running the weekly gcc snapshots as I was between 4.0 and 4.1.

So, everything on my system has been compiled with (I think) at least a
4.1 release candidate or newer, but I haven't done a full --emptytree
since 4.1.1 was released and unmasked, I know.  Thus, particularly since
I'm having that mysterious problem, it's time to do one, and see if the
problem disappears. =8^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Unexpected side effect of GCC 4

[Duncan]

Peter Humphrey <prh@gotadsl.co.uk> posted
200611060848.16269.prh@gotadsl.co.uk, excerpted below, on  Mon, 06 Nov
2006 08:48:16 +0000:

> It's gentoolkit-0.2.3_pre2. I know it's deprecated, but that was done far
> too early, as at the time equery couldn't do half the things qpkg did.
> Definitely a case of too much haste.

OK, that's what I'm using.  I agree they cut it off a bit early, but
between equery and esearch, I think there's full functionality now.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Peter Humphrey]

On Monday 06 November 2006 08:48, Peter Humphrey wrote:
> On Monday 06 November 2006 07:50, Duncan wrote:
> > What version of gentoolkit do you have?  qpkg is deprecated and has
> > been since at at least gentoolkit 0.2.2-rc1, merged here back on May
> > 27, according to my logs.
>
> It's gentoolkit-0.2.3_pre2. I know it's deprecated, but that was done far
                                     ^^^^
qpkg, that is.

> too early, as at the time equery couldn't do half the things qpkg did.
> Definitely a case of too much haste.
>
> > Even if you are using an older version, consider switching to equery
> > instead, as qpkg hasn't been under development for some time and may
> > therefore have strange bugs.
>
> I do use equery more now, and sooner or later will probably drop qpkg
> quietly. I'm certainly not complaining of bugs in it.
>
> > I'd say this is probably one example, tho it's still working sort of
> > correctly for you.
>
> Quite so.
>
> --
> Rgds
> Peter

-- 
Rgds
Peter Humphrey
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Unexpected side effect of GCC 4

[Peter Humphrey]

On Sunday 05 November 2006 08:45, Peter Humphrey wrote:

> Now that I've finally managed to build a full system using GCC 4.1.1 on
> the workstation, I find its rsync traffic has leapt to full speed with no
> collisions to speak of. I'm not complaining, mind, but this is a puzzle.
> As far as I know I haven't changed anything else - certainly the firewall
> box has been running continuously throughout.

This morning, however, it's back to crippled rsync. I have changed 
something - I've put /tmp on tmpfs as in another thread. Surely that can't 
have done it?

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Unexpected side effect of GCC 4

[Duncan]

Peter Humphrey <prh@gotadsl.co.uk> posted
200611081014.09754.prh@gotadsl.co.uk, excerpted below, on  Wed, 08 Nov
2006 10:14:09 +0000:

> This morning, however, it's back to crippled rsync. I have changed 
> something - I've put /tmp on tmpfs as in another thread. Surely that can't 
> have done it?

Stranger and stranger.  I certainly wish you luck in figuring it out, and
please let us know what it was if/when you do.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Peter Humphrey]

On Monday 06 November 2006 15:51, Duncan wrote:

> If you have bios entries for configuring it [...] we could try to work
> thru it.  

I don't have those BIOS controls I'm afraid.

> Here, as I'm the only human user, I don't have to be [...] strict on
> security.  To keep things simple, /var/tmp is a symlink to /tmp, so I
> don't have to worry about a tmpfs for both dirs.  You'll want to set the
> following in make.conf:
>
> PKG_TMPDIR
> PORTAGE_TMPDIR
> PORTAGE_TMPFS

I decided on an even simpler approach. As I don't really want all the other 
stuff in /var/tmp to evaporate every time I shut down, I put /tmp on 
tmpfs, with this entry in /etc/fstab:
	tmpfs	/tmp		tmpfs	nodev,nosuid,size=85%		0 0

Then I added these entries to /etc/make.conf:
	BUILD_PREFIX="/tmp/portage/build"	# not sure I need this
	CCACHE_DIR="/var/tmp/ccache"	# to make sure it doesn't go looking in /tmp
	PORTAGE_TMPDIR="/tmp"

> Note that setting [PORTAGE_TMPFS] to a small, say 50 meg, tmpfs, is useful
> even if you aren't setting PORTAGE_TMPDIR on tmpfs.  It's used for
> quick/small stuff like lockfiles and the like.  The portage docs suggest
> setting it to /dev/shm, an LSB standard location for such things.  I have
> a separate (max 50 meg as I said) tmpfs mounted at /dev/shm and followed
> the recommendation to point PORTAGE_TMPFS at it.

Where did you find this recommendation? I can't find any reference to it.

> Of course, you'll only need PKG_TMPDIR if you have FEATURES=buildpkg set
> or otherwise deal with binary packages.

Same question.

> The disk spends most of its time idle, and you can watch the disk activity
> light and tell when the kernel's cache flush writes kick in, as it blinks
> red a couple times every few seconds.

My SATA disks don't show up on the disk activity LED, but I do have gkrellm 
permanently on my desktop and it shows disk activity. I'll watch it with 
interest  :-)

Thanks for the help.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Unexpected side effect of GCC 4

[Duncan]

Peter Humphrey <prh@gotadsl.co.uk> posted
200611081119.07931.prh@gotadsl.co.uk, excerpted below, on  Wed, 08 Nov
2006 11:19:07 +0000:

> I decided on an even simpler approach. As I don't really want all the other 
> stuff in /var/tmp to evaporate every time I shut down, I put /tmp on 
> tmpfs, with this entry in /etc/fstab:

> 	tmpfs	/tmp		tmpfs	nodev,nosuid,size=85%		0 0

Hmm...  I wasn't aware size could be set as a percentage of memory.  man
mount says size is in bytes, and offers the suffixes k, m and g, with no
mention of the % suffix, tho it does say the default is half of memory.

Does the 85% actually work as implied?  If so, I just learned something
new. =8^)
 
> Then I added these entries to /etc/make.conf:
> 	BUILD_PREFIX="/tmp/portage/build"	# not sure I need this

I don't think you do.  I don't have that var set here.

> 	CCACHE_DIR="/var/tmp/ccache" # to make sure it doesn't go looking in/tmp
>       PORTAGE_TMPDIR="/tmp"
> 
>> Note that setting [PORTAGE_TMPFS] to a small, say 50 meg, tmpfs, is
>> useful even if you aren't setting PORTAGE_TMPDIR on tmpfs.  It's used
>> for quick/small stuff like lockfiles and the like.  The portage docs
>> suggest setting it to /dev/shm, an LSB standard location for such
>> things.  I have a separate (max 50 meg as I said) tmpfs mounted at
>> /dev/shm and followed the recommendation to point PORTAGE_TMPFS at it.
> 
> Where did you find this recommendation? I can't find any reference to
> it.

PORTAGE_TMPFS is a fairly new setting that may not have made it to stable
portage yet. It doesn't appear to have made it into the make.conf manpage
yet, but from make.conf.example:

# PORTAGE_TMPFS is a location where portage may create temporary files.
#     If specified, portage will use this directory whenever possible
#     for all rapid operations such as lockfiles and transient data.
#     It is _highly_ recommended that this be a tmpfs or ramdisk. Do not
#     set this to anything that does not give a significant performance
#     enhancement and proper FS compliance for locks and read/write.
#     /dev/shm is a glibc mandated tmpfs, and should be a reasonable
#     setting for all linux kernel+glibc based systems.
#PORTAGE_TMPFS="/dev/shm"

>> Of course, you'll only need PKG_TMPDIR if you have FEATURES=buildpkg
>> set or otherwise deal with binary packages.
> 
> Same question.

Hmm... I don't see any documentation on this.  A google search on
PKG_TMPDIR turns up mostly BSD references (here you can see portage's BSD
heritage, Daniel Robbins found the concept there and liked it so created
portage for Gentoo when he started it).  However, adding the terms portage
or gentoo to the search helps.  There are a bit less than a hundred hits
in each of those two cases.  They are interesting but I don't see anything
authoritative.

I must have picked it up elsewhere, probably from either the gentoo-dev
or gentoo-portage lists, or from gentoo-user, which I followed for a few
months as a newbie.

Anyway, PKG_TMPDIR/binpkgs is used temporarily when merging binary
packages.  Thus, it'll be used with FEATURES=buildpkg, which I have set
and recommend as a very useful feature, as well as when a previously built
binary package is merged using -k or -K.  (The buildpkg FEATURE doesn't do
the usual ebuild qmerge step, but rather merges from the binary package
just created, thus testing it, using PKG_TMPDIR in the process.)  Note
that you only need to point the var at /tmp, as portage automatically uses
the binpkgs subdir of whatever it's pointed at, just as it uses the
portage subdir of PORTAGE_TMPDIR during the build process.

Interesting it doesn't seem to be documented, and PORTAGE_TMPFS is in
make.conf.example, but not in the make.conf manpage.  I should file a bug.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Re: Unexpected side effect of GCC 4

[Peter Humphrey]

On Wednesday 08 November 2006 15:17, Duncan wrote:
> Peter Humphrey <prh@gotadsl.co.uk> posted
> > I decided on an even simpler approach. As I don't really want all the
> > other stuff in /var/tmp to evaporate every time I shut down, I put /tmp
> > on tmpfs, with this entry in /etc/fstab:
> >
> > 	tmpfs	/tmp		tmpfs	nodev,nosuid,size=85%		0 0
>
> Does the 85% actually work as implied?  If so, I just learned something
> new. =8^)

Can't say yet. I got the info 
from /usr/src/linux/Documentation/filesystems/tmpfs.txt.

> > Then I added these entries to /etc/make.conf:
> > 	BUILD_PREFIX="/tmp/portage/build"	# not sure I need this
>
> I don't think you do.  I don't have that var set here.

I was being cautious, having seen hints that setting a directory name 
doesn't necessarily also set any others below it.

> PORTAGE_TMPFS is a fairly new setting that may not have made it to stable
> portage yet. It doesn't appear to have made it into the make.conf manpage
> yet, but from make.conf.example:

[...]

Thanks - there's always one more place to look :-(

> Interesting [that PKG_TMPDIR] doesn't seem to be documented, and
> PORTAGE_TMPFS is in make.conf.example, but not in the make.conf manpage. 
> I should file a bug.

Perhaps. I may also complain about the inconsistency that some directory 
specifications are literal while others point to a directory above where 
data will actually be used.

-- 
Rgds
Peter
-- 
gentoo-amd64@gentoo.org mailing list