[gentoo-user] Puzzled with duration of chromium emerge under profile 17.0

[gentoo-user] Puzzled with duration of chromium emerge under profile 17.0

[Mick]

[-- Attachment #1.1: Type: text/plain, Size: 1673 bytes --]

I discovered that building Chromium with gcc-6.4.0 is taking an inordinately 
longer time on a laptop with 1st gen i7 and 4G of RAM, e.g.: 

     Wed Sep 27 17:36:53 2017 >>> www-client/chromium-61.0.3163.100
       merge time: 6 hours, 40 minutes and 50 seconds.

     Thu Nov  9 17:44:58 2017 >>> www-client/chromium-62.0.3202.89
       merge time: 8 hours, 12 minutes and 30 seconds.

-->switch to gcc-6.4.0

     Mon Dec  4 11:39:36 2017 >>> www-client/chromium-62.0.3202.89
       merge time: 20 hours, 2 minutes and 4 seconds.

     Wed Dec  6 18:41:13 2017 >>> www-client/chromium-62.0.3202.94
       merge time: 22 hours, 47 minutes and 35 seconds.


but not so on another older and lesser Intel(R) Core(TM)2 Duo CPU P7550  @ 
2.26GHz, also with 4G RAM:

     Wed Sep 27 22:25:32 2017 >>> www-client/chromium-61.0.3163.100
       merge time: 11 hours, 46 minutes and 18 seconds.

     Thu Nov  9 22:09:59 2017 >>> www-client/chromium-62.0.3202.89
       merge time: 13 hours, 16 minutes and 41 seconds.

-->switch to gcc-6.4.0

     Sat Dec  2 21:00:59 2017 >>> www-client/chromium-62.0.3202.89
       merge time: 15 hours, 35 minutes and 50 seconds.

     Mon Dec  4 03:44:12 2017 >>> www-client/chromium-62.0.3202.94
       merge time: 15 hours, 40 minutes and 18 seconds.


Any idea why this is happening?  I attach emerge info of the i7 in case you 
can spot something which may be causing this exponential increase in emerge 
times.  BTW, on the i7 I had to increase swap because the 4,200,960 KiB swap 
partition was not enough to complete the compilation of version 62.0.3202.89, 
even after I shut down all applications and exited X.  O_O

-- 
Regards,
Mick

[-- Attachment #1.2: chromium_info.txt --]
[-- Type: text/plain, Size: 7430 bytes --]

~ $ emerge --info www-client/chromium
Portage 2.3.13 (python 3.5.4-final-0, default/linux/amd64/17.0/desktop/plasma, g
cc-6.4.0, glibc-2.25-r9, 4.12.12-gentoo x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-4.12.12-gentoo-x86_64-Intel-R-_Core-TM-_i7_CPU_Q_720_@_1.60G
Hz-with-gentoo-2.4.1
KiB Mem:     4032296 total,   2050332 free
KiB Swap:    4200960 total,   4200960 free
Timestamp of repository gentoo: Wed, 06 Dec 2017 18:15:01 +0000
Head commit of repository gentoo: f2378f105da3bbdc56fe40323b97dd31044b8dcc
sh bash 4.3_p48-r1
ld GNU ld (Gentoo 2.29.1 p3) 2.29.1
app-shells/bash:          4.3_p48-r1::gentoo
dev-lang/perl:            5.24.3::gentoo
dev-lang/python:          2.7.14::gentoo, 3.5.4::gentoo
dev-util/cmake:           3.8.2::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.4.1-r2::gentoo
sys-apps/openrc:          0.34.11::gentoo
sys-apps/sandbox:         2.10-r4::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69::gentoo
sys-devel/automake:       1.15.1-r1::gentoo
sys-devel/binutils:       2.29.1-r1::gentoo
sys-devel/gcc:            6.4.0::gentoo
sys-devel/gcc-config:     1.8-r1::gentoo
sys-devel/libtool:        2.4.6-r3::gentoo
sys-devel/make:           4.2.1::gentoo
sys-kernel/linux-headers: 4.4::gentoo (virtual/os-headers)
sys-libs/glibc:           2.25-r9::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://10.10.10.2/gentoo-portage
    priority: -1000
    sync-rsync-extra-opts: --exclude-from=/etc/portage/rsync_excludes

bar
    location: /var/lib/layman/bar
    masters: gentoo
    priority: 50

brother-overlay
    location: /var/lib/layman/brother-overlay
    masters: gentoo
    priority: 50

psix-overlay
    location: /var/lib/layman/psix-overlay
    masters: gentoo
    priority: 50

local
    location: /usr/local/portage
    masters: gentoo
    priority: 100

Installed sets: @enlightenment
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config
/usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/
fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /et
c/terminfo"
CXXFLAGS="-march=native -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--quiet-build=n"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild
-locks fail-clean fixlafiles merge-sync multilib-strict network-sandbox news par
allel-fetch preserve-libs protect-owned sandbox sfperms splitdebug strict unknow
n-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox user
sync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://10.10.10.2:1024/ http://mirror.qubenet.net/mirror/gentoo/
 rsync://rsync.mirrorservice.org/distfiles.gentoo.org/ http://ftp.snt.utwente.nl
/pub/os/linux/gentoo ftp://ftp.snt.utwente.nl/pub/os/linux/gentoo http://www.mir
rorservice.org/sites/distfiles.gentoo.org/"
LANG="en_GB.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j5 -l12.8"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="--exclude-from=/etc/portage/rsync_excludes"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-
times --compress --force --whole-file --delete --stats --human-readable --timeou
t=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X a52 aac aacplus acl acpi activities alsa amd64 amr apng avx berkdb blueto
oth branding bzip2 cairo cdda cddb cdparanoia cdr chroot cli consolekit cracklib
 crypt css cups cxx dbus declarative dell dri dts dvd dvdr emboss encode exif fa
ac fam ffmpeg firefox flac fortran fts3 gbm gdbm gif glamor gpm hddtemp hpijs ic
onv imagemagick ipv6 irda jpeg kde kipi kwallet lcms ldap libnotify libv4l2 live
 lm_sensors logrotate mad mmx mmxext mng modules mp3 mp4 mpeg multilib ncurses n
etwork-cron new-hpcups nls nodrm nptl nsplugin obex ogg opengl openmp opus pam p
ango pcre pdf phonon plasma png policykit popcnt ppds pulseaudio qml qt3support
qt5 quicktime rdesktop readline redland rtmp sdl seccomp semantic-desktop sessio
n shout smime spell sqlite sse sse2 sse4_1 sse4_2 ssl ssse3 startup-notification
 svg tcpd tiff truetype udev udisks unicode upower usb v4l2 vaapi vdpau vnc vorb
is vpx wayland widgets win32codecs wps wxwidgets x264 xattr xcb xcomposite xiner
ama xml xv xvid xvmc zip zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp a
tiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda
-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpc
i" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth
_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz
_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi c
gid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache
 filter headers include info log_config logio mem_cache mime mime_magic negotiat
ion rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CAL
LIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump
author" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_
FLAGS_X86="mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GP
SD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock is
ync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm10
4v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="s
ynaptics evdev" KERNEL="linux" L10N="en-GB en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer pdfimport" LINGUAS="en_GB en" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-0" POSTGRES_TARGETS="postgres9_5" PYTHON_SINGLE_TARGET="python3_5" PYTHON_TARGETS="python2_7 python3_5" RUBY_TARGETS="ruby22" USERLAND="GNU" VIDEO_CARDS="radeon r600" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS

=================================================================
                        Package Settings
=================================================================

www-client/chromium-62.0.3202.94::gentoo was built with the following:
USE="cups hangouts (pic) proprietary-codecs pulseaudio suid system-ffmpeg -component-build -custom-cflags -gnome-keyring -kerberos (-neon) (-selinux) (-system-icu) (-system-libvpx) (-tcmalloc) -widevine" ABI_X86="(64)" L10N="en-GB -am -ar -bg -bn -ca -cs -da -de -el -es -es-419 -et -fa -fi -fil -fr -gu -he -hi -hr -hu -id -it -ja -kn -ko -lt -lv -ml -mr -ms -nb -nl -pl -pt-BR -pt-PT -ro -ru -sk -sl -sr -sv -sw -ta -te -th -tr -uk -vi -zh-CN -zh-TW"
CXXFLAGS="-march=native -O2 -pipe -fno-delete-null-pointer-checks"


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Puzzled with duration of chromium emerge under profile 17.0

[Alan McKinnon]

On 06/12/2017 21:10, Mick wrote:
> I discovered that building Chromium with gcc-6.4.0 is taking an inordinately 
> longer time on a laptop with 1st gen i7 and 4G of RAM, e.g.: 
> 
>      Wed Sep 27 17:36:53 2017 >>> www-client/chromium-61.0.3163.100
>        merge time: 6 hours, 40 minutes and 50 seconds.
> 
>      Thu Nov  9 17:44:58 2017 >>> www-client/chromium-62.0.3202.89
>        merge time: 8 hours, 12 minutes and 30 seconds.
> 
> -->switch to gcc-6.4.0
> 
>      Mon Dec  4 11:39:36 2017 >>> www-client/chromium-62.0.3202.89
>        merge time: 20 hours, 2 minutes and 4 seconds.
> 
>      Wed Dec  6 18:41:13 2017 >>> www-client/chromium-62.0.3202.94
>        merge time: 22 hours, 47 minutes and 35 seconds.
> 
> 
> but not so on another older and lesser Intel(R) Core(TM)2 Duo CPU P7550  @ 
> 2.26GHz, also with 4G RAM:
> 
>      Wed Sep 27 22:25:32 2017 >>> www-client/chromium-61.0.3163.100
>        merge time: 11 hours, 46 minutes and 18 seconds.
> 
>      Thu Nov  9 22:09:59 2017 >>> www-client/chromium-62.0.3202.89
>        merge time: 13 hours, 16 minutes and 41 seconds.
> 
> -->switch to gcc-6.4.0
> 
>      Sat Dec  2 21:00:59 2017 >>> www-client/chromium-62.0.3202.89
>        merge time: 15 hours, 35 minutes and 50 seconds.
> 
>      Mon Dec  4 03:44:12 2017 >>> www-client/chromium-62.0.3202.94
>        merge time: 15 hours, 40 minutes and 18 seconds.
> 
> 
> Any idea why this is happening?  I attach emerge info of the i7 in case you 
> can spot something which may be causing this exponential increase in emerge 
> times.  BTW, on the i7 I had to increase swap because the 4,200,960 KiB swap 
> partition was not enough to complete the compilation of version 62.0.3202.89, 
> even after I shut down all applications and exited X.  O_O
> 


Pure gut feel and intuition and nothing else leads me to look in two places:

You use -march=native on the i7 so I assume the same on the Core2? Those
are rather different processors, and google is fond of optimizing deeply
for specific cases (common to all browsers I think). You'd have to ask a
chromium hacker but I'd say the odds are good there are serious
optimizations for i7 that stress your compiler out muchly.

Add to that your i7 is RAM-constrained so you compensate with swap,
which is easily 50,000 times slower with sucky latency. When you use a
disk as RAM, performance tanks. Well, usually it causes a cascade effect
and stuff blows up, but if it completes it will have done so slowly.

If you at all can, shove lots more RAM in that i7. These days RAM is
cheap and it's always by first performance tweak, then SSD.

-- 
Alan McKinnon
alan.mckinnon@gmail.com

[gentoo-user] Re: Puzzled with duration of chromium emerge under profile 17.0

[Grant Edwards]

On 2017-12-06, Mick <michaelkintzios@gmail.com> wrote:

> I discovered that building Chromium with gcc-6.4.0 is taking an inordinately 
> longer time on a laptop with 1st gen i7 and 4G of RAM, e.g.: 
>
>      Wed Sep 27 17:36:53 2017 >>> www-client/chromium-61.0.3163.100
>        merge time: 6 hours, 40 minutes and 50 seconds.
>
>      Thu Nov  9 17:44:58 2017 >>> www-client/chromium-62.0.3202.89
>        merge time: 8 hours, 12 minutes and 30 seconds.
>
> -->switch to gcc-6.4.0
>
>      Mon Dec  4 11:39:36 2017 >>> www-client/chromium-62.0.3202.89
>        merge time: 20 hours, 2 minutes and 4 seconds.
>
>      Wed Dec  6 18:41:13 2017 >>> www-client/chromium-62.0.3202.94
>        merge time: 22 hours, 47 minutes and 35 seconds.

Did the CPU throttling stuff somehow get broken when you updated to
gcc-6.4.0?  After updating a bunch of stuff a month or two back, I
noticed that builds suddently took 4X as long.  I finally realized
that I had broken the CPU throttling feature and my laptop was always
running at 400MHz and not ramping up to 2.<whatever> GHz when doing
things like compiling large packages.

-- 
Grant Edwards               grant.b.edwards        Yow! I'm using my X-RAY
                                  at               VISION to obtain a rare
                              gmail.com            glimpse of the INNER
                                                   WORKINGS of this POTATO!!

Re: [gentoo-user] Puzzled with duration of chromium emerge under profile 17.0

[Mick]

[-- Attachment #1: Type: text/plain, Size: 1659 bytes --]

On Wednesday, 6 December 2017 20:13:50 GMT Alan McKinnon wrote:
> On 06/12/2017 21:10, Mick wrote:
> > I discovered that building Chromium with gcc-6.4.0 is taking an
> > inordinately> 
> > longer time on a laptop with 1st gen i7 and 4G of RAM, e.g.:
[snip...]

> Pure gut feel and intuition and nothing else leads me to look in two places:
> 
> You use -march=native on the i7 so I assume the same on the Core2? Those
> are rather different processors, and google is fond of optimizing deeply
> for specific cases (common to all browsers I think). You'd have to ask a
> chromium hacker but I'd say the odds are good there are serious
> optimizations for i7 that stress your compiler out muchly.

Yes, I run -march=native on both.

> Add to that your i7 is RAM-constrained so you compensate with swap,
> which is easily 50,000 times slower with sucky latency. 

Yes, it's ridiculously slow!  :-(


> When you use a
> disk as RAM, performance tanks. Well, usually it causes a cascade effect
> and stuff blows up, but if it completes it will have done so slowly.
> 
> If you at all can, shove lots more RAM in that i7. These days RAM is
> cheap and it's always by first performance tweak, then SSD.

I know that compiling in RAM would be done in a fraction of the time.  The 
thing is, this is a 8 year old laptop and I am resisting throwing good money 
after bad.  I had a quick look a few months ago and good quality memory will 
cost me around £60.  With the battery shot and the keyboard on its way out, 
I'd rather put the money towards more memory for a newer PC, sometime in the 
next year.  ;-)

-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: Puzzled with duration of chromium emerge under profile 17.0

[Mick]

[-- Attachment #1.1: Type: text/plain, Size: 1104 bytes --]

On Wednesday, 6 December 2017 21:01:25 GMT Grant Edwards wrote:
> On 2017-12-06, Mick <michaelkintzios@gmail.com> wrote:
> > I discovered that building Chromium with gcc-6.4.0 is taking an
> > inordinately> 
> > longer time on a laptop with 1st gen i7 and 4G of RAM, e.g.:
[snip ...]

> Did the CPU throttling stuff somehow get broken when you updated to
> gcc-6.4.0?  After updating a bunch of stuff a month or two back, I
> noticed that builds suddently took 4X as long.  I finally realized
> that I had broken the CPU throttling feature and my laptop was always
> running at 400MHz and not ramping up to 2.<whatever> GHz when doing
> things like compiling large packages.

Hmm ... I have not changed anything related to CPU throttling I can remember.  
Which reminds me, perhaps I should 'make clean' my kernel now that I have 
switched to gcc-6.4.0?

i7z shows turbo mode kicks in too and hyperthreading is on (attached).  Under 
turbo it jumps up to 2.8GHz.  So cpu throttling is probably not the cause of 
my problem, unless it throttles more now than it used to do before?  :-/
 
-- 
Regards,
Mick

[-- Attachment #1.2: i7z.txt --]
[-- Type: text/plain, Size: 1502 bytes --]

Cpu speed from cpuinfo 1595.00Mhz
cpuinfo might be wrong if cpufreq is enabled. To guess correctly try estimating via tsc
Linux's inbuilt cpu_khz code emulated now
True Frequency (without accounting Turbo) 1595 MHz
  CPU Multiplier 12x || Bus clock frequency (BCLK) 132.92 MHz

Socket [0] - [physical cores=4, logical cores=8, max online cores ever=4]
  TURBO ENABLED on 4 Cores, Hyper Threading ON
  Max Frequency without considering Turbo 1727.92 MHz (132.92 x [13])
  Max TURBO Multiplier (if Enabled) with 1/2/3/4 Cores is  21x/18x/13x/13x
  Real Current Frequency 2564.92 MHz [132.92 x 19.30] (Max of below)
        Core [core-id]  :Actual Freq (Mult.)	  C0%   Halt(C1)%  C3 %   C6 %   C7 %  Temp	 VCore
        Core 1 [0]:	  1666.27 (12.54x)	12.5    5.72    22.8    58.4	   0    62	0.0000
        Core 2 [1]:	  1615.74 (12.16x)	11.1    6.62    12.8    69.3	   0    62	0.0000
        Core 3 [2]:	  1747.37 (13.15x)	14.5    5.95    13.9    64.3	   0    62	0.0000
        Core 4 [3]:	  2564.92 (19.30x)	99.4	   0	   0	   0	   0    72	0.0000



C0 = Processor running without halting
C1 = Processor running with halts (States >C0 are power saver modes with cores idling)
C3 = Cores running with PLL turned off and core cache turned off
C6, C7 = Everything in C3 + core state saved to last level cache, C7 is deeper than C6
  Above values in table are in percentage over the last 1 sec
[core-id] refers to core-id number in /proc/cpuinfo
'Garbage Values' message printed when garbage values are read


[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[gentoo-user] Re: Puzzled with duration of chromium emerge under profile 17.0

[Grant Edwards]

On 2017-12-06, Mick <michaelkintzios@gmail.com> wrote:
> On Wednesday, 6 December 2017 21:01:25 GMT Grant Edwards wrote:
>> On 2017-12-06, Mick <michaelkintzios@gmail.com> wrote:
>> > I discovered that building Chromium with gcc-6.4.0 is taking an
>> > inordinately> 
>> > longer time on a laptop with 1st gen i7 and 4G of RAM, e.g.:
> [snip ...]
>
>> Did the CPU throttling stuff somehow get broken when you updated to
>> gcc-6.4.0?
[...]
> Hmm ... I have not changed anything related to CPU throttling I can remember.  
> Which reminds me, perhaps I should 'make clean' my kernel now that I have 
> switched to gcc-6.4.0?

Shouldn't need to.  I think my throttling got broken when I updated
kernels.

> i7z shows turbo mode kicks in too and hyperthreading is on (attached).  Under 
> turbo it jumps up to 2.8GHz.  So cpu throttling is probably not the cause of 
> my problem, unless it throttles more now than it used to do before?  :-/

If throttling were broken it would be pretty obvious: when you do 'cat
/proc/cpuinfo' in the middle of an emerge, it would show 500MHz (or
whatever) as the core frequency.  [It still took me half a day to
find out that's what was wrong.]

-- 
Grant Edwards               grant.b.edwards        Yow! I want to dress you
                                  at               up as TALLULAH BANKHEAD and
                              gmail.com            cover you with VASELINE and
                                                   WHEAT THINS ...