[gentoo-user] systemd very slow to compile?

[gentoo-user] systemd very slow to compile?

[walt]

My very old and slow ~amd64 machine took 3 hours and 45-minutes to
compile systemd-226 today.  I was curious to know why it was taking so
long to finish, so I used 'top' to see what was happening.

Turns out that two instances of 'sh' were each using 15-30% of CPU for
a total of 30-60% (the machine has two CPUs).  cc1 never even showed up
in 'top' although the compiler was obviously compiling code because the
build did eventually finish.

I tried the same on a faster 4-core machine and I could see much the
same thing happening during the systemd build.

Can anyone else reproduce what I'm seeing?  Is this 'normal'?

Re: [gentoo-user] systemd very slow to compile?

[Mike Gilbert]

On Fri, Sep 11, 2015 at 6:08 PM, walt <w41ter@gmail.com> wrote:
> My very old and slow ~amd64 machine took 3 hours and 45-minutes to
> compile systemd-226 today.  I was curious to know why it was taking so
> long to finish, so I used 'top' to see what was happening.
>
> Turns out that two instances of 'sh' were each using 15-30% of CPU for
> a total of 30-60% (the machine has two CPUs).  cc1 never even showed up
> in 'top' although the compiler was obviously compiling code because the
> build did eventually finish.
>
> I tried the same on a faster 4-core machine and I could see much the
> same thing happening during the systemd build.
>
> Can anyone else reproduce what I'm seeing?  Is this 'normal'?

I have noticed that systemd takes quite a long time to build, but I
have never spent any effort looking into exactly why.

If you figure anything out, please file a bug with your findings.

Re: [gentoo-user] systemd very slow to compile?

[Marc Joliet]

[-- Attachment #1.1: Type: text/plain, Size: 5994 bytes --]

=2D-nextPart10531075.nJdPiP2ksp
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="utf-8"

On Friday 11 September 2015 15:08:54 walt wrote:
>My very old and slow ~amd64 machine took 3 hours and 45-minutes to
>compile systemd-226 today.

Just out of curiosity: exactly how old?  My dual-core amd64 system is almost 9 
years old now, and systemd compiles in about 6 minutes:

# genlop -t systemd
 * sys-apps/systemd

     Fri Feb 20 21:54:48 2015 >>> sys-apps/systemd-216-r3
       merge time: 6 minutes and 16 seconds.

     Sun Feb 22 18:14:19 2015 >>> sys-apps/systemd-216-r3
       merge time: 25 seconds.

     Mon Apr 27 18:54:06 2015 >>> sys-apps/systemd-218-r3
       merge time: 6 minutes and 5 seconds.

     Sun Jul 12 00:28:48 2015 >>> sys-apps/systemd-218-r3
       merge time: 6 minutes and 2 seconds.

     Tue Aug 11 22:54:55 2015 >>> sys-apps/systemd-218-r3
       merge time: 6 minutes and 5 seconds.

     Mon Sep  7 23:57:50 2015 >>> sys-apps/systemd-218-r3
       merge time: 12 minutes and 30 seconds.

(My system might have been pretty busy on that last one, and the second one 
must have been a binpkg merge.)

>I was curious to know why it was taking so
>long to finish, so I used 'top' to see what was happening.
>
>Turns out that two instances of 'sh' were each using 15-30% of CPU for
>a total of 30-60% (the machine has two CPUs).  cc1 never even showed up
>in 'top' although the compiler was obviously compiling code because the
>build did eventually finish.
>
>I tried the same on a faster 4-core machine and I could see much the
>same thing happening during the systemd build.
>
>Can anyone else reproduce what I'm seeing?  Is this 'normal'?

Perhaps it is a problem only in the build system of newer systemd versions?  
In any case, you can see my USE flags in the attached file for comparison 
(sorry, I couldn't get KMail -- which I am still getting used to -- to stop 
it's automatic line wrapping just for those lines, if it even supports that).

HTH
=2D- 
Marc Joliet
=2D-
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup
=2D- 
Marc Joliet
=2D-
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup
=2D- 
Marc Joliet
=2D-
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup
=2D- 
Marc Joliet
=2D-
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

=2D-nextPart10531075.nJdPiP2ksp
Content-Disposition: inline; filename="systemd use flags.txt"
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="utf-8"; name="systemd use flags.txt"

# equery uses systemd
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for sys-apps/systemd-218-r3:
 U I
 + + abi_x86_32                     : 32-bit (x86) libraries
 + + acl                            : Add support for Access Control Lists
 - - audit                          : Enable support for sys-process/audit
 - - cryptsetup                     : Enable cryptsetup tools (includes unit generator for crypttab)
 - - curl                           : Enable support for uploading journals; required to build systemd-import/systemd-pull
 - - doc                            : Add extra documentation (API, Javadoc, etc). It is recommended to enable per package instead of globally
 - - elfutils                       : Enable coredump stacktraces in the journal
 - - gcrypt                         : Enable sealing of journal files using gcrypt; required to build systemd-import/systemd-pull
 + + gudev                          : enable libudev gobject interface
 - - http                           : Enable embedded HTTP server in journald
 + + idn                            : Enable support for Internationalized Domain Names
 + + introspection                  : Add support for GObject based introspection
 - - kdbus                          : Connect to kernel dbus (KDBUS) instead of userspace dbus if available
 + + kmod                           : Enable kernel module loading via sys-apps/kmod
 + + lz4                            : Enable lz4 compression for the journal
 + + lzma                           : Support for LZMA (de)compression algorithm
 + + pam                            : Add support for PAM (Pluggable Authentication Modules) - DANGEROUS to arbitrarily flip
 - - python                         : Add optional support/bindings for the Python language
 + + python_single_target_python2_7 : Build for Python 2.7 only
 - - python_single_target_python3_3 : Build for Python 3.3 only
 - - python_single_target_python3_4 : Build for Python 3.4 only
 + + python_targets_python2_7       : Build with Python 2.7
 - - python_targets_python3_3       : Build with Python 3.3
 + + python_targets_python3_4       : Build with Python 3.4
 - - qrcode                         : Enable qrcode output support in journal
 + + seccomp                        : Enable seccomp (secure computing mode) to perform system call filtering at runtime to increase security of programs
 + + ssl                            : Add support for Secure Socket Layer connections
 + + sysv-utils                     : Install sysvinit compatibility symlinks and manpages for init, telinit, halt, poweroff, reboot, runlevel, and shutdown
 - - terminal                       : Enable experimental userspace virtual terminal support
 - - test                           : Workaround to pull in packages needed to run with FEATURES=test. Portage-2.1.2 handles this internally, so don't set it in make.conf/package.use anymore
 - - vanilla                        : Disable Gentoo-specific behavior and compatibility quirks
 - - xkb                            : Validate XKB keymap in logind

=2D-nextPart10531075.nJdPiP2ksp--
This is a multi-part message in MIME format.

[-- Attachment #1.2: Type: text/plain, Size: 2383 bytes --]

On Friday 11 September 2015 15:08:54 walt wrote:
>My very old and slow ~amd64 machine took 3 hours and 45-minutes to
>compile systemd-226 today.

Just out of curiosity: exactly how old?  My dual-core amd64 system is almost 9 
years old now, and systemd compiles in about 6 minutes:

# genlop -t systemd
 * sys-apps/systemd

     Fri Feb 20 21:54:48 2015 >>> sys-apps/systemd-216-r3
       merge time: 6 minutes and 16 seconds.

     Sun Feb 22 18:14:19 2015 >>> sys-apps/systemd-216-r3
       merge time: 25 seconds.

     Mon Apr 27 18:54:06 2015 >>> sys-apps/systemd-218-r3
       merge time: 6 minutes and 5 seconds.

     Sun Jul 12 00:28:48 2015 >>> sys-apps/systemd-218-r3
       merge time: 6 minutes and 2 seconds.

     Tue Aug 11 22:54:55 2015 >>> sys-apps/systemd-218-r3
       merge time: 6 minutes and 5 seconds.

     Mon Sep  7 23:57:50 2015 >>> sys-apps/systemd-218-r3
       merge time: 12 minutes and 30 seconds.

(My system might have been pretty busy on that last one, and the second one 
must have been a binpkg merge.)

>I was curious to know why it was taking so
>long to finish, so I used 'top' to see what was happening.
>
>Turns out that two instances of 'sh' were each using 15-30% of CPU for
>a total of 30-60% (the machine has two CPUs).  cc1 never even showed up
>in 'top' although the compiler was obviously compiling code because the
>build did eventually finish.
>
>I tried the same on a faster 4-core machine and I could see much the
>same thing happening during the systemd build.
>
>Can anyone else reproduce what I'm seeing?  Is this 'normal'?

Perhaps it is a problem only in the build system of newer systemd versions?  
In any case, you can see my USE flags in the attached file for comparison 
(sorry, I couldn't get KMail -- which I am still getting used to -- to stop 
it's automatic line wrapping just for those lines, if it even supports that).

HTH
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup
-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #1.3: systemd use flags.txt --]
[-- Type: text/plain, Size: 3208 bytes --]

# equery uses systemd
[ Legend : U - final flag setting for installation]
[        : I - package is installed with flag     ]
[ Colors : set, unset                             ]
 * Found these USE flags for sys-apps/systemd-218-r3:
 U I
 + + abi_x86_32                     : 32-bit (x86) libraries
 + + acl                            : Add support for Access Control Lists
 - - audit                          : Enable support for sys-process/audit
 - - cryptsetup                     : Enable cryptsetup tools (includes unit generator for crypttab)
 - - curl                           : Enable support for uploading journals; required to build systemd-import/systemd-pull
 - - doc                            : Add extra documentation (API, Javadoc, etc). It is recommended to enable per package instead of globally
 - - elfutils                       : Enable coredump stacktraces in the journal
 - - gcrypt                         : Enable sealing of journal files using gcrypt; required to build systemd-import/systemd-pull
 + + gudev                          : enable libudev gobject interface
 - - http                           : Enable embedded HTTP server in journald
 + + idn                            : Enable support for Internationalized Domain Names
 + + introspection                  : Add support for GObject based introspection
 - - kdbus                          : Connect to kernel dbus (KDBUS) instead of userspace dbus if available
 + + kmod                           : Enable kernel module loading via sys-apps/kmod
 + + lz4                            : Enable lz4 compression for the journal
 + + lzma                           : Support for LZMA (de)compression algorithm
 + + pam                            : Add support for PAM (Pluggable Authentication Modules) - DANGEROUS to arbitrarily flip
 - - python                         : Add optional support/bindings for the Python language
 + + python_single_target_python2_7 : Build for Python 2.7 only
 - - python_single_target_python3_3 : Build for Python 3.3 only
 - - python_single_target_python3_4 : Build for Python 3.4 only
 + + python_targets_python2_7       : Build with Python 2.7
 - - python_targets_python3_3       : Build with Python 3.3
 + + python_targets_python3_4       : Build with Python 3.4
 - - qrcode                         : Enable qrcode output support in journal
 + + seccomp                        : Enable seccomp (secure computing mode) to perform system call filtering at runtime to increase security of programs
 + + ssl                            : Add support for Secure Socket Layer connections
 + + sysv-utils                     : Install sysvinit compatibility symlinks and manpages for init, telinit, halt, poweroff, reboot, runlevel, and shutdown
 - - terminal                       : Enable experimental userspace virtual terminal support
 - - test                           : Workaround to pull in packages needed to run with FEATURES=test. Portage-2.1.2 handles this internally, so don't set it in make.conf/package.use anymore
 - - vanilla                        : Disable Gentoo-specific behavior and compatibility quirks
 - - xkb                            : Validate XKB keymap in logind

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-user] systemd very slow to compile?

[Peter Humphrey]

On Sunday 13 September 2015 13:16:29 Marc Joliet wrote:

> I couldn't get KMail -- which I am still getting used to -- to stop it's
> automatic line wrapping just for those lines, if it even supports that

You can switch it on or off, but it applies to the whole message. What I do if 
I want a line not wrapped is to go through the rest of the message and insert 
a line break in place of each line-end space, then switch wrapping off. 
Laborious if it's a long message, but it does at least work.

Welcome to the Brave New World of a mostly good MUA. :)

-- 
Rgds
Peter

Re: [gentoo-user] systemd very slow to compile?

[Marc Joliet]

[-- Attachment #1: Type: text/plain, Size: 1262 bytes --]

On Sunday 13 September 2015 13:57:04 Peter Humphrey wrote:
>On Sunday 13 September 2015 13:16:29 Marc Joliet wrote:
>> I couldn't get KMail -- which I am still getting used to -- to stop it's
>> automatic line wrapping just for those lines, if it even supports that
>
>You can switch it on or off, but it applies to the whole message. What I do
>if I want a line not wrapped is to go through the rest of the message and
>insert a line break in place of each line-end space, then switch wrapping
>off. Laborious if it's a long message, but it does at least work.

Yeah, I was worried about that :( .  Personally, I think I'll stick to using 
attachments if I need the formatting to stay absolutely the same, although 
that has the downside of not being as easily quotable.

>Welcome to the Brave New World of a mostly good MUA. :)

Since it's off-topic, I'll stick to saying this much: I'm liking a lot of it, 
but not all of it (but claws-mail isn't perfect, either).  Using KMail is part 
of my "let's see what running a full KDE desktop is like" experiment, which 
has been pretty successful for the most part.

-- 
Marc Joliet
--
"People who think they know everything really annoy those of us who know we
don't" - Bjarne Stroustrup

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [gentoo-user] systemd very slow to compile?

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1068 bytes --]

On Sun, 13 Sep 2015 13:16:29 +0200, Marc Joliet wrote:

> On Friday 11 September 2015 15:08:54 walt wrote:
> >My very old and slow ~amd64 machine took 3 hours and 45-minutes to
> >compile systemd-226 today.  
> 
> Just out of curiosity: exactly how old?  My dual-core amd64 system is
> almost 9 years old now, and systemd compiles in about 6 minutes:
> 
> # genlop -t systemd
>  * sys-apps/systemd
>      Mon Sep  7 23:57:50 2015 >>> sys-apps/systemd-218-r3
>        merge time: 12 minutes and 30 seconds.
> 
> (My system might have been pretty busy on that last one, and the second
> one must have been a binpkg merge.)

> Perhaps it is a problem only in the build system of newer systemd
> versions?

It appears to have only occurred with systemd-226, which is why you are
not seeing it. I won't post my emerge times because portage was
building Chromium at the same time, but even so they were hours in 
excess of previous builds.


-- 
Neil Bothwick

"There's more to life than sex, beer and computers.
Not a lot more admittedly..."

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]