public inbox for gentoo-user@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-user] Delays while building Libre Office.
@ 2013-05-02 16:27 Alan Mackenzie
  2013-05-02 16:33 ` Michael Mol
                   ` (5 more replies)
  0 siblings, 6 replies; 15+ messages in thread
From: Alan Mackenzie @ 2013-05-02 16:27 UTC (permalink / raw
  To: gentoo-user

Hi, Gentoo.

I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes on my
2.6 GHz quad core Athlon 2.  It used to take about an hour.

Watching the build, it became evident that the first 50 minutes or so
was taken up by several hundred mkdir operations (more precisely, mkdir
-p <long path>).  Some of these mkdir's would take, perhaps, a minute to
execute.  All the while, top showed make taking 100% of one core.

There seems to be something suboptimal here.  Has anybody else seen
this, or does anybody have any ideas how to fix the problem?

-- 
Alan Mackenzie (Nuremberg, Germany).


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Delays while building Libre Office.
  2013-05-02 16:27 [gentoo-user] Delays while building Libre Office Alan Mackenzie
@ 2013-05-02 16:33 ` Michael Mol
  2013-05-02 16:58   ` Alan Mackenzie
  2013-05-02 16:49 ` Philip Webb
                   ` (4 subsequent siblings)
  5 siblings, 1 reply; 15+ messages in thread
From: Michael Mol @ 2013-05-02 16:33 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 971 bytes --]

On 05/02/2013 12:27 PM, Alan Mackenzie wrote:
> Hi, Gentoo.
> 
> I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes on my
> 2.6 GHz quad core Athlon 2.  It used to take about an hour.
> 
> Watching the build, it became evident that the first 50 minutes or so
> was taken up by several hundred mkdir operations (more precisely, mkdir
> -p <long path>).  Some of these mkdir's would take, perhaps, a minute to
> execute.  All the while, top showed make taking 100% of one core.
> 
> There seems to be something suboptimal here.  Has anybody else seen
> this, or does anybody have any ideas how to fix the problem?

Long delays suggest a timeout of some sort.

First thing I'd look at is the filesystem underneath, and the disk
underneath that.

Second thing I'd look at is to see if permissions checks might be
bouncing through something like kerberos, samba or ldap. Do you have any
single-signon things configured on that machine?


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 555 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Delays while building Libre Office.
  2013-05-02 16:27 [gentoo-user] Delays while building Libre Office Alan Mackenzie
  2013-05-02 16:33 ` Michael Mol
@ 2013-05-02 16:49 ` Philip Webb
  2013-05-04  7:42 ` Stroller
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 15+ messages in thread
From: Philip Webb @ 2013-05-02 16:49 UTC (permalink / raw
  To: gentoo-user

130502 Alan Mackenzie wrote:
> I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes
> on my 2.6 GHz quad core Athlon 2.  It used to take about an hour.

I've been using 4.0.0.1 w/o problems since 130113 : 4.0.2.2 is in testing.
It took  1 h 10 m  & used  c 3,5 GB  for temporary storage.
You might try it & report the result.

You have to be careful if you mix stable/testing for system pkgs,
but using the testing version of something well-maintained,
like LO or KDE, shouldn't cause problems normally.

-- 
========================,,============================================
SUPPORT     ___________//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT    `-O----------O---'   purslowatchassdotutorontodotca



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Delays while building Libre Office.
  2013-05-02 16:33 ` Michael Mol
@ 2013-05-02 16:58   ` Alan Mackenzie
  2013-05-02 17:15     ` Michael Mol
  0 siblings, 1 reply; 15+ messages in thread
From: Alan Mackenzie @ 2013-05-02 16:58 UTC (permalink / raw
  To: gentoo-user

On Thu, May 02, 2013 at 12:33:37PM -0400, Michael Mol wrote:
> On 05/02/2013 12:27 PM, Alan Mackenzie wrote:
> > Hi, Gentoo.

> > I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes on my
> > 2.6 GHz quad core Athlon 2.  It used to take about an hour.

> > Watching the build, it became evident that the first 50 minutes or so
> > was taken up by several hundred mkdir operations (more precisely, mkdir
> > -p <long path>).  Some of these mkdir's would take, perhaps, a minute to
> > execute.  All the while, top showed make taking 100% of one core.

> > There seems to be something suboptimal here.  Has anybody else seen
> > this, or does anybody have any ideas how to fix the problem?

> Long delays suggest a timeout of some sort.

OK.  As a matter of interest, some of the mkdirs executed relatively
quickly - perhaps in 0.5 seconds.  I never saw the screen whizzing by as
I ought to have done, though.

> First thing I'd look at is the filesystem underneath, and the disk
> underneath that.

My /var is an ext3 LVM partition, doubled up on a RAID-1 disk array.  In
the middle of the mkdiring, I checked there were enough inodes free
(there were).  I've no reason to suspect the disk drives might be flaky.

> Second thing I'd look at is to see if permissions checks might be
> bouncing through something like kerberos, samba or ldap. Do you have any
> single-signon things configured on that machine?

I've not got kerberos or samba installed.  I appear to have ldap
(whatever that might be ;-).  ls -lurt /usr/bin/ldap* shows these
binaries were last accessed (?used) on 2012-03-14.

What exactly do you mean by "single-signon"?

-- 
Alan Mackenzie (Nuremberg, Germany).


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Delays while building Libre Office.
  2013-05-02 16:58   ` Alan Mackenzie
@ 2013-05-02 17:15     ` Michael Mol
  2013-05-02 18:47       ` Alan Mackenzie
  0 siblings, 1 reply; 15+ messages in thread
From: Michael Mol @ 2013-05-02 17:15 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 3057 bytes --]

On 05/02/2013 12:58 PM, Alan Mackenzie wrote:
> On Thu, May 02, 2013 at 12:33:37PM -0400, Michael Mol wrote:
>> On 05/02/2013 12:27 PM, Alan Mackenzie wrote:
>>> Hi, Gentoo.
> 
>>> I've just built libreoffice-3.6.6.2 and it took 2 hours 10
>>> minutes on my 2.6 GHz quad core Athlon 2.  It used to take about
>>> an hour.
> 
>>> Watching the build, it became evident that the first 50 minutes
>>> or so was taken up by several hundred mkdir operations (more
>>> precisely, mkdir -p <long path>).  Some of these mkdir's would
>>> take, perhaps, a minute to execute.  All the while, top showed
>>> make taking 100% of one core.
> 
>>> There seems to be something suboptimal here.  Has anybody else
>>> seen this, or does anybody have any ideas how to fix the
>>> problem?
> 
>> Long delays suggest a timeout of some sort.
> 
> OK.  As a matter of interest, some of the mkdirs executed relatively 
> quickly - perhaps in 0.5 seconds.  I never saw the screen whizzing by
> as I ought to have done, though.

Hm.

> 
>> First thing I'd look at is the filesystem underneath, and the disk 
>> underneath that.
> 
> My /var is an ext3 LVM partition, doubled up on a RAID-1 disk array.

How full is the ext3 partition? What options do you have enabled on it?
(e.g. dir indexing?)


> In the middle of the mkdiring, I checked there were enough inodes
> free (there were).  I've no reason to suspect the disk drives might
> be flaky.

Well, you kinda do, now; you have evidence that at least some disk
access is unusually slow. Check dmesg for disk I/O errors (unlikely to
be reported at this point; I'm sure you checked whether your RAID was in
a degraded state), and run commanded smartctl tests on the disks.

> 
>> Second thing I'd look at is to see if permissions checks might be 
>> bouncing through something like kerberos, samba or ldap. Do you
>> have any single-signon things configured on that machine?
> 
> I've not got kerberos or samba installed.  I appear to have ldap 
> (whatever that might be ;-).  ls -lurt /usr/bin/ldap* shows these 
> binaries were last accessed (?used) on 2012-03-14.

It would be more a question of whether they were tied into PAM.

> 
> What exactly do you mean by "single-signon"?

Well, that was a slip of the tongue. More "central auth". I was
wondering if there were any features installed on your system that are
designed to check authorization against a server somewhere. (i.e. you
can use an LDAP directory to centrally manage things like users, groups,
etc.)

Technically, single-signon combines authorization checks with persistent
authentication checks. Examples of this include kerberos, web session
cookies and some uses of OAuth; once you're authenticated, the mechanism
ensures you don't need to authenticate to another server in the same
auth realm so long as your existing session hasn't expired. But this is
less likely to be related to your problem than something seeking to ask
a server if you have authorization to access something.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 555 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Delays while building Libre Office.
  2013-05-02 17:15     ` Michael Mol
@ 2013-05-02 18:47       ` Alan Mackenzie
  2013-05-02 18:52         ` Michael Mol
  0 siblings, 1 reply; 15+ messages in thread
From: Alan Mackenzie @ 2013-05-02 18:47 UTC (permalink / raw
  To: gentoo-user

On Thu, May 02, 2013 at 01:15:58PM -0400, Michael Mol wrote:
> On 05/02/2013 12:58 PM, Alan Mackenzie wrote:
> > On Thu, May 02, 2013 at 12:33:37PM -0400, Michael Mol wrote:
> >> On 05/02/2013 12:27 PM, Alan Mackenzie wrote:
> >>> Hi, Gentoo.

> >>> I've just built libreoffice-3.6.6.2 and it took 2 hours 10
> >>> minutes on my 2.6 GHz quad core Athlon 2.  It used to take about
> >>> an hour.

> >>> Watching the build, it became evident that the first 50 minutes
> >>> or so was taken up by several hundred mkdir operations (more
> >>> precisely, mkdir -p <long path>).  Some of these mkdir's would
> >>> take, perhaps, a minute to execute.  All the while, top showed
> >>> make taking 100% of one core.

> >>> There seems to be something suboptimal here.  Has anybody else
> >>> seen this, or does anybody have any ideas how to fix the
> >>> problem?

> >> Long delays suggest a timeout of some sort.

> > OK.  As a matter of interest, some of the mkdirs executed relatively 
> > quickly - perhaps in 0.5 seconds.  I never saw the screen whizzing by
> > as I ought to have done, though.

> Hm.


> >> First thing I'd look at is the filesystem underneath, and the disk 
> >> underneath that.

> > My /var is an ext3 LVM partition, doubled up on a RAID-1 disk array.

> How full is the ext3 partition? What options do you have enabled on it?
> (e.g. dir indexing?)

root@acm ~ # df /var
Filesystem         1K-blocks    Used Available Use% Mounted on
/dev/mapper/vg-var  12385456 1959860   9796580  17% /var

> > In the middle of the mkdiring, I checked there were enough inodes
> > free (there were).  I've no reason to suspect the disk drives might
> > be flaky.

> Well, you kinda do, now;

The reason I say this is that building the last ?one/two/three versions
of libreoffice also gave me this grief, but I haven't noticed anything
else amiss.

> you have evidence that at least some disk access is unusually slow.
> Check dmesg for disk I/O errors (unlikely to be reported at this point;

Nothing awry in dmesg.

> I'm sure you checked whether your RAID was in a degraded state),

cat /proc/mdstat shows everything in order.

> and run commanded smartctl tests on the disks.

That I haven't done, yet.

> >> Second thing I'd look at is to see if permissions checks might be 
> >> bouncing through something like kerberos, samba or ldap. Do you
> >> have any single-signon things configured on that machine?

> > I've not got kerberos or samba installed.  I appear to have ldap 
> > (whatever that might be ;-).  ls -lurt /usr/bin/ldap* shows these 
> > binaries were last accessed (?used) on 2012-03-14.

> It would be more a question of whether they were tied into PAM.

OK.  I'm sadly ignorant about PAM.  :-(

> > What exactly do you mean by "single-signon"?

> Well, that was a slip of the tongue. More "central auth". I was
> wondering if there were any features installed on your system that are
> designed to check authorization against a server somewhere. (i.e. you
> can use an LDAP directory to centrally manage things like users, groups,
> etc.)

Not that I know of.  My machine is a mere desktop connected via a
router/modem to the net.  I'd have no reason to install any auth stuff.

> Technically, single-signon combines authorization checks with persistent
> authentication checks. Examples of this include kerberos, web session
> cookies and some uses of OAuth; once you're authenticated, the mechanism
> ensures you don't need to authenticate to another server in the same
> auth realm so long as your existing session hasn't expired. But this is
> less likely to be related to your problem than something seeking to ask
> a server if you have authorization to access something.

If this were the case, what would libreoffice's build need to ask that no
other package stumbles over?

-- 
Alan Mackenzie (Nuremberg, Germany).


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Delays while building Libre Office.
  2013-05-02 18:47       ` Alan Mackenzie
@ 2013-05-02 18:52         ` Michael Mol
  0 siblings, 0 replies; 15+ messages in thread
From: Michael Mol @ 2013-05-02 18:52 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 4324 bytes --]

On 05/02/2013 02:47 PM, Alan Mackenzie wrote:
> On Thu, May 02, 2013 at 01:15:58PM -0400, Michael Mol wrote:
>> On 05/02/2013 12:58 PM, Alan Mackenzie wrote:
>>> On Thu, May 02, 2013 at 12:33:37PM -0400, Michael Mol wrote:
>>>> On 05/02/2013 12:27 PM, Alan Mackenzie wrote:
>>>>> Hi, Gentoo.
> 
>>>>> I've just built libreoffice-3.6.6.2 and it took 2 hours 10
>>>>> minutes on my 2.6 GHz quad core Athlon 2.  It used to take about
>>>>> an hour.
> 
>>>>> Watching the build, it became evident that the first 50 minutes
>>>>> or so was taken up by several hundred mkdir operations (more
>>>>> precisely, mkdir -p <long path>).  Some of these mkdir's would
>>>>> take, perhaps, a minute to execute.  All the while, top showed
>>>>> make taking 100% of one core.
> 
>>>>> There seems to be something suboptimal here.  Has anybody else
>>>>> seen this, or does anybody have any ideas how to fix the
>>>>> problem?
> 
>>>> Long delays suggest a timeout of some sort.
> 
>>> OK.  As a matter of interest, some of the mkdirs executed relatively 
>>> quickly - perhaps in 0.5 seconds.  I never saw the screen whizzing by
>>> as I ought to have done, though.
> 
>> Hm.
> 
> 
>>>> First thing I'd look at is the filesystem underneath, and the disk 
>>>> underneath that.
> 
>>> My /var is an ext3 LVM partition, doubled up on a RAID-1 disk array.
> 
>> How full is the ext3 partition? What options do you have enabled on it?
>> (e.g. dir indexing?)
> 
> root@acm ~ # df /var
> Filesystem         1K-blocks    Used Available Use% Mounted on
> /dev/mapper/vg-var  12385456 1959860   9796580  17% /var
> 
>>> In the middle of the mkdiring, I checked there were enough inodes
>>> free (there were).  I've no reason to suspect the disk drives might
>>> be flaky.
> 
>> Well, you kinda do, now;
> 
> The reason I say this is that building the last ?one/two/three versions
> of libreoffice also gave me this grief, but I haven't noticed anything
> else amiss.
> 
>> you have evidence that at least some disk access is unusually slow.
>> Check dmesg for disk I/O errors (unlikely to be reported at this point;
> 
> Nothing awry in dmesg.
> 
>> I'm sure you checked whether your RAID was in a degraded state),
> 
> cat /proc/mdstat shows everything in order.
> 
>> and run commanded smartctl tests on the disks.
> 
> That I haven't done, yet.
> 
>>>> Second thing I'd look at is to see if permissions checks might be 
>>>> bouncing through something like kerberos, samba or ldap. Do you
>>>> have any single-signon things configured on that machine?
> 
>>> I've not got kerberos or samba installed.  I appear to have ldap 
>>> (whatever that might be ;-).  ls -lurt /usr/bin/ldap* shows these 
>>> binaries were last accessed (?used) on 2012-03-14.
> 
>> It would be more a question of whether they were tied into PAM.
> 
> OK.  I'm sadly ignorant about PAM.  :-(

If you've just got a single box, it's very unlikely this is your problem.

> 
>>> What exactly do you mean by "single-signon"?
> 
>> Well, that was a slip of the tongue. More "central auth". I was
>> wondering if there were any features installed on your system that are
>> designed to check authorization against a server somewhere. (i.e. you
>> can use an LDAP directory to centrally manage things like users, groups,
>> etc.)
> 
> Not that I know of.  My machine is a mere desktop connected via a
> router/modem to the net.  I'd have no reason to install any auth stuff.
> 
>> Technically, single-signon combines authorization checks with persistent
>> authentication checks. Examples of this include kerberos, web session
>> cookies and some uses of OAuth; once you're authenticated, the mechanism
>> ensures you don't need to authenticate to another server in the same
>> auth realm so long as your existing session hasn't expired. But this is
>> less likely to be related to your problem than something seeking to ask
>> a server if you have authorization to access something.
> 
> If this were the case, what would libreoffice's build need to ask that no
> other package stumbles over?
> 

My presumption there was that this was a very recent thing, and LO's
time-to-build makes it easier to observe.

Anyway, floor's open to anyone else who might have an idea.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 555 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Delays while building Libre Office.
  2013-05-02 16:27 [gentoo-user] Delays while building Libre Office Alan Mackenzie
  2013-05-02 16:33 ` Michael Mol
  2013-05-02 16:49 ` Philip Webb
@ 2013-05-04  7:42 ` Stroller
  2013-05-04 10:12 ` Florian Philipp
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 15+ messages in thread
From: Stroller @ 2013-05-04  7:42 UTC (permalink / raw
  To: gentoo-user


On 2 May 2013, at 17:27, Alan Mackenzie wrote:
> ...
> Watching the build, it became evident that the first 50 minutes or so
> was taken up by several hundred mkdir operations (more precisely, mkdir
> -p <long path>).  Some of these mkdir's would take, perhaps, a minute to
> execute.  All the while, top showed make taking 100% of one core.

I think that at least you need to write a short shell script in which $i, $j, $k & $l each number 0 - 9, and which you create 1000 or 100,000 directories in the format /1/2/3/4/5 (or /a/b/c/d/e/).

I'm not sure if it's equally good to record the number of seconds since the epoc into a variable as the script launches as it i to just run the `time` command.

The former would allow you to calculate how long it's taken to create a directory (or 100 of them). You can compare how long it takes to delete directories and how long it takes to create and delete empty files (or files containing a single character).

I think you need to run this script on all your filesystems, and compare the results when emerge is and isn't running.

Ideally you'd hope to be able to reproduce this slow directory creation behaviour using only your shell script. 

I think it's really easy to create such a shell script as this and to compare the results. I appreciate that adding complexities to it might be a chore, but it might be that you can reproduce the behaviour simply by creating 1000 directories with a single bash loop. I think it's pretty important to rule that out before assuming that the problem is with portage. 

Stroller.



^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Delays while building Libre Office.
  2013-05-02 16:27 [gentoo-user] Delays while building Libre Office Alan Mackenzie
                   ` (2 preceding siblings ...)
  2013-05-04  7:42 ` Stroller
@ 2013-05-04 10:12 ` Florian Philipp
  2013-05-05 20:38   ` Alan Mackenzie
  2013-05-04 11:57 ` Jörg Schaible
  2013-05-05 20:34 ` walt
  5 siblings, 1 reply; 15+ messages in thread
From: Florian Philipp @ 2013-05-04 10:12 UTC (permalink / raw
  To: gentoo-user

[-- Attachment #1: Type: text/plain, Size: 996 bytes --]

Am 02.05.2013 18:27, schrieb Alan Mackenzie:
> Hi, Gentoo.
> 
> I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes on my
> 2.6 GHz quad core Athlon 2.  It used to take about an hour.
> 
> Watching the build, it became evident that the first 50 minutes or so
> was taken up by several hundred mkdir operations (more precisely, mkdir
> -p <long path>).  Some of these mkdir's would take, perhaps, a minute to
> execute.  All the while, top showed make taking 100% of one core.
> 
> There seems to be something suboptimal here.  Has anybody else seen
> this, or does anybody have any ideas how to fix the problem?
> 

I'm seeing the same behavior here.
libreoffice-3.6.6.2 on a mostly stable system.
btrfs on lvm on dmcrypt on a hybrid disk.
kernel 3.7.5 pf-sources.

I don't think is has anything to do with the file system or hardware.
make is eating 200M memory and uses 100% CPU, not mkdir. I guess the
script is just buggy.

Regards,
Florian Philipp


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [gentoo-user] Re: Delays while building Libre Office.
  2013-05-02 16:27 [gentoo-user] Delays while building Libre Office Alan Mackenzie
                   ` (3 preceding siblings ...)
  2013-05-04 10:12 ` Florian Philipp
@ 2013-05-04 11:57 ` Jörg Schaible
  2013-05-05 20:34 ` walt
  5 siblings, 0 replies; 15+ messages in thread
From: Jörg Schaible @ 2013-05-04 11:57 UTC (permalink / raw
  To: gentoo-user

Alan Mackenzie wrote:

> Hi, Gentoo.
> 
> I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes on my
> 2.6 GHz quad core Athlon 2.  It used to take about an hour.
> 
> Watching the build, it became evident that the first 50 minutes or so
> was taken up by several hundred mkdir operations (more precisely, mkdir
> -p <long path>).  Some of these mkdir's would take, perhaps, a minute to
> execute.  All the while, top showed make taking 100% of one core.
> 
> There seems to be something suboptimal here.  Has anybody else seen
> this, or does anybody have any ideas how to fix the problem?

Just to test the file system, you may copy a large tree with "cp -l" within 
the same partition, since it creates only hard links, no data is actually 
copied.

- Jörg



^ permalink raw reply	[flat|nested] 15+ messages in thread

* [gentoo-user] Re: Delays while building Libre Office.
  2013-05-02 16:27 [gentoo-user] Delays while building Libre Office Alan Mackenzie
                   ` (4 preceding siblings ...)
  2013-05-04 11:57 ` Jörg Schaible
@ 2013-05-05 20:34 ` walt
  5 siblings, 0 replies; 15+ messages in thread
From: walt @ 2013-05-05 20:34 UTC (permalink / raw
  To: gentoo-user

On 05/02/2013 09:27 AM, Alan Mackenzie wrote:
> Hi, Gentoo.
> 
> I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes on my
> 2.6 GHz quad core Athlon 2.  It used to take about an hour.
> 
> Watching the build, it became evident that the first 50 minutes or so
> was taken up by several hundred mkdir operations (more precisely, mkdir
> -p <long path>).  Some of these mkdir's would take, perhaps, a minute to
> execute.  All the while, top showed make taking 100% of one core.
> 
> There seems to be something suboptimal here.  Has anybody else seen
> this, or does anybody have any ideas how to fix the problem?

I've been seeing it for a long time.  I even filed a bug report, which
vapier confirmed and he renamed my bug report "ebuild spends too much
time polling ${obscure-socket-name}".

I just looked for the bug report but I can't find it.  Eventually the
problem went away and I assumed he fixed it, but maybe not -- your
problem seems identical.

Come to think of it, maybe the problem disappeared because I've been
using libreoffice-4 for quite a while now, and lo-4 never had that
bug.

Or maybe some old update of portage fixed the problem for me, dunno.




^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Delays while building Libre Office.
  2013-05-04 10:12 ` Florian Philipp
@ 2013-05-05 20:38   ` Alan Mackenzie
  2013-05-06 23:22     ` [gentoo-user] " walt
  0 siblings, 1 reply; 15+ messages in thread
From: Alan Mackenzie @ 2013-05-05 20:38 UTC (permalink / raw
  To: gentoo-user

Hi, Florian.

On Sat, May 04, 2013 at 12:12:51PM +0200, Florian Philipp wrote:
> Am 02.05.2013 18:27, schrieb Alan Mackenzie:
> > Hi, Gentoo.

> > I've just built libreoffice-3.6.6.2 and it took 2 hours 10 minutes on my
> > 2.6 GHz quad core Athlon 2.  It used to take about an hour.

> > Watching the build, it became evident that the first 50 minutes or so
> > was taken up by several hundred mkdir operations (more precisely, mkdir
> > -p <long path>).  Some of these mkdir's would take, perhaps, a minute to
> > execute.  All the while, top showed make taking 100% of one core.

> > There seems to be something suboptimal here.  Has anybody else seen
> > this, or does anybody have any ideas how to fix the problem?


> I'm seeing the same behavior here.
> libreoffice-3.6.6.2 on a mostly stable system.
> btrfs on lvm on dmcrypt on a hybrid disk.
> kernel 3.7.5 pf-sources.

> I don't think is has anything to do with the file system or hardware.
> make is eating 200M memory and uses 100% CPU, not mkdir. I guess the
> script is just buggy.

Thanks for the report.  I think the build script is buggy, too.  There is
something about our systems, possibly some use flag, which is driving
make crazy.

There doesn't appear to be any action in emerge which unpacks the build
files (or even the entire source) of a package for perusal.  This is a
shame.

At this point, I have to ask myself whether spending time trying to debug
it (or even to report the bug usefully) is better than simply tolerating
the long build time.  It doesn't seem like there'll be an easy solution.

> Regards,
> Florian Philipp

-- 
Alan Mackenzie (Nuremberg, Germany).


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [gentoo-user] Re: Delays while building Libre Office.
  2013-05-05 20:38   ` Alan Mackenzie
@ 2013-05-06 23:22     ` walt
  2013-05-06 23:37       ` Michael Hampicke
  0 siblings, 1 reply; 15+ messages in thread
From: walt @ 2013-05-06 23:22 UTC (permalink / raw
  To: gentoo-user

On 05/05/2013 01:38 PM, Alan Mackenzie wrote:

> There doesn't appear to be any action in emerge which unpacks the build
> files (or even the entire source) of a package for perusal.  This is a
> shame.

You should become familiar with the 'ebuild' command, which I use very
often when trying to debug a package, e.g.

#ebuild /usr/portage/app-office/libreoffice/libreoffice-3.6.6.2.ebuild unpack

To completely build and install a package you need to follow that command
with the 'configure', 'compile', 'install', and 'qmerge' steps (man ebuild).

This method allows you to resume an emerge without starting from zero.
It saves a lot of pain when dealing with a package like libreoffice :)


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [gentoo-user] Re: Delays while building Libre Office.
  2013-05-06 23:22     ` [gentoo-user] " walt
@ 2013-05-06 23:37       ` Michael Hampicke
  2013-05-07  1:34         ` walt
  0 siblings, 1 reply; 15+ messages in thread
From: Michael Hampicke @ 2013-05-06 23:37 UTC (permalink / raw
  To: gentoo-user

Am 07.05.2013 01:22, schrieb walt:
> On 05/05/2013 01:38 PM, Alan Mackenzie wrote:
> 
>> There doesn't appear to be any action in emerge which unpacks the build
>> files (or even the entire source) of a package for perusal.  This is a
>> shame.
> 
> You should become familiar with the 'ebuild' command, which I use very
> often when trying to debug a package, e.g.
> 
> #ebuild /usr/portage/app-office/libreoffice/libreoffice-3.6.6.2.ebuild unpack
> 
> To completely build and install a package you need to follow that command
> with the 'configure', 'compile', 'install', and 'qmerge' steps (man ebuild).
> 

If you are lazy like my, and don't want to execute all steps manually,
just use ebuild <package.ebuild> merge :)


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [gentoo-user] Re: Delays while building Libre Office.
  2013-05-06 23:37       ` Michael Hampicke
@ 2013-05-07  1:34         ` walt
  0 siblings, 0 replies; 15+ messages in thread
From: walt @ 2013-05-07  1:34 UTC (permalink / raw
  To: gentoo-user

On 05/06/2013 04:37 PM, Michael Hampicke wrote:
> Am 07.05.2013 01:22, schrieb walt:
>> On 05/05/2013 01:38 PM, Alan Mackenzie wrote:
>>
>>> There doesn't appear to be any action in emerge which unpacks the build
>>> files (or even the entire source) of a package for perusal.  This is a
>>> shame.
>>
>> You should become familiar with the 'ebuild' command, which I use very
>> often when trying to debug a package, e.g.
>>
>> #ebuild /usr/portage/app-office/libreoffice/libreoffice-3.6.6.2.ebuild unpack
>>
>> To completely build and install a package you need to follow that command
>> with the 'configure', 'compile', 'install', and 'qmerge' steps (man ebuild).
>>
> 
> If you are lazy like me,

No way!  I'm lazier than you :p

> and don't want to execute all steps manually,
> just use ebuild <package.ebuild> merge :)

The reason it's nice to know the individual steps is that there can be bugs
at any of those steps along the way (usually during the configure or compile
phases) and if the bug is simple enough you can edit the broken files and
then restart the merge.  Often someone will add a patch to an existing bug
report, which you can apply to a failed emerge and then restart the compile
phase without starting from scratch.

I'm guessing you know all of this already, but Alan and others may not know.
The more debuggers, the merrier :)



^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2013-05-07  1:35 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-05-02 16:27 [gentoo-user] Delays while building Libre Office Alan Mackenzie
2013-05-02 16:33 ` Michael Mol
2013-05-02 16:58   ` Alan Mackenzie
2013-05-02 17:15     ` Michael Mol
2013-05-02 18:47       ` Alan Mackenzie
2013-05-02 18:52         ` Michael Mol
2013-05-02 16:49 ` Philip Webb
2013-05-04  7:42 ` Stroller
2013-05-04 10:12 ` Florian Philipp
2013-05-05 20:38   ` Alan Mackenzie
2013-05-06 23:22     ` [gentoo-user] " walt
2013-05-06 23:37       ` Michael Hampicke
2013-05-07  1:34         ` walt
2013-05-04 11:57 ` Jörg Schaible
2013-05-05 20:34 ` walt

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox