* [gentoo-user] portage sandbox path-depth limit ?
@ 2018-10-30 6:30 Håkon Alstadheim
2018-10-30 9:01 ` Mick
0 siblings, 1 reply; 5+ messages in thread
From: Håkon Alstadheim @ 2018-10-30 6:30 UTC (permalink / raw
To: gentoo-user@lists.gentoo.org
I'm having fun enabling "test" in FEATURES on my gentoo-desktop. One
interesting failure, that brings to mind build failures I have had in
the past:
Building sys-apps/mlocate-0.26-r2, I get
43: updatedb: Very deep hierarchy FAILED
(updatedb.at:261)
Trying to reproduce, as root I do "make check" in the work/mlocate-0.26/
, and the test passes.
43: updatedb: Very deep hierarchy ok
I'd really like to get to the bottom of this, as I believe it must have
the same root-cause as issues I have had compiling large packages such
as firefox.
Re-running both the emerge and the make check, I get the same results.
emerge fails, make check succeeds. I made a local copy of the ebuild and
inserted a "ulimit -a" in pre_src_test.
ulimit from root-shell:
# ulimit -a
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 59958
max locked memory (kbytes, -l) 16384
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 10000
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
ulimit from emerge:
>>> Source compiled.
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 59958
max locked memory (kbytes, -l) 16384
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 9788
cpu time (seconds, -t) unlimited
max user processes (-u) 10000
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
>>> Test phase: sys-apps/mlocate-0.26-r2
I have plenty of space in my portage temp directory (/pt):
# df -hT ./
Filsystem Type Størrelse Brukt Tilgj. Bruk% Montert på
/dev/xvdc ext4 163G 8,0G 147G 6% /pt
Portage temp is at /pt due to the earlier mentioned issues with firefox.
At my wits end here. Anyone ?
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [gentoo-user] portage sandbox path-depth limit ?
2018-10-30 6:30 [gentoo-user] portage sandbox path-depth limit ? Håkon Alstadheim
@ 2018-10-30 9:01 ` Mick
2018-10-30 12:29 ` Håkon Alstadheim
0 siblings, 1 reply; 5+ messages in thread
From: Mick @ 2018-10-30 9:01 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 3579 bytes --]
On Tuesday, 30 October 2018 06:30:23 GMT Håkon Alstadheim wrote:
> I'm having fun enabling "test" in FEATURES on my gentoo-desktop. One
> interesting failure, that brings to mind build failures I have had in
> the past:
>
> Building sys-apps/mlocate-0.26-r2, I get
>
> 43: updatedb: Very deep hierarchy FAILED
> (updatedb.at:261)
>
> Trying to reproduce, as root I do "make check" in the work/mlocate-0.26/
> , and the test passes.
>
> 43: updatedb: Very deep hierarchy ok
>
> I'd really like to get to the bottom of this, as I believe it must have
> the same root-cause as issues I have had compiling large packages such
> as firefox.
>
> Re-running both the emerge and the make check, I get the same results.
> emerge fails, make check succeeds. I made a local copy of the ebuild and
> inserted a "ulimit -a" in pre_src_test.
>
> ulimit from root-shell:
>
> # ulimit -a
> core file size (blocks, -c) unlimited
> data seg size (kbytes, -d) unlimited
> scheduling priority (-e) 0
> file size (blocks, -f) unlimited
> pending signals (-i) 59958
> max locked memory (kbytes, -l) 16384
> max memory size (kbytes, -m) unlimited
> open files (-n) 1024
> pipe size (512 bytes, -p) 8
> POSIX message queues (bytes, -q) 819200
> real-time priority (-r) 0
> stack size (kbytes, -s) 8192
> cpu time (seconds, -t) unlimited
> max user processes (-u) 10000
> virtual memory (kbytes, -v) unlimited
> file locks (-x) unlimited
>
> ulimit from emerge:
> >>> Source compiled.
>
> core file size (blocks, -c) unlimited
> data seg size (kbytes, -d) unlimited
> scheduling priority (-e) 0
> file size (blocks, -f) unlimited
> pending signals (-i) 59958
> max locked memory (kbytes, -l) 16384
> max memory size (kbytes, -m) unlimited
> open files (-n) 1024
> pipe size (512 bytes, -p) 8
> POSIX message queues (bytes, -q) 819200
> real-time priority (-r) 0
> stack size (kbytes, -s) 9788
> cpu time (seconds, -t) unlimited
> max user processes (-u) 10000
> virtual memory (kbytes, -v) unlimited
> file locks (-x) unlimited
>
> >>> Test phase: sys-apps/mlocate-0.26-r2
>
> I have plenty of space in my portage temp directory (/pt):
>
> # df -hT ./
> Filsystem Type Størrelse Brukt Tilgj. Bruk% Montert på
> /dev/xvdc ext4 163G 8,0G 147G 6% /pt
>
> Portage temp is at /pt due to the earlier mentioned issues with firefox.
>
> At my wits end here. Anyone ?
I have not looked or used the test FEATURES of portage, but have also noticed
over time certain packages fail to build on machines with low RAM. As low
here I consider anything less than 4G. It seems each thread is now consuming
so much memory they cumulatively use up all RAM available and then start
swapping endlessly until the compilation invariably fails. Increasingly more
and more packages have been suffering from this, the last two I noticed are
qtwebkit and qtwebengine.
My solution has been to create a package.env file in which I specify MAKEOPTS
limiting the number of jobs and average load for any of these packages which
chew up all the RAM.
--
Regards,
Mick
[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [gentoo-user] portage sandbox path-depth limit ?
2018-10-30 9:01 ` Mick
@ 2018-10-30 12:29 ` Håkon Alstadheim
2018-11-01 0:09 ` Andrew Savchenko
0 siblings, 1 reply; 5+ messages in thread
From: Håkon Alstadheim @ 2018-10-30 12:29 UTC (permalink / raw
To: gentoo-user
Den 30. okt. 2018 10:01, skrev Mick:
> On Tuesday, 30 October 2018 06:30:23 GMT Håkon Alstadheim wrote:
>> I'm having fun enabling "test" in FEATURES on my gentoo-desktop. One
>> interesting failure, that brings to mind build failures I have had in
>> the past:
>>
>> Building sys-apps/mlocate-0.26-r2, I get
>>
>> 43: updatedb: Very deep hierarchy FAILED
>> (updatedb.at:261)
>>
>> Trying to reproduce, as root I do "make check" in the work/mlocate-0.26/
>> , and the test passes.
>>
>> 43: updatedb: Very deep hierarchy ok
>>
>> I'd really like to get to the bottom of this, as I believe it must have
>> the same root-cause as issues I have had compiling large packages such
>> as firefox.
>>
>> Re-running both the emerge and the make check, I get the same results.
>> emerge fails, make check succeeds. I made a local copy of the ebuild and
>> inserted a "ulimit -a" in pre_src_test.
>>
>> ulimit from root-shell:
>>
>> # ulimit -a
>> core file size (blocks, -c) unlimited
>> data seg size (kbytes, -d) unlimited
>> scheduling priority (-e) 0
>> file size (blocks, -f) unlimited
>> pending signals (-i) 59958
>> max locked memory (kbytes, -l) 16384
>> max memory size (kbytes, -m) unlimited
>> open files (-n) 1024
>> pipe size (512 bytes, -p) 8
>> POSIX message queues (bytes, -q) 819200
>> real-time priority (-r) 0
>> stack size (kbytes, -s) 8192
>> cpu time (seconds, -t) unlimited
>> max user processes (-u) 10000
>> virtual memory (kbytes, -v) unlimited
>> file locks (-x) unlimited
>>
>> ulimit from emerge:
>>>>> Source compiled.
>> core file size (blocks, -c) unlimited
>> data seg size (kbytes, -d) unlimited
>> scheduling priority (-e) 0
>> file size (blocks, -f) unlimited
>> pending signals (-i) 59958
>> max locked memory (kbytes, -l) 16384
>> max memory size (kbytes, -m) unlimited
>> open files (-n) 1024
>> pipe size (512 bytes, -p) 8
>> POSIX message queues (bytes, -q) 819200
>> real-time priority (-r) 0
>> stack size (kbytes, -s) 9788
>> cpu time (seconds, -t) unlimited
>> max user processes (-u) 10000
>> virtual memory (kbytes, -v) unlimited
>> file locks (-x) unlimited
>>
>>>>> Test phase: sys-apps/mlocate-0.26-r2
>> I have plenty of space in my portage temp directory (/pt):
>>
>> # df -hT ./
>> Filsystem Type Størrelse Brukt Tilgj. Bruk% Montert på
>> /dev/xvdc ext4 163G 8,0G 147G 6% /pt
>>
>> Portage temp is at /pt due to the earlier mentioned issues with firefox.
>>
>> At my wits end here. Anyone ?
> I have not looked or used the test FEATURES of portage, but have also noticed
> over time certain packages fail to build on machines with low RAM. As low
> here I consider anything less than 4G. It seems each thread is now consuming
> so much memory they cumulatively use up all RAM available and then start
> swapping endlessly until the compilation invariably fails. Increasingly more
> and more packages have been suffering from this, the last two I noticed are
> qtwebkit and qtwebengine.
>
> My solution has been to create a package.env file in which I specify MAKEOPTS
> limiting the number of jobs and average load for any of these packages which
> chew up all the RAM.
Memory should not be a problem here. Fails with only that one emerge
running,
succeeds if run directly as root, or with FEATURES="-sandbox -usersandbox".
Memory is >14GB:
# vmstat
procs -----------memory---------- ---swap-- -----io---- -system--
------cpu-----
r b swpd free buff cache si so bi bo in cs us sy
id wa st
3 4 28416 6904608 174112 4616144 0 0 65 266 13 4 10
2 84 4 0
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [gentoo-user] portage sandbox path-depth limit ?
2018-10-30 12:29 ` Håkon Alstadheim
@ 2018-11-01 0:09 ` Andrew Savchenko
2018-11-01 0:18 ` Andrew Savchenko
0 siblings, 1 reply; 5+ messages in thread
From: Andrew Savchenko @ 2018-11-01 0:09 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 4557 bytes --]
On Tue, 30 Oct 2018 13:29:59 +0100 Håkon Alstadheim wrote:
>
> Den 30. okt. 2018 10:01, skrev Mick:
> > On Tuesday, 30 October 2018 06:30:23 GMT Håkon Alstadheim wrote:
> >> I'm having fun enabling "test" in FEATURES on my gentoo-desktop. One
> >> interesting failure, that brings to mind build failures I have had in
> >> the past:
> >>
> >> Building sys-apps/mlocate-0.26-r2, I get
> >>
> >> 43: updatedb: Very deep hierarchy FAILED
> >> (updatedb.at:261)
> >>
> >> Trying to reproduce, as root I do "make check" in the work/mlocate-0.26/
> >> , and the test passes.
> >>
> >> 43: updatedb: Very deep hierarchy ok
> >>
> >> I'd really like to get to the bottom of this, as I believe it must have
> >> the same root-cause as issues I have had compiling large packages such
> >> as firefox.
> >>
> >> Re-running both the emerge and the make check, I get the same results.
> >> emerge fails, make check succeeds. I made a local copy of the ebuild and
> >> inserted a "ulimit -a" in pre_src_test.
> >>
> >> ulimit from root-shell:
> >>
> >> # ulimit -a
> >> core file size (blocks, -c) unlimited
> >> data seg size (kbytes, -d) unlimited
> >> scheduling priority (-e) 0
> >> file size (blocks, -f) unlimited
> >> pending signals (-i) 59958
> >> max locked memory (kbytes, -l) 16384
> >> max memory size (kbytes, -m) unlimited
> >> open files (-n) 1024
> >> pipe size (512 bytes, -p) 8
> >> POSIX message queues (bytes, -q) 819200
> >> real-time priority (-r) 0
> >> stack size (kbytes, -s) 8192
> >> cpu time (seconds, -t) unlimited
> >> max user processes (-u) 10000
> >> virtual memory (kbytes, -v) unlimited
> >> file locks (-x) unlimited
> >>
> >> ulimit from emerge:
> >>>>> Source compiled.
> >> core file size (blocks, -c) unlimited
> >> data seg size (kbytes, -d) unlimited
> >> scheduling priority (-e) 0
> >> file size (blocks, -f) unlimited
> >> pending signals (-i) 59958
> >> max locked memory (kbytes, -l) 16384
> >> max memory size (kbytes, -m) unlimited
> >> open files (-n) 1024
> >> pipe size (512 bytes, -p) 8
> >> POSIX message queues (bytes, -q) 819200
> >> real-time priority (-r) 0
> >> stack size (kbytes, -s) 9788
> >> cpu time (seconds, -t) unlimited
> >> max user processes (-u) 10000
> >> virtual memory (kbytes, -v) unlimited
> >> file locks (-x) unlimited
> >>
> >>>>> Test phase: sys-apps/mlocate-0.26-r2
> >> I have plenty of space in my portage temp directory (/pt):
> >>
> >> # df -hT ./
> >> Filsystem Type Størrelse Brukt Tilgj. Bruk% Montert på
> >> /dev/xvdc ext4 163G 8,0G 147G 6% /pt
> >>
> >> Portage temp is at /pt due to the earlier mentioned issues with firefox.
> >>
> >> At my wits end here. Anyone ?
> > I have not looked or used the test FEATURES of portage, but have also noticed
> > over time certain packages fail to build on machines with low RAM. As low
> > here I consider anything less than 4G. It seems each thread is now consuming
> > so much memory they cumulatively use up all RAM available and then start
> > swapping endlessly until the compilation invariably fails. Increasingly more
> > and more packages have been suffering from this, the last two I noticed are
> > qtwebkit and qtwebengine.
> >
> > My solution has been to create a package.env file in which I specify MAKEOPTS
> > limiting the number of jobs and average load for any of these packages which
> > chew up all the RAM.
> Memory should not be a problem here. Fails with only that one emerge
> running,
> succeeds if run directly as root, or with FEATURES="-sandbox -usersandbox".
>
> Memory is >14GB:
> # vmstat
> procs -----------memory---------- ---swap-- -----io---- -system--
> ------cpu-----
> r b swpd free buff cache si so bi bo in cs us sy
> id wa st
> 3 4 28416 6904608 174112 4616144 0 0 65 266 13 4 10
> 2 84 4 0
It is possible that you hit directory loop. What lstree says on
that dir? Anyway, report this to sandbox devs.
Best regards,
Andrew Savchenko
[-- Attachment #2: Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [gentoo-user] portage sandbox path-depth limit ?
2018-11-01 0:09 ` Andrew Savchenko
@ 2018-11-01 0:18 ` Andrew Savchenko
0 siblings, 0 replies; 5+ messages in thread
From: Andrew Savchenko @ 2018-11-01 0:18 UTC (permalink / raw
To: gentoo-user
[-- Attachment #1: Type: text/plain, Size: 902 bytes --]
On Thu, 1 Nov 2018 03:09:51 +0300 Andrew Savchenko wrote:
> On Tue, 30 Oct 2018 13:29:59 +0100 Håkon Alstadheim wrote:
> >
> > Den 30. okt. 2018 10:01, skrev Mick:
[...]
> > Memory should not be a problem here. Fails with only that one emerge
> > running,
> > succeeds if run directly as root, or with FEATURES="-sandbox -usersandbox".
> >
> > Memory is >14GB:
> > # vmstat
> > procs -----------memory---------- ---swap-- -----io---- -system--
> > ------cpu-----
> > r b swpd free buff cache si so bi bo in cs us sy
> > id wa st
> > 3 4 28416 6904608 174112 4616144 0 0 65 266 13 4 10
> > 2 84 4 0
>
> It is possible that you hit directory loop. What lstree says on
> that dir? Anyway, report this to sandbox devs.
Sorry, `tree -l | grep recursive`.
Best regards,
Andrew Savchenko
[-- Attachment #2: Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2018-11-01 0:18 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-10-30 6:30 [gentoo-user] portage sandbox path-depth limit ? Håkon Alstadheim
2018-10-30 9:01 ` Mick
2018-10-30 12:29 ` Håkon Alstadheim
2018-11-01 0:09 ` Andrew Savchenko
2018-11-01 0:18 ` Andrew Savchenko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox