[gentoo-portage-dev] dead emerge processes and/or lockfiles

[gentoo-portage-dev] dead emerge processes and/or lockfiles

[Brian Dolbec]

I've read in several forum posts lately about emerge not running and
the problem comes down to dead emerge processes and remaining lockfiles.

Perhaps we should make an emaint module to search for and fix these.
It should be easy enough.


Quoting 2 posts from this forum thread: 

https://forums.gentoo.org/viewtopic-t-885066.html?sid=95544407ed19693f2e5dcae094efd977

adamf663
n00b
n00b


Joined: 08 Mar 2007
Posts: 11

PostPosted: Sat Sep 12, 2015 4:52 pm    Post subject: a solution	Reply with quote
I had something similar after messing around with the '--jobs' option. No emerge would run after that. 
When I went to clear out /var/tmp/portage, I discovered a lockfile. 
Ran lsof and found an orphaned emerge. Killed it and emerges started running properly again.
Back to top	
View user's profile Send private message

ravloony
n00b
n00b


Joined: 04 Feb 2005
Posts: 54
Location: France
PostPosted: Fri Jan 15, 2016 9:38 am    Post subject: Re: a solution	Reply with quote
adamf663 wrote:
I had something similar after messing around with the '--jobs' option. No emerge would run after that. 
When I went to clear out /var/tmp/portage, I discovered a lockfile. 
Ran lsof and found an orphaned emerge. Killed it and emerges started running properly again.


Just happened to me. Thanks for the writeup!
_________________
No sig yet, sig ebuild up soon :-)
-- 
Brian Dolbec <dolsen>

Re: [gentoo-portage-dev] dead emerge processes and/or lockfiles

[Zac Medico]

On 01/17/2016 09:06 AM, Brian Dolbec wrote:
> 
> I've read in several forum posts lately about emerge not running and
> the problem comes down to dead emerge processes and remaining lockfiles.
> 
> Perhaps we should make an emaint module to search for and fix these.
> It should be easy enough.

It would be nicer if we fixed whatever issue(s) cause the emerge
processes to hang up. How would the emaint module distinguish a "good"
emerge process from a "bad" one? I suppose you could strace it to see if
it has any activity.
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] dead emerge processes and/or lockfiles

[Joshua Kinard]

On 01/17/2016 15:01, Zac Medico wrote:
> On 01/17/2016 09:06 AM, Brian Dolbec wrote:
>>
>> I've read in several forum posts lately about emerge not running and
>> the problem comes down to dead emerge processes and remaining lockfiles.
>>
>> Perhaps we should make an emaint module to search for and fix these.
>> It should be easy enough.
> 
> It would be nicer if we fixed whatever issue(s) cause the emerge
> processes to hang up. How would the emaint module distinguish a "good"
> emerge process from a "bad" one? I suppose you could strace it to see if
> it has any activity.
> 

I've been playing around with Gentoo/FreeBSD and have been noticing that emerge
is leaving orphaned processes behind on that platform.  Seems to be
ecompressdir getting hung up.  emerge itself just moves on, but after I
accumulated ~5 of those stuck ecompressdir processes in a single run, I kill
-9'ed them all.  Didn't see side-effects similar to what's described in the
original post, but the way to detect this issue might be to look for orphaned
children processes lacking a parent PID, then reap them.

--J

Re: [gentoo-portage-dev] dead emerge processes and/or lockfiles

[Joshua Kinard]

On 01/18/2016 07:37, Joshua Kinard wrote:
> On 01/17/2016 15:01, Zac Medico wrote:
>> On 01/17/2016 09:06 AM, Brian Dolbec wrote:
>>>
>>> I've read in several forum posts lately about emerge not running and
>>> the problem comes down to dead emerge processes and remaining lockfiles.
>>>
>>> Perhaps we should make an emaint module to search for and fix these.
>>> It should be easy enough.
>>
>> It would be nicer if we fixed whatever issue(s) cause the emerge
>> processes to hang up. How would the emaint module distinguish a "good"
>> emerge process from a "bad" one? I suppose you could strace it to see if
>> it has any activity.
>>
> 
> I've been playing around with Gentoo/FreeBSD and have been noticing that emerge
> is leaving orphaned processes behind on that platform.  Seems to be
> ecompressdir getting hung up.  emerge itself just moves on, but after I
> accumulated ~5 of those stuck ecompressdir processes in a single run, I kill
> -9'ed them all.  Didn't see side-effects similar to what's described in the
> original post, but the way to detect this issue might be to look for orphaned
> children processes lacking a parent PID, then reap them.


Updating my FreeBSD VM again, I captured one of the error messages that's
leading to these orphaned ecompressdir processes:

/usr/lib/portage/python3.5/ebuild-helpers/ecompressdir: cannot make pipe for
process substitution: File exists
/usr/lib/portage/python3.5/ebuild-helpers/ecompressdir: line 72:
/ramfs/portage/sys-freebsd/boot0-10.3/temp/sh-np-1865519000: ambiguous redirect
ecompressdir: bzip2 -9 /usr/share/man
 * The ebuild phase 'install' with pid 32075 appears to have left an orphan
 * process running in the background.


And a second one:
/ramfs/portage/._portage_reinstall_.pesqhjhn/bin/ebuild-helpers/ecompressdir:
cannot make pipe for process substitution: File exists
/ramfs/portage/._portage_reinstall_.pesqhjhn/bin/ebuild-helpers/ecompressdir:
line 72: /ramfs/portage/sys-apps/grep-2.25/temp/sh-np-474708936: ambiguous redirect
ecompressdir: bzip2 -9 /usr/share/man
ecompressdir: bzip2 -9 /usr/share/info
ecompressdir: bzip2 -9 /usr/share/doc
 * The ebuild phase 'install' with pid 60185 appears to have left an orphan
 * process running in the background.

Not sure the exact cause.  Any additional info I can provide?

--J

Re: [gentoo-portage-dev] dead emerge processes and/or lockfiles

[Zac Medico]

On 04/24/2016 04:50 PM, Joshua Kinard wrote:
> On 01/18/2016 07:37, Joshua Kinard wrote:
>> On 01/17/2016 15:01, Zac Medico wrote:
>>> On 01/17/2016 09:06 AM, Brian Dolbec wrote:
>>>>
>>>> I've read in several forum posts lately about emerge not running and
>>>> the problem comes down to dead emerge processes and remaining lockfiles.
>>>>
>>>> Perhaps we should make an emaint module to search for and fix these.
>>>> It should be easy enough.
>>>
>>> It would be nicer if we fixed whatever issue(s) cause the emerge
>>> processes to hang up. How would the emaint module distinguish a "good"
>>> emerge process from a "bad" one? I suppose you could strace it to see if
>>> it has any activity.
>>>
>>
>> I've been playing around with Gentoo/FreeBSD and have been noticing that emerge
>> is leaving orphaned processes behind on that platform.  Seems to be
>> ecompressdir getting hung up.  emerge itself just moves on, but after I
>> accumulated ~5 of those stuck ecompressdir processes in a single run, I kill
>> -9'ed them all.  Didn't see side-effects similar to what's described in the
>> original post, but the way to detect this issue might be to look for orphaned
>> children processes lacking a parent PID, then reap them.
> 
> 
> Updating my FreeBSD VM again, I captured one of the error messages that's
> leading to these orphaned ecompressdir processes:
> 
> /usr/lib/portage/python3.5/ebuild-helpers/ecompressdir: cannot make pipe for
> process substitution: File exists
> /usr/lib/portage/python3.5/ebuild-helpers/ecompressdir: line 72:
> /ramfs/portage/sys-freebsd/boot0-10.3/temp/sh-np-1865519000: ambiguous redirect
> ecompressdir: bzip2 -9 /usr/share/man
>  * The ebuild phase 'install' with pid 32075 appears to have left an orphan
>  * process running in the background.
> 
> 
> And a second one:
> /ramfs/portage/._portage_reinstall_.pesqhjhn/bin/ebuild-helpers/ecompressdir:
> cannot make pipe for process substitution: File exists
> /ramfs/portage/._portage_reinstall_.pesqhjhn/bin/ebuild-helpers/ecompressdir:
> line 72: /ramfs/portage/sys-apps/grep-2.25/temp/sh-np-474708936: ambiguous redirect
> ecompressdir: bzip2 -9 /usr/share/man
> ecompressdir: bzip2 -9 /usr/share/info
> ecompressdir: bzip2 -9 /usr/share/doc
>  * The ebuild phase 'install' with pid 60185 appears to have left an orphan
>  * process running in the background.
> 
> Not sure the exact cause.  Any additional info I can provide?
> 
> --J

Looks like a problem with bash. Make sure your bash has the fix for this
issue:

https://bugs.gentoo.org/show_bug.cgi?id=447810

What version of bash is it? Maybe try some other versions.
-- 
Thanks,
Zac

Re: [gentoo-portage-dev] dead emerge processes and/or lockfiles

[Joshua Kinard]

On 04/24/2016 20:09, Zac Medico wrote:
> On 04/24/2016 04:50 PM, Joshua Kinard wrote:
>> On 01/18/2016 07:37, Joshua Kinard wrote:
>>> On 01/17/2016 15:01, Zac Medico wrote:
>>>> On 01/17/2016 09:06 AM, Brian Dolbec wrote:
>>>>>
>>>>> I've read in several forum posts lately about emerge not running and
>>>>> the problem comes down to dead emerge processes and remaining lockfiles.
>>>>>
>>>>> Perhaps we should make an emaint module to search for and fix these.
>>>>> It should be easy enough.
>>>>
>>>> It would be nicer if we fixed whatever issue(s) cause the emerge
>>>> processes to hang up. How would the emaint module distinguish a "good"
>>>> emerge process from a "bad" one? I suppose you could strace it to see if
>>>> it has any activity.
>>>>
>>>
>>> I've been playing around with Gentoo/FreeBSD and have been noticing that emerge
>>> is leaving orphaned processes behind on that platform.  Seems to be
>>> ecompressdir getting hung up.  emerge itself just moves on, but after I
>>> accumulated ~5 of those stuck ecompressdir processes in a single run, I kill
>>> -9'ed them all.  Didn't see side-effects similar to what's described in the
>>> original post, but the way to detect this issue might be to look for orphaned
>>> children processes lacking a parent PID, then reap them.
>>
>>
>> Updating my FreeBSD VM again, I captured one of the error messages that's
>> leading to these orphaned ecompressdir processes:
>>
>> /usr/lib/portage/python3.5/ebuild-helpers/ecompressdir: cannot make pipe for
>> process substitution: File exists
>> /usr/lib/portage/python3.5/ebuild-helpers/ecompressdir: line 72:
>> /ramfs/portage/sys-freebsd/boot0-10.3/temp/sh-np-1865519000: ambiguous redirect
>> ecompressdir: bzip2 -9 /usr/share/man
>>  * The ebuild phase 'install' with pid 32075 appears to have left an orphan
>>  * process running in the background.
>>
>>
>> And a second one:
>> /ramfs/portage/._portage_reinstall_.pesqhjhn/bin/ebuild-helpers/ecompressdir:
>> cannot make pipe for process substitution: File exists
>> /ramfs/portage/._portage_reinstall_.pesqhjhn/bin/ebuild-helpers/ecompressdir:
>> line 72: /ramfs/portage/sys-apps/grep-2.25/temp/sh-np-474708936: ambiguous redirect
>> ecompressdir: bzip2 -9 /usr/share/man
>> ecompressdir: bzip2 -9 /usr/share/info
>> ecompressdir: bzip2 -9 /usr/share/doc
>>  * The ebuild phase 'install' with pid 60185 appears to have left an orphan
>>  * process running in the background.
>>
>> Not sure the exact cause.  Any additional info I can provide?
>>
>> --J
> 
> Looks like a problem with bash. Make sure your bash has the fix for this
> issue:
> 
> https://bugs.gentoo.org/show_bug.cgi?id=447810
> 
> What version of bash is it? Maybe try some other versions.

Latest version in ~arch, bash-4.3_p42-r2.

Doesn't appear to be completely tied to FreeBSD, either, as there's this
unanswered topic on the forums from Nov 2015:
https://forums-lb.gentoo.org/viewtopic-t-1032574.html?sid=5d7566d09a49ba06124032598d3ad362

Just looks like FreeBSD trips it up far more often, as I've only seen it there.

--J

Re: [gentoo-portage-dev] dead emerge processes and/or lockfiles

[Joshua Kinard]

On 04/24/2016 20:21, Joshua Kinard wrote:
> On 04/24/2016 20:09, Zac Medico wrote:
>> On 04/24/2016 04:50 PM, Joshua Kinard wrote:
>>> On 01/18/2016 07:37, Joshua Kinard wrote:
>>>> On 01/17/2016 15:01, Zac Medico wrote:
>>>>> On 01/17/2016 09:06 AM, Brian Dolbec wrote:
>>>>>>
>>>>>> I've read in several forum posts lately about emerge not running and
>>>>>> the problem comes down to dead emerge processes and remaining lockfiles.
>>>>>>
>>>>>> Perhaps we should make an emaint module to search for and fix these.
>>>>>> It should be easy enough.
>>>>>
>>>>> It would be nicer if we fixed whatever issue(s) cause the emerge
>>>>> processes to hang up. How would the emaint module distinguish a "good"
>>>>> emerge process from a "bad" one? I suppose you could strace it to see if
>>>>> it has any activity.
>>>>>
>>>>
>>>> I've been playing around with Gentoo/FreeBSD and have been noticing that emerge
>>>> is leaving orphaned processes behind on that platform.  Seems to be
>>>> ecompressdir getting hung up.  emerge itself just moves on, but after I
>>>> accumulated ~5 of those stuck ecompressdir processes in a single run, I kill
>>>> -9'ed them all.  Didn't see side-effects similar to what's described in the
>>>> original post, but the way to detect this issue might be to look for orphaned
>>>> children processes lacking a parent PID, then reap them.
>>>
>>>
>>> Updating my FreeBSD VM again, I captured one of the error messages that's
>>> leading to these orphaned ecompressdir processes:
>>>
>>> /usr/lib/portage/python3.5/ebuild-helpers/ecompressdir: cannot make pipe for
>>> process substitution: File exists
>>> /usr/lib/portage/python3.5/ebuild-helpers/ecompressdir: line 72:
>>> /ramfs/portage/sys-freebsd/boot0-10.3/temp/sh-np-1865519000: ambiguous redirect
>>> ecompressdir: bzip2 -9 /usr/share/man
>>>  * The ebuild phase 'install' with pid 32075 appears to have left an orphan
>>>  * process running in the background.
>>>
>>>
>>> And a second one:
>>> /ramfs/portage/._portage_reinstall_.pesqhjhn/bin/ebuild-helpers/ecompressdir:
>>> cannot make pipe for process substitution: File exists
>>> /ramfs/portage/._portage_reinstall_.pesqhjhn/bin/ebuild-helpers/ecompressdir:
>>> line 72: /ramfs/portage/sys-apps/grep-2.25/temp/sh-np-474708936: ambiguous redirect
>>> ecompressdir: bzip2 -9 /usr/share/man
>>> ecompressdir: bzip2 -9 /usr/share/info
>>> ecompressdir: bzip2 -9 /usr/share/doc
>>>  * The ebuild phase 'install' with pid 60185 appears to have left an orphan
>>>  * process running in the background.
>>>
>>> Not sure the exact cause.  Any additional info I can provide?
>>>
>>> --J
>>
>> Looks like a problem with bash. Make sure your bash has the fix for this
>> issue:
>>
>> https://bugs.gentoo.org/show_bug.cgi?id=447810
>>
>> What version of bash is it? Maybe try some other versions.
> 
> Latest version in ~arch, bash-4.3_p42-r2.
> 
> Doesn't appear to be completely tied to FreeBSD, either, as there's this
> unanswered topic on the forums from Nov 2015:
> https://forums-lb.gentoo.org/viewtopic-t-1032574.html?sid=5d7566d09a49ba06124032598d3ad362
> 
> Just looks like FreeBSD trips it up far more often, as I've only seen it there.
> 
> --J

Took some more digging, but here's our bug for it.  Does appear to be mostly
FreeBSD-related:
https://bugs.gentoo.org/show_bug.cgi?id=574426

Doesn't answer the question of how it happened in that one Linux case, but w/o
additional information, sounds like it's a remote corner case on Linux and hard
to reproduce.

--J