[gentoo-user] dhcpd always shows "crashed" even though it's running

[gentoo-user] dhcpd always shows "crashed" even though it's running

[Mike Edenfield]

For some reason, whenever I check the status of my startup scripts, 
dhcpd registers as "crashed". However, dhcpd is up and running and 
working fine. Normally I don't worry about it, but on those occasions 
where dhcpd does stop working, it's hard to tell if it's "fixed" or not.

What makes rc-status think something is crashed, and how can I fix this?

basement log # rc-status -v | grep crashed
  dhcpd [  crashed  ]
basement log # ps aux | grep dhcpd
root      2214  0.0  0.0   8268   876 pts/0    S+   19:47   0:00 grep 
--colour=auto dhcpd
dhcp      2648  0.0  0.6  30028 12136 ?        Ss   Aug29   0:00 
/usr/sbin/dhcpd -cf /etc/dhcp/dhcpd.conf -q -pf /var/run/dhcp/dhcpd.pid 
-lf /var/lib/dhcp/dhcpd.leases -user dhcp -group dhcp -chroot 
/chroot/dhcp enp0s7

Re: [gentoo-user] dhcpd always shows "crashed" even though it's running

[Fernando Rodriguez]

On Thursday, September 03, 2015 8:09:02 PM Mike Edenfield wrote:
> For some reason, whenever I check the status of my startup scripts, 
> dhcpd registers as "crashed". However, dhcpd is up and running and 
> working fine. Normally I don't worry about it, but on those occasions 
> where dhcpd does stop working, it's hard to tell if it's "fixed" or not.
> 
> What makes rc-status think something is crashed, and how can I fix this?
> 
> basement log # rc-status -v | grep crashed
>   dhcpd [  crashed  ]
> basement log # ps aux | grep dhcpd
> root      2214  0.0  0.0   8268   876 pts/0    S+   19:47   0:00 grep 
> --colour=auto dhcpd
> dhcp      2648  0.0  0.6  30028 12136 ?        Ss   Aug29   0:00 
> /usr/sbin/dhcpd -cf /etc/dhcp/dhcpd.conf -q -pf /var/run/dhcp/dhcpd.pid 
> -lf /var/lib/dhcp/dhcpd.leases -user dhcp -group dhcp -chroot 
> /chroot/dhcp enp0s7
> 
> 

This is just a guess but it could be the permissions on the pid file on 
/chroot/dhcp/var/run/dhcp/. So stop the daemon, delete the file, check that the 
directory is owned by dhcp:dhcp and start the daemon again.

-- 
Fernando Rodriguez

Re: [gentoo-user] dhcpd always shows "crashed" even though it's running

[Rich Freeman]

On Thu, Sep 3, 2015 at 8:09 PM, Mike Edenfield <kutulu@kutulu.org> wrote:
> For some reason, whenever I check the status of my startup scripts, dhcpd
> registers as "crashed". However, dhcpd is up and running and working fine.
> Normally I don't worry about it, but on those occasions where dhcpd does
> stop working, it's hard to tell if it's "fixed" or not.
>
> What makes rc-status think something is crashed, and how can I fix this?
>

Not to sidetrack the discussion, but this is one of the areas where
systemd does shine, in a sense.  If systemd thinks the service is
down, it is definitely down, because by default when systemd thinks a
service is down it kills anything it ever spawned (and it can
auto-restart if configured to do so).  So, this forces to you
configure the unit correctly so that you don't have these kinds of
maybe-running-maybe-not situations.

It is a bit like having strong types and stricter build-time error
checking.  It makes it a little harder to be lazier but saves you as
the user from the lazy developer.

As far as openrc goes, I suspect it is a pid file issue of some kind.
If a process goes and forks without putting the right pid in the file
then there is no way for openrc to detect this.

-- 
Rich

Re: [gentoo-user] dhcpd always shows "crashed" even though it's running

[Dale]

Fernando Rodriguez wrote:
> On Thursday, September 03, 2015 8:09:02 PM Mike Edenfield wrote:
>> For some reason, whenever I check the status of my startup scripts, 
>> dhcpd registers as "crashed". However, dhcpd is up and running and 
>> working fine. Normally I don't worry about it, but on those occasions 
>> where dhcpd does stop working, it's hard to tell if it's "fixed" or not.
>>
>> What makes rc-status think something is crashed, and how can I fix this?
>>
>> basement log # rc-status -v | grep crashed
>>   dhcpd [  crashed  ]
>> basement log # ps aux | grep dhcpd
>> root      2214  0.0  0.0   8268   876 pts/0    S+   19:47   0:00 grep 
>> --colour=auto dhcpd
>> dhcp      2648  0.0  0.6  30028 12136 ?        Ss   Aug29   0:00 
>> /usr/sbin/dhcpd -cf /etc/dhcp/dhcpd.conf -q -pf /var/run/dhcp/dhcpd.pid 
>> -lf /var/lib/dhcp/dhcpd.leases -user dhcp -group dhcp -chroot 
>> /chroot/dhcp enp0s7
>>
>>
> This is just a guess but it could be the permissions on the pid file on 
> /chroot/dhcp/var/run/dhcp/. So stop the daemon, delete the file, check that the 
> directory is owned by dhcp:dhcp and start the daemon again.
>


I don't know if this will help or not but don't forget the zap option. 

root@fireball / # /etc/init.d/dhcpcd  <tab twice>
broken   ineed    iuse     needsme  pause    restart  start    status  
stop     usesme   zap     
root@fireball / # /etc/init.d/dhcpcd 

It's been a long time since I used it but if I recall correctly that
resets the status.  I think it stops and deletes any files that stores
its run status. 

If that doesn't apply, just ignore me.  Heck, a lot of people ignore
me.  lol 

Dale

:-)  :-) 

Re: [gentoo-user] dhcpd always shows "crashed" even though it's running

[Fernando Rodriguez]

On Thursday, September 03, 2015 8:47:34 PM Dale wrote:
> Fernando Rodriguez wrote:
> > On Thursday, September 03, 2015 8:09:02 PM Mike Edenfield wrote:
> >> For some reason, whenever I check the status of my startup scripts, 
> >> dhcpd registers as "crashed". However, dhcpd is up and running and 
> >> working fine. Normally I don't worry about it, but on those occasions 
> >> where dhcpd does stop working, it's hard to tell if it's "fixed" or not.
> >>
> >> What makes rc-status think something is crashed, and how can I fix this?
> >>
> >> basement log # rc-status -v | grep crashed
> >>   dhcpd [  crashed  ]
> >> basement log # ps aux | grep dhcpd
> >> root      2214  0.0  0.0   8268   876 pts/0    S+   19:47   0:00 grep 
> >> --colour=auto dhcpd
> >> dhcp      2648  0.0  0.6  30028 12136 ?        Ss   Aug29   0:00 
> >> /usr/sbin/dhcpd -cf /etc/dhcp/dhcpd.conf -q -pf /var/run/dhcp/dhcpd.pid 
> >> -lf /var/lib/dhcp/dhcpd.leases -user dhcp -group dhcp -chroot 
> >> /chroot/dhcp enp0s7
> >>
> >>
> > This is just a guess but it could be the permissions on the pid file on 
> > /chroot/dhcp/var/run/dhcp/. So stop the daemon, delete the file, check that 
the 
> > directory is owned by dhcp:dhcp and start the daemon again.
> >
> 
> 
> I don't know if this will help or not but don't forget the zap option. 
> 
> root@fireball / # /etc/init.d/dhcpcd  <tab twice>
> broken   ineed    iuse     needsme  pause    restart  start    status  
> stop     usesme   zap     
> root@fireball / # /etc/init.d/dhcpcd 
> 
> It's been a long time since I used it but if I recall correctly that
> resets the status.  I think it stops and deletes any files that stores
> its run status. 
> 
> If that doesn't apply, just ignore me.  Heck, a lot of people ignore
> me.  lol 
> 
> Dale
> 
> :-)  :-) 
> 

It does. I knew about it but never used it and didn't know what it'll do if 
the permissions are wrong so I thought I'd minimize the chances of being 
wrong.

-- 
Fernando Rodriguez

Re: [gentoo-user] dhcpd always shows "crashed" even though it's running

[Dale]

Fernando Rodriguez wrote:
> On Thursday, September 03, 2015 8:47:34 PM Dale wrote:
>
> I don't know if this will help or not but don't forget the zap option. 
>
> root@fireball / # /etc/init.d/dhcpcd  <tab twice>
> broken   ineed    iuse     needsme  pause    restart  start    status  
> stop     usesme   zap     
> root@fireball / # /etc/init.d/dhcpcd 
>
> It's been a long time since I used it but if I recall correctly that
> resets the status.  I think it stops and deletes any files that stores
> its run status. 
>
> If that doesn't apply, just ignore me.  Heck, a lot of people ignore
> me.  lol 
>
> Dale
>
> :-)  :-) 
>
> It does. I knew about it but never used it and didn't know what it'll do if 
> the permissions are wrong so I thought I'd minimize the chances of being 
> wrong.
>

I wasn't sure if it would help or not.  I know it has worked in the past
for me but it has been while.  I usually tell the service to stop, make
sure any processes are dead, with kill command if needed, and then use
the zap thing.  Maybe try that as a last resort if nothing else. 

I just didn't want to not mention it and it turn out to be just what was
needed.  ;-)

Dale

:-)  :-)

Re: [gentoo-user] dhcpd always shows "crashed" even though it's running

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 1667 bytes --]

On Friday 04 Sep 2015 03:46:04 Dale wrote:
> Fernando Rodriguez wrote:
> > On Thursday, September 03, 2015 8:47:34 PM Dale wrote:
> > 
> > I don't know if this will help or not but don't forget the zap option.
> > 
> > root@fireball / # /etc/init.d/dhcpcd  <tab twice>
> > broken   ineed    iuse     needsme  pause    restart  start    status
> > stop     usesme   zap
> > root@fireball / # /etc/init.d/dhcpcd
> > 
> > It's been a long time since I used it but if I recall correctly that
> > resets the status.  I think it stops and deletes any files that stores
> > its run status.
> > 
> > If that doesn't apply, just ignore me.  Heck, a lot of people ignore
> > me.  lol
> > 
> > Dale
> > 
> > :-)  :-)
> > 
> > It does. I knew about it but never used it and didn't know what it'll do
> > if the permissions are wrong so I thought I'd minimize the chances of
> > being wrong.
> 
> I wasn't sure if it would help or not.  I know it has worked in the past
> for me but it has been while.  I usually tell the service to stop, make
> sure any processes are dead, with kill command if needed, and then use
> the zap thing.  Maybe try that as a last resort if nothing else.
> 
> I just didn't want to not mention it and it turn out to be just what was
> needed.  ;-)
> 
> Dale
> 
> :-)  :-)

You can increase its verbosity in /etc/init.d/dhcpcd, so that the logs show 
more of what is happening to cause the crash.

Also, here at least, I have /run/dhcpcd/ with its subdirectories as well as 
/run/dhcpcd-enp11s0.pid both owned by root:root, but this is a laptop and the 
dhcpcd is launched by ifplugd.

-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] dhcpd always shows "crashed" even though it's running

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 520 bytes --]

On Fri, 4 Sep 2015 08:06:18 +0100, Mick wrote:

> Also, here at least, I have /run/dhcpcd/ with its subdirectories as
> well as /run/dhcpcd-enp11s0.pid both owned by root:root, but this is a
> laptop and the dhcpcd is launched by ifplugd.

Using ifplugd to manage your connection can conflict with the openrc
stuff. If you install ifplugd but do not configure it to run, openrc will
use it itself, avoiding any such conflicts.


-- 
Neil Bothwick

PCMCIA: People Can't Memorize Computer Industry Acronyms

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-user] dhcpd always shows "crashed" even though it's running

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 757 bytes --]

On Friday 04 Sep 2015 09:48:00 Neil Bothwick wrote:
> On Fri, 4 Sep 2015 08:06:18 +0100, Mick wrote:
> > Also, here at least, I have /run/dhcpcd/ with its subdirectories as
> > well as /run/dhcpcd-enp11s0.pid both owned by root:root, but this is a
> > laptop and the dhcpcd is launched by ifplugd.
> 
> Using ifplugd to manage your connection can conflict with the openrc
> stuff. If you install ifplugd but do not configure it to run, openrc will
> use it itself, avoiding any such conflicts.

I am a bit confused about your statement not to configure ifplugd to run.  
There's no /etc/conf.d or /etc/init.d file for ifplugd per se, if that's what 
you mean, but in /etc/conf.d/net I have:

ifplugd_enp11s0="-t 2 -u 3

-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [gentoo-user] dhcpd always shows "crashed" even though it's running

[Fernando Rodriguez]

On Friday, September 04, 2015 8:06:18 AM Mick wrote:
> On Friday 04 Sep 2015 03:46:04 Dale wrote:
> > Fernando Rodriguez wrote:
> > > On Thursday, September 03, 2015 8:47:34 PM Dale wrote:
> > > 
> > > I don't know if this will help or not but don't forget the zap option.
> > > 
> > > root@fireball / # /etc/init.d/dhcpcd  <tab twice>
> > > broken   ineed    iuse     needsme  pause    restart  start    status
> > > stop     usesme   zap
> > > root@fireball / # /etc/init.d/dhcpcd
> > > 
> > > It's been a long time since I used it but if I recall correctly that
> > > resets the status.  I think it stops and deletes any files that stores
> > > its run status.
> > > 
> > > If that doesn't apply, just ignore me.  Heck, a lot of people ignore
> > > me.  lol
> > > 
> > > Dale
> > > 
> > > :-)  :-)
> > > 
> > > It does. I knew about it but never used it and didn't know what it'll do
> > > if the permissions are wrong so I thought I'd minimize the chances of
> > > being wrong.
> > 
> > I wasn't sure if it would help or not.  I know it has worked in the past
> > for me but it has been while.  I usually tell the service to stop, make
> > sure any processes are dead, with kill command if needed, and then use
> > the zap thing.  Maybe try that as a last resort if nothing else.
> > 
> > I just didn't want to not mention it and it turn out to be just what was
> > needed.  ;-)
> > 
> > Dale
> > 
> > :-)  :-)
> 
> You can increase its verbosity in /etc/init.d/dhcpcd, so that the logs show 
> more of what is happening to cause the crash.
> 
> Also, here at least, I have /run/dhcpcd/ with its subdirectories as well as 
> /run/dhcpcd-enp11s0.pid both owned by root:root, but this is a laptop and 
the 
> dhcpcd is launched by ifplugd.
> 
> 

dhcpd is getting confused with dhcpd... But now that you mentioned that my pid 
file for dhcpd is also owned by root:root, but the directory is owned by 
dhcp:dhcp and the daemon runs as dhcp.

-- 
Fernando Rodriguez

Re: [gentoo-user] dhcpd always shows "crashed" even though it's running

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 768 bytes --]

On Fri, 4 Sep 2015 10:10:46 +0100, Mick wrote:

> > Using ifplugd to manage your connection can conflict with the openrc
> > stuff. If you install ifplugd but do not configure it to run, openrc
> > will use it itself, avoiding any such conflicts.  
> 
> I am a bit confused about your statement not to configure ifplugd to
> run. There's no /etc/conf.d or /etc/init.d file for ifplugd per se, if
> that's what you mean, but in /etc/conf.d/net I have:
> 
> ifplugd_enp11s0="-t 2 -u 3

That's the correct way to do it, although I didn't even need to give
options. Ifplugd can be run standalone, as on other distros, I read your
post to imply that's what you were doing.


-- 
Neil Bothwick

Bang on the LEFT side of your computer to restart Windows

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [gentoo-user] dhcpd always shows "crashed" even though it's running

[Mike Edenfield]

On 9/3/2015 8:59 PM, Fernando Rodriguez wrote:
> On Thursday, September 03, 2015 8:09:02 PM Mike Edenfield wrote:

>> What makes rc-status think something is crashed, and how can I fix this?
>>
>> basement log # rc-status -v | grep crashed
>>    dhcpd [  crashed  ]
>> basement log # ps aux | grep dhcpd
>> root      2214  0.0  0.0   8268   876 pts/0    S+   19:47   0:00 grep
>> --colour=auto dhcpd
>> dhcp      2648  0.0  0.6  30028 12136 ?        Ss   Aug29   0:00
>> /usr/sbin/dhcpd -cf /etc/dhcp/dhcpd.conf -q -pf /var/run/dhcp/dhcpd.pid
>> -lf /var/lib/dhcp/dhcpd.leases -user dhcp -group dhcp -chroot
>> /chroot/dhcp enp0s7
>>
>>
>
> This is just a guess but it could be the permissions on the pid file on
> /chroot/dhcp/var/run/dhcp/. So stop the daemon, delete the file, check that the
> directory is owned by dhcp:dhcp and start the daemon again.
>

That was a good guess -- I did find something else unrelated wrong with 
the log file permissions :) But it didn't help here.

The directory is owned by dhcp:dhcp, and when I stop the service, the 
pid file is deleted automatically, which I assume means the permissions 
are correct:

basement log # dir /chroot/dhcp/var/run/dhcp
total 8
drwxr-xr-x 2 dhcp dhcp 4096 Sep  4 07:21 ./
drwxr-xr-x 3 root root 4096 Oct  4  2009 ../
basement log # /etc/init.d/dhcpd start
  * Starting chrooted dhcpd ... 
                                            [ ok ]
basement log # dir /chroot/dhcp/var/run/dhcp
total 12
drwxr-xr-x 2 dhcp dhcp 4096 Sep  4 07:21 ./
drwxr-xr-x 3 root root 4096 Oct  4  2009 ../
-rw-r--r-- 1 root root    6 Sep  4 07:21 dhcpd.pid
basement log # rc-status -v | grep crashed
  dhcpd       [ crashed  ]

Also, just for reference, I already tried "zap"; that didn't help 
because "stop" puts it correctly into the stopped state anyway.

--Mike

Re: [gentoo-user] dhcpd always shows "crashed" even though it's running

[Fernando Rodriguez]

On Friday, September 04, 2015 7:25:31 AM Mike Edenfield wrote:
> On 9/3/2015 8:59 PM, Fernando Rodriguez wrote:
> > On Thursday, September 03, 2015 8:09:02 PM Mike Edenfield wrote:
> 
> >> What makes rc-status think something is crashed, and how can I fix this?
> >>
> >> basement log # rc-status -v | grep crashed
> >>    dhcpd [  crashed  ]
> >> basement log # ps aux | grep dhcpd
> >> root      2214  0.0  0.0   8268   876 pts/0    S+   19:47   0:00 grep
> >> --colour=auto dhcpd
> >> dhcp      2648  0.0  0.6  30028 12136 ?        Ss   Aug29   0:00
> >> /usr/sbin/dhcpd -cf /etc/dhcp/dhcpd.conf -q -pf /var/run/dhcp/dhcpd.pid
> >> -lf /var/lib/dhcp/dhcpd.leases -user dhcp -group dhcp -chroot
> >> /chroot/dhcp enp0s7
> >>
> >>
> >
> > This is just a guess but it could be the permissions on the pid file on
> > /chroot/dhcp/var/run/dhcp/. So stop the daemon, delete the file, check that 
the
> > directory is owned by dhcp:dhcp and start the daemon again.
> >
> 
> That was a good guess -- I did find something else unrelated wrong with 
> the log file permissions :) But it didn't help here.
> 
> The directory is owned by dhcp:dhcp, and when I stop the service, the 
> pid file is deleted automatically, which I assume means the permissions 
> are correct:
> 
> basement log # dir /chroot/dhcp/var/run/dhcp
> total 8
> drwxr-xr-x 2 dhcp dhcp 4096 Sep  4 07:21 ./
> drwxr-xr-x 3 root root 4096 Oct  4  2009 ../
> basement log # /etc/init.d/dhcpd start
>   * Starting chrooted dhcpd ... 
>                                             [ ok ]
> basement log # dir /chroot/dhcp/var/run/dhcp
> total 12
> drwxr-xr-x 2 dhcp dhcp 4096 Sep  4 07:21 ./
> drwxr-xr-x 3 root root 4096 Oct  4  2009 ../
> -rw-r--r-- 1 root root    6 Sep  4 07:21 dhcpd.pid
> basement log # rc-status -v | grep crashed
>   dhcpd       [ crashed  ]
> 

After doing that again cat the pid file and compare it to the PID for dhcpd.
If it looks right you can try copying to /var/run/dhcp/ and run rc-status 
again, if it works this time then portage is looking for the pid file outside 
the chroot. You set it up using the DHCPD_CHROOT in /etc/conf.d/dhcpd right?
I don't use that option since I use apparmor but it looks like the init script 
will do the right thing in traccking the pid file if setup correctly. Are you 
using the latest version (may need to run etc-update)?

-- 
Fernando Rodriguez

Re: [gentoo-user] dhcpd always shows "crashed" even though it's running

[Mike Edenfield]

On 9/4/2015 3:06 AM, Mick wrote:

> You can increase its verbosity in /etc/init.d/dhcpcd, so that the logs show
> more of what is happening to cause the crash.
>
> Also, here at least, I have /run/dhcpcd/ with its subdirectories as well as
> /run/dhcpcd-enp11s0.pid both owned by root:root, but this is a laptop and the
> dhcpcd is launched by ifplugd.

This isn't dhcpcd (the client), it's dhcpd (the server), and the problem 
is, it's *not* crashing. It's running fine, and the service logs have 
nothing unusual in them.

It's only rc-status that seems to be confused.

--Mike