[gentoo-user] openpty() failing with UNIX98 ptys

[gentoo-user] openpty() failing with UNIX98 ptys

[Mike Edenfield]

At some point recently, one of my systems has begun having problems
allocating pseudo-terminals via the UNIX98 pty scheme. I am using the same
kernel configuration I've had for years, and running the latest ~amd64
version of all the relevant packages. The problem manifests itself on any
program that attempts to allocate a pseudo-terminal, including portage and
openssh. I first noticed the problem when I could no longer ssh into the
server because it would not allocate a pty. 

I have the latest udev installed, and udev-mount is running on boot. Both
/dev and /dev/pts are mounted, and /dev/ptmx exists and is world-readable:

basement package.use # mount | grep /dev
/dev/root on / type ext3
(rw,seclabel,noatime,errors=continue,barrier=1,data=writeback)
devpts on /dev/pts type devpts
(rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
shm on /dev/shm type tmpfs (rw,seclabel,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs
(rw,seclabel,nosuid,relatime,size=10240k,nr_inodes=248584,mode=755)

basement package.use # ls -alF /dev/ptmx /dev/pts
crw-rw-rw-. 1 root tty  5, 2 Jan 26 13:18 /dev/ptmx

/dev/pts:
total 0
drwxr-xr-x.  2 root root    40 Jan 26 13:18 ./
drwxr-xr-x. 10 root root 13300 Jan 26 13:18 ../

When I trace sshd's attempt to open a new pty, I see it doing this:

* open /dev/ptmx
* stat /dev/pts
* stat /dev
* try (and fail) to open /dev/ptyp0

Since I know that last bit is openssh trying to open an old-style BSD pty, I
can only assume that something is going wrong trying to allocate the pty the
correct way.

For the time being I've added BSD pty support into my kernel and everything
seems to be working now, but I'm at a loss as to what I did to break things
in the first place.

[gentoo-user] Re: openpty() failing with UNIX98 ptys

[walt]

On 01/26/2013 08:46 PM, Mike Edenfield wrote:
> At some point recently, one of my systems has begun having problems
> allocating pseudo-terminals via the UNIX98 pty scheme. I am using the same
> kernel configuration I've had for years, and running the latest ~amd64
> version of all the relevant packages. The problem manifests itself on any
> program that attempts to allocate a pseudo-terminal, including portage and
> openssh. I first noticed the problem when I could no longer ssh into the
> server because it would not allocate a pty. 
> 
> I have the latest udev installed, and udev-mount is running on boot. Both
> /dev and /dev/pts are mounted, and /dev/ptmx exists and is world-readable:
> 
> basement package.use # mount | grep /dev
> /dev/root on / type ext3
> (rw,seclabel,noatime,errors=continue,barrier=1,data=writeback)
> devpts on /dev/pts type devpts
> (rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
> shm on /dev/shm type tmpfs (rw,seclabel,nosuid,nodev,noexec,relatime)
> udev on /dev type devtmpfs
> (rw,seclabel,nosuid,relatime,size=10240k,nr_inodes=248584,mode=755)
> 
> basement package.use # ls -alF /dev/ptmx /dev/pts
> crw-rw-rw-. 1 root tty  5, 2 Jan 26 13:18 /dev/ptmx
> 
> /dev/pts:
> total 0
> drwxr-xr-x.  2 root root    40 Jan 26 13:18 ./
> drwxr-xr-x. 10 root root 13300 Jan 26 13:18 ../
> 
> When I trace sshd's attempt to open a new pty, I see it doing this:
> 
> * open /dev/ptmx
> * stat /dev/pts
> * stat /dev
> * try (and fail) to open /dev/ptyp0
> 
> Since I know that last bit is openssh trying to open an old-style BSD pty, I
> can only assume that something is going wrong trying to allocate the pty the
> correct way.
> 
> For the time being I've added BSD pty support into my kernel and everything
> seems to be working now, but I'm at a loss as to what I did to break things
> in the first place.

Do you have any files left in /usr/lib/udev ?  If so, try re-emerging the
package that owns those files.

Re: [gentoo-user] openpty() failing with UNIX98 ptys

[Alan McKinnon]

On Sat, 26 Jan 2013 23:46:22 -0500
"Mike Edenfield" <kutulu@kutulu.org> wrote:

> At some point recently, one of my systems has begun having problems
> allocating pseudo-terminals via the UNIX98 pty scheme. I am using the
> same kernel configuration I've had for years, and running the latest
> ~amd64 version of all the relevant packages. The problem manifests
> itself on any program that attempts to allocate a pseudo-terminal,
> including portage and openssh. I first noticed the problem when I
> could no longer ssh into the server because it would not allocate a
> pty. 
> 
> I have the latest udev installed, and udev-mount is running on boot.
> Both /dev and /dev/pts are mounted, and /dev/ptmx exists and is
> world-readable:
> 
> basement package.use # mount | grep /dev
> /dev/root on / type ext3
> (rw,seclabel,noatime,errors=continue,barrier=1,data=writeback)
> devpts on /dev/pts type devpts
> (rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
> shm on /dev/shm type tmpfs (rw,seclabel,nosuid,nodev,noexec,relatime)
> udev on /dev type devtmpfs
> (rw,seclabel,nosuid,relatime,size=10240k,nr_inodes=248584,mode=755)
> 
> basement package.use # ls -alF /dev/ptmx /dev/pts
> crw-rw-rw-. 1 root tty  5, 2 Jan 26 13:18 /dev/ptmx
> 
> /dev/pts:
> total 0
> drwxr-xr-x.  2 root root    40 Jan 26 13:18 ./
> drwxr-xr-x. 10 root root 13300 Jan 26 13:18 ../
> 
> When I trace sshd's attempt to open a new pty, I see it doing this:
> 
> * open /dev/ptmx
> * stat /dev/pts
> * stat /dev
> * try (and fail) to open /dev/ptyp0
> 
> Since I know that last bit is openssh trying to open an old-style BSD
> pty, I can only assume that something is going wrong trying to
> allocate the pty the correct way.
> 
> For the time being I've added BSD pty support into my kernel and
> everything seems to be working now, but I'm at a loss as to what I
> did to break things in the first place.

I had something similar (details are different though):

All my virt consoles went away and I couldn't get to them after X
starts. Ctrl-Alt-F1 left the X screen as-is and it wouldn't blank and
give me the KMS framebuffer. Ctrl-Alt-F7 brought X back.

In my case it's kernel 3.7 - no version of gentoo-sources-3.7-* worked
and 3.6.11 works fine.

What kernel are you on?
Have you tested this on 3.6?



-- 
Alan McKinnon
alan.mckinnon@gmail.com

RE: [gentoo-user] openpty() failing with UNIX98 ptys

[Mike Edenfield]

> From: Alan McKinnon [mailto:alan.mckinnon@gmail.com]
> Sent: Sunday, January 27, 2013 1:49 PM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] openpty() failing with UNIX98 ptys
> 
> On Sat, 26 Jan 2013 23:46:22 -0500
> "Mike Edenfield" <kutulu@kutulu.org> wrote:
> 
> > I have the latest udev installed, and udev-mount is running on
> boot.
> > Both /dev and /dev/pts are mounted, and /dev/ptmx exists and is
> > world-readable:
> >
> > basement package.use # mount | grep /dev
> > /dev/root on / type ext3
> > (rw,seclabel,noatime,errors=continue,barrier=1,data=writeback)
> > devpts on /dev/pts type devpts
> > (rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
> > shm on /dev/shm type tmpfs
> (rw,seclabel,nosuid,nodev,noexec,relatime)
> > udev on /dev type devtmpfs
> > (rw,seclabel,nosuid,relatime,size=10240k,nr_inodes=248584,mode=755)
> >
> > basement package.use # ls -alF /dev/ptmx /dev/pts
> > crw-rw-rw-. 1 root tty  5, 2 Jan 26 13:18 /dev/ptmx
> >
> > /dev/pts:
> > total 0
> > drwxr-xr-x.  2 root root    40 Jan 26 13:18 ./
> > drwxr-xr-x. 10 root root 13300 Jan 26 13:18 ../
> >
> > When I trace sshd's attempt to open a new pty, I see it doing this:
> >
> > * open /dev/ptmx
> > * stat /dev/pts
> > * stat /dev
> > * try (and fail) to open /dev/ptyp0
> >
> > Since I know that last bit is openssh trying to open an old-style
> BSD
> > pty, I can only assume that something is going wrong trying to
> > allocate the pty the correct way.
> >
> > For the time being I've added BSD pty support into my kernel and
> > everything seems to be working now, but I'm at a loss as to what I
> > did to break things in the first place.
> 
> I had something similar (details are different though):

> In my case it's kernel 3.7 - no version of gentoo-sources-3.7-*
> worked
> and 3.6.11 works fine.
> 
> What kernel are you on?
> Have you tested this on 3.6?

I first notice the problem on 3.4.2, upgraded to 3.6.4 and the problem
persisted. I have not upgraded to 3.7 to see if it's still a problem.

--Mike

[gentoo-user] Re: openpty() failing with UNIX98 ptys

[walt]

On 01/27/2013 10:48 AM, Alan McKinnon wrote:
> I had something similar (details are different though):
> 
> All my virt consoles went away and I couldn't get to them after X
> starts. Ctrl-Alt-F1 left the X screen as-is and it wouldn't blank and
> give me the KMS framebuffer. Ctrl-Alt-F7 brought X back.
> 
> In my case it's kernel 3.7 - no version of gentoo-sources-3.7-* worked
> and 3.6.11 works fine.

I'm not seeing that problem.  You could do a diff on the /boot/config* files
and look for missing stuff in your 3.7 kernels.

Re: [gentoo-user] openpty() failing with UNIX98 ptys

[Peter Humphrey]

On Sunday 27 January 2013 04:46:22 Mike Edenfield wrote:
> At some point recently, one of my systems has begun having problems
> allocating pseudo-terminals via the UNIX98 pty scheme. I am using the
> same kernel configuration I've had for years, and running the latest
> ~amd64 version of all the relevant packages. The problem manifests
> itself on any program that attempts to allocate a pseudo-terminal,
> including portage and openssh. I first noticed the problem when I could
> no longer ssh into the server because it would not allocate a pty.
> 
> I have the latest udev installed, and udev-mount is running on boot. Both
> /dev and /dev/pts are mounted, and /dev/ptmx exists and is
> world-readable:

What does 'grep devtmpfs /etc/fstab' reveal? (Long shot - I misread the 
latest news article and changed one of the tmpfs fields to devtmpfs and got 
similar results to what you describe.)

-- 
Peter

RE: [gentoo-user] openpty() failing with UNIX98 ptys

[Mike Edenfield]

> -----Original Message-----
> From: Peter Humphrey [mailto:peter@humphrey.ukfsn.org]
> Sent: Sunday, January 27, 2013 9:51 PM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] openpty() failing with UNIX98 ptys
> 
> On Sunday 27 January 2013 04:46:22 Mike Edenfield wrote:
> > At some point recently, one of my systems has begun having problems
> > allocating pseudo-terminals via the UNIX98 pty scheme. I am using the
> > same kernel configuration I've had for years, and running the latest
> > ~amd64 version of all the relevant packages. The problem manifests
> > itself on any program that attempts to allocate a pseudo-terminal,
> > including portage and openssh. I first noticed the problem when I could
> > no longer ssh into the server because it would not allocate a pty.
> >
> > I have the latest udev installed, and udev-mount is running on boot.
Both
> > /dev and /dev/pts are mounted, and /dev/ptmx exists and is
> > world-readable:
> 
> What does 'grep devtmpfs /etc/fstab' reveal? (Long shot - I misread the
> latest news article and changed one of the tmpfs fields to devtmpfs and
got
> similar results to what you describe.)

I don't have any devtmpfs in my /etc/fstab; I think I removed a lot of
pseudo-mount points a while back when I added udev-mount to my runlevel.

--Mike