[gentoo-user] ssh rekeying slow ?

[gentoo-user] ssh rekeying slow ?

[Stefan G. Weichinger]

When I ssh into a server in my basement, this takes way more time than
usual.

I don't have a clue what might have changed ... aside from usual
updating. I rebuilt and restarted openssh down there without a change.

This is a bit annoying when logging in and using git to pull/push stuff
from/to there.

Does anyone have an idea what I could do to fix that?

Stefan

demo ->

$ ssh -v root@mythtv

OpenSSH_6.6.1, OpenSSL 1.0.1h 5 Jun 2014
debug1: Reading configuration data /home/sgw/.ssh/config
debug1: /home/sgw/.ssh/config line 33: Applying options for mythtv
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to mythtv [2001:15c0:65ff:8742:219:99ff:fee8:2343]
port 22.
debug1: fd 3 clearing O_NONBLOCK
debug1: Connection established.
debug1: identity file /home/sgw/.ssh/id_rsa type 1
debug1: identity file /home/sgw/.ssh/id_rsa-cert type -1
debug1: identity file /home/sgw/.ssh/id_dsa type -1
debug1: identity file /home/sgw/.ssh/id_dsa-cert type -1
debug1: identity file /home/sgw/.ssh/id_ecdsa type -1
debug1: identity file /home/sgw/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/sgw/.ssh/id_ed25519 type -1
debug1: identity file /home/sgw/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1p1-hpn14v4
debug1: Remote protocol version 2.0, remote software version
OpenSSH_6.6p1-hpn14v4
debug1: match: OpenSSH_6.6p1-hpn14v4 pat OpenSSH_6.5*,OpenSSH_6.6*
compat 0x14000000
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: AUTH STATE IS 0
debug1: REQUESTED ENC.NAME is 'aes128-ctr'
debug1: kex: server->client aes128-ctr hmac-md5-etm@openssh.com none
debug1: REQUESTED ENC.NAME is 'aes128-ctr'
debug1: kex: client->server aes128-ctr hmac-md5-etm@openssh.com none
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA
07:f3:16:2b:e9:64:87:fa:df:14:70:dc:03:60:5a:3c
debug1: Host 'mythtv' is known and matches the ECDSA host key.
debug1: Found key in /home/sgw/.ssh/known_hosts:168
debug1: ssh_ecdsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/sgw/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 277
debug1: Single to Multithread CTR cipher swap - client request
debug1: Authentication succeeded (publickey).
Authenticated to mythtv ([2001:15c0:65ff:8742:219:99ff:fee8:2343]:22).
debug1: HPN to Non-HPN Connection
debug1: Final hpn_buffer_size = 2097152
debug1: HPN Disabled: 0, HPN Buffer Size: 2097152
debug1: channel 0: new [client-session]
debug1: Enabled Dynamic Window Scaling
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: need rekeying
debug1: SSH2_MSG_KEXINIT sent
debug1: rekeying in progress
debug1: rekeying in progress
debug1: rekeying in progress
debug1: rekeying in progress
debug1: enqueue packet: 80
debug1: rekeying in progress
debug1: SSH2_MSG_KEXINIT received
debug1: AUTH STATE IS 1
debug1: REQUESTED ENC.NAME is 'aes128-ctr'
debug1: kex: server->client aes128-ctr hmac-md5-etm@openssh.com none
debug1: REQUESTED ENC.NAME is 'aes128-ctr'
debug1: kex: client->server aes128-ctr hmac-md5-etm@openssh.com none
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA
07:f3:16:2b:e9:64:87:fa:df:14:70:dc:03:60:5a:3c
debug1: Host 'mythtv' is known and matches the ECDSA host key.
debug1: Found key in /home/sgw/.ssh/known_hosts:168
debug1: ssh_ecdsa_verify: signature correct
debug1: set_newkeys: rekeying
debug1: spawned a thread
debug1: spawned a thread
debug1: dequeue packet: 80
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: set_newkeys: rekeying
debug1: spawned a thread
debug1: spawned a thread
debug1: SSH2_MSG_NEWKEYS received
debug1: Sending environment.
debug1: Sending env LANG = de_DE.UTF-8

[gentoo-user] Re: ssh rekeying slow ?

[James]

Stefan G. Weichinger <lists <at> xunil.at> writes:


> When I ssh into a server in my basement, this takes way more time than
> usual.
> Does anyone have an idea what I could do to fix that?


ssh has an ordered array of negotiations between systems that are related
to the version numbers of ssh and the other configurations. There is
usually a mismatch, when it takes "too long" to start a session,
in my experience.

I did not look at the specifics you posted.

hth,
James

Re: [gentoo-user] Re: ssh rekeying slow ?

[Stefan G. Weichinger]

Am 25.06.2014 20:30, schrieb James:
> Stefan G. Weichinger <lists <at> xunil.at> writes:
> 
> 
>> When I ssh into a server in my basement, this takes way more time than
>> usual.
>> Does anyone have an idea what I could do to fix that?
> 
> 
> ssh has an ordered array of negotiations between systems that are related
> to the version numbers of ssh and the other configurations. There is
> usually a mismatch, when it takes "too long" to start a session,
> in my experience.
> 
> I did not look at the specifics you posted.

both servers/machines run net-misc/openssh-6.6.1_p1 ... re-compiled
right today.

Re: [gentoo-user] Re: ssh rekeying slow ?

[Alan McKinnon]

On 25/06/2014 20:41, Stefan G. Weichinger wrote:
> Am 25.06.2014 20:30, schrieb James:
>> Stefan G. Weichinger <lists <at> xunil.at> writes:
>>
>>
>>> When I ssh into a server in my basement, this takes way more time than
>>> usual.
>>> Does anyone have an idea what I could do to fix that?
>>
>>
>> ssh has an ordered array of negotiations between systems that are related
>> to the version numbers of ssh and the other configurations. There is
>> usually a mismatch, when it takes "too long" to start a session,
>> in my experience.
>>
>> I did not look at the specifics you posted.
> 
> both servers/machines run net-misc/openssh-6.6.1_p1 ... re-compiled
> right today.

I've also noticed slowdowns recently, I think it's the new ciphers likes
ecdsa. Try this:

Connect using ssh -vvv and examine the output to find which of the
various ciphers and algorithms are used once connection is achieved. On
the client, add those configuration options for the server to
ssh_config. You should notice a speed up on the next attempt as unused
methods will be skipped

man 5 ssh_config

has all the details



-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Re: ssh rekeying slow ?

[Stefan G. Weichinger]

Am 25.06.2014 21:49, schrieb Alan McKinnon:

> I've also noticed slowdowns recently, I think it's the new ciphers likes
> ecdsa. Try this:
> 
> Connect using ssh -vvv and examine the output to find which of the
> various ciphers and algorithms are used once connection is achieved. On
> the client, add those configuration options for the server to
> ssh_config. You should notice a speed up on the next attempt as unused
> methods will be skipped
> 
> man 5 ssh_config
> 
> has all the details

;-)

thanks, Alan.

Did you already find out what options to set?

Aside from that, I wonder why we as users have to do that and why it
isn't set up "as good as possible" by the coders of openssh.

I will see if I can figure out what to do ...

Stefan

Re: [gentoo-user] Re: ssh rekeying slow ?

[Alan McKinnon]

On 25/06/2014 23:10, Stefan G. Weichinger wrote:
> Am 25.06.2014 21:49, schrieb Alan McKinnon:
> 
>> I've also noticed slowdowns recently, I think it's the new ciphers likes
>> ecdsa. Try this:
>>
>> Connect using ssh -vvv and examine the output to find which of the
>> various ciphers and algorithms are used once connection is achieved. On
>> the client, add those configuration options for the server to
>> ssh_config. You should notice a speed up on the next attempt as unused
>> methods will be skipped
>>
>> man 5 ssh_config
>>
>> has all the details
> 
> ;-)
> 
> thanks, Alan.
> 
> Did you already find out what options to set?

No, only you can do that. You have to run ssh -vvv and eyeball the
output, see what your machines are using. Then add those config settings
to ssh_config

> 
> Aside from that, I wonder why we as users have to do that and why it
> isn't set up "as good as possible" by the coders of openssh.

Because the openssh developers have no idea what you set up and cannot
possibly know. The phrase "as good as possible" has no meaning here as
the options out there in the wild as whatever they happen to be.


> I will see if I can figure out what to do ...

ssh -vvv

then look




-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] Re: ssh rekeying slow ?

[Stefan G. Weichinger]

Am 25.06.2014 23:31, schrieb Alan McKinnon:

> Because the openssh developers have no idea what you set up and cannot
> possibly know. The phrase "as good as possible" has no meaning here as
> the options out there in the wild as whatever they happen to be.

Having users installing their software with the default config isn't
that wild or unpredictable for them, I assume.

anyway

Stefan

Re: [gentoo-user] Re: ssh rekeying slow ?

[Stefan G. Weichinger]

Am 25.06.2014 23:10, schrieb Stefan G. Weichinger:

> I will see if I can figure out what to do ...

To me it looks as if my issue is related to this line in the logs:

Jun 25 23:30:45 mythtv sshd[5387]: pam_systemd(sshd:session): Failed to
create session: Connection timed out

hmm ...

Re: [gentoo-user] Re: ssh rekeying slow ?

[Stefan G. Weichinger]

Am 25.06.2014 23:31, schrieb Stefan G. Weichinger:
> Am 25.06.2014 23:10, schrieb Stefan G. Weichinger:
> 
>> I will see if I can figure out what to do ...
> 
> To me it looks as if my issue is related to this line in the logs:
> 
> Jun 25 23:30:45 mythtv sshd[5387]: pam_systemd(sshd:session): Failed to
> create session: Connection timed out
> 
> hmm ...
> 
yes.

edited /etc/pam.d/system-auth and commented this line (to be disabled):

#-session        optional        pam_systemd.so

Immediate logins now.

Other people on the web face(d) that as well, according to google.

S

Re: [gentoo-user] Re: ssh rekeying slow ?

[Mick]

[-- Attachment #1: Type: Text/Plain, Size: 1649 bytes --]

On Wednesday 25 Jun 2014 22:10:42 Stefan G. Weichinger wrote:
> Am 25.06.2014 21:49, schrieb Alan McKinnon:
> > I've also noticed slowdowns recently, I think it's the new ciphers likes
> > ecdsa. Try this:
> > 
> > Connect using ssh -vvv and examine the output to find which of the
> > various ciphers and algorithms are used once connection is achieved. On
> > the client, add those configuration options for the server to
> > ssh_config. You should notice a speed up on the next attempt as unused
> > methods will be skipped
> > 
> > man 5 ssh_config
> > 
> > has all the details
> 
> ;-)
> 
> thanks, Alan.
> 
> Did you already find out what options to set?
> 
> Aside from that, I wonder why we as users have to do that and why it
> isn't set up "as good as possible" by the coders of openssh.

Because the "as good as possible" datum is being redefined post Snowden.


> I will see if I can figure out what to do ...

The Better Crypto team suggest:

Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-
gcm@openssh.com,aes256-ctr,aes128-ctr

MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-
etm@openssh.com,hmac-sha2-512,hmac-sha2-256,hmac-ripemd160

KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-
sha256,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1

The above may be OTT for ssh connections between machines within a trusted 
LAN.  As has already been mentioned if you choose your favourite crypto and 
strip out all the rest, then the negotiation ought to be faster between modern 
PCs.

-- 
Regards,
Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [gentoo-user] Re: ssh rekeying slow ?

[covici]

James <wireless@tampabay.rr.com> wrote:

> Stefan G. Weichinger <lists <at> xunil.at> writes:
> 
> 
> > When I ssh into a server in my basement, this takes way more time than
> > usual.
> > Does anyone have an idea what I could do to fix that?
> 
> 
> ssh has an ordered array of negotiations between systems that are related
> to the version numbers of ssh and the other configurations. There is
> usually a mismatch, when it takes "too long" to start a session,
> in my experience.
> 
> I did not look at the specifics you posted.


I had a problem like that and solved it by  changine UseDNS no
because it is trying to look for reverse dns pointers.  This is done on
the hosts /etc/ssh/sshd_config .


-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

         John Covici
         covici@ccs.covici.com

Re: [gentoo-user] Re: ssh rekeying slow ?

[Stefan G. Weichinger]

Am 25.06.2014 23:45, schrieb covici@ccs.covici.com:

> I had a problem like that and solved it by  changine UseDNS no
> because it is trying to look for reverse dns pointers.  This is done on
> the hosts /etc/ssh/sshd_config .

Tried/tested a few hours ago. No change.

pam_systemd is (or seems to be) the reason, see my other posting.

Stefan

Re: [gentoo-user] Re: ssh rekeying slow ?

[Stefan G. Weichinger]

Am 26.06.2014 00:20, schrieb Stefan G. Weichinger:

> pam_systemd is (or seems to be) the reason, see my other posting.

maybe it would be also solved by upgrading to the (in terms of gentoo)
unstable version 214 of systemd:

# equery b pam_systemd.so

 * Searching for pam_systemd.so ...
sys-apps/systemd-212-r5 (/lib64/security/pam_systemd.so)

I will check tomorrow or so, late here.

Stefan

Re: [gentoo-user] Re: ssh rekeying slow ?

[covici]

Stefan G. Weichinger <lists@xunil.at> wrote:

> Am 25.06.2014 23:45, schrieb covici@ccs.covici.com:
> 
> > I had a problem like that and solved it by  changine UseDNS no
> > because it is trying to look for reverse dns pointers.  This is done on
> > the hosts /etc/ssh/sshd_config .
> 
> Tried/tested a few hours ago. No change.
> 
> pam_systemd is (or seems to be) the reason, see my other posting.
hmmm, I don't even have that file, I guess I am glad.

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

         John Covici
         covici@ccs.covici.com

Re: [gentoo-user] ssh rekeying slow ?

[Dale]

Stefan G. Weichinger wrote:
> When I ssh into a server in my basement, this takes way more time than
> usual.
>
> I don't have a clue what might have changed ... aside from usual
> updating. I rebuilt and restarted openssh down there without a change.
>
> This is a bit annoying when logging in and using git to pull/push stuff
> from/to there.
>
> Does anyone have an idea what I could do to fix that?
>
> Stefan
>

I ran into a issue like this once a long time ago.  I had something
wrong with my hosts file if I recall correctly.  It never did make sense
as to how it messed things up but after fixing that, it worked fine. 
So, I'd look at the hosts file and see if anything is amiss there.  I'm
pretty sure that is the file that was messed up tho. 

Hope that helps. 

Dale

:-)  :-) 

Re: [gentoo-user] ssh rekeying slow ?

[Stefan G. Weichinger]

Am 26.06.2014 06:07, schrieb Dale:

> I ran into a issue like this once a long time ago.  I had something
> wrong with my hosts file if I recall correctly.  It never did make sense
> as to how it messed things up but after fixing that, it worked fine. 
> So, I'd look at the hosts file and see if anything is amiss there.  I'm
> pretty sure that is the file that was messed up tho. 
> 
> Hope that helps. 

thanks for the suggestion.
I don't see anything strange in the hosts file(s).

For now I keep pam_systemd commented out. Maybe I upgrade to systemd 214
on that server ... ssh-ing to my main workstation works fine with that
line in /etc/pam.d/system-auth ... so maybe it's related to the release
of systemd.

Stefan

Re: [gentoo-user] ssh rekeying slow ?

[Alan McKinnon]

On 26/06/2014 12:45, Stefan G. Weichinger wrote:
> Am 26.06.2014 06:07, schrieb Dale:
> 
>> I ran into a issue like this once a long time ago.  I had something
>> wrong with my hosts file if I recall correctly.  It never did make sense
>> as to how it messed things up but after fixing that, it worked fine. 
>> So, I'd look at the hosts file and see if anything is amiss there.  I'm
>> pretty sure that is the file that was messed up tho. 
>>
>> Hope that helps. 
> 
> thanks for the suggestion.
> I don't see anything strange in the hosts file(s).
> 
> For now I keep pam_systemd commented out. Maybe I upgrade to systemd 214
> on that server ... ssh-ing to my main workstation works fine with that
> line in /etc/pam.d/system-auth ... so maybe it's related to the release
> of systemd.
> 
> Stefan
> 
> 
> 
> 


Is your delay about 30 seconds?
If so, that's almost certain to be related to dns lookups (30 seconds
being the magic timeout that almost everything seems to use)

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] ssh rekeying slow ?

[Stefan G. Weichinger]

Am 26.06.2014 12:54, schrieb Alan McKinnon:

> Is your delay about 30 seconds?

hmm, I think it was shorter ... but around that, yes.

> If so, that's almost certain to be related to dns lookups (30 seconds
> being the magic timeout that almost everything seems to use)

Which hosts might it be then?
The one on the server I want to reach?

reverse-lookup for the contacting clients IP?

I already activated:

UseDNS no


S

Re: [gentoo-user] ssh rekeying slow ?

[Alan McKinnon]

On 26/06/2014 15:12, Stefan G. Weichinger wrote:
> Am 26.06.2014 12:54, schrieb Alan McKinnon:
> 
>> Is your delay about 30 seconds?
> 
> hmm, I think it was shorter ... but around that, yes.
> 
>> If so, that's almost certain to be related to dns lookups (30 seconds
>> being the magic timeout that almost everything seems to use)
> 
> Which hosts might it be then?
> The one on the server I want to reach?
> 
> reverse-lookup for the contacting clients IP?



I don't know, I can't see your logs.



-- 
Alan McKinnon
alan.mckinnon@gmail.com