OpenSSH public key problem with Solaris 10

[Erich Weiler]

Hi ya'll-

I've got this odd openssh problem with Solaris 10 I was hoping someone
could shed some light on. Not sure if it is a bug... Basically I'm
trying to use pubkeys as an auth method, but am having issues. I can
log in using passwords no problem, but as soon as it notices a matching
public key it closes the connection. I ran the sshd server (on Solaris
10 box) in debug mode and got this output when I tried to log in:

% sshd -d
debug1: sshd version OpenSSH_4.3p2
debug1: private host key: #0 type 0 RSA1
debug1: read PEM private key done: type RSA
debug1: private host key: #1 type 1 RSA
debug1: read PEM private key done: type DSA
debug1: private host key: #2 type 2 DSA
debug1: rexec_argv[0]='/usr/local/openssh.10/sbin/sshd'
debug1: rexec_argv[1]='-d'
debug1: Bind to port 22 on ::.
Server listening on :: port 22.
debug1: Bind to port 22 on 0.0.0.0.
Server listening on 0.0.0.0 port 22.
Generating 768 bit RSA key.
RSA key generation complete.
debug1: fd 6 clearing O_NONBLOCK
debug1: Server will not fork when running in debugging mode.
debug1: rexec start in 6 out 6 newsock 6 pipe -1 sock 11
debug1: inetd sockets after dupping: 4, 4
Connection from 128.114.48.86 port 49490
debug1: Client protocol version 2.0; client software version OpenSSH_3.8.1p1
debug1: match: OpenSSH_3.8.1p1 pat OpenSSH_3.*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-1.99-OpenSSH_4.3
debug1: permanently_set_uid: 22/22
debug1: list_hostkey_types: ssh-rsa,ssh-dss
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: client->server aes128-cbc hmac-md5 none
debug1: kex: server->client aes128-cbc hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST received
debug1: SSH2_MSG_KEX_DH_GEX_GROUP sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_INIT
debug1: SSH2_MSG_KEX_DH_GEX_REPLY sent
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: KEX done
debug1: userauth-request for user weiler service ssh-connection method none
debug1: attempt 0 failures 0
debug1: PAM: initializing for "weiler"
debug1: PAM: setting PAM_RHOST to "banshee.cse.ucsc.edu"
debug1: PAM: setting PAM_TTY to "ssh"
Failed none for weiler from 128.114.48.86 port 49490 ssh2
Failed none for weiler from 128.114.48.86 port 49490 ssh2
debug1: userauth-request for user weiler service ssh-connection method
publickey
debug1: attempt 1 failures 1
debug1: test whether pkalg/pkblob are acceptable
debug1: temporarily_use_uid: 3495/100 (e=0/0)
debug1: trying public key file /cse/tstaff/weiler/.ssh/authorized_keys
debug1: restore_uid: 0/0
debug1: temporarily_use_uid: 3495/100 (e=0/0)
debug1: trying public key file /cse/tstaff/weiler/.ssh/authorized_keys2
debug1: matching key found: file
/cse/tstaff/weiler/.ssh/authorized_keys2, line 2
Found matching RSA key: 4d:c0:33:3b:dd:75:89:bb:d1:36:e7:17:2b:85:34:9c
debug1: restore_uid: 0/0
Postponed publickey for weiler from 128.114.48.86 port 49490 ssh2
debug1: userauth-request for user weiler service ssh-connection method
publickey
debug1: attempt 2 failures 1
debug1: temporarily_use_uid: 3495/100 (e=0/0)
debug1: trying public key file /cse/tstaff/weiler/.ssh/authorized_keys
debug1: restore_uid: 0/0
debug1: temporarily_use_uid: 3495/100 (e=0/0)
debug1: trying public key file /cse/tstaff/weiler/.ssh/authorized_keys2
debug1: matching key found: file
/cse/tstaff/weiler/.ssh/authorized_keys2, line 2
Found matching RSA key: 4d:c0:33:3b:dd:75:89:bb:d1:36:e7:17:2b:85:34:9c
debug1: restore_uid: 0/0
debug1: ssh_rsa_verify: signature correct
debug1: do_pam_account: called
Access denied for user weiler by PAM account configuration
debug1: do_cleanup
debug1: PAM: cleanup
Failed publickey for weiler from 128.114.48.86 port 49490 ssh2
debug1: do_cleanup
debug1: PAM: cleanup
%

Again, If I move my public key out of the way and try to log in with a
password it works fine. Since it mentions my PAM configuration, here's
my /etc/pam.conf file:

login auth requisite pam_authtok_get.so.1
login auth required pam_unix_cred.so.1
login auth sufficient pam_unix_auth.so.1
login auth sufficient pam_krb5.so.1
login auth sufficient pam_ldap.so.1
#
dtsession auth sufficient pam_unix_auth.so.1
dtsession auth sufficient pam_krb5.so.1
dtsession auth sufficient pam_ldap.so.1
#
# rlogin service (explicit because of pam_rhost_auth)
#
rlogin auth sufficient pam_rhosts_auth.so.1
rlogin auth requisite pam_authtok_get.so.1
rlogin auth required pam_dhkeys.so.1
rlogin auth required pam_unix_cred.so.1
rlogin auth required pam_unix_auth.so.1
#
# Kerberized rlogin service
#
krlogin auth required pam_unix_cred.so.1
krlogin auth binding pam_krb5.so.1
krlogin auth required pam_unix_auth.so.1
#
# rsh service (explicit because of pam_rhost_auth,
# and pam_unix_auth for meaningful pam_setcred)
#
rsh auth sufficient pam_rhosts_auth.so.1
rsh auth required pam_unix_cred.so.1
#
# Kerberized rsh service
#
krsh auth required pam_unix_cred.so.1
krsh auth binding pam_krb5.so.1
krsh auth required pam_unix_auth.so.1
#
# Kerberized telnet service
#
ktelnet auth required pam_unix_cred.so.1
ktelnet auth binding pam_krb5.so.1
ktelnet auth required pam_unix_auth.so.1
#
# PPP service (explicit because of pam_dial_auth)
#
ppp auth requisite pam_authtok_get.so.1
ppp auth required pam_dhkeys.so.1
ppp auth required pam_unix_cred.so.1
ppp auth required pam_unix_auth.so.1
ppp auth required pam_dial_auth.so.1
#
# Default definitions for Authentication management
# Used when service name is not explicitly mentioned for authentication
#
other auth requisite pam_authtok_get.so.1
other auth required pam_unix_cred.so.1
other auth sufficient pam_unix_auth.so.1
other auth sufficient pam_krb5.so.1
other auth sufficient pam_ldap.so.1
#
# passwd command (explicit because of a different authentication module)
#
passwd auth sufficient pam_passwd_auth.so.1
passwd auth sufficient pam_ldap.so.1
#
# cron service (explicit because of non-usage of pam_roles.so.1)
#
cron account required pam_unix_account.so.1
#
# Default definition for Account management
# Used when service name is not explicitly mentioned for account management
#
passwd account sufficient pam_unix_account.so.1
passwd account sufficient pam_ldap.so.1
#
other account sufficient pam_unix_account.so.1
other account sufficient pam_ldap.so.1
other account sufficient pam_krb5.so.1
#
# Default definition for Session management
# Used when service name is not explicitly mentioned for session management
#
other session sufficient pam_unix_session.so.1
other session sufficient pam_ldap.so.1
other session sufficient pam_krb5.so.1
#
# Default definition for Password management
# Used when service name is not explicitly mentioned for password management
other password requisite pam_authtok_get.so.1
other password requisite pam_authtok_check.so.1
other password required pam_authtok_store.so.1

Would any of you guys happen to have a clue as to where I'm going wrong?
Thanks a million in advance!

ciao, erich

[Darren Tucker]

Post by Erich Weiler
Hi ya'll-
I've got this odd openssh problem with Solaris 10 I was hoping someone
could shed some light on. Not sure if it is a bug... Basically I'm
trying to use pubkeys as an auth method, but am having issues. I can
log in using passwords no problem, but as soon as it notices a matching
public key it closes the connection. I ran the sshd server (on Solaris

[...]

Post by Erich Weiler
Found matching RSA key: 4d:c0:33:3b:dd:75:89:bb:d1:36:e7:17:2b:85:34:9c
debug1: restore_uid: 0/0
debug1: ssh_rsa_verify: signature correct
debug1: do_pam_account: called
Access denied for user weiler by PAM account configuration

[...]

What's happening is that sshd is successfully authenticating via
public-key.

It then tries to check the account status via PAM which fails, because you
have Kerberos modules in your PAM config but public-key authentication
does not provide the Kerberos credentials required for the module to
perform those checks, and thus it fails.

If you don't use Kerberos then you can comment out the Kerberos account
(and probably session) modules. (You might want to create a "sshd"
service in the PAM config specifically for it.) If you do use Kerberos
then I'm not sure what your options are.

[Erich Weiler]

Arrg. Yup, I need Kerberos to work in this case. Of course it works
when a password is entered, but the public key thing would be very nice.
Annoyingly enough this works under linux (redhat/fedora). I guess
Sun's kerberos PAM module is somewhat lacking in functionality.

How annoying of Sun!

Thanks for the reply in any case.

Post by Darren Tucker

Post by Erich Weiler
Hi ya'll-
I've got this odd openssh problem with Solaris 10 I was hoping someone
could shed some light on. Not sure if it is a bug... Basically I'm
trying to use pubkeys as an auth method, but am having issues. I can
log in using passwords no problem, but as soon as it notices a matching
public key it closes the connection. I ran the sshd server (on Solaris

[...]

Post by Erich Weiler
Found matching RSA key: 4d:c0:33:3b:dd:75:89:bb:d1:36:e7:17:2b:85:34:9c
debug1: restore_uid: 0/0
debug1: ssh_rsa_verify: signature correct
debug1: do_pam_account: called
Access denied for user weiler by PAM account configuration

[...]
What's happening is that sshd is successfully authenticating via
public-key.
It then tries to check the account status via PAM which fails, because you
have Kerberos modules in your PAM config but public-key authentication
does not provide the Kerberos credentials required for the module to
perform those checks, and thus it fails.
If you don't use Kerberos then you can comment out the Kerberos account
(and probably session) modules. (You might want to create a "sshd"
service in the PAM config specifically for it.) If you do use Kerberos
then I'm not sure what your options are.

[Douglas E. Engert]

Post by Erich Weiler
Arrg. Yup, I need Kerberos to work in this case. Of course it works
when a password is entered, but the public key thing would be very nice.
Annoyingly enough this works under linux (redhat/fedora). I guess
Sun's kerberos PAM module is somewhat lacking in functionality.

The Solaris 10 sshd has a nice PAM feature, in that it
will use a different pam service name depending on the auth used.
For example: sshd-password, sshd-kdbint, sshd-pubkey, sshd-gssapi ...
The sshd_config can override these too.

Thus you can skip the pam_krb5 for pubkey.

OpenSSH might want to consider a similiar feature.

Post by Erich Weiler
How annoying of Sun!
Thanks for the reply in any case.

Post by Darren Tucker

Post by Erich Weiler
Hi ya'll-
I've got this odd openssh problem with Solaris 10 I was hoping someone
could shed some light on. Not sure if it is a bug... Basically I'm
trying to use pubkeys as an auth method, but am having issues. I can
log in using passwords no problem, but as soon as it notices a matching
public key it closes the connection. I ran the sshd server (on Solaris

[...]

Post by Erich Weiler
Found matching RSA key: 4d:c0:33:3b:dd:75:89:bb:d1:36:e7:17:2b:85:34:9c
debug1: restore_uid: 0/0
debug1: ssh_rsa_verify: signature correct
debug1: do_pam_account: called
Access denied for user weiler by PAM account configuration

[...]
What's happening is that sshd is successfully authenticating via
public-key.
It then tries to check the account status via PAM which fails, because you
have Kerberos modules in your PAM config but public-key authentication
does not provide the Kerberos credentials required for the module to
perform those checks, and thus it fails.
If you don't use Kerberos then you can comment out the Kerberos account
(and probably session) modules. (You might want to create a "sshd"
service in the PAM config specifically for it.) If you do use Kerberos
then I'm not sure what your options are.

[Erich Weiler]

Upon further investigation I discovered that the root of my problem
actually lies with the fact that Solaris's pam_ldap module does not
allow account information to be read without valid credentials. It does
not consider an ssh key auth to be a valid cred set, but it does
consider a password to be (obviously).

Linux pam_ldap (or PADL pam_ldap) works fine, which is why this setup is
working on my linux boxes.

This is apparently a documented issue and they are working on fixing it.
I'm bugging the Sun engineers about it now. Turns out it has nothing
to do with kerberos. Thanks a million for replying in any case!

-erich

Post by Douglas E. Engert

Post by Erich Weiler
Arrg. Yup, I need Kerberos to work in this case. Of course it works
when a password is entered, but the public key thing would be very
nice. Annoyingly enough this works under linux (redhat/fedora). I
guess Sun's kerberos PAM module is somewhat lacking in functionality.

The Solaris 10 sshd has a nice PAM feature, in that it
will use a different pam service name depending on the auth used.
For example: sshd-password, sshd-kdbint, sshd-pubkey, sshd-gssapi ...
The sshd_config can override these too.
Thus you can skip the pam_krb5 for pubkey.
OpenSSH might want to consider a similiar feature.

Post by Erich Weiler
How annoying of Sun!
Thanks for the reply in any case.

Post by Darren Tucker

Post by Erich Weiler
Hi ya'll-
I've got this odd openssh problem with Solaris 10 I was hoping
someone could shed some light on. Not sure if it is a bug...
Basically I'm trying to use pubkeys as an auth method, but am having
issues. I can log in using passwords no problem, but as soon as it
notices a matching public key it closes the connection. I ran the
sshd server (on Solaris 10 box) in debug mode and got this output

[...]

Post by Erich Weiler
Found matching RSA key: 4d:c0:33:3b:dd:75:89:bb:d1:36:e7:17:2b:85:34:9c
debug1: restore_uid: 0/0
debug1: ssh_rsa_verify: signature correct
debug1: do_pam_account: called
Access denied for user weiler by PAM account configuration

[...]
What's happening is that sshd is successfully authenticating via
public-key.
It then tries to check the account status via PAM which fails, because you
have Kerberos modules in your PAM config but public-key authentication
does not provide the Kerberos credentials required for the module to
perform those checks, and thus it fails.
If you don't use Kerberos then you can comment out the Kerberos account
(and probably session) modules. (You might want to create a "sshd"
service in the PAM config specifically for it.) If you do use Kerberos
then I'm not sure what your options are.

[Darren Tucker]

Post by Douglas E. Engert
The Solaris 10 sshd has a nice PAM feature, in that it
will use a different pam service name depending on the auth used.
For example: sshd-password, sshd-kdbint, sshd-pubkey, sshd-gssapi ...
The sshd_config can override these too.
Thus you can skip the pam_krb5 for pubkey.
OpenSSH might want to consider a similiar feature.

I've seen that mentioned earlier (here or elsewhere) and it's also
been listed in OpenSSH's TODO file for a long time (originally from
Solar Designer). It's not a bad idea, but the catch is that it would
require either another compile or run time button and/or a migration
hassle.

This is because there's no (sane) way to tell which PAM services are
available: pam_get_item(handle, PAM_SERVICE, [...]) will return the
service name you asked for, not the service name that you actually got
(which makes it kinda useless, since you already know what you asked
for).

Thus there is no way to, eg, try "sshd-kbdint" and fall back to "sshd"
if it's not available (in the first instance you'll get the "other"
service but have no way of knowing it).

This is the case on at least Sun and LinuxPAM implementations. You can
confirm this behaviour on other platforms with my PAM test tool:
http://www.zip.com.au/~dtucker/patches/#pamtest

$ sudo ./pam-test-harness -s some-random-service
[...]
pam_start(some-random-service, (NULL), &conv, &pamh) = 0 (Success)
pam_get_item(pamh, PAM_SERVICE, ...) = 0 (Success)
PAM_SERVICE = some-random-service (unchanged)

[Senthil Kumar]

Post by Darren Tucker
I've seen that mentioned earlier (here or elsewhere) and it's also
been listed in OpenSSH's TODO file for a long time (originally from
Solar Designer). It's not a bad idea, but the catch is that it would
require either another compile or run time button and/or a migration
hassle.

Yes, its mentioned here and the previous discussion can be reached at
http://marc.theaimsgroup.com/?l=openssh-unix-dev&m=110844606628771&w=2
I have an old patch, will update it and send it to the community if there
are interest on it.

Thanks,
Senthil Kumar.

[Erich Weiler]

Hi Senthil,

I'm using OpenSSH 4.3p2, if you have a patch that would work on that
version to allow Solaris's pam_ldap to let someone in with public keys
via SSH then I'm all over it. If you do post it, just let me know how
to patch the code and I'll be eternally grateful.

ciao, erich

Post by Senthil Kumar

Post by Darren Tucker
I've seen that mentioned earlier (here or elsewhere) and it's also
been listed in OpenSSH's TODO file for a long time (originally from
Solar Designer). It's not a bad idea, but the catch is that it would
require either another compile or run time button and/or a migration
hassle.

Yes, its mentioned here and the previous discussion can be reached at
http://marc.theaimsgroup.com/?l=openssh-unix-dev&m=110844606628771&w=2
I have an old patch, will update it and send it to the community if
there are interest on it.
Thanks,
Senthil Kumar.

[Douglas E. Engert]

Post by Darren Tucker

Post by Douglas E. Engert
The Solaris 10 sshd has a nice PAM feature, in that it
will use a different pam service name depending on the auth used.
For example: sshd-password, sshd-kdbint, sshd-pubkey, sshd-gssapi ...
The sshd_config can override these too.
Thus you can skip the pam_krb5 for pubkey.
OpenSSH might want to consider a similiar feature.

I've seen that mentioned earlier (here or elsewhere) and it's also
been listed in OpenSSH's TODO file for a long time (originally from
Solar Designer). It's not a bad idea, but the catch is that it would
require either another compile or run time button and/or a migration
hassle.
This is because there's no (sane) way to tell which PAM services are
available: pam_get_item(handle, PAM_SERVICE, [...]) will return the
service name you asked for, not the service name that you actually got
(which makes it kinda useless, since you already know what you asked
for).
Thus there is no way to, eg, try "sshd-kbdint" and fall back to "sshd"
if it's not available (in the first instance you'll get the "other"
service but have no way of knowing it).

Yes that's to bad and sounds like a bug. But even without this change how
do you know now that "other" is not being used? i.e. the admin did not setup
a "sshd" pam entry.


Even Solaris 10 is misleading as the man page for sshd_config
says you can change these from the default of sshd:

PamSvcForNone
PamSvcForPassword
PamSvcForKbdInt
PamSvcForOther

But the man page for sshd says it uses sshd-none, sshd-password, sshd-kdbint,
sshd-pubkey, sshd-hostbased, sshd-gssapi which it appears to do. (I have
only tried the sshd-kbdint and sshd-gssapi.)

Post by Darren Tucker
This is the case on at least Sun and LinuxPAM implementations. You can
http://www.zip.com.au/~dtucker/patches/#pamtest
$ sudo ./pam-test-harness -s some-random-service
[...]
pam_start(some-random-service, (NULL), &conv, &pamh) = 0 (Success)
pam_get_item(pamh, PAM_SERVICE, ...) = 0 (Success)
PAM_SERVICE = some-random-service (unchanged)

So could OpenSSH have the sshd_config options set to sshd? This would then
allow the admin the flexability and responsibility to update both sshd_config
and pam.conf to match.

If falling back to "other" is a problem, then maybe "other" should
always fail or at least log that it is being used when not expected.

Using the bug you discribe above, A pam_other_fail.so called only from "other"
could use pam_get_item, and if did not return "other" it could fail. This would
then force the admin to explicitly setup pam for each service.