OpenSSH key problem with OpenSSL 1.1.0i

[mancha]

Hi.

A user on #openssh on ***@freenode (nickname Gisle) reported problems
with an OpenSSH RSA private key (OpenSSL-style PEM) after upgrading from
OpenSSL 1.1.0g to OpenSSL 1.1.0i.

*** CAVEAT LECTOR: OpenSSH doesn't yet support OpenSSL 1.1.0+ ***

By the time I realized OpenSSH doesn't support this configuration, I had
debugged the problem. Gisle's distrib, Arch Linux, patches OpenSSH to
allow this combination [1].

My findings might help the OpenSSH team prepare its OpenSSL 1.1.0+
support and help distros already building against OpenSSL 1.1.0+ via
distro patches:

As of OpenSSL 1.1.0i, zero-length passwords are allowed [2]. So,
handlers like PEM_do_header() no longer return PEM_R_BAD_PASSWORD_READ
when passed an encrypted PEM and an empty password.

If an OpenSSH key doesn't successfully decrypt with an empty password,
then it doesn't matter. OpenSSH merely gets a "bad decrypt" error from
OpenSSL rather than a "bad password read" but both get mapped to
SSH_ERR_KEY_WRONG_PASSPHRASE.

However, problems arise when a key "successfully" decrypts with an empty
password. In those cases OpenSSH gets a set of ASN.1 decoding errors
that it maps to SSH_ERR_INVALID_FORMAT. I've created an example of such
a key (see attached: poc.pem with password "mancha").

You can trigger the behavior by trying to use poc.pem for public key
authentication or by trying to change its passphrase (i.e. ssh-keygen
-pf poc.pem).

Relevant OpenSSH code can be found in several places, such as:

--------sshconnect2.c--------
1363 snprintf(prompt, sizeof prompt,
1364 "Enter passphrase for key '%.100s': ", id->filename);
1365 for (i = 0; i <= options.number_of_password_prompts; i++) {
1366 if (i == 0)
1367 passphrase = "";
1368 else {
1369 passphrase = read_passphrase(prompt, 0);
1370 if (*passphrase == '\0') {
1371 debug2("no passphrase given, try next key");
1372 free(passphrase);
1373 break;
1374 }
1375 }
1376 switch ((r = sshkey_load_private_type(KEY_UNSPEC, id->filename,
-----------------------------

and

--------ssh-keygen.c---------
1333 /* Try to load the file with empty passphrase. */
1334 r = sshkey_load_private(identity_file, "", &private, &comment);
1335 if (r == SSH_ERR_KEY_WRONG_PASSPHRASE) {
1336 if (identity_passphrase)
1337 old_passphrase = xstrdup(identity_passphrase);
1338 else
1339 old_passphrase =
1340 read_passphrase("Enter old passphrase: ",
1341 RP_ALLOW_STDIN);
1342 r = sshkey_load_private(identity_file, old_passphrase,
1343 &private, &comment);
1344 explicit_bzero(old_passphrase, strlen(old_passphrase));
1345 free(old_passphrase);
1346 if (r != 0)
1347 goto badkey;
1348 } else if (r != 0) {
1349 badkey:
1350 fatal("Failed to load key %s: %s", identity_file, ssh_err(r));
1351 }
-----------------------------

Cheers,

--mancha (https://twitter.com/mancha140)

PS While preparing this report, Gisle pointed me to a forum of Arch
Linux users who seem to be discussing this [3].

[1] https://git.archlinux.org/svntogit/packages.git/tree/trunk/openssl-1.1.0.patch?h=packages/openssh
[2] https://github.com/openssl/openssl/commit/c82c3462267afd
[3] https://bugs.archlinux.org/task/59730