gpgsm: Wrong length when parsing octetstring in constructed encoding + definite length
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	thgoebel
	Jan 20 2022, 8:21 PM

Description

gpgsm: Wrong length when parsing octetstring in constructed encoding + definite length

a.k.a. "Cannot import QuoVadis-issued PKCS#12", a.k.a. "Failed to decrypt encrypted certificate".

TLDR: the length of the data value seems to be calculated to wrongly include header fields when constructed encoding and definite length are used.

Short steps to reproduce:

Get a PKCS#12 archive (certificate + private key) with certain choices of BER encoding.
Run gpgsm --import test.pfx (or try to import it in the Kleopatra UI).

Versions:

$ gpgsm --version
gpgsm (GnuPG) 2.2.19
libgcrypt 1.8.5
libksba 1.3.5-unknown
[...]
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.3 LTS
Release:        20.04
Codename:       focal
$ uname -a
Linux thore 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Expected behaviour:

Certificate is imported.

Actual behaviour:

The certificate import fails:

$ gpgsm --import test.pfx
gpgsm: enabled debug flags: ipc
gpgsm: DBG: chan_4 <- OK Pleased to meet you, process 25716
gpgsm: DBG: connection to agent established
gpgsm: DBG: chan_4 -> RESET
gpgsm: DBG: chan_4 <- OK
gpgsm: DBG: chan_4 -> OPTION ttyname=/dev/pts/6
gpgsm: DBG: chan_4 <- OK
gpgsm: DBG: chan_4 -> OPTION ttytype=xterm-256color
gpgsm: DBG: chan_4 <- OK
gpgsm: DBG: chan_4 -> OPTION display=:0
gpgsm: DBG: chan_4 <- OK
gpgsm: DBG: chan_4 -> OPTION xauthority=/tmp/xauth-1000-_0
gpgsm: DBG: chan_4 <- OK
gpgsm: DBG: chan_4 -> OPTION putenv=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
gpgsm: DBG: chan_4 <- OK
gpgsm: DBG: chan_4 -> OPTION lc-ctype=en_US.UTF-8
gpgsm: DBG: chan_4 <- OK
gpgsm: DBG: chan_4 -> OPTION lc-messages=en_US.UTF-8
gpgsm: DBG: chan_4 <- OK
gpgsm: DBG: chan_4 -> GETINFO version
gpgsm: DBG: chan_4 <- D 2.2.19
gpgsm: DBG: chan_4 <- OK
gpgsm: DBG: chan_4 -> OPTION allow-pinentry-notify
gpgsm: DBG: chan_4 <- OK
gpgsm: DBG: chan_4 -> GET_PASSPHRASE --data -- X X X Please+enter+the+passphrase+to+unprotect+the+PKCS#12+object.
gpgsm: DBG: chan_4 <- INQUIRE PINENTRY_LAUNCHED 25718 gnome3 1.1.0 /dev/pts/6 xterm-256color :0
gpgsm: DBG: chan_4 -> END
gpgsm: DBG: chan_4 <- D test
gpgsm: DBG: chan_4 <- OK
gpgsm: 2376 bytes of 3DES encrypted text
gpgsm: 1632 bytes of RC2 encrypted text
gpgsm: decryption failed; trying charset 'ISO-8859-1'
gpgsm: decryption failed; trying charset 'ISO-8859-15'
gpgsm: decryption failed; trying charset 'ISO-8859-2'
gpgsm: decryption failed; trying charset 'ISO-8859-3'
gpgsm: decryption failed; trying charset 'ISO-8859-4'
gpgsm: decryption failed; trying charset 'ISO-8859-5'
gpgsm: decryption failed; trying charset 'ISO-8859-6'
gpgsm: decryption failed; trying charset 'ISO-8859-7'
gpgsm: decryption failed; trying charset 'ISO-8859-8'
gpgsm: decryption failed; trying charset 'ISO-8859-9'
gpgsm: decryption failed; trying charset 'KOI8-R'
gpgsm: decryption failed; trying charset 'IBM437'
gpgsm: decryption failed; trying charset 'IBM850'
gpgsm: decryption failed; trying charset 'EUC-JP'
gpgsm: decryption failed; trying charset 'BIG5'
gpgsm: encryptedData error at "outer.outer.seq", offset 2
gpgsm: possibly bad passphrase given
gpgsm: error at "bag.encryptedData", offset 2585
gpgsm: error parsing or decrypting the PKCS#12 file
gpgsm: total number processed: 0
secmem usage: 2400/16384 bytes in 1 blocks

Yes, there is a cleartext password in this log, and it is "test".

Long steps to reproduce

I got my hands on such a specifically-formatted archive through ETH Zurich's
PKI portal, which in turn obtains them from QuoVadis.
This is an issue, because it makes it rather hard to obtain a test archive that
contains test keys (we would need to jump through hoops at QuoVadis support).
I also didn't find an easy way to reproduce the ASN1 structure of this archive
with openssl.

Therefore throughout this report I use a test.pfx created as follows:

Disassemble: der2ascii -i quovadis-production.pfx -o test.ascii (from [0]).
der2ascii -i openssl-self-signed.pfx -o self.ascii
Manually fiddle with the hex values in the OCTET_STRINGs and copy the values from self.ascii over to test.ascii.
Reassemble: ascii2der -i test.ascii -o test.pfx

This way, we preserve the ASN1 structure of the PFX QuoVadis created.
But now it contains non-confidential, self-signed test data, protected
with the password "test".

Note that since we use ascii2der all the length fields in the BER are set correctly.
The final MAC is NOT correct.
The hashes that integrity-protect the key and cert bags are correct.
You can see this in the log (gpgsm: 2376 bytes of 3DES encrypted text, i.e. the
keybag can be decrypted), or by running pk12util -l test.pfx (from [1]).
So for the purpose of reproducing the problem, this should suffice.

If you need a better pfx file to reproduce this, I can *privately* provide my revoked,
but-still-production pfx file.

Digging into the problem

Looking into the structure of the test.pfx:

$ openssl asn1parse -i -inform der -in test.pfx -strparse 30
    0:d=0  hl=4 l=4243 cons: SEQUENCE          
    4:d=1  hl=4 l=2532 cons:  SEQUENCE          
    8:d=2  hl=2 l=   9 prim:   OBJECT            :pkcs7-data
   19:d=2  hl=4 l=2517 cons:   cont [ 0 ]        
   23:d=3  hl=4 l=2513 prim:    OCTET STRING      [HEX DUMP]:308209CD30....6500720074
 2540:d=1  hl=4 l=1703 cons:  SEQUENCE          
 2544:d=2  hl=2 l=   9 prim:   OBJECT            :pkcs7-encryptedData
 2555:d=2  hl=4 l=1688 cons:   cont [ 0 ]        
 2559:d=3  hl=4 l=1684 cons:    SEQUENCE          
 2563:d=4  hl=2 l=   1 prim:     INTEGER           :00
 2566:d=4  hl=4 l=1677 cons:     SEQUENCE          
 2570:d=5  hl=2 l=   9 prim:      OBJECT            :pkcs7-data
 2581:d=5  hl=2 l=  28 cons:      SEQUENCE          
 2583:d=6  hl=2 l=  10 prim:       OBJECT            :pbeWithSHA1And40BitRC2-CBC
 2595:d=6  hl=2 l=  14 cons:       SEQUENCE          
 2597:d=7  hl=2 l=   8 prim:        OCTET STRING      [HEX DUMP]:5024C63D9644A885
 2607:d=7  hl=2 l=   2 prim:        INTEGER           :0800
 2611:d=5  hl=4 l=1632 cons:      cont [ 0 ]        
 2615:d=6  hl=4 l= 500 prim:       OCTET STRING      [HEX DUMP]:B243AD2C37....1C9890FD3F
 3119:d=6  hl=4 l=1124 prim:       OCTET STRING      [HEX DUMP]:F76CD48CE0....3854B5AC26

If you step down at offset 30+27=57, you will see the 2376 bytes of the keybag that gpgsm
successfully decrypted.

At offset 30+2615 is the data of the certificate. As seen in the log above, gpgsm
tries to decrypt "1632 bytes of RC2 encrypted text".

This is wrong, the cert bag is only 1624 bytes long.

Inspecting the original openssl-self-signed.pfx confirms this (note that here
openssl swapped the order of the cert and key bags):

$ openssl asn1parse -i -inform der -in openssl-self-signed.pfx -strparse 30
    0:d=0  hl=4 l=4200 cons: SEQUENCE          
    4:d=1  hl=4 l=1695 cons:  SEQUENCE          
    8:d=2  hl=2 l=   9 prim:   OBJECT            :pkcs7-encryptedData
   19:d=2  hl=4 l=1680 cons:   cont [ 0 ]        
   23:d=3  hl=4 l=1676 cons:    SEQUENCE          
   27:d=4  hl=2 l=   1 prim:     INTEGER           :00
   30:d=4  hl=4 l=1669 cons:     SEQUENCE          
   34:d=5  hl=2 l=   9 prim:      OBJECT            :pkcs7-data
   45:d=5  hl=2 l=  28 cons:      SEQUENCE          
   47:d=6  hl=2 l=  10 prim:       OBJECT            :pbeWithSHA1And40BitRC2-CBC
   59:d=6  hl=2 l=  14 cons:       SEQUENCE          
   61:d=7  hl=2 l=   8 prim:        OCTET STRING      [HEX DUMP]:5024C63D9644A885
   71:d=7  hl=2 l=   2 prim:        INTEGER           :0800
   75:d=5  hl=4 l=1624 prim:      cont [ 0 ]        
 1703:d=1  hl=4 l=2497 cons:  SEQUENCE          
 1707:d=2  hl=2 l=   9 prim:   OBJECT            :pkcs7-data
 1718:d=2  hl=4 l=2482 cons:   cont [ 0 ]        
 1722:d=3  hl=4 l=2478 prim:    OCTET STRING      [HEX DUMP]:[omitted]

The extra 8 bytes seem to come from the two identifier and length octets that come as
part of the constructed encoding with definite length.

Others can parse this correctly:

pk12util [1] from Mozilla's NSS can read it (For my QuoVadis production PFX there is no MAC warning.):

$ sudo apt install libnss3-tools # on Ubuntu 20.04 it installs 2:3.49.1-1ubuntu1.6
[...]
$ pk12util -l test.pfx 
Enter password for PKCS12 file:     # test
pk12util: PKCS12 decode not verified: SEC_ERROR_PKCS12_INVALID_MAC: Unable to import. Invalid MAC. Incorrect password or corrupt file.
[...PFX content...]

openssl can read it too:

$ openssl version
OpenSSL 1.1.1f  31 Mar 2020
$ openssl pkcs12 -in test.pfx
Enter Import Password:     # test
[...PFX content...]

I attached the openssl-self-signed.pfx that I generated myself, as well as the
test.pfx that contains the cert and key from the openssl-self-signed.pfx
but mapped onto the ASN1 structure as QuoVadis created it for my ETH PFX archive.

test.pfx4 KBDownload

openssl-self-signed.pfx4 KBDownload

Let me know if you need more information.

[0] https://github.com/google/der-ascii
[1] https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/tools/NSS_Tools_pk12util

Details

External Link: https://bugs.kde.org/show_bug.cgi?id=448723
Version: 2.2.19

Revisions and Commits

rG GnuPG
	rG5f694dc0be99 sm: Adding missing stuff to the PKCS#12 parser rewrite.
	rG0c7dffe99d3f sm: Fix parsing encrypted data.
	rG6c50834c0905 sm: Fix parsing encrypted data.

Related Objects

Mentioned In: T7213: PKCS #12 import fails on broken P12 files which MS accepts
rGa6dad932f429: sm: Complete rewrite of the PKCS#12 parser
rG5f694dc0be99: sm: Adding missing stuff to the PKCS#12 parser rewrite.
T5949: Release GnuPG 2.2.36
T5928: Release GnuPG 2.2.35
T5743: Release GnuPG 2.3.5

Event Timeline

thgoebel created this task.Jan 20 2022, 8:21 PM

thgoebel created this object in space S1 Public.

Please try again with a recent version of GnuPG (2.2.33 or 2.3.4) and libksba (1.6.0) and reopen this bug if the problem persists.

Sorry about that! Maybe consider adding a line "Check that the issue persists in the latest version" to the "How to file a bug". I was going through this list, so it would have caught me.

Building from git with tag 2.3.4 checked out, I can still reproduce the issue:

$ gpgsm --version
gpgsm (GnuPG) 2.3.4
libgcrypt 1.9.4-beta212
libksba 1.6.1-beta9
Copyright (C) 2021 Free Software Foundation, Inc.
License GNU GPL-3.0-or-later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Home: /home/test/.gnupg
Supported algorithms:
Cipher: 3DES, AES128, AES192, AES256, SERPENT128, SERPENT192, SERPENT256, SEED, CAMELLIA128, CAMELLIA192, CAMELLIA256
Pubkey: RSA, ECC, ECC
Hash: MD5, SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224, WHIRLPOOL

$ gpgsm --import test.pfx 
gpgsm: WARNING: server 'gpg-agent' is older than us (2.2.27 < 2.3.4)
gpgsm: Note: Outdated servers may lack important security fixes.
gpgsm: Note: Use the command "gpgconf --kill all" to restart them.
gpgsm: decryption failed; trying charset 'ISO-8859-1'
gpgsm: decryption failed; trying charset 'ISO-8859-15'
gpgsm: decryption failed; trying charset 'ISO-8859-2'
gpgsm: decryption failed; trying charset 'ISO-8859-3'
gpgsm: decryption failed; trying charset 'ISO-8859-4'
gpgsm: decryption failed; trying charset 'ISO-8859-5'
gpgsm: decryption failed; trying charset 'ISO-8859-6'
gpgsm: decryption failed; trying charset 'ISO-8859-7'
gpgsm: decryption failed; trying charset 'ISO-8859-8'
gpgsm: decryption failed; trying charset 'ISO-8859-9'
gpgsm: decryption failed; trying charset 'KOI8-R'
gpgsm: decryption failed; trying charset 'IBM437'
gpgsm: decryption failed; trying charset 'IBM850'
gpgsm: decryption failed; trying charset 'EUC-JP'
gpgsm: decryption failed; trying charset 'BIG5'
gpgsm: encryptedData error at "outer.outer.seq", offset 2
gpgsm: possibly bad passphrase given
gpgsm: error at "bag.encryptedData", offset 2585
gpgsm: error parsing or decrypting the PKCS#12 file
gpgsm: total number processed: 0

thgoebel reopened this task as Open.Jan 21 2022, 4:11 PM

• werner triaged this task as Normal priority.Jan 21 2022, 9:42 PM

• werner added projects: gnupg (gpg22), S/MIME.

matteocorti added a subscriber: matteocorti.Jan 24 2022, 10:35 AM

• gniibe claimed this task.Mar 7 2022, 11:25 AM

• gniibe updated the task description. (Show Details)Mar 8 2022, 1:52 AM

I located the cause; Current implementation cannot parse the data like:

2611:d=5  hl=4 l=1632 cons:      cont [ 0 ]        
2615:d=6  hl=4 l= 500 prim:       OCTET STRING
3119:d=6  hl=4 l=1124 prim:       OCTET STRING

Here is a patch to show the place in question:

diff --git a/sm/minip12.c b/sm/minip12.c
index 820e0d6b0..18d6fa0d2 100644
--- a/sm/minip12.c
+++ b/sm/minip12.c
@@ -362,7 +362,7 @@ cram_octet_string (const unsigned char *input, size_t *length,
   if (!output)
     goto bailout;
 
-  for (;;)
+  while (n)
     {
       if (parse_tag (&s, &n, &ti))
         goto bailout;
@@ -933,6 +933,19 @@ parse_bag_encrypted_data (const unsigned char *buffer, size_t length,
       r_consumed = NULL; /* Ugly hack to not update that value any further. */
       ti.length = n;
     }
+  else if (ti.class == CLASS_CONTEXT && ti.tag == 0 && ti.is_constructed)
+    {
+      where = "octets-rc2or3des-ciphertext";
+      n = ti.length;
+      cram_buffer = cram_octet_string ( p, &n, &consumed);
+      if (!cram_buffer)
+        goto bailout;
+      p = p_start = cram_buffer;
+      if (r_consumed)
+        *r_consumed = consumed;
+      r_consumed = NULL; /* Ugly hack to not update that value any further. */
+      ti.length = n;
+    }
   else if (ti.class == CLASS_CONTEXT && ti.tag == 0 && ti.length )
     ;
   else