Page MenuHome GnuPG
Create Task
Maniphest T3768

Decryption of RSA public key encrypted session key packet fails when encrypted session key packet length is not equal to RSA key modulus size
Closed, ResolvedPublic
Actions

Description

Hi,

Here is the use case :

Given an identity "some-key" with a RSA master key of keyid 0x0xBC3979CD0985291D and size 2048 bits
And its encryption subkey of keyid 0x0xBD31C81AB1654B76

When i proceed to GPG encryption of the file "testdgr.txt" with recipient "some-key"

Then i sometimes get pubkey encrypted session key packet having an MPI indicating a 2040 bits data size packet
And the effective RSA encrypted session key packet size is 255 bytes length in the GPG file

Looking at the RFC 4880, just before RSA encryption of the session key, a PKCS#1 V1.5 padding is applied , which should produce a result of length k (k being the RSA key modulus size here).

Following the use case above, when i proceed to file GPG decryption, i observe that the encrypted session key packet submitted for RSA decryption is effectively of 255 bytes length.
This decryption operation always fails when the RSA keys are protected through a Hardware Security Module, and decryption is delegated to this HSM through PKCS#11 API.
Actually, it fails because the submitted packet length is lower thant the RSA key modulus size.

A full sample

testdgr.tgz9 KBDownload
is attached with :

  • testdgr.txt : plain file,
  • testdgr.2040bits.gpg : GPG encrypted file,
  • list-packets.res : result of "gpg --list-packets" command,
  • testdgr.2040bits.gpg.analysis.txt : manual analysis of the public key encrypted session key packet,
  • gen_2040bitlen_pubkeyencpacket.sh : a bash script to generate a 2040 bits data packet,
  • .gnupg : GPG home files used fo this use case (pubring, secring, ...)

Reproduced with GPG 1.4.12, 2.3.0

Sincerely yours,

Damien

Details

Version
1.4.12, 2.3.0

Event Timeline

dams50 created this task.Feb 1 2018, 7:40 PM
dams50 created this object in space S1 Public.
dams50 updated the task description. (Show Details)
dams50 edited projects, added gnupg (gpg14); removed gnupg (gpg23).Feb 1 2018, 7:44 PM
dams50 updated the task description. (Show Details)
• werner added a subscriber: • werner.Feb 2 2018, 2:38 PM

What kind of hardware token?

dams50 added a comment.EditedFeb 2 2018, 3:29 PM

Our HSM is a certified FIPS 140-2, sec level3, hardware module, exposing a PKCS#11 v2.30 spec compliant API.

About RSA related supported mechanism, our HSM documentation indicates :

mechanism functionsencrypt/decryptsign/verify
CKM_RSA_PKCSyesyes
CKM_RSA_PKCS_OAEPyesno
CKM_RSA_PKCS_PSSnoyes
dams50 added a comment.Feb 3 2018, 6:18 PM

Some enlightenments here because i may have not mention some info in the first place :

By "decryption fails", i meant that the PKCS11 module actually returns a CKR_ENCRYPTED_DATA_LEN_RANGE error which, according to PKCS11 spec, means "the input ciphertext data cannot be decrypted because it has an inappropriate length".

Back to our case, "input ciphertext data" here refers to the 255 bytes encrypted session key value extracted from the PGP pubkey session key encrypted packet (see joined file "testdgr.2040bits.gpg.analysis.txt" in "testdgr.tgz" attachment).

Please note that, some other tests have showed that for GPG encrypted files having a session key value size in the range [2041 - 2048], session key is successfully decrypted by the HSM/PKCS11.

Anyway, my point here, rather than the use of an HSM through PKCS11, is :
at GPG encryption, shouldn't the RSA encryption of session key value (that is : pkcs1V1.5 padding of one-octet algorithm id + raw session key + two-octet checksum) always produce a 256 bytes result for a RSA 2048 bits modulus ?

Assuming that, then what happended to the missing 256th byte in the GPG file ?

It's like :

  • the session key value has a significant size of 2040 bit (255 bytes), giving the "07 F8" 2-octets MPI length field,
  • the rsa encryption of this session key value has produces a 256 bytes result,
  • but only 255 bytes are written in the GPG file, making the PKCS11 unhappy when submitting this 255 bytes to RSA PKCS1 decryption
  • and yet, when switching from a PKCS11 module to a local crypto module (like the simple Java JCE API), the decryption succeeds
dams50 added a comment.Feb 5 2018, 11:38 AM

FYI : when submitting a buffer composed of

  • a leading 00 byte,
  • the 255 bytes encrypted session key value

to HSM/PKCS11 for decyption, decrypt returns without any errors, and returned plain session key is the one expected.

Well, thinking about it, MPI is just a big number after all...

dams50 closed this task as Resolved.Feb 5 2018, 4:01 PM
bichhoriya added a subscriber: bichhoriya.May 5 2019, 5:05 PM
This comment was removed by bichhoriya.