Failing to decrypt due to missing MDC
Open, NormalPublic

Description

There is one contact for whom I always fail to decrypt their emails (using gpgol).

So I downloaded the encryptedPart separately (using webmail), and try to decrypt the file. This also failed.

I then turned on debugging in gpgme (level 9), and found that although the decryption claimed to fail, I could read the entire contents of the file in the debug file. ie it had de-crypted, but erroneously said that it could not.

I am reluctant to send you the file, or the debug output without some form of redaction. Is there anything obvious I can send you that would assist? I don't know if it is relevant, but the PGP version of the file is BCPG v1.54

Details

Commits
rO9f7ec6eb2962: Add distinct error for legacy nomdc
Version
Kleopatra 3.0.1-gpg4win-3.0.2
Lloyd created this task.Jan 4 2018, 3:38 PM
werner added a subscriber: werner.Jan 4 2018, 7:13 PM

I guess that the MDC indicated a broken encryption or no MDC was used at all. Can you pleae run the decryption of the file on the commandline? Assuming that thar the file is msg.eml you do:

gpg -d -v -o msg.txt msg.eml

Does gpg print any kind of warning?

Lloyd added a comment.Jan 4 2018, 9:38 PM

Tried that, and it complained that the gpg-agent was not running. Now Kleopatra fails to , constantly trying to load certificate cache. Self test fails on UiServer Connectivity. Was fine up to that point.

Lloyd added a comment.Jan 5 2018, 10:02 AM

Ignore my previous comment - seems that if I'm off our corporate network, I have the issue. Back on the network, Kleopatra is ok, and the gpg command completes. (I suspect that there is a firewall rule required, as the firewall is only enabled off network.)

The gpg command gives the following output:

gpg: using character set 'CP850'
gpg: armor: BEGIN PGP MESSAGE
gpg: armor header: Version: BCPG v1.54
# off=0 ctb=85 tag=1 hlen=3 plen=268
:pubkey enc packet: version 3, algo 1, keyid 355F64A5B288D1D9
        data: [2047 bits]
gpg: public key is 355F64A5B288D1D9
gpg: pinentry launched (18400 qt 1.1.0 - - -)
gpg: public key encrypted data: good DEK
# off=271 ctb=c9 tag=9 hlen=3 plen=1879 new-ctb
:encrypted data packet:
        length: 1879
gpg: encrypted with 2048-bit RSA key, ID 355F64A5B288D1D9, created 2017-06-16
      "Xxxx Yyyy<xxxx.yyy@xxx.xxx.uk>"
gpg: AES256 encrypted data
# off=292 ctb=a3 tag=8 hlen=1 plen=0 indeterminate
:compressed packet: algo=1
# off=294 ctb=cb tag=11 hlen=3 plen=3182 new-ctb
:literal data packet:
        mode b (62), created 1512744656, name="_CONSOLE",
        raw data: 3168 bytes
gpg: Note: sender requested "for-your-eyes-only"
gpg: WARNING: message was not integrity protected
gpg: decryption forced to fail
werner edited projects, added kleopatra, FAQ; removed Bug Report, gpg4win.Jan 5 2018, 11:20 AM
werner added a subscriber: aheinecke.

The last line shows that gpg decided that to return a failure because the message does not use the MDC scheme. Since the introduction of modern algorithms with a _blocklength_ of 128 bit (e.g. AES) gpg always uses the MDC encryption system even if it is not announced by the respective key flags. The reason for theses algorithms are newer than the MDC system and thus we can expect that all applications supporting AES will also support MDC.

For whatever reasons Bouncy Castle did not implement MDC despite that it was already agreed upon in the WG in 2000 and released with PGP 7 and GnuPG 1.0.2 in falls 2000. This was before the RSA patent expired and the export restrictions were lifted.

Not supporting MDC opens the path to somewhat tricky but possible attacks on the ciphertext using the decryption tool as an oracle. For example Enigmails auto-decrypt features makes this possible. This is why we finally decided to let gpg, and GPGME, tell the caller that the decryption failed if MDC was not used with a modern cipher.

I would suggest to report this to Bouncy Castle.

The only thing we can do is to let Kleopatra display a dedicated diagnostic. @aheinecke: What do you think?

Lloyd added a comment.Jan 5 2018, 2:04 PM

Forgive me if I'm completely off the mark here. In no way do I claim to fully understand gpg etc.

I presume that gpg is still following RFC4880. Reading that, it implies that although the decoding of MDC is mandated, the creation is optional. From 5.13:

An implementation MUST support decrypting these packets and
SHOULD prefer generating them to the older Symmetrically Encrypted
Data packet when possible.

Or have I got this wrong?
If it is not mandated to generate the MDC, should gpg allow the user to override the error?

werner added a comment.Jan 5 2018, 4:56 PM

Thanks for asking. We may need to put this into the FAQ, so here is my answer:

A specification is not everything and implementations sometimes even need to ignore requirements. RFC4880 is over 10 years old and had a long drafting process. New research showed that there are real world threats for non authenticated messages and thus gpg now makes the MDC a hard requirement for all algorithms for which we can assume that MDC has to be used. We hesitate to require the MDC also for old algorithms (3DES, CAST5) because a lot of data has been encrypted using them in the first years of OpenPGP.

The option

--ignore-mdc-error
       This option changes a MDC integrity protection failure into a warning.  This can  be
       useful if a message is partially corrupt, but it is necessary to get as much data as
       possible out of the corrupt message.  However, be aware that a MDC protection  fail‐
       ure may also mean that the message was tampered with intentionally by an attacker.

can be used as a workaround but the real solution is to fix compliant-according-to-specs-only implementations. FWIW, we recently received a draft of a paper for a novel attack which can only be avoided with the MDC.

Lloyd added a comment.Jan 5 2018, 5:17 PM

OK. Thank you for that.

I appreciate the dangers. Whilst I try and persuade the sender to deal with the issue at their end, is there anyway to include this option in GpgOL on a temporary basis?

werner triaged this task as Normal priority.Jan 6 2018, 11:47 AM
werner assigned this task to aheinecke.

Andre, I assign this to you. If you don't think that a better warning in Kleopatra is needed, please close the report.

werner renamed this task from Failing to decrypt to Failing to decrypt due to missing MDC.Jan 6 2018, 11:49 AM
In T3714#109045, @Lloyd wrote:

I appreciate the dangers. Whilst I try and persuade the sender to deal with the issue at their end, is there anyway to include this option in GpgOL on a temporary basis?

In the folder %APPDATA%\gnupg create a file named gpg.conf (or edit it if it exists) and put the line "ignore-mdc-error" in there. This should globally set this option and gpgol will also respect this.

I'll keep it open as I want to have better error handling for this.

Lloyd added a comment.Jan 8 2018, 11:25 AM

In the folder %APPDATA%\gnupg create a file named gpg.conf (or edit it if it exists) and put the line "ignore-mdc-error" in there. This should globally set this option and gpgol will also respect this.

Thanks. That did the trick.

As far as I can see GnuPG does not emit appropriate status lines:

./run-decrypt --status --verbose --openpgp /tmp/nomdc.asc                   1
status_cb: ENC_TO 8CC999BDAA45C71F 1 0
status_cb: KEY_CONSIDERED 94A5C9A03C2FE5CA3B095D8E1FDF723CF462B6B1 0
status_cb: KEY_CONSIDERED 94A5C9A03C2FE5CA3B095D8E1FDF723CF462B6B1 0
status_cb: DECRYPTION_KEY E18955A9626007093BDC2ED28CC999BDAA45C71F 94A5C9A03C2FE5CA3B095D8E1FDF723CF462B6B1 u
status_cb: KEY_CONSIDERED 94A5C9A03C2FE5CA3B095D8E1FDF723CF462B6B1 0
status_cb: BEGIN_DECRYPTION 
status_cb: DECRYPTION_COMPLIANCE_MODE 23
status_cb: DECRYPTION_INFO 0 9
status_cb: PLAINTEXT 62 1516277928 
status_cb: PLAINTEXT_LENGTH 4
status_cb: DECRYPTION_FAILED 
status_cb: END_DECRYPTION 
status_cb:  
run-decrypt: decrypt failed: Decryption failed

So the only way for me to handle this in Kleopatra would be:

  • If I get Plaintext even if GPGME reports Decryption Failed -> It's an MDC warning that is not reported?

^ This is so ugly that I won't implement it but rather have a status line NO_MDC from GnuPG and then parse that into a dedicated error code in GPGME.

There can't be an MDC warning if MDC is not used ;-)

DECRYPTION_INFO 0 9

The 0 tells you that there is no MDC. I have not checked whether we make this available in the GPGME API. I will check that anyway in somedays because I am currently adding AEAD support to gpg which will eventually replace MDC.

I have not checked whether we make this available in the GPGME API

Well they are exposed by status_cb :-P. This gives me an excuse to add the status lines in GpgME++

I know it's against the architecture of GPGME but if used with care it adds some flexibility for users of GPGME.

werner added a comment.Jun 1 2018, 1:51 AM

I justed commited some gadgets to gpgme which might be helpful But please show warnings etc before you use that new option.

It's nice. Although for now I've only added a message in the legacy_cipher_nomdc case:

"Data is not integrity protected. Decrypting it could be a security problem. (no MDC)"

GpgOL is not really prepared to parse a mail twice so a message box for decrypt anyway is not super easy. I also don't think that it would make much sense as the user probably would not understand the implications of such a question.

What I find missing here is that I still don't get a proper error when trying to decrypt a message with GPGME that was created with --disable-mdc

werner added a comment.Jun 6 2018, 4:30 PM

BTW, you now need to use --rfc2440 to create a non-mdc message for testing.