S/MIME - Encrypt Option produces broken Mails in Sent Folder and leads to crash of GpgOL
Closed, ResolvedPublic

Description

Using GpgOL from Gpg4win 3.0.0:
An S/MIME encrypted email that has been successfully send, cannot be opened in the send folder.

Originally this report was broader and about:
Several Users report, that they can't decrypt mails that are encrypted and signed via GpgOL, or that encrypted and signed files via GpgEX are corrupt.

This may also cause issue in T3441.

  • Broken Mails - 1
  • Broken Files - 1 2

The Logs we received until now to analyze the issue further:

I recieved the Log File of a user which may helps analyzing this problem further

This doesn't seem just to affect E-Mails but also File-Encryption.

We had several reports in the last couple of days about Problems with file encryption. However, the Person who sent me the log about this issue with Mails, had also issues with encrypting Files via S/MIME.

Signieren => ok

Verschlüsseln, egal ob nur an sich selbst oder zusätzlich an einen Fremden
=> ok

Signieren + Verschlüsseln => Resultierende Datei ist viel zu klein, ca 3 Kb,
scheint nur der Verschlüsselungsheader enthalten, nicht aber die eigentliche
Datei. Beim Entschlüssen und Prüfen wird eine leere Datei (0 Byte) mit
korrektem Namen angelegt, die wohl eigentlich die ehemals zu
verschlüsselnden Daten enthalten sollte.

Weiterhin ist aufgefallen:
Importiert man einen eigenen SMime-Schlüssel (pub/sec), ist die Ausgabe von
gpgsm -K ok.

Importiert man danach zusätzlich nur einen fremden Pub-Key, tauchen bei der
Ausgabe von gpgsm -K Meldungen mit
gpgsm: DBG: chan_0x000000d8 <- ERR 67108881 Kein geheimer Schl³ssel <GPG Age
auf!?


Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. Alle Rechte vorbehalten.

C:\Users\CLEANED>gpgsm -K
gpgsm: enabled debug flags: x509 mpi crypto memory cache memstat hashing ipc
gpgsm: DBG: get_keygrip for public key
gpgsm: DBG: keygrip= CF CA 58 44 82 22 EC AA F7 7E EF 8C C4 5F 0D 6D B4 E4 1

gpgsm: DBG: chan_0x000000d8 <- OK Pleased to meet you
gpgsm: DBG: connection to agent established
gpgsm: DBG: chan_0x000000d8 -> RESET
gpgsm: DBG: chan_0x000000d8 <- OK
gpgsm: DBG: chan_0x000000d8 -> GETINFO version
gpgsm: DBG: chan_0x000000d8 <- D 2.2.1
gpgsm: DBG: chan_0x000000d8 <- OK
gpgsm: DBG: chan_0x000000d8 -> OPTION allow-pinentry-notify
gpgsm: DBG: chan_0x000000d8 <- OK
gpgsm: DBG: chan_0x000000d8 -> HAVEKEY CFCA58448222ECAAF77EEF8CC45F0D6DB4E41
gpgsm: DBG: chan_0x000000d8 <- ERR 67108881 Kein geheimer Schl³ssel <GPG Age
gpgsm: DBG: get_keygrip for public key
gpgsm: DBG: keygrip= 02 07 A2 0D 02 EF 3B C6 8F F8 5A CF 46 6C 51 04 69 D8 F

gpgsm: DBG: chan_0x000000d8 -> HAVEKEY 0207A20D02EF3BC68FF85ACF466C510469D8F
gpgsm: DBG: chan_0x000000d8 <- ERR 67108881 Kein geheimer Schl³ssel <GPG Age
gpgsm: DBG: get_keygrip for public key
gpgsm: DBG: keygrip= 07 58 5C 31 C9 A6 F4 F7 F3 38 6C 7B 2E ED 06 F2 70 94 D

gpgsm: DBG: chan_0x000000d8 -> HAVEKEY 07585C31C9A6F4F7F3386C7B2EED06F27094D
gpgsm: DBG: chan_0x000000d8 <- OK

werner added a subscriber: werner.Oct 10 2017, 8:01 AM

The output from gpgsm -K in the last quote is perfectly okay. -K works by iterating over all public keys and checking for each public key whether the private key part is also available. If the private key is not available gpg-agent returns an error.

The log file shows that gpgex (or explorer) crashes.

werner removed a subscriber: werner.
JochenSaalfeld added a subscriber: tstreibl.

There are more Logfiles:

I have both types of certificates stored in kleopatra; S/MIME from StartCOM and OpenPGP created by Kleopatra.

In the system settings outlook is not configured to sign with S/MIME. I don't know where to configure GPGOL to make the right choice between the two signatures. I would expect GPGOL to pop up a message box when sending the email if certificate situation is unclear - at least not to crash.

Please keep in mind that I have this ribbon buck (see T3441), too.

Please find attached log files from keopatra and gpgol.

So far we could recreate the following issues:

  1. Mails encrypted with S/MIME are stored with "No Data" in the sent EMail folder, but arrive properly at the recipients (you will recieve a readable copy, if you add yourself to the list of recipients). This Issue breaks the GpgOL Plugin after some time which is leading to the described Problem.
  1. Files that are Signed and Encrypted to a S/MIME Certificate is broken. When you select a file and encrypt and sign it to a recipient, only a detached signature will be created and the Encrpyted file is missing. (Very similar to Issue 1, but file based).
  1. Detached S/MIME Signatures can not be handled via Kleopatra/GpgEx. When having a detached signature, Kleopatra tries to decrypt it and overwrites the original file with a 0-Byte file. No verfication of the signature is done.

Since it is very likely that the Issues with Encrypting and Signing Files correlates with the EMail (and the followed breaking of GpgOL) issue, we try to get Kleopatra fixed first and then investigate if the rest of the issues, which are similar are then fixed as well.

  1. Files that are Signed and Encrypted to a S/MIME Certificate is broken. When you select a file and encrypt and sign it to a recipient, only a detached signature will be created and the Encrpyted file is missing. (Very similar to Issue 1, but file based).

https://phabricator.kde.org/D8368

  1. Detached S/MIME Signatures can not be handled via Kleopatra/GpgEx. When having a detached signature, Kleopatra tries to decrypt it and overwrites the original file with a 0-Byte file. No verfication of the signature is done.

https://phabricator.kde.org/D8376

But the changes do not fix the Issue, that S/MIME encrypted mails are correctly stored in the Mail sent folder.

  1. Mails encrypted with S/MIME are stored with "No Data" in the sent EMail folder, but arrive properly at the recipients (you will recieve a readable copy, if you add yourself to the list of recipients). This Issue breaks the GpgOL Plugin after some time which is leading to the described Problem.

I installed gpg4win 2.3.4 and tried if this error persisted already in older Versions of Gpg4win. I couldn't reproduce it with an older Version, because S/MIME Encryption was done via NO-MIME (aka PGP/INLINE). But I was able to decrypt messages that were S/MIME encrypted with gpg4win 3.0.0 successfully that were put in the IMAP "Sent" Folder. The conclusion is, that messages in the sent folder have the correct content but aren't decrypted correctly.

Since this is a bug that is related to two different parts of the gpg4win package, this bug now only cares about the GpgOL Issue, that GpgOL crashes and cant decrypt messages from the sent folder that are encrypted with S/MIME. All File Based Issues are belonging to Kleopatra are documentet in the KDE Phabricator (https://phabricator.kde.org/T7310).

JochenSaalfeld renamed this task from S/MIME - Encrypt and Sign Option produces broken Files and Mails to S/MIME - Encrypt Option produces broken Mails in Sent Folder and leads to crash of GpgOL.Oct 24 2017, 11:14 AM
JochenSaalfeld removed projects: kleopatra, gpgex.
JochenSaalfeld added subscribers: bernhard, emanuel.

I tried to melt it down to a specific beta release with which it isn't possible anymore to decrypt send emails.

The latest beta that worked was gpg4win3.0.0-beta187 (with gpgol1.4.0) from 07.06.16 (12:33).

The next beta with which it didn't work anymore was gpg4win3.0.0-beta194(with gpgol2.0.0-beta172) from 15.11.16 (16:42).

So the introduction of this behavior boils down to that time window.

gpg4win3.0.0-beta187 contains gpgol-1.4.0, so the last official stable release before version 2.0.0. It should be identical to 1.4.0 tip from git.

gpg4win3.0.0-beta194 contains gpgol-beta172 which was created on 2016-11-15 15:19 (The earliest published beta is just a day younger, so not that big difference). That Version of gpgol is based on this commit. Since there was no release between 1.4 and 2.0 it has to be somewhere between the 172 commits between the 1.4.0 tip and that shipped version. That Version was the first version that was published with a gpgol 2.0.0 beta. In that change there are 164 files changed, 11596 insertions(+) and 4534 deletions(-). It may take some time to get through this.

bernhard updated the task description. (Show Details)Oct 25 2017, 2:40 PM
bernhard claimed this task.Oct 25 2017, 2:43 PM

This week I'm trying to make progress with this issue.

Yesterday I could reproduce that emails in the "send" folder cannot be decrypted anymore.

Used S/MIME and encryption only. The sender could decrypt.
Gpg4win 3.0.0 DE on Window 10 DE with Outlook 2016.

Still have to see the crash part.
I've got a crash after turning S/MIME one, but not when trying to decrypt the "send" folder message.

Comparing the gpgol.log files in the case of OpenPGP decryption (successful) and S/MIME decryption in send folder (failing).

-> Idea: decryption is tried with the wrong protocol (OpenPGP, where it should be S/MIME).

Next steps:

  1. compare with successful S/MIME decryption.
  2. build gpgol

method

A command like the following helps to compare to gpgol.logs by cutting away the time and the process uid number:

cut --delimiter=/ --fields=3--d/  gpgol-20171026-3-openpgp-decr.log >cut/gpgol-20171026-3-openpgp-decr.log

cut/gpgol-20171026-3-openpgp-decr.log

application-events.cpp:Invoke: ItemLoad event. Getting object.
application-events.cpp:Invoke: Creating mail object for item: 00000222A0C9C3B0
mapihelp.cpp:mapi_change_message_class: have override message class
mapihelp.cpp:mapi_change_message_class: checking message class `IPM.Note.GpgOL.MultipartEncrypted'
mapihelp.cpp:mapi_change_message_class: saving old message class
mapihelp.cpp:mapi_change_message_class: setting message class to `IPM.Note.GpgOL.MultipartEncrypted'
mapihelp.cpp:mapi_get_message_type: have override message class
mapihelp.cpp:mapi_create_attach_table: message has 1 attachments
mapihelp.cpp:mapi_create_attach_table: attachment info:
    41093 mt=3 fname=`gpgolXXX.dat' ct=`multipart/signed' ct_parms=`(null)'

mail.cpp:do_parsing: preparing the parser for: 000002229FAAFF70
parsecontroller.cpp:parse:000002229FAD8DE0 decrypt: 1 verify: 0 with protocol: OpenPGP sender: test1@example.com

parsecontroller.cpp:parse:000002229FAD8DE0 decrypt / verify done.
parsecontroller.cpp:parse:000002229FAD8DE0: decrypt err: 0 verify err: 0
parsecontroller.cpp:parse:348: tracepoint
Decrypt / Verify result: GpgME::DecryptionResult(
 error:                GpgME::Error(0 (Success))
 fileName:             <null>
 unsupportedAlgorithm: <null>
 isWrongKeyUsage:      0
 isDeVs                1
 recipients:

cut/gpgol-20171025-smime-send.log

application-events.cpp:Invoke: ItemLoad event. Getting object.
application-events.cpp:Invoke: Creating mail object for item: 000001D8601E0820
mapihelp.cpp:mapi_change_message_class: have override message class
mapihelp.cpp:mapi_change_message_class: checking message class `IPM.Note.GpgOL.OpaqueEncrypted'
mapihelp.cpp:mapi_change_message_class: saving old message class
mapihelp.cpp:mapi_change_message_class: setting message class to `IPM.Note.GpgOL.OpaqueEncrypted'
mapihelp.cpp:mapi_get_message_type: have override message class
mapihelp.cpp:mapi_create_attach_table: message has 1 attachments
mapihelp.cpp:mapi_create_attach_table: attachment info:
    40677 mt=3 fname=`gpgolXXX.dat' ct=`multipart/signed' ct_parms=`(null)'

mail.cpp:do_parsing: preparing the parser for: 000001D85A0FDFE0
parsecontroller.cpp:parse:000001D85A0A8380 decrypt: 1 verify: 0 with protocol: OpenPGP sender: test1@example.com
parsecontroller.cpp:parse:000001D85A0A8380 decrypt / verify done.
parsecontroller.cpp:parse:000001D85A0A8380: decrypt err: 58 verify err: 58
parsecontroller.cpp:parse:348: tracepoint
Decrypt / Verify result: GpgME::DecryptionResult(
 error:                GpgME::Error(117440570 (No data))
 fileName:             <null>
 unsupportedAlgorithm: <null>
 isWrongKeyUsage:      0
 isDeVs                0
 recipients:

When receiving an S/MIME mail that is encrypted, the successful log looks like:

gpgol-20171030-2-smime-decr.log

11:34:30/3120/application-events.cpp:Invoke: ItemLoad event. Getting object.
11:34:30/3120/application-events.cpp:Invoke: Creating mail object for item: 0000018F081A6F80
11:34:30/3120/mapihelp.cpp:mapi_change_message_class: checking message class `IPM.Note.GpgOL.OpaqueEncrypted'
11:34:31/3120/mapihelp.cpp:mapi_create_attach_table: message has 1 attachments
11:34:31/3120/mapihelp.cpp:mapi_create_attach_table: attachment info:
11:34:31/3120/»·41157 mt=1 fname=`smime.p7m' ct=`application/pkcs7-mime' ct_parms=`(null)'

1:34:31/6592/mail.cpp:do_parsing: preparing the parser for: 0000018F065F76C0
11:34:31/6592/parsecontroller.cpp:parse:0000018F065C9C70 decrypt: 1 verify: 0 with protocol: CMS sender: bernhard@intevation.de
11:34:31/6592/mimedataprovider.cpp:t2body: Collecting text body.
11:34:31/6592/parsecontroller.cpp:parse:0000018F065C9C70 decrypt / verify done.
11:34:31/6592/parsecontroller.cpp:parse:0000018F065C9C70: decrypt err: 0 verify err: 0
11:34:31/6592/parsecontroller.cpp:parse:348: tracepoint
11:34:31/6592/Decrypt / Verify result: GpgME::DecryptionResult(
 error:                GpgME::Error(0 (Success))
 fileName:             <null>
 unsupportedAlgorithm: <null>
 isWrongKeyUsage:      0
 isDeVs                1
 recipients:
bernhard reassigned this task from bernhard to aheinecke.Nov 10 2017, 3:11 PM
aheinecke triaged this task as Unbreak Now! priority.Nov 10 2017, 4:07 PM

This indeed is a mixup of the protocol detection and likely a regression from a fix for exchange support. (On Exchange emails from exchange to exchange look the same as sent mails as both don't go through the MIME conversion)

Working on it.

aheinecke changed the task status from Open to Testing.Nov 10 2017, 6:04 PM
aheinecke reassigned this task from aheinecke to JochenSaalfeld.
aheinecke added a subscriber: aheinecke.
  1. Mails encrypted with S/MIME are stored with "No Data" in the sent EMail folder, but arrive properly at the recipients (you will recieve a readable copy, if you add yourself to the list of recipients). This Issue breaks the GpgOL Plugin after some time which is leading to the described Problem.

Fixed with 474cc15d8e331c9def298dbbfe3b99e6c8cf8035

Should we close this now as we have tasks for the other issues?

@aheinecke Regarding closing: I'd say that we should have a test on this one and then close it for only the refocussed "send-folder problem".
Can you provide an updated gpgol.dll drop in replacement?
Some of the users in the forum may be willing to test as well.

A new binary for GpgOL can be found under: http://files.gpg4win.org/Beta/gpgol/2.0.2-beta8/ or for http://files.gpg4win.org/Beta/gpgol/2.0.2-beta8_x64/

Shutdown Outlook and replace your gpgol.dll in the gpg4win install folder to update to the beta version.

Hi,

tested your new .dll. Created a new email. Choosed "sign". Pasted an email adress from outlook address book into the "an" field. Outlook crashs. Took me 2 seconds to test. What the hell are you testing?

Starting Outlook still brings up the "

Starting Outlook still bring up the "Fehler in der Benutzeroberfläche von XML von "GpgOL .....Unbekannte Office.Steuerelemente-ID: TabComposerTool" Message Box.

Sorry for my inpatience...but it's a little bit hard to understand why the above, very simple test procedure obviously isn't reproducible on your systems.

Another unsolved bug: Reveiced an smime signed but unencrypted email. First time after starting outlook email got validated and afer endless waiting (approx. 30 sec) contense was shown. Restarting Outlook later and revisiting this email will restart the verification process. This time the message "Bitte warten Sie während die Nachricht entschlüsselt / geprüft wird..." stays forever. No chance to read this email again.

Starting Outlook still bring up the "Fehler in der Benutzeroberfläche von XML von "GpgOL .....Unbekannte Office.Steuerelemente-ID: TabComposerTool" Message Box.

Sorry for my inpatience...but it's a little bit hard to understand why the above, very simple test procedure obviously isn't reproducible on your systems.

This will be fixed in the next release, the Task for this is still open ( T3441 ) I'm currently waiting on a confirmation there that the error happens on Outlook 2010. A version I don't have in my development setup (And yes I should probably set up a VM with this version, too). But you are right this will be a simple fix. The error is harmless though and only shown to users that have enabled an Add in development Option.

Another unsolved bug: Reveiced an smime signed but unencrypted email. First time after starting outlook email got validated and afer endless waiting (approx. 30 sec) contense was shown. Restarting Outlook later and revisiting this email will restart the verification process. This time the message "Bitte warten Sie während die Nachricht entschlüsselt / geprüft wird..." stays forever. No chance to read this email again.

For S/MIME checks for CRL's can take a long time as it depends on network communication between you and the Certificate issuer (cacert especially is known as very problematic in that regard). 30 Seconds even sounds like a timeout there. You could try to disable crl checks in Kleopatra settings for S/MIME. This should speed up the process.

If somehow the "Bitte warten" message does not disappear you could try to view another mail and reopen this mail later.

@tstreibl thanks for helping with your feedback!
Please note that this issue (T3442) still has the status "testing", because we want to perform some more tests before declaring it resolved. :)

  1. Mails encrypted with S/MIME are stored with "No Data" in the sent EMail folder, but arrive properly at the recipients (you will recieve a readable copy, if you add yourself to the list of recipients). This Issue breaks the GpgOL Plugin after some time which is leading to the described Problem.

Fixed with 474cc15d8e331c9def298dbbfe3b99e6c8cf8035

Tested with Gpg4win-3.0.0-beta17 with GpgOL-2.0.2-beta8 on Windows 10 (64bit) with Outlook 2016 and it worked there. I encountered no crashes and the sent mail had the correct ribbons ontop of the mail.

Tested with Gpg4win-3.0.0-beta17 with GpgOL-2.0.2-beta8 on Windows 7 (64bit) with Outlook 2010 and it doesn't work here. It is still the same symptomatic.

With the Release of Gpg4win 3.0.1 the Error doesn't appear anymore while testing with Windows 10 (64bit) with Outlook 2016 and Windows 7 (64bit) with Outlook 2010.

May @tstreibl confirm this?

Hello Jochen,

I can confirm.

-The Message Box "Fehler in der Benutzeroberfläche von XML von "GpgOL .....Unbekannte Office.Steuerelemente-ID: TabComposerTool" doesn not show up anymore when starting outlook

-Copy Paste of email adresses from outlook adress book to mailitem works fine. No crash any more.

Added a new issue with signatures in 3.0.1

aheinecke closed this task as Resolved.Nov 27 2017, 8:07 AM

Thanks for the test!