Page MenuHome GnuPG

GpgOL (Gpg4Win 3.1.24) / Error in parsing mail-headers (empty mail-body without correct decoded encryption-scheme) when using gpgol.dll 2.5.4 (gpgol.dll 2.5.0 from 3.1.16 works)
Closed, ResolvedPublic

Description

When installing GpgOL 2.5.4 (by Gpg4Win 3.1.24), and doing a test for mail-encryption on Windows 10 x64 21H2 <=> 20H2 (HTML- or Plaintext), Outlook shows no mail-body and no trust-state (see image, on top right corner):

When replacing the original gpgol.dll from Gpg4Win 3.1.16 or doing a reinstall of Gpg4Win 3.1.16 or 3.1.23, everything works fine (correct parsing and decoding):

The error occurs, when using GpgOL 2.5.4 (from the Gpg4Win 3.1.24-Build); when using GpgOL from 3.1.16 or 3.1.23, NO ERROR occurs.

On both machines, we're using Outlook (16.0.0.5257.1000) MSO (16.0.5361.1000) 64Bit

To "quick-and-dirty" reproduce / eliminate the error we just have to directly replace the "gpgol.dll" in the appropriate directories.

  • gpgol.dll.2.5.0.18451 (Gpg4Win 3.1.16) -- works --
  • gpgol.dll.2.5.3.21842 (Gpg4Win 3.1.23) -- works --
  • gpgol.dll.2.5.4.37473 (Gpg4Win 3.1.24) -- ! FAILS ! --

I did some further testing (HTML-Mail or Plaintext-Mail is irrelevant):

On the Win10REL20H2 - system we're using "MCAFEE ENDPOINT PROTECTION WITH EPO-Server" ...

Maybe this could be an issue because all encrypted mail from this system is not "decodable" by version 2.5.4, only when we're using gpgol.dll <2.5.4 (i. e. 2.5.3 or 2.5.0, etc.) !!

But what are the special differences between these gpgol-versions and why does the decoding works with the older gpgol-versions ?

Win10REL20H2 GpGOL 2.5.3 or 2.5.4 (McAfee Endpoint Protection)
  => Win10REL21H2 GpgOL 2.5.4 (MS-Defender -standard-)
    => FAILED !

Win10REL20H2 GpGOL 2.5.4 or 2.5.4 (McAfee Endpoint Protection)
  => Win10REL21H2 GpgOL 2.5.4 (MS-Defender -standard-)
    => FAILED !

Win10REL20H2 GpGOL 2.5.3 or 2.5.4 (McAfee Endpoint Protection)
  => Win10REL21H2 GpgOL 2.5.3 (MS-Defender -standard-)
    => OK !

Win10REL21H2 GpgOL 2.5.3 or 2.5.4 (MS-Defender -standard-)
  => Win10REL20H2 GpGOL 2.5.3 or 2.5.4 (McAfee Endpoint Protection) 
    => OK !
  • On the next hours i will do a test by temporarily deactivating the McAfee-Engine in order to rule out any influence on the process.**
  • Then i will post that result here ...**

Here is the (i hope so) disarmed gpgol.log.txt (i hope, i have removed all sensitive data):

Be patient ... and best regards,

Veit Berwig

Details

Version
2.5.4

Revisions and Commits

Event Timeline

vitusb created this object in space S1 Public.

Mh, fuck?

We have tested this a lot of course. But I will have to analyze your logs. Thanks.

I do not have a mind to really analyze this today, but when the checkbox in the logging options for "include data" is not set. There should be no much as an IP Address or Fingerprint mentioned in the logs. This was important to me and if you find that there are issues with that it would be a different bug also.

Yeah the error would lie in here I think:

20:48:08/1748/DBG_OOM/mail.cpp:preProcessMessage_m: GetBaseMessage OK for 000002ad043d1120.
20:48:08/1748/mapihelp.cpp:mapi_change_message_class: checking message class `IPM.Note'
20:48:08/1748/DBG_OOM/oomhelp.cpp:gpgol_openProperty:73 AddRef on 000002ad0243ccd0
20:48:08/1748/oomhelp.cpp:gpgol_openProperty: OpenProperty on 000002ad045dce98 prop 7d001e result 000002ad0243ccd0
20:48:08/1748/DBG_MEM/mapihelp.cpp:mapi_get_header:647: Object: 000002ad0243ccd0 released ref: 0·
20:48:08/1748/DBG_DATA/Parsing header: Return-Path: <--- disarmed sensitive data --->

--- disarmed sensitive data ---

20:48:08/1748/mapihelp.cpp:parse_header_data:Failed to parse headers.
20:48:08/1748/ERROR/mapihelp.cpp:mapi_get_message_content_type: failed to parse headers
20:48:08/1748/mapihelp.cpp:change_message_class_ipm_note: content type is 'null'
20:48:08/1748/oomhelp.cpp:gpgol_openProperty: OpenProperty failed hr=0x8004010f MAPI_E_NOT_FOUND
20:48:08/1748/mapihelp.cpp:mapi_get_body_as_stream: OpenProperty tag=815d0102 failed: hr=0x8004010f
20:48:08/1748/oomhelp.cpp:gpgol_openProperty: OpenProperty failed hr=0x8004010f MAPI_E_NOT_FOUND
20:48:08/1748/mapihelp.cpp:mapi_get_body_as_stream: OpenProperty failed: hr=0x8004010f
20:48:08/1748/mapihelp.cpp:get_msgcls_from_pgp_lines: Failed to get body ASCII stream.
20:48:08/1748/oomhelp.cpp:gpgol_openProperty: OpenProperty failed hr=0x8004010f MAPI_E_NOT_FOUND
20:48:08/1748/ERROR/mapihelp.cpp:get_msgcls_from_pgp_lines: Failed to get  w_body stream. : hr=0x8004010f
20:48:08/1748/mapihelp.cpp:mapi_change_message_class Message is not a crypto message or already in the right class.

Hrumpf. How can that be changed between our versions....

Should i create a new log without "include data" ?

No, I was just meaning that you should not have to disarm your logs when include data is not set.

At this moment I have no idea what your problem could be. I have to think about this. 2.5.3 and 2.5.4 have no differences in the parser for mails and nearly no differences in the creator of mails. Also we have tested this before the release of course. If you have any idea why this issue might occur to you and not to our testers, please share. We test IMAP / EXCHANGE (from office 365) and mails sent to self and to / from others and look at the mails in different Mail clients like KMail.

Here is another Test:

When i send encrypted mail from ...

Win10REL20H2 GpGOL 2.5.3 or 2.5.4 (McAfee Endpoint Protection) ...

... to => Win10REL21H2 GpgOL 2.5.3 (MS-Defender -standard-)
        ... => this works ... because 2.5.3 decodes without error.

... the mail resides in my inbox ...

... NOW ... when i change the dll-version on the receiving machine from 2.5.3 to 2.5.4 ... then 2.5.4 !! may still open this mail without error !! (pinentry launches). => is this because version 2.5.3 decoded the incoming data flawlessly before ?

I'm waiting for my "Administration" to temporary disable "McAfee Endpoint-Protection" (ATP, etc.) for checking, if this could be an issue ... but i don't understand, why the older versions of "gpgol.dll" (2.5.0, 2.5.3, etc.) are unaffected, when McAffe should be the issue ...

Anyway ... here are the additional logs:

Best regards ...

Here some further investigations ...

We deactivated the McAfee EPO-Client services on the relevant managed source-system Win10REL20H2 (with an EXCHANGE/MAPI connection), mentioned above ...

... but the errors in parsing / decoding the headers on the target-system Win10REL21H2 (with a GMX POP/IMAP connection) were allways the same.

So, i activated now the GpgOL-Logging on the source-system (EXCHANGE/MAPI) in order to log the sending-process and there we have found some exceptions (details in logfile):

Details of Connection:

Client-System:

Win1020H2 (McAfee-EPO-Client, Exchange / MAPI Connection)

Server-Version:

Exchange MAPI/HTTP Connectivity Endpoint
Exchange Server 2019 / Version: 15.2.1118.9

... on the other system (GMX IMAP/POP3) we also did a logging of the sending-process and no exceptions appeared in this logfile:

When we sent only singed e-mail (gpgol-2.5.4), this worked in both directions !

Best regards,
Veit Berwig

aheinecke added a project: Restricted Project.

Hello,
many thanks for the detailed report, I have given it some time to analyze and think I understand it:

With your testmail, when I moved the mail to a test client with KMail decryption worked.
But on another system it did not work.

I think the problem is that exchange converted the encrypted data in an ms-tnef attachment and did not provide the transport headers through MAPI but instead in the ms-tnef attachment part.
In your logs the problem happens here:

10:24:15/11060/mapihelp.cpp:change_message_class_ipm_note: content type is 'application/ms-tnef'
10:24:15/11060/oomhelp.cpp:gpgol_openProperty: OpenProperty failed hr=0x8004010f MAPI_E_NOT_FOUND
10:24:15/11060/mapihelp.cpp:mapi_get_body_as_stream: OpenProperty tag=84050102 failed: hr=0x8004010f

Tried to get the transport header, this failed,

10:24:15/11060/DBG_OOM/oomhelp.cpp:gpgol_openProperty:73 AddRef on 19c07b80
10:24:15/11060/oomhelp.cpp:gpgol_openProperty: OpenProperty on 1985cab4 prop 1000001e result 19c07b80
10:24:15/11060/DBG_MEM/mapihelp.cpp:get_msgcls_from_pgp_lines:882: Object: 19c07b80 released ref: 0

Opened the body of the mail.

10:24:15/11060/mapihelp.cpp:get_msgcls_from_pgp_lines: Detected non whitespace M before a PGP Marker

Did not find the -----BEGIN PGP Marker in the message body so it is not PGP/Inline.

10:24:15/11060/DBG_OOM/mapihelp.cpp:get_first_attach_mime_tag:1110 AddRef on 195d76c8
10:24:15/11060/DBG_OOM/mapihelp.cpp:get_first_attach_mime_tag:1150 AddRef on 1978802c
10:24:15/11060/DBG_MEM/mapihelp.cpp:get_first_attach_mime_tag:1164: Object: 1978802c released ref: 0 
10:24:15/11060/DBG_MEM/mapihelp.cpp:get_first_attach_mime_tag:1166: Object: 195d76c8 released ref: 0 
10:24:15/11060/mapihelp.cpp:mapi_change_message_class Message is not a crypto message or already in the right class.

Now it is done and will not look at this mail again. The reason for this is that ms-tnef handling previously somewhat worked because of behavior I thought is a bug. 5fd467a00d3ffa6c1ca83e9a248f4c01d77bbe72 I did not think of the MS-TNEF case that this might be required.

I am still not really sure how to trigger this ms-tnef (winmail.dat) conversion or when that happens. In KMail through IMAP I saw the mail correctly (same account without moving). In Outlook 2019 after moving I saw it correctly. In Outlook 2019 in the same folder I saw it as winmail.dat. We need to get this included in our pre release tests. It is probably some exchange configuration or it happens when it is transported between the Exchange servers.

And yes opening the mail once with 2.5.3 would fix the issue if my analysis is correct because then it will have marked the mail correctly.

Good news is that I can reproduce the bug in our testlab by connecting an account via IMAP to exchange. Our other IMAP tests have intermediates like dovecot. The fix for this will be fairly simple but first I wanted to ensure that we could reproduce it for future testing of releases as this is a case that should have been covered.

I think what I saw and reproduced (and now fixed) was a different issue though. 5fd467a00d3ffa6c1ca83e9a248f4c01d77bbe72 broke IMAP connections for GpgOL in general. So we definitely will make a new, at least minor GnuPG VS-Desktop release. But first we need to reproduce and also fix your issue.

If you could try: https://files.gpg4win.org/Beta/gpgol/2.5.5-beta2/x64/ (Source tarball in the directory above, signed by my key) Doc for this can be found here: https://wiki.gnupg.org/TroubleShooting#Manually_update_GpgOL_to_a_beta

I think it will not help.

If you could enable data debugging though (Include Data) in a log. And send it to me by private mail, that could help to make another targeted fix. Although as I said, I am only happy with closing this issue when we find out how to reproduce it.

If you could try: https://files.gpg4win.org/Beta/gpgol/2.5.5-beta2/x64/ (Source tarball in the directory above, signed by my key)
If you could enable data debugging though (Include Data) in a log. And send it to me

Already done :-)

GpgOL-2.5.5-beta2 works !
... For details look at your InBox :-)

Best Regards
Veit Berwig

I did a build of Gpg4Win 3.1.24 with Andre's provided patch :-)

Till now, it works fine with my IMAP-inbox ;-)
   I keep testing ... be patient ...

Slightly modified ...

  • Makefile.am:
@@ -47,2 +47,3 @@
              patches/glib/glib-2-fixes.patch \
+             patches/gpgol/0001-Fix-IMAP-access-to-encrypted-mails.patch \
              patches/kconfig/0001-Read-defaults-from-Windows-registry.patch \

Best regards ...
Veit Berwig

"Fix IMAP access to encrypted mails" - patch still not applied in codebase 3.1.25 ...

Referring to Andre's Fix IMAP access to encrypted mails - patch, it was not applied to version 3.1.24 of Gpg4Win because the release-date was after the release of 3.1.24. So i thought the patch above would be applied to the Gpg4Win-codebase as of 3.1.25 because the GpgOL-codebase is still GpgOL 2.5.4 without the patch above, but that was not implemented.

So here i've created 2 patch-files (included a fixed "Makefile.am") in order to apply this patch to codebase 3.1.24 and 3.1.25:

I'm afraid the patch wasn't applied to "GnuPG VS Desktop" either; without this patch encrypted communication with IMAP mailboxes seems impossible unless an alternative solution exists. :-(

Best regards,
Veit Berwig

werner added a subscriber: werner.

Please note that gpg4win 3.1 is not anymore maintained. Gpg4win 4.0.4 is the currrent release and comes with the IMAP fix. We do not have a single GnuPG VS-Desktop customer using IMAP and thus having the fix only in the next VSD version seems to be okay.

Thanks for testing.

Please note that gpg4win 3.1 is not anymore maintained.

As far as i can see, the 3.1.xx-codebase ist still the codebase for "GnuPG VS Desktop". Here "gpgol" 2.5.4 ist still used.

Gpg4win 4.0.4 is the currrent release and comes with the IMAP fix.

Yes, here in "Gpg4win 4.0.4" "gpgol 2.5.5" with the IPMAP-fix is used, but that's not the official approved version by the "BSI" as far as i know ...

The BSI has only approved the 3.1.16 release (documented by SHA-checksums in their "SecOPs" to the approved product) and its sub-releases . Release 4.x.xx was never approved here ... so "GnuPG VS Dsktop" uses still the 3.x.xx-codebase as far as i know ...

We do not have a single GnuPG VS-Desktop customer using IMAP ...

How can you know ?
It should be possible to communicate encrypted with external contacts based on IMAP if supported by the software.

Best Regards ...

ebo moved this task from Restricted Project Column to Restricted Project Column on the Restricted Project board.Jul 24 2023, 2:12 PM