Windows: Unable to import CRL
Closed, ResolvedPublic

Description

Steps to reproduce:

Take the latest gpg4win-beta build.
Download CRL list (e.g. http://ca.intevation.org/crls/email-ca-2013.crl)
Open Kleopatra
Change lange to german
Choose Extras > Import CRL from File
Navigate to File and Import
-> Error

aheinecke claimed this task.Mar 3 2017, 1:40 PM
werner added a subscriber: werner.Mar 6 2017, 12:45 PM

Does this work on the command line?

I tried it on two different autohrities.

https://ssl.intevation.de/:

    gpgsm -v --import F:\zertifikate\SMIME\emailca2013.crl
    gpgsm: no issuer found in certificate
    gpgsm: Grundlegende Zertifikatprüfungen fehlgeschlagen - nicht importiert
    gpgsm: no issuer found in certificate
    gpgsm: Grundlegende Zertifikatprüfungen fehlgeschlagen - nicht importiert
    gpgsm: ksba_cert_hash failed: Kein Wert
    ksba: ber-decoder: node `?': TLV length too large
    gpgsm: gesamte verarbeitete Anzahl: 2
    gpgsm:                        nicht importiert: 2

https://www.rz.uni-osnabrueck.de/dienste/unios_ca/index.html:

    gpgsm -v --import Downloads\cacrl.crl
    gpgsm: unknown hash algorithm '?'
    gpgsm: certificate has a BAD signature: Allgemeiner Fehler
    gpgsm: Grundlegende Zertifikatprüfungen fehlgeschlagen - nicht importiert
    gpgsm: gesamte verarbeitete Anzahl: 1
    gpgsm:                        nicht importiert: 1

Seem to be different errors, but it doesnt work from command-line, too.

But the error from the front-end is different, when I tried importing via
commandline first:

Beim Importieren der Sperrliste ist ein Fehler aufgetreten. Die Ausgabe von

GpgSM lautet:

gpgsm: unsupported inquiry 'SENDCERT_SKI

93E3D83226DAD5F14AA5914AE0EA4BE2A20CCFE1 /CN=DFN-Verein Certification Authority
2,OU=DFN-PKI,O=Verein zur Foerderung eines Deutschen Forschungsnetzes e. V.,C=DE'

    gpgsm: response of dirmngr: Unbekanntes IPC "Inquire"

The error is the same for both CAs (except tfor the inquiry details).

JochenSaalfeld removed aheinecke as the assignee of this task.Mar 13 2017, 9:55 AM

To make this more precise I think above might actually be more then one bug.

Let's make this bug about the failure to encrypt to @intevation certificates with the current EMail CA's CRL. Works on Linux, fails on Windows. To reproduce:

Grab my cert:

and import it.

Then with no special config:

echo foo | gpgsm -er aheinecke@intevation.de

gpgsm: certificate #17/CN=Email CA 2013,O=Intevation GmbH,C=DE
gpgsm: checking the CRL failed: Invalid CRL object
gpgsm: can't encrypt to 'aheinecke@intevation.de': Invalid CRL object

aheinecke renamed this task from Unable to import CRL from file with Kleopatra to Windows: Unable to import CRL.Apr 3 2017, 5:48 PM
aheinecke updated the task description. (Show Details)
justus claimed this task.Apr 19 2017, 1:34 PM
justus added a subscriber: justus.

For me it fails a bit differently:

$ export GNUPGHOME=$(mktemp -d)
$ sm/gpgsm --import ../tests/gpgsm/issue2984.pem
gpgsm: NOTE: THIS IS A DEVELOPMENT VERSION!
gpgsm: It is only intended for test purposes and should NOT be
gpgsm: used in a production environment or with production keys!
gpgsm: keybox '/tmp/tmp.hkulRz0FlX/pubring.kbx' created
gpgsm: importing common certificates '/home/teythoon/repos/g10/local/share/gnupg/com-certs.pem'
gpgsm: total number processed: 2
gpgsm:               imported: 2
gpgsm: total number processed: 3
gpgsm:               imported: 3
$ echo foo | sm/gpgsm -er aheinecke@intevation.de
gpgsm: NOTE: THIS IS A DEVELOPMENT VERSION!
gpgsm: It is only intended for test purposes and should NOT be
gpgsm: used in a production environment or with production keys!
gpgsm: certificate #17/CN=Email CA 2013,O=Intevation GmbH,C=DE
gpgsm: no CRL found for certificate
gpgsm: certificate #04/CN=Root CA 2010,O=Intevation GmbH,C=DE
gpgsm: no CRL found for certificate
gpgsm: root certificate is not marked trusted
gpgsm: fingerprint=11:B9:1B:31:EE:09:E0:84:4D:25:4E:58:7A:65:CE:51:84:F3:6B:70
gpgsm: DBG: BEGIN Certificate 'issuer':
gpgsm: DBG:      serial: 00
gpgsm: DBG:   notBefore: 2010-03-12 19:54:02
gpgsm: DBG:    notAfter: 2020-03-12 19:54:02
gpgsm: DBG:      issuer: CN=Root CA 2010,O=Intevation GmbH,C=DE
gpgsm: DBG:     subject: CN=Root CA 2010,O=Intevation GmbH,C=DE
gpgsm: DBG:   hash algo: 1.2.840.113549.1.1.5
gpgsm: DBG:   SHA1 Fingerprint: 11:B9:1B:31:EE:09:E0:84:4D:25:4E:58:7A:65:CE:51:84:F3:6B:70
gpgsm: DBG: END Certificate
gpgsm: after checking the fingerprint, you may want to add it manually to the list of trusted certificates.
gpgsm: can't encrypt to 'aheinecke@intevation.de': Not trusted

The trouble is that I don't understand the problem. I just don't know how gpgsm works, it is this weird gpg-like program that is likely beautiful on the inside.

Noteworthy is that this works in the environment created by the test suite. Here is the config:

$ make -Ctests/gpgsm check XTESTS=shell.scm
make: Entering directory '/home/teythoon/repos/g10/gnupg/obj/tests/gpgsm'
LC_ALL=C EXEEXT= PATH=../gpgscm:/home/teythoon/repos/g10/local/bin:/home/teythoon/.guix-profile/bin::/home/teythoon/.local/bin:/home/teythoon/Applications/.bin:/usr/bin:/home/teythoon/opt/e17/bin:/usr/local/bin:/bin:/usr/local/games:/usr/games srcdir=/home/teythoon/repos/g10/gnupg/obj/../tests/gpgsm objdir=/home/teythoon/repos/g10/gnupg/obj GPGSCM_PATH=/home/teythoon/repos/g10/gnupg/obj/../tests/gpgscm:/home/teythoon/repos/g10/gnupg/obj/../tests/openpgp:/home/teythoon/repos/g10/gnupg/obj/../tests/gpgsm /home/teythoon/repos/g10/gnupg/obj/tests/gpgscm/gpgscm \
  /home/teythoon/repos/g10/gnupg/obj/../tests/gpgsm/run-tests.scm --parallel shell.scm
PASS: setup.scm 
$ cat gpgsm.conf
disable-crl-checks
faked-system-time 1008241200
$ cat gpg-agent.conf
pinentry-program /home/teythoon/repos/g10/gnupg/obj/tests/openpgp/fake-pinentry

Aha! disable-crl-checks.

Note: This is Windows only!

The error you had just means that the root certificate of Intevation is not trusted in your environment. In an interactive environment the option allow-mark-trusted should have brought up a pinentry asking you to trust the root certificate.

Ok, after adding allow-mark-trusted and configuring a pinentry, I get a failure once, a second try succeeds:

$ sm/gpgsm -er aheinecke@intevation.de
gpgsm: NOTE: THIS IS A DEVELOPMENT VERSION!
gpgsm: It is only intended for test purposes and should NOT be
gpgsm: used in a production environment or with production keys!
gpgsm: certificate #17/CN=Email CA 2013,O=Intevation GmbH,C=DE
gpgsm: no CRL found for certificate
gpgsm: certificate #04/CN=Root CA 2010,O=Intevation GmbH,C=DE
gpgsm: no CRL found for certificate
gpgsm: root certificate is not marked trusted
gpgsm: fingerprint=11:B9:1B:31:EE:09:E0:84:4D:25:4E:58:7A:65:CE:51:84:F3:6B:70
gpgsm: root certificate has now been marked as trusted
gpgsm: can't encrypt to 'aheinecke@intevation.de': No CRL known
$ sm/gpgsm -er aheinecke@intevation.de
gpgsm: NOTE: THIS IS A DEVELOPMENT VERSION!
gpgsm: It is only intended for test purposes and should NOT be
gpgsm: used in a production environment or with production keys!
foo
gpgsm: encrypted data created
datenbrei

Yes I see this behavior on Linux, too (needs two tries). I have not yet reported a specific issue but this as it does not hurt my usecase (Kleopatra / GpgOL) because there you are asked when you import the key / when a keylist --with-validation is done if you want to trust the root certificate. Which happens before the certificate is used in an operation.
The problem for me is that CRL checks, at least against our current CRL fail (on windows) but not on linux.

gniibe added a subscriber: gniibe.Apr 20 2017, 6:30 AM

GPG_ERR_INV_CRL_OBJ is only possible by libksba.
I'd suggest enabling debug option for dirmngr by .gnupg/dirmngr.conf:

log-file SOMEWHERE
debug-level {basic,advanced,expert,guru} # Chose one
debug-all

to investigate what's going on.

Let's create a test case that demonstrates the problem. Andre, is it ok to include your certificate in the test suite? How can I mark the root certificate as trusted without pinentry interaction?

Before the gpg-agent is started you need to create a file trustlist.txt:

echo "11:B9:1B:31:EE:09:E0:84:4D:25:4E:58:7A:65:CE:51:84:F3:6B:70 S" > $GNUPGHOME/trustlist.txt

(See gnupg/doc/examples/trustlist.txt)

It's Ok to use my certificate but the test would need faked-system-time to avoid expiration problems.

For a testcase it's probably better to use the CRL from a file (to avoid networking) and let dirmngr parse that with dirmngr --load-crl. The CRL's for the root CA 2010 and EMail CA2013 are available at https://ssl.intevation.de/

Under Linux this works for me but this fails for me even under GNU/Linux with the error:

aheinecke@esus /tmp> dirmngr --debug-level guru --verbose --load-crl email-ca-2013.crl
dirmngr[14141]: NOTE: this is a development version!
dirmngr[14141]: enabled debug flags: x509 crypto memory cache memstat hashing ipc dns network lookup extprog
dirmngr[14141]: error opening '/tmp/tmp.Izp3iglIP7/dirmngr_ldapservers.conf': No such file or directory
dirmngr[14141.0]: permanently loaded certificates: 176
dirmngr[14141.0]:     runtime cached certificates: 0
dirmngr[14141.0]:            trusted certificates: 176 (175,0,0,1)
dirmngr[14141.0]: update times of this CRL: this=20170406T195040 next=20170428T195040
dirmngr[14141.0]: DBG: find_cert_bysubject: certificate not in cache
dirmngr[14141.0]: DBG: get_cert_local called w/o context
dirmngr[14141.0]: DBG: find_cert_bysubject: certificate not returned by caller - doing lookup
dirmngr[14141.0]: error fetching certificate by subject: Configuration error
dirmngr[14141.0]: crl_parse_insert failed: Missing certificate

I'm not sure if I'm using it wrong. I Imported my Certificate (including the chain) loaded the root ca's crl from file (this worked) and then tried the email ca.

The man page of dirmngr suggests to use gpgsm --call-dirmngr loadcrl but this then fails with:

gpgsm --call-dirmngr loadcrl /tmp/email-ca-2013.crl 
gpgsm: unsupported inquiry 'ISTRUSTED 11B91B31EE09E0844D254E587A65CE5184F36B70'
gpgsm: response of dirmngr: Unknown IPC inquire

I'll try now if I can get to the Invalid CRL Object on Windows using files only.

On Windows I can reproduce the problem this way:

C:\Users\aheinecke>mkdir c:\tmpcrl

C:\Users\aheinecke>set GNUPGHOME=c:\tmpcrl

C:\Users\aheinecke>gpgsm --import x:\src\crltest\aheinecke-2016.pem
gpgsm: keybox 'c:\tmpcrl\pubring.kbx' created
gpgsm: total number processed: 3
gpgsm:               imported: 3

C:\Users\aheinecke>dirmngr --load-crl x:\src\crltest\root-ca-2010.crl
dirmngr[1988]: error opening 'c:\tmpcrl\dirmngr_ldapservers.conf': No such file or directory
dirmngr[1988]: permanently loaded certificates: 78
dirmngr[1988]:     runtime cached certificates: 0
dirmngr[1988]:            trusted certificates: 78 (77,0,0,1)
dirmngr[1988]: failed to open cache dir file 'c:\tmpcrl\crls.d\DIR.txt': No such file or directo
dirmngr[1988]: creating directory 'c:\tmpcrl\crls.d'
dirmngr[1988]: new cache dir file 'c:\tmpcrl\crls.d\DIR.txt' created
dirmngr[1988]: root certificate is not marked trusted
dirmngr[1988]: fingerprint=11B91B31EE09E0844D254E587A65CE5184F36B70
dirmngr[1988]: DBG: BEGIN Certificate 'issuer':
dirmngr[1988]: DBG:      serial: 00
dirmngr[1988]: DBG:   notBefore: 2010-03-12 19:54:02
dirmngr[1988]: DBG:    notAfter: 2020-03-12 19:54:02
dirmngr[1988]: DBG:      issuer: CN=Root CA 2010,O=Intevation GmbH,C=DE
dirmngr[1988]: DBG:     subject: CN=Root CA 2010,O=Intevation GmbH,C=DE
dirmngr[1988]: DBG:         aka: <ca@intevation.de>
dirmngr[1988]: DBG:         aka: (3:uri24:http://ca.intevation.org)
dirmngr[1988]: DBG:   hash algo: 1.2.840.113549.1.1.5
dirmngr[1988]: DBG:   SHA1 fingerprint: 11B91B31EE09E0844D254E587A65CE5184F36B70
dirmngr[1988]: DBG: END Certificate

C:\Users\aheinecke>dirmngr --load-crl x:\src\crltest\email-ca-2013.crl
dirmngr[3968]: error opening 'c:\tmpcrl\dirmngr_ldapservers.conf': No such file or directory
dirmngr[3968]: permanently loaded certificates: 78
dirmngr[3968]:     runtime cached certificates: 0
dirmngr[3968]:            trusted certificates: 78 (77,0,0,1)
dirmngr[3968]: root certificate is not marked trusted
dirmngr[3968]: fingerprint=11B91B31EE09E0844D254E587A65CE5184F36B70
dirmngr[3968]: DBG: BEGIN Certificate 'issuer':
dirmngr[3968]: DBG:      serial: 00
dirmngr[3968]: DBG:   notBefore: 2010-03-12 19:54:02
dirmngr[3968]: DBG:    notAfter: 2020-03-12 19:54:02
dirmngr[3968]: DBG:      issuer: CN=Root CA 2010,O=Intevation GmbH,C=DE
dirmngr[3968]: DBG:     subject: CN=Root CA 2010,O=Intevation GmbH,C=DE
dirmngr[3968]: DBG:         aka: <ca@intevation.de>
dirmngr[3968]: DBG:         aka: (3:uri24:http://ca.intevation.org)
dirmngr[3968]: DBG:   hash algo: 1.2.840.113549.1.1.5
dirmngr[3968]: DBG:   SHA1 fingerprint: 11B91B31EE09E0844D254E587A65CE5184F36B70
dirmngr[3968]: DBG: END Certificate
dirmngr[3968]: ksba_crl_parse failed: Invalid CRL object
dirmngr[3968]: crl_parse_insert failed: Invalid CRL object
dirmngr[3968]: crl_cache_insert via DP failed: Invalid CRL object
dirmngr[3968]: error checking validity of CRL issuer certificate: Invalid CRL object
dirmngr[3968]: crl_parse_insert failed: Invalid CRL object

I manually parse email-ca-2013.crl:

30 82 02 84 ; SEQUENCE
   30 6e    ;   SEQUENCE
      30 0d ;     SEQUENCE
            06 09 ; OBJECT_ID
                  2a 86 48 86 f7 0d 01 01 05
            05 00 ; NULL
      30 3f ;     SEQUENCE
            31 0b ; SET
                  30 09 ; SEQUENCE
                        06 03 ; OBJECT_ID
			      55 04 06
			13 02 ; PRINTABLE_STRING "DE"
                              44 45
            31 18 ; SET
                  30 16 ; SEQUENCE
                        06 03 ; OBJECT_ID
                              55 04 0a
                        0c 0f ; UTF8_STRING "Intevation GmbH"
                              49 6e 74 65 76 61 74 69 6f 6e 20 47 6d 62 48
            31 16 ; SET
                  30 14 ; SEQUENCE
                        06 03 ; OBJECT_ID
                              55 04 03
                        0c 0d ; UTF8_STRING "Email CA 2013"
                              45 6d 61 69 6c 20 43 41 20 32 30 31 33 
      17 0d       ; UTC_TIME
            31 37 30 34 30 36 31 39 35 30 34 30 5a
      17 0d       ; UTC_TIME
            31 37 30 34 32 38 31 39 35 30 34 30 5a
      30 0d;     SEQUENCE
            06 09 ; OBJECT_ID
                  2a 86 48 86 f7 0d 01 01 05
            05 00 ; NULL
   03 82 02 01;  BIT_STRING (The signature)
            ...

Parsing output by openssl:

$ openssl crl -inform DER -noout -in email-ca-2013.crl -text
Certificate Revocation List (CRL):
        Version 1 (0x0)
    Signature Algorithm: sha1WithRSAEncryption
        Issuer: /C=DE/O=Intevation GmbH/CN=Email CA 2013
        Last Update: Apr  6 19:50:40 2017 GMT
        Next Update: Apr 28 19:50:40 2017 GMT
No Revoked Certificates.
    Signature Algorithm: sha1WithRSAEncryption
         71:bb:23:88:f6:6c:41:a6:97:0e:a5:d4:ac:98:78:80:e5:87:
         ...

It seems for me that libksba doesn't assume no CRL when parsing.

I don't think that is the case because the CRL works on Linux and t-crl-parser from libksba correctly parses the crl on both Windows and Linux.

This is what I noticed. This patch makes dirmngr and t-crl-parser both use same reader:

Thanks for looking into it but sadly this did not fix the problem. I'll try to attach a debugger or trace with printf tomorrow where libksba thinks the CRL is invalid and how this differs from the success case with t-crl-parser.
So I'm claiming this task again for that.
But I also think that a unit test that does an S/MIME import / trustlist change / CRL Imports and then an encryption without crl-checks disabled would be useful.

Ok I think I understand it better now. The problem is not with the email-ca file (that explains why the test works) but appears to be in the ldap fetching code which is different on Windows. Even if we loaded the CRL for the root CA previously dirmngr still fetches it over LDAP and tries to parse it. This parsing of the root ca's crl is what fails.
Under GNU/Linux the dirmngr_ldap wrapper is used (at least on my system) while for windows since 2.1 we use the ldap-wrapper-ce.

I confirmed through debug output that the error occurs in libksba/src/crl.c line 865 parse_to_next_update which is the first thing done when a CRL parsing is started. This is because the check fails because _ksba_ber_read_tl reads a 0 byte and not the expected 0x30 byte.

In ldap-wrapper-ce.c I added debug output to confirm why. The data is there properly written into the buffer but the read happens at the end of the buffer instead of the beginning. Eg. The CRL is 647 byte long. But If I add debug output in buffer_get_data in the first read to show the value of cookie->buffer_pos it is 647 because there is no distinction between the write / read position and it's not reset after writing the buffer. And as it is supposed to be a ring buffer using the same value for read and write pos does not appear to make sense to me.

I don't understand how this could ever have worked or what the supposed mechanism should be to reset the buffer pos. I think I saw CRL's working at some point on Windows but with all the CRL problems we had that might not be true because I often disabled crl checks or at least the ldap-dp in my test setups. I tried it now with 2.1.0, 2.1.10, 2.1.15 and all showed this problem, but maybe our LDAP server behaves differently now and there is some hidden magic in ksba or the ldap-wrapper that I have not seen.

I'm adding a Differential which fixes the problem for me but please Review the buffer_get_data and buffer_put_data functions in ldap-wrapper-ce. For example I don't see how cookie->buffer_len can really work there because it is only modified on write and not on read.

Or maybe we can get rid of ldap-wrapper-ce and use more similar code to GNU/Linux on Windows?

aheinecke reassigned this task from aheinecke to wk.May 2 2017, 10:18 AM

Assigned to werner as the corresponding differential is waiting review and a general look at how we do ldap fetching on windows is also best done by werner.

aheinecke closed this task as Resolved.Jul 24 2017, 1:14 PM

The diff was commited. A general review of the ldap fetching on Windows is still pending but I think this can be resolved for now and we revisit this topic when we see new problems.

I think LDAP CRL's that are larger then the buffer size won't work as the buffer is not really a ring buffer but only pretends to be one imo. But I have not tested and don't care atm.