I've compiled current master and it works for the testcase. But when I start
kleopatra and it runs the keylist/verify dirmngr now crashes.

Can be triggered with gpgme: run-keylist --validate --cms

It crashes at different points but it never gets through all my certificates.

An example of the debug output that is collected before it crashes (differs
between crashes):

2015-06-16 19:09:15 dirmngr[9303.1] no CRL available for issuer id
18F071EAAC08885C9434A7DE1DB3AFC30F27DD32
2015-06-16 19:09:15 dirmngr[9303.1] DBG: chan_1 -> INQUIRE SENDCERT
2015-06-16 19:09:15 dirmngr[9303.1] DBG: chan_1 <- [ 44 20 30 82 04 7d 30 82 03
65 a0 03 02 01 02 02 ...(982 byte(s) skipped) ]
2015-06-16 19:09:15 dirmngr[9303.1] DBG: chan_1 <- [ 44 20 14 34 6d f5 07 c2 04
86 4a ba a1 71 50 b0 ...(187 byte(s) skipped) ]
2015-06-16 19:09:15 dirmngr[9303.1] DBG: chan_1 <- END
2015-06-16 19:09:15 dirmngr[9303.1] checking distribution points
2015-06-16 19:09:15 dirmngr[9303.1] no distribution point - trying issuer name
2015-06-16 19:09:15 dirmngr[9303.1] fetching CRL from default location
2015-06-16 19:09:15 dirmngr[9303.1] ldap wrapper 10199 started (reader
0x00007f6a580337a0)
2015-06-16 19:09:15 dirmngr[9303.0] ldap wrapper 10198 ready: exitcode=1
2015-06-16 19:09:15 dirmngr[9303.0] ldap worker stati:
2015-06-16 19:09:15 dirmngr[9303.0] c=0x00007f6a58033740 pid=10199/10199
rdr=0x00007f6a580337a0 ctrl=0x00007f6a580008c0/1 la=1434474555 rdy=0
2015-06-16 19:09:15 dirmngr[9303.0] c=0x00007f6a58022520 pid=-1/10198
rdr=0x0000000000000000 ctrl=0x0000000000000000/0 la=1434474554 rdy=1
2015-06-16 19:09:15 dirmngr[9303.0] dirmngr_ldap[10199]: processing url 'ldap://'
2015-06-16 19:09:15 dirmngr[9303.0] dirmngr_ldap[10199]: host
'directory.verisign.com'
2015-06-16 19:09:15 dirmngr[9303.0] dirmngr_ldap[10199]: port 389
2015-06-16 19:09:15 dirmngr[9303.0] dirmngr_ldap[10198]: processing url 'ldap://'
2015-06-16 19:09:15 dirmngr[9303.0] dirmngr_ldap[10198]: host
'directory.verisign.com'
2015-06-16 19:09:15 dirmngr[9303.0] dirmngr_ldap[10198]: port 389
2015-06-16 19:09:15 dirmngr[9303.0] dirmngr_ldap[10199]: DN
'1.2.840.113549.1.9.1=#4865696E65636B656E40676D61696C2E636F6D,CN=Common
Name,ST=Some-State,C=DE'
2015-06-16 19:09:15 dirmngr[9303.0] dirmngr_ldap[10199]: filter
'objectClass=*'
2015-06-16 19:09:15 dirmngr[9303.0] dirmngr_ldap[10199]: attr
'certificateRevocationList'
2015-06-16 19:09:15 dirmngr[9303.0] dirmngr_ldap[10198]: DN
'1.2.840.113549.1.9.1=#4865696E65636B656E40676D61696C2E636F6D,CN=Common
Name,ST=Some-State,C=DE'
2015-06-16 19:09:15 dirmngr[9303.0] dirmngr_ldap[10198]: filter
'objectClass=*'
2015-06-16 19:09:15 dirmngr[9303.0] dirmngr_ldap[10198]: attr
'certificateRevocationList'
2015-06-16 19:09:15 dirmngr[9303.0] dirmngr_ldap[10198]: searching 'ldap://'
failed: No such object
2015-06-16 19:09:15 dirmngr[9303.0] ldap worker stati:
2015-06-16 19:09:15 dirmngr[9303.0] c=0x00007f6a58033740 pid=10199/10199
rdr=0x00007f6a580337a0 ctrl=0x00007f6a580008c0/1 la=1434474555 rdy=0
2015-06-16 19:09:15 dirmngr[9303.0] c=0x00007f6a58022520 pid=-1/10198
rdr=0x0000000000000000 ctrl=0x0000000000000000/0 la=1434474555 rdy=1
2015-06-16 19:09:15 dirmngr[9303.0] dirmngr_ldap[10199]: searching 'ldap://'
failed: No such object
2015-06-16 19:09:15 dirmngr[9303.0] ldap wrapper 10199 ready: exitcode=1
2015-06-16 19:09:15 dirmngr[9303.0] ldap worker stati:
2015-06-16 19:09:15 dirmngr[9303.0] c=0x00007f6a58033740 pid=-1/10199
rdr=0x0000000000000000 ctrl=0x00007f6a580008c0/1 la=1434474555 rdy=1

Backtrace for this log (also differs):

#0 0x00007f6a69109cc9 in GI_raise (sig=sig@entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007f6a6910d0d8 in GI_abort () at abort.c:89
#2 0x00007f6a69146394 in __libc_message (do_abort=do_abort@entry=1,

fmt=fmt@entry=0x7f6a69254b28 "*** Error in `%s': %s: 0x%s ***\n") at

../sysdeps/posix/libc_fatal.c:175
#3 0x00007f6a6915266e in malloc_printerr (ptr=<optimized out>,
str=0x7f6a69254cf0 "double free or corruption (fasttop)", action=1)

at malloc.c:4996

#4 _int_free (av=<optimized out>, p=<optimized out>, have_lock=0) at malloc.c:3840
#5 0x00007f6a69d4fd2d in ?? () from /opt/gnupg/lib/libgcrypt.so.20
#6 0x0000000000428802 in ldap_wrapper (ctrl=ctrl@entry=0x7f6a580008c0,
reader=reader@entry=0x7f6a64c05cd0,

argv=argv@entry=0x7f6a64c05a80) at ldap-wrapper.c:772

#7 0x000000000042218c in run_ldap_wrapper (ctrl=ctrl@entry=0x7f6a580008c0,
multi_mode=multi_mode@entry=0, proxy=0x0,

host=<optimized out>, port=<optimized out>, user=<optimized out>, pass=0x0, 
dn=dn@entry=0x7f6a58001d50

"1.2.840.113549.1.9.1=#4865696E65636B656E40676D61696C2E636F6D,CN=Common
Name,ST=Some-State,C=DE",

filter=filter@entry=0x44dcc1 "objectClass=*", attr=attr@entry=0x44b50a

"certificateRevocationList", url=url@entry=0x0,

reader=reader@entry=0x7f6a64c05cd0, ignore_timeout=0) at ldap.c:191

#8 0x00000000004228ea in attr_fetch_ldap (ctrl=0x7f6a580008c0,

dn=0x7f6a58001d50

"1.2.840.113549.1.9.1=#4865696E65636B656E40676D61696C2E636F6D,CN=Common
Name,ST=Some-State,C=DE",

attr=attr@entry=0x44b50a "certificateRevocationList",

reader=reader@entry=0x7f6a64c05cd0) at ldap.c:287
#9 0x0000000000414aed in crl_fetch_default (ctrl=ctrl@entry=0x7f6a580008c0,

issuer=issuer@entry=0x7f6a58001d50

"1.2.840.113549.1.9.1=#4865696E65636B656E40676D61696C2E636F6D,CN=Common
Name,ST=Some-State,C=DE", reader=reader@entry=0x7f6a64c05cd0) at crlfetch.c:319
#10 0x000000000041439d in crl_cache_reload_crl (ctrl=ctrl@entry=0x7f6a580008c0,
cert=0x7f6a58002740) at crlcache.c:2554
#11 0x000000000040e1d5 in inquire_cert_and_load_crl (ctx=0x7f6a58000950) at
server.c:589
#12 cmd_isvalid (ctx=0x7f6a58000950, line=<optimized out>) at server.c:901
#13 0x00007f6a6a23e96a in ?? () from /opt/gnupg/lib/libassuan.so.0
#14 0x00007f6a6a23ed49 in assuan_process () from /opt/gnupg/lib/libassuan.so.0
#15 0x000000000040edc7 in start_command_handler (fd=fd@entry=1) at server.c:2243
#16 0x000000000040ada5 in start_connection_thread (arg=arg@entry=0x1) at
dirmngr.c:1937
#17 0x00007f6a69908dbc in thread_start (startup_arg=<optimized out>) at npth.c:265
#18 0x00007f6a696f1182 in start_thread (arg=0x7f6a64c06700) at pthread_create.c:312
#19 0x00007f6a691cd47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

A different backtrace:

#0 0x00007fe9e9805cc9 in GI_raise (sig=sig@entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007fe9e98090d8 in GI_abort () at abort.c:89
#2 0x00007fe9e9842394 in __libc_message (do_abort=do_abort@entry=1,

fmt=fmt@entry=0x7fe9e9950b28 "*** Error in `%s': %s: 0x%s ***\n") at

../sysdeps/posix/libc_fatal.c:175
#3 0x00007fe9e984dac2 in malloc_printerr (ptr=<optimized out>,
str=0x7fe9e994cbfc "corrupted double-linked list", action=1)

at malloc.c:4996

#4 malloc_consolidate (av=av@entry=0x7fe9d8000020) at malloc.c:4165
#5 0x00007fe9e984edf8 in _int_malloc (av=0x7fe9d8000020, bytes=1025) at
malloc.c:3423
#6 0x00007fe9e98517b0 in GI_libc_malloc (bytes=1025) at malloc.c:2891
#7 0x00007fe9ea44ad11 in ?? () from /opt/gnupg/lib/libgcrypt.so.20
#8 0x00007fe9ea44bc19 in ?? () from /opt/gnupg/lib/libgcrypt.so.20
#9 0x00007fe9ea93b284 in init_membuf (maxlen=0, initiallen=<optimized out>,
mb=0x7fe9e53018e0, ctx=0x7fe9d8000950)

at assuan-inquire.c:64

#10 assuan_inquire (ctx=ctx@entry=0x7fe9d8000950, keyword=keyword@entry=0x44774b
"SENDCERT",

r_buffer=r_buffer@entry=0x7fe9e5301d50,

r_length=r_length@entry=0x7fe9e5301d60, maxlen=maxlen@entry=0) at
assuan-inquire.c:169
#11 0x000000000040dfca in inquire_cert_and_load_crl (ctx=0x7fe9d8000950) at
server.c:567
#12 cmd_isvalid (ctx=0x7fe9d8000950, line=<optimized out>) at server.c:901
#13 0x00007fe9ea93a96a in dispatch_command (ctx=0x7fe9d8000950, line=<optimized
out>, linelen=<optimized out>)

at assuan-handler.c:675

#14 0x00007fe9ea93ad49 in process_request (ctx=0x7fe9d8000950) at
assuan-handler.c:871
#15 assuan_process (ctx=0x7fe9d8000950) at assuan-handler.c:894
#16 0x000000000040edc7 in start_command_handler (fd=fd@entry=6) at server.c:2243
#17 0x000000000040ada5 in start_connection_thread (arg=arg@entry=0x6) at
dirmngr.c:1937
#18 0x00007fe9ea004dbc in thread_start (startup_arg=<optimized out>) at npth.c:265
#19 0x00007fe9e9ded182 in start_thread (arg=0x7fe9e5302700) at pthread_create.c:312
#20 0x00007fe9e98c947d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

By killing I meant sending SIGTERM (15) through the kill command.

But
"gpgconf --kill dirmngr" also does not kill the dirmngr. Is this problem not
reproducible for you?

kill -9 kills it of course.

Just to point this problem out again (still exists with current master of
course). The CRL checks during a normal start of kleopatra on my keyring leave
55 dirmngr zombies.

This problem is not really bad for me as I am using the attached Patch. Still
after 3 months I'd appreciate a reaction / review.

Can you let me know when you can take a look at this or was my assignment wrong
here? (If so please change it)

This is a pretty major bug imho that would leave our application servers
(without manual intervention) if we would deploy 2.1 in our company. As such it
is blocking our adoption of 2.1.

I would appreciate some kind of reaction / confirmation on this issue.

During deployment of gnupg 2.1.3 this bug was still noticed and I can still
reproduce it with git master. -> back to chatting

I must have messed up the test in T1921 (aheinecke on Apr 08 2015, 04:36 PM / Roundup). Probably by using a different
sysconfig dir for that test. Apologies for that.

From looking at the error (I don't have gcc 5 at hand) this looks to me like a
problem in the stdc++ library.

It appears that the basic_string implementation wants to put the templates class
(QChar) into a Union and it fails because it has a non-trivial constructor and
this is not allowed.

As this currently works, either the stdc++ library does this differently or gcc
does not check that rule.

Depending on the GCC Version during build configuration I guess we could add
-std=gnu++11 as the error message suggests if the GCC mayor version is > 5

Afaik this is not a pinentry-qt issue as the style looks ok under Unity (ubuntu)
and Windows.

I want to investigate why that is the case and figure out what the problem is
exactly (other KDE password entries in the same style environment look slightly
better) so I left this open to remind me.

I've clarified the title.

This was fixed by:

http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=commit;h=6619ead2cfd2abcb95b66dc70622fdeef624fb8a

using the test described in T1921 (aheinecke on Mar 10 2015, 06:13 PM / Roundup) there are no longer duplicated certificates
shown after the verify command.

Thanks!

I can't reproduce this problem neither in our company setup nor on a vanilla debian.

I've placed the .der files in the correct directories
/var/lib/dirmngr/extra-certs and /etc/dirmngr/trusted-certs

gpgsm --import aheinecke.der

Dirmngr output shows that the LOOKUP Issuer and Intermediate -Cert are not not
found in the dirmngr cache and they are not imported into the keyring.

This is probably another bug that hid this issue in the past.

Changing component to gnupg with topic dirmngr in the hope to get more
visibility and because dirmngr is part of the gnupg repo in 2.1

And the testscript I used to test this.

D289: 587_0001-gpgtar-Fix-extracting-files-with-size-512.patch

In gpgtar-extract.c extract_regular

  for (n=0; n < hdr->nrecords;)
    {
      err = read_record (stream, record);
      if (err)
        goto leave;
      n++;
      nbytes = (n < hdr->nrecords)? RECORDSIZE : (hdr->size % RECORDSIZE);

^ this does not work for the last header if hdr->size size is a multiple of 512.
In that case the last record will not be written.

Please check my attached patch which fixes the problem.

To further minimize the test case:

dd if=/dev/urandom of=testfile count=1024 bs=1024
./gpgtar --encrypt --skip-crypto -- testfile > test.tar
./gpgtar --decrypt --skip-crypto -- test.tar
diff ./test.tar_1_/testfile ./testfile

Binary files ./test.tar_1_/testfile and ./testfile differ

last 512 bytes of testfile are missing after extracting it with gpgtar.

Further tracked this problem down to be a gpgtar extraction issue.

Calling gpgtar on the decrypted archive with the same command kleopatra uses:

C:\Users\aheinecke\Desktop>type gpg-archive.tar |"c:\Program
Files\GNU\GnuPG\gpgtar.exe" --openpgp --skip-crypto --set-filename
C:/Users/aheinecke/Desktop/gpg-archive.tar.gpg --decrypt -- -

Produces the corrupted binary. Copying this archive to a GNU/Linux system and
extracting the tarball with GNU Tar produces a valid binary.

Next test on GNU/Linux with:

./gpgtar --version

gpgtar (GnuPG) 2.1.3-beta4

./gpgtar --openpgp --skip-crypto \

--set-filename /home/aheinecke/arbeit/gpg4win/gpg-archive.tar.gpg \
--decrypt -- /home/aheinecke/arbeit/gpg4win/gpg-archive.tar

sha1sum gpg-archive.tar_1_/gpg2.exe

2d387c8fb53d105c31e4cc2ec186e70a365b0c65 gpg-archive.tar_1_/gpg2.exe

tar -fx /home/aheinecke/arbeit/gpg4win/gpg-archive.tar
sha1sum gpg2.exe

54c8c2ec1083943e556255f76ff8f58e623c5b27 gpg2.exe

The second one is correct.

Patch still needs to be applied upstream but this is tracked in another issue.
-> Resolved

D283: 555_0001-dirmngr-Clean-up-spawned-dirmngr_ldap-processes.patch

This problem is due to ldap_wrapper creating processes with gnupg_spawn_process
but while gnupg_spawn_process states that you have to call gnupg_wait_process
and gnupg_release_process afterwards this is not done in the ldap_wrapper.

Sometimes release is called but never wait to get the exit status of the spawned
process and remove the zombie.

If release is not called this will also leak a handle on Windows.

Attached Patch moves the process cleanup in it's own function and calls that
function from where the process should be terminated and cleaned up.

My test for this:
export GNUPGHOME=$(mktemp -d)
echo "11:B9:1B:31:EE:09:E0:84:4D:25:4E:58:7A:65:CE:51:84:F3:6B:70 S" >
$GNUPGHOME/trustlist.txt
gpgsm --verify signed-smime-test.asc

Works for me now. Thanks again. -> resolved.

Thanks!
I'll test it. Any idea what could have caused this corruption in the first place?

Uh sorry, yes it terminates after over a minute. Sorry I should have waited
longer but 100% CPU for over a minute is quite a lot of calculations ;-).
Changed the title.

Windows does not allow file names with a '*'. I'm not sure on what level but Its
ok not to handle this case.
I don't expect any problems for internal usage. Keep in mind that this is a
regression, we had wildcard expansion before we made the switch to mingw-w64.

We also don't need this in gpgwrap as gpgwrap just passes the argument on and it
will be expanded in the process itself.

But I actually like the idea to do the wildcard expansion in kleowrap / gpgwrap.
This way it would be contained in Gpg4win and we catch all our "user exposed"
processes. Ok?

Well just gpg would be enough imho as this is by far the most prominent command
line tool.

On the other hand it might be more prudent for us to hack / patch it just in the
gpg4win build to have it enabled globally for all tools we ship so that it is
more consistent. This would mean patching the compiler tough which we tried to
avoid so far.

I would be fine with moving this patch to the version independet gnupg2 patches
in gpg4win as it is kind of a "distribution" option forced upon gpg4win by the
compiler we are currently using.

Werner: If you agree please give a short ping here and I'll move the patch /
close the issue.

No this was on "the master of the day"

And with the dead server detection the case for "localhost lookup" already got
better.

But you could look at npth src/npth.c
I am pretty sure that npth_enter and npth_leave modify errno and that this
causes at least npth_connect not to set errno as expected.

Additionally to T1665 (wk on Jul 03 2014, 11:13 AM / Roundup) (outlining that a trust path to the global SSL companies
is available and thus resolving this):

https://files.gpg4win.org is verified by a certificate that is available over
https://ssl.intevation.de/ this site is "verified" by one of the preinstalled
companies. (You are hopefully aware that you just have to send them some bucks
and some unsigned mails with an @intevation.de address claiming that you are
intevation.de to get such a certificate)

We also bought a certificate for codesigning so that in Windows itself you get
an assurance that one of the >100 Root CA's in their certificate program earned
some money from us ;-)

Please check the openpgp signatures or the checksums in our release
announcements and decide for yourself if you trust us. We can just buy your
trust otherwise.

This should have been resolved a long time ago. There was a KDE bug about this
but I can't find it anymore.

I had another go at this bug this evening. I had a keyserver with reproducable
failures (while I still could use it in gpg1). And suddenly during debugging it
all changed and worked flawlessly. I was down to npth_connect and after I had
added debug output in there it began to work (and kept working after removing
the debug output again, hrmpf)

With regards to the test case from T1773 (aheinecke on Nov 26 2014, 10:35 PM / Roundup). This now (after e8c0ed7 ) returns a
dead host.

Btw. I think the error message could be improved for dead hosts.

gpg2 --keyserver hkp://127.0.0.1 --search foobar
gpg: no keyserver known (use option --keyserver)
gpg: keyserver search failed: No keyserver available

Should be something like "No reachable keyserver found"

Assigned this bug to me to at least provide a clearer example.

Thanks for fixing the 127.0.0.1 lookup error :)

Should be fixed now.

The next version will no longer include the generated moc files.
It's not really a patch to backport (as you requested this in your mailing list).

In quilt you can just do something like:
quilt new remove-broken-moc-files.patch
quilt add qt4/*.moc
rm qt4/*.moc
quilt refresh

I also ran into this problem with our (intevation's) debian packaging.

Just removing the .moc files worked as they were correctly generated
automatically (as they should be).

I'll commit a fix not to include them in the dist package anymore.

Thanks!

Applied:
http://git.gnupg.org/cgi-bin/gitweb.cgi?p=pinentry.git;a=commit;h=484465338ea6f5f12eb1341ca0ef84aa79224ed4

Werner: I think the source tarball you distribute should be changed not to
include the .moc files as they depend on the qt version used when you generate
the tarballs.

MOC should be detected during configure (and otherwise pinentry-qt should be
disabled) and MOC should be used to generate the moc files. If this does not
work as expected this might have been caused by my limited Make and autotools
skills ;-)

confirmed

I've changed the category to gpa, adjusted the topic and version to 2.2.3
As you've already described the problem together with GPA here I think this is
better then opening a new bug.

I'll also no longer call this critical as the original data loss problem
(Encrypting files where one has an umlaut -> kleo thinks its a success and
deletes the original) Should be resolved.

The fix in GPA should be fairly easy. Some conversion from native to utf-8 on
input and utf-8 to native on output. So I'm taking this issue.

Werner: Could you please take a look at the patch for gpgtar. I will probably
propose something quite similar for GPA. Not real unicode support but at least
for 8 bit filenames.

As I wrote in T1774 (aheinecke on Nov 27 2014, 05:42 PM / Roundup)
I reproduced this Bug with 2.1.1-beta45