I'm going to write some documentation about the programmatic use of GnuPG.

Applied with commit 0a90f87799 to master. I will backport and release 1.7.5 today.

The Gentoo bug report for this has a
proposed fix, correcting a typo (EGAIN-

EAGAIN) in an autoconf script.

https://bugs.gentoo.org/show_bug.cgi?
id=602502#c5

@werner, have you looked into the patch?

AC_CHECK_FUNCS([mmap]) on Fedora fails to find mmap() due to missing -fPIC.
/usr/bin/ld: /tmp/ccvNyAcN.o: relocation R_X86_64_PC32 against undefined symbol
`mmap@@GLIBC_2.2.5' can not be used when making a shared object; recompile with
-fPIC

We have -fPIC somewhere in default CFLAGS, so just resotre user CFLAGS
before making checks for functions.

I'm glad that git has this fixed. Well, then the actual problem is that it is
broken in release.

Even being gentoo user, I cannot install gnupg from git easily (there's no live
ebuild for gnupg yet). So users will suffer from this until you make next
release and distros maintainers update packages.

So regarding functional tests for shell utils... Any suggestion how to arrange
that? Or would you review whatever I come up with?

I improved our test suite so that it detects this problem.

This is indeed a bug in libgcrypt. Thanks for the report.

/bin/pwd does produce the full name off the current directory, but it doesn't
canonicalize it, taking into account symlinks, the way $(readlink -f) does.
Thus, the comparisons may fail, and that's what happened in the case of our
build.

I just tried it with the current version from git and I see no real problems.
The only annoyance is that you need to enter the passpharse (or no passphrase)
for each subkey.

"Can you please compile gpg with debugging symbols"...

Sorry, I am not currently setup to compile GnuPG and all its dependencies and I'm not
even sure of the details as to how to go about doing so. As I mentioned I am installing
pre-compiled binaries compiled server side by the homebrew project which installs those
binaries.

I would imagine the GnuPG project has an OS X development machine to test/debug on. No?

If you have specific changes you would want me to make to the homebrew recipe I linked
to I can try to do that.

I just released Libgcrypt 1.7.4 - whcih should fix that bug.

No info recevied and thus closing. You should switch to 1.7 (and soon 1.8)
anyway. A lot of things have been fixed since 1.6.

FWIW, we do not see any problem with current Libgcrypt in our CUI system running
under Sierra.

Given that the patent expired I consider touching that comment not important.

Thanks for the feedback! Can you please compile gpg with debugging symbols, add
a break point on log_debug in string_to_ulong (in g10/tofu.c), and then do 'run
--verify ts.txt'. When you hit the breakpoint, please do a 'bt full', print out
the value of "string" and "tail" (using gdb's 'p' command), and repeat (continue
execution using 'c').

Thanks!

FYI, here is the homebrew formula that is used to compile GnuPG

https://github.com/Homebrew/homebrew-versions/blob/master/gnupg21.rb#L46

Hmmm. So since I filed this bug I happened to do a key transition so I started with a
brand new gnupg dir. So in trying to replicate this again I was starting from scratch.
I imported the key and downloaded the signed file from the gist I sent you. I still see
the same output! This leads me to wonder if there is something different about how the
tofu code compiles when installed on OS X via homebrew?? The gnupg installation didn't
change, but my whole .gnupg dir is new.

$ gpg2 --verify ts.txt
gpg: Signature made Wed Nov 23 23:08:29 2016 PST
gpg: using DSA key 0x6F3B2E6AB748A8F8
gpg: Good signature from "TrueTimeStamp <signing-department@TrueTimeStamp.org>"
[marginal]
gpg: DBG: tofu.c:2772: strtoul failed for DB returned string (tail=): Invalid argument
gpg: DBG: tofu.c:2774: strtoul failed for DB returned string (tail=): Invalid argument
gpg: signing-department@truetimestamp.org: Verified 1 signature in the past

0 seconds, and encrypted 0 messages.

gpg: Warning: we've only seen one message signed using this key and user id!
gpg: Warning: you have yet to encrypt a message to this key!
gpg: Warning: if you think you've seen more signatures by this key and user

id, then this key might be a forgery!  Carefully examine the email address
for small variations.  If the key is suspect, then use
  gpg --tofu-policy bad 83289060F40DED088CF246B56F3B2E6AB748A8F8
to mark it as being bad.

gpg: WARNING: This key is not certified with sufficiently trusted signatures!
gpg: It is not certain that the signature belongs to the owner.
Primary key fingerprint: 8328 9060 F40D ED08 8CF2 46B5 6F3B 2E6A B748 A8F8

In general a bug tracker is not a help line. Anyway, please describe you
environment (OS, NFS mounts etc) and give an exact description on what you did.

I tested with the GnuPG version 2.0.30 (GPG4WIn) as well as the current 2.1.16
Windows binaries. SCdaemon was running but was unable to get exclusive card access.
Why?
The Cisco Network Manager as well as Cisco Anyconnect VPN did both gain shared
card access (they were not told to do so!). I needed both programs to get access
to the university network.

Uninstalling both Programs and restarting did resolve the issue. To find the
two offenders I used Process Explorer (Processes for all users) and used the
Find Handle or DLL functon with the search term "SCARD". All crosschecked all
Processes (except for scdaemon which sould access the card) and Services
(svchost) to be only scdaemon aswell as the services to be Windows internal.
To determine the inital issue I used
https://sourceforge.net/projects/pcsctracker/ which told me the status of my
Yubikey (as Present,InUse -> Shared Access).

As a suggestion I like to see the experimental option to change the accessmode
from exclusive to shared on the commandline (If for example the other
application cannot be uninstalled).

Backported to LIBGCRYPT-1-7-BRANCH

I have now pushed a change to Libgcrypt master to implement auto-extending of
secre memory pools. Commit b6870cf but there are two cother commits which this
is based upon. My test shows that I can now decrypt a message encrypted to the
test-hugekey.key.

I will port this back to Libgcrypt 1.7.

Which version of GnuPG are you using? Do you have scdaemon?

I will try out the idea of extending the secmem pool even if that means no mlock.

ah right, "ulimit -l" says 64 (kbytes) on my Linux system as well. According to
mlock(2) that's since kernel 2.6.9.

So i think it's worth adopting the supplied patch as a workaround at least (i
can confirm that it resolves the specific use case described in T2857 (dkg on Dec 05 2016, 05:47 PM / Roundup)), and i
agree with you that we should extend libgcrypt to extend secure memory allocation.

it's not clear to me that swap is outside the trust boundary anyway these days,
and modern systems should prefer encrypted swap where possible.

The secmem has two goals:

• Avoid swapping out tehse pages. Thus the mlock.
• Making sure that on free the memory is zeroized.

mlock requires root privileges and thus a special init sequence is required
(install as setuid(root) and gpg-agent drops the privileges direct after
allocating and mlocking the secmem). In the old times, and probably still today
on non-Linux platforms, this is still required. However, Linux turned to
allowing any process to mlock a certain amount (64k on my box).

I tend to suggest that we extend Libgcrypt to extend the secure memory
allocation by not using mlocked memory but keeping the the seroization feature.
The second option from T2857 (wk on Dec 05 2016, 07:11 PM / Roundup).

is the only goal of the secure memory to keep the RAM from being written to
swap, or are there other goals of secure memory? why is it unlikely that a new
block of memory can be mlock'd? what are the consequences of the new block not
being mlock'd? will it still be treated as secure memory?

crashing in the event that we run out of secure memory is simply not acceptable
these days, especially in a model where we have persistent long-term daemons
that people expect to remain running.

I just posted 0001-agent-Respect-enable-large-secmem.patch to gnupg-devel:

https://lists.gnupg.org/pipermail/gnupg-devel/2016-December/032285.html

D400: 927_0001-agent-Respect-enable-large-secmem.patch

Thanks! I tried reproducing this issue with your tofu.db (using HEAD), but I
didn't see the warning:

$ gpg --verify /tmp/TrueTimeStamp-certificate-4793.txt
gpg: Signature made Thu 24 Nov 2016 08:08:29 AM CET
gpg: using DSA key 6F3B2E6AB748A8F8
gpg: Good signature from "TrueTimeStamp <signing-department@TrueTimeStamp.org>"
[marginal]
gpg: signing-department@truetimestamp.org: Verified 2 signatures in the past

12 days.  Encrypted 0 messages.

gpg: Warning: you have yet to encrypt a message to this key!
gpg: Warning: if you think you've seen more signatures by this key and user

id, then this key might be a forgery!  Carefully examine the email address
for small variations.  If the key is suspect, then use
  gpg --tofu-policy bad 83289060F40DED088CF246B56F3B2E6AB748A8F8
to mark it as being bad.

gpg: WARNING: This key is not certified with sufficiently trusted signatures!
gpg: It is not certain that the signature belongs to the owner.
Primary key fingerprint: 8328 9060 F40D ED08 8CF2 46B5 6F3B 2E6A B748 A8F8

Most likely, this is because when you verifies the message, the error was fixed.
Can you confirm this for me by trying to reproduce the error with your current
tofu.db? If there is no error, could you send me a copy of the tofu.db from
before the initial verification?

Thanks!

Already fixed in 4db9a425644dccaf81b51ebc97b32a9cc21941a4. Duplicate of T2848.

This is a duplicate of bug 2848

Yeah, I saw the Debian bug report. Unfortunately there is no easy
solution to this except for rejecting the use of large secret keys.

The problem here is that the big number library needs to allocate from
a limited secure memory region (32 KiB by default) and terminates on
allocation failure. I know that this is sub-optimal but we are doing
this for 19 years now. Checking for an error after each low-level big
number operation would make the code unreadable and will introduce
bugs. Ideas on what to do:

• On secure memory allocation failure, call the out-of-memory handler which may then free other memory (or purchase new memory). This can be done in the application.
• On secure memory allocation failure, allocate a new block of secure memory and allocate from that one. There are two disadvantages: a) It is unlikely that the new block can be mlock'd. b) A free will be a a little bit slower because it needs to check the list of secure memory blocks and not just one address range. The address range check is needed so that we can figure out whether the freed address is in the secmem range and needs to zeroed out. This requires a new Libgcrypt version, though no ABI change.
• We have a ./configure option --enable-large-secmem which sets 64k instead of 32k aside for the secmem. This is currently only used in gpg to enable gpg's --enable-large-rsa option. Given that in 2.1 we use the gpg-agent for the secret key operations we should have the same options in gpg-agent. However, it is only a kludge, but one we once agreed upon to silence some pretty vocal experts on key size.

I just tried with latest gpgol (beta 204) and it now seems to work. So bug is
solved already! :)

fwiw, i'm seeing this too, over at https://bugs.debian.org/846953 , for a user
with an insanely large (10240-bit) RSA key when it is locked with a passphrase.

I'm attaching such an example secret key (with passphrase "abc123"), and you can
trigger the crash with:

gpg --batch --yes --import test-hugekey.key
echo test | gpg -r 861A97D02D4EE690A125DCC156CC9789743D4A89

--encrypt --armor --trust-model=always --batch --yes --output data.gpg

            gpg --decrypt data.gpg

While i think it's fair to say that we need to have some limits on the sizes of
keys we can handle, gpg-agent should not crash when asked to deal with
extra-large keys, it should fail gracefully and return a sensible error code.

The only viable solution will be to export the key secret key after key
generation, append that to the %secring given file and delete the key from
gpg-agent's store. Recall that the agent needs to know the secret key so that
gpg is abale to create the self-signatures. Adding a dedicated cache for this
would complicate the gpg-agent code a lot.

Little update with latest 1.0.0 release:

– nothing new regarding pinentry-gnome3, still working nicely;
– nothing new regarding pinentry-gtk-2, but I know why it doesn’t take into
account my dead keys anymore and it’s not an issue on pinentry side;
– thanks to the new “show passphrase” button, I’ve been able to figure where the
issue lies with pinentry-qt: while invoked in the terminal, it does take into
account my dead keys, but while invoked via Thunderbird/Enigmail, it does not
(altought pinentry-gnome3 does).

So I suppose this is in fact an issue with Enigmail… Any hints on what they
could be doing wrong so that I can report this to them?

It's been a year since last update. Still an issue. Maybe someone should send
an email to gnupg-commits-owner@gnupg.org ?

tofu.db sent via encrypted email today.

In general, parallel operations aren't great, but I find that such bad
performance surprising.

If you update a key, only that key's effective policy is rechecked, not all
keys. But, the effective policy of conflicting keys is always rechecked.

Hi,

I think that your assumption is that the local keyring is somehow trusted. In

that case, I think it make sense that deleted keys would clear conflicts.

No, not really I don't think trust plays a role here. It's just a way I think
users may resolve conflicts when they don't know about policies or how things
work internally.

I'm curious when you think people delete keys. My intuition is that it is not

a very common pattern. Do you have any thoughts on this?

As an example: You get a new lock in your front door. Would you remove the key
for the old lock from your keyring or would you paint the old one red as a
marker not to use the old key.

I know that this is not totally applicable because the old key can still be used
for verification etc. But I think that this is the intuitive behavior if a key
changes.

Maybe if GUI offers conflict resolution better this might not be such a big deal
but currently (Kleopatra does not yet have conflict resolution) but for me
during tofu testing I thought I could resolve a the conflict by deleting one of
the keys and found the behavior unexpected.

I encourage you to first try and find a consensus before implementing a

different policy at the higher level.

Indeed. Let's try :-)

No need to apologize for the dup; I was just noting it here for the record.

I think that your assumption is that the local keyring is somehow trusted. In
that case, I think it make sense that deleted keys would clear conflicts.

I'm curious when you think people delete keys. My intuition is that it is not a
very common pattern. Do you have any thoughts on this?

I encourage you to first try and find a consensus before implementing a
different policy at the higher level.

Sorry for the duplicated bug. Although the other issue is older I got more
response here so I keep the discussion here.

In my Optionion it's completely natural for a User to think (I thought this):

• Oh I have two keys that are in conflict: I'll delete the bad one then I don't

have a conflict anymore.

This is very intuitive behavior.

I'm not looking for a solution that works for me but for a solution that I think
would work for other users.

So for me your response ("what you should do") would mean that in Kleopatra on
Key deletion I would need to check for conflicting keys and change their policy
to auto again. Maybe even mark the deleted key as bad before deletion. I would
much prefer it if GnuPG handled this. For me it seems just like an unhandled
state as the error messages also indicate. "Key not found" etc so It's a bug or
maybe missing feature / functionality.

Fwiw I don't see how this can be consistent with WoT behavior as I don't think
WoT has a comparable problem. Can you explain what a comparable problem in WoT is?

If you meant hat the validity of all keys is not updated immediately after key
deletion, and you had some ownertrust to the deleted key ok yes thats probably
also another issue. :-)

Duplicate of T2859

Thanks for reporting this! Unfortunately, I'm not able to reproduce this. I
hope you can help me figure out what is wrong. Would you be willing to share
your tofu.db with me? Feel free to send it to me directly
(8F17777118A33DDA9BA48E62AACB3243630052D9); it contains some privacy sensitive
information (namely, who you communicate with).

Thanks!

This issue has also been reported in https://bugs.gnupg.org/gnupg/Issue2859

Werner replied there and I agree with his conclusion.

Note: this is a dup of T2742

I tend to agree with Werner: if we discover a conflict, it needs to be resolved
and deleting a key is not a sufficient resolution.

Another user reported the same problem on IRC. It seems it is Arch Linux
specific but we don't known for sure. The latest test with re-building
Libgcrypt w/o any special options didn't changed anything.

I need top be able to replicate the problem before I can come up with a solution.

That is consistent with the WoT behaviour. Deleting a key is no solution to a
faked key. It might be re-imported as any time.

What you should do instead is to disable the key so that it won't be used again.

While testing with tofu enabled I sometimes see that some actions take very
long. (>1minute)

Like importing a key in Kleopatra where Kleopatra does an import and starts a
keylist afterwards / in parallel.

I'll try to reproduce this on the command line. Just doing a simple import on
the command line is quick.

Do you have any hint what can take so long?
Like a trigger that would cause a rechecks for cross signatures?