--quick-keygen fixed in dd3dde07a9a46130ac01d849f8edf0566e44f11f.

The default expiration interval has been discussed on the mailing list. There
was a rough consensus on two years, which has been challenged by Neal who thinks
it is too short given the current state of the tools, but the ensuing discussion
did not revolve around the time span, so I'm keeping my two years for now. In
any case, it is easy to adjust.

I decided to not change the --full-key-gen, because a) the user asked for it, b)
changing that requires breaking up a large chunk of translated text, and I do
not want to do that right now (a release is imminent).

I just tried it with the current version from git and I see no real problems.
The only annoyance is that you need to enter the passpharse (or no passphrase)
for each subkey.

"Can you please compile gpg with debugging symbols"...

Sorry, I am not currently setup to compile GnuPG and all its dependencies and I'm not
even sure of the details as to how to go about doing so. As I mentioned I am installing
pre-compiled binaries compiled server side by the homebrew project which installs those
binaries.

I would imagine the GnuPG project has an OS X development machine to test/debug on. No?

If you have specific changes you would want me to make to the homebrew recipe I linked
to I can try to do that.

I just released Libgcrypt 1.7.4 - whcih should fix that bug.

Partially addressed in d568a1561642ed9b7b7b6282b86c56786d10a956.

Thanks for the feedback! Can you please compile gpg with debugging symbols, add
a break point on log_debug in string_to_ulong (in g10/tofu.c), and then do 'run
--verify ts.txt'. When you hit the breakpoint, please do a 'bt full', print out
the value of "string" and "tail" (using gdb's 'p' command), and repeat (continue
execution using 'c').

Thanks!

This would emit the "content-encryption key", as specified in
https://tools.ietf.org/html/rfc5652#section-6.3

FYI, here is the homebrew formula that is used to compile GnuPG

https://github.com/Homebrew/homebrew-versions/blob/master/gnupg21.rb#L46

Hmmm. So since I filed this bug I happened to do a key transition so I started with a
brand new gnupg dir. So in trying to replicate this again I was starting from scratch.
I imported the key and downloaded the signed file from the gist I sent you. I still see
the same output! This leads me to wonder if there is something different about how the
tofu code compiles when installed on OS X via homebrew?? The gnupg installation didn't
change, but my whole .gnupg dir is new.

$ gpg2 --verify ts.txt
gpg: Signature made Wed Nov 23 23:08:29 2016 PST
gpg: using DSA key 0x6F3B2E6AB748A8F8
gpg: Good signature from "TrueTimeStamp <signing-department@TrueTimeStamp.org>"
[marginal]
gpg: DBG: tofu.c:2772: strtoul failed for DB returned string (tail=): Invalid argument
gpg: DBG: tofu.c:2774: strtoul failed for DB returned string (tail=): Invalid argument
gpg: signing-department@truetimestamp.org: Verified 1 signature in the past

0 seconds, and encrypted 0 messages.

gpg: Warning: we've only seen one message signed using this key and user id!
gpg: Warning: you have yet to encrypt a message to this key!
gpg: Warning: if you think you've seen more signatures by this key and user

id, then this key might be a forgery!  Carefully examine the email address
for small variations.  If the key is suspect, then use
  gpg --tofu-policy bad 83289060F40DED088CF246B56F3B2E6AB748A8F8
to mark it as being bad.

gpg: WARNING: This key is not certified with sufficiently trusted signatures!
gpg: It is not certain that the signature belongs to the owner.
Primary key fingerprint: 8328 9060 F40D ED08 8CF2 46B5 6F3B 2E6A B748 A8F8

Okay, I implemented --status-fd for gpg-wks-client.

Regarding a return code as text lines: Do you need this due to the double-fork
we use in gpgme?

I think so, at least I did not find a way to return an exit code from
gpgme_op_spawn.

If we provide this we should resort to the GnuPG standard
which is to required --status-fd N to print
[GNUPG:] ERROR ....
okay?

Yes. In that case i could use op_spawn with status-fd 2 and would get the error
I think.

Okay, try "gpg-wks-cleint --check ADDR" So see details use -v.

Regarding a return code as text lines: Do you need this due to the double-fork
we use in gpgme? If we provide this we should resort to the GnuPG standard which is
to required --status-fd N to print
[GNUPG:] ERROR ....
okay?

In general a bug tracker is not a help line. Anyway, please describe you
environment (OS, NFS mounts etc) and give an exact description on what you did.

I tested with the GnuPG version 2.0.30 (GPG4WIn) as well as the current 2.1.16
Windows binaries. SCdaemon was running but was unable to get exclusive card access.
Why?
The Cisco Network Manager as well as Cisco Anyconnect VPN did both gain shared
card access (they were not told to do so!). I needed both programs to get access
to the university network.

Uninstalling both Programs and restarting did resolve the issue. To find the
two offenders I used Process Explorer (Processes for all users) and used the
Find Handle or DLL functon with the search term "SCARD". All crosschecked all
Processes (except for scdaemon which sould access the card) and Services
(svchost) to be only scdaemon aswell as the services to be Windows internal.
To determine the inital issue I used
https://sourceforge.net/projects/pcsctracker/ which told me the status of my
Yubikey (as Present,InUse -> Shared Access).

As a suggestion I like to see the experimental option to change the accessmode
from exclusive to shared on the commandline (If for example the other
application cannot be uninstalled).

Backported to LIBGCRYPT-1-7-BRANCH

I have now pushed a change to Libgcrypt master to implement auto-extending of
secre memory pools. Commit b6870cf but there are two cother commits which this
is based upon. My test shows that I can now decrypt a message encrypted to the
test-hugekey.key.

I will port this back to Libgcrypt 1.7.

Which version of GnuPG are you using? Do you have scdaemon?

Would you mind to write to gnupg-devel and ask for comments on your proposal?
In particular on how long the default expiration time shall be. 12, 18, or 24
months?

I will try out the idea of extending the secmem pool even if that means no mlock.

ah right, "ulimit -l" says 64 (kbytes) on my Linux system as well. According to
mlock(2) that's since kernel 2.6.9.

So i think it's worth adopting the supplied patch as a workaround at least (i
can confirm that it resolves the specific use case described in T2857 (dkg on Dec 05 2016, 05:47 PM / Roundup)), and i
agree with you that we should extend libgcrypt to extend secure memory allocation.

it's not clear to me that swap is outside the trust boundary anyway these days,
and modern systems should prefer encrypted swap where possible.

The secmem has two goals:

• Avoid swapping out tehse pages. Thus the mlock.
• Making sure that on free the memory is zeroized.

mlock requires root privileges and thus a special init sequence is required
(install as setuid(root) and gpg-agent drops the privileges direct after
allocating and mlocking the secmem). In the old times, and probably still today
on non-Linux platforms, this is still required. However, Linux turned to
allowing any process to mlock a certain amount (64k on my box).

I tend to suggest that we extend Libgcrypt to extend the secure memory
allocation by not using mlocked memory but keeping the the seroization feature.
The second option from T2857 (wk on Dec 05 2016, 07:11 PM / Roundup).

is the only goal of the secure memory to keep the RAM from being written to
swap, or are there other goals of secure memory? why is it unlikely that a new
block of memory can be mlock'd? what are the consequences of the new block not
being mlock'd? will it still be treated as secure memory?

crashing in the event that we run out of secure memory is simply not acceptable
these days, especially in a model where we have persistent long-term daemons
that people expect to remain running.

I just posted 0001-agent-Respect-enable-large-secmem.patch to gnupg-devel:

https://lists.gnupg.org/pipermail/gnupg-devel/2016-December/032285.html

D400: 927_0001-agent-Respect-enable-large-secmem.patch

Thanks! I tried reproducing this issue with your tofu.db (using HEAD), but I
didn't see the warning:

$ gpg --verify /tmp/TrueTimeStamp-certificate-4793.txt
gpg: Signature made Thu 24 Nov 2016 08:08:29 AM CET
gpg: using DSA key 6F3B2E6AB748A8F8
gpg: Good signature from "TrueTimeStamp <signing-department@TrueTimeStamp.org>"
[marginal]
gpg: signing-department@truetimestamp.org: Verified 2 signatures in the past

12 days.  Encrypted 0 messages.

gpg: Warning: you have yet to encrypt a message to this key!
gpg: Warning: if you think you've seen more signatures by this key and user

id, then this key might be a forgery!  Carefully examine the email address
for small variations.  If the key is suspect, then use
  gpg --tofu-policy bad 83289060F40DED088CF246B56F3B2E6AB748A8F8
to mark it as being bad.

gpg: WARNING: This key is not certified with sufficiently trusted signatures!
gpg: It is not certain that the signature belongs to the owner.
Primary key fingerprint: 8328 9060 F40D ED08 8CF2 46B5 6F3B 2E6A B748 A8F8

Most likely, this is because when you verifies the message, the error was fixed.
Can you confirm this for me by trying to reproduce the error with your current
tofu.db? If there is no error, could you send me a copy of the tofu.db from
before the initial verification?

Thanks!

Already fixed in 4db9a425644dccaf81b51ebc97b32a9cc21941a4. Duplicate of T2848.

This is a duplicate of bug 2848

Yeah, I saw the Debian bug report. Unfortunately there is no easy
solution to this except for rejecting the use of large secret keys.

The problem here is that the big number library needs to allocate from
a limited secure memory region (32 KiB by default) and terminates on
allocation failure. I know that this is sub-optimal but we are doing
this for 19 years now. Checking for an error after each low-level big
number operation would make the code unreadable and will introduce
bugs. Ideas on what to do:

• On secure memory allocation failure, call the out-of-memory handler which may then free other memory (or purchase new memory). This can be done in the application.
• On secure memory allocation failure, allocate a new block of secure memory and allocate from that one. There are two disadvantages: a) It is unlikely that the new block can be mlock'd. b) A free will be a a little bit slower because it needs to check the list of secure memory blocks and not just one address range. The address range check is needed so that we can figure out whether the freed address is in the secmem range and needs to zeroed out. This requires a new Libgcrypt version, though no ABI change.
• We have a ./configure option --enable-large-secmem which sets 64k instead of 32k aside for the secmem. This is currently only used in gpg to enable gpg's --enable-large-rsa option. Given that in 2.1 we use the gpg-agent for the secret key operations we should have the same options in gpg-agent. However, it is only a kludge, but one we once agreed upon to silence some pretty vocal experts on key size.

fwiw, i'm seeing this too, over at https://bugs.debian.org/846953 , for a user
with an insanely large (10240-bit) RSA key when it is locked with a passphrase.

I'm attaching such an example secret key (with passphrase "abc123"), and you can
trigger the crash with:

gpg --batch --yes --import test-hugekey.key
echo test | gpg -r 861A97D02D4EE690A125DCC156CC9789743D4A89

--encrypt --armor --trust-model=always --batch --yes --output data.gpg

            gpg --decrypt data.gpg

While i think it's fair to say that we need to have some limits on the sizes of
keys we can handle, gpg-agent should not crash when asked to deal with
extra-large keys, it should fail gracefully and return a sensible error code.

Thanks!

--quick-set-expire now available.

I'll take the --quick-set-expire command. -wk

The only viable solution will be to export the key secret key after key
generation, append that to the %secring given file and delete the key from
gpg-agent's store. Recall that the agent needs to know the secret key so that
gpg is abale to create the self-signatures. Adding a dedicated cache for this
would complicate the gpg-agent code a lot.

That is becuase we consider a mail address to be a unique indentifier and thus
and algorithm to figure out the best matching makes sense. Other kinds of user
IDS not not need to be unique and should at best return an ambigious key error.
Well, expired keys and such should be sorted out, though.

tofu.db sent via encrypted email today.

In general, parallel operations aren't great, but I find that such bad
performance surprising.

If you update a key, only that key's effective policy is rechecked, not all
keys. But, the effective policy of conflicting keys is always rechecked.

Hi,

I think that your assumption is that the local keyring is somehow trusted. In

that case, I think it make sense that deleted keys would clear conflicts.

No, not really I don't think trust plays a role here. It's just a way I think
users may resolve conflicts when they don't know about policies or how things
work internally.

I'm curious when you think people delete keys. My intuition is that it is not

a very common pattern. Do you have any thoughts on this?

As an example: You get a new lock in your front door. Would you remove the key
for the old lock from your keyring or would you paint the old one red as a
marker not to use the old key.

I know that this is not totally applicable because the old key can still be used
for verification etc. But I think that this is the intuitive behavior if a key
changes.

Maybe if GUI offers conflict resolution better this might not be such a big deal
but currently (Kleopatra does not yet have conflict resolution) but for me
during tofu testing I thought I could resolve a the conflict by deleting one of
the keys and found the behavior unexpected.

I encourage you to first try and find a consensus before implementing a

different policy at the higher level.

Indeed. Let's try :-)

No need to apologize for the dup; I was just noting it here for the record.

I think that your assumption is that the local keyring is somehow trusted. In
that case, I think it make sense that deleted keys would clear conflicts.

I'm curious when you think people delete keys. My intuition is that it is not a
very common pattern. Do you have any thoughts on this?

I encourage you to first try and find a consensus before implementing a
different policy at the higher level.

Sorry for the duplicated bug. Although the other issue is older I got more
response here so I keep the discussion here.

In my Optionion it's completely natural for a User to think (I thought this):

• Oh I have two keys that are in conflict: I'll delete the bad one then I don't

have a conflict anymore.

This is very intuitive behavior.

I'm not looking for a solution that works for me but for a solution that I think
would work for other users.

So for me your response ("what you should do") would mean that in Kleopatra on
Key deletion I would need to check for conflicting keys and change their policy
to auto again. Maybe even mark the deleted key as bad before deletion. I would
much prefer it if GnuPG handled this. For me it seems just like an unhandled
state as the error messages also indicate. "Key not found" etc so It's a bug or
maybe missing feature / functionality.

Fwiw I don't see how this can be consistent with WoT behavior as I don't think
WoT has a comparable problem. Can you explain what a comparable problem in WoT is?

If you meant hat the validity of all keys is not updated immediately after key
deletion, and you had some ownertrust to the deleted key ok yes thats probably
also another issue. :-)

Duplicate of T2859

Thanks for reporting this! Unfortunately, I'm not able to reproduce this. I
hope you can help me figure out what is wrong. Would you be willing to share
your tofu.db with me? Feel free to send it to me directly
(8F17777118A33DDA9BA48E62AACB3243630052D9); it contains some privacy sensitive
information (namely, who you communicate with).

Thanks!

This issue has also been reported in https://bugs.gnupg.org/gnupg/Issue2859

Werner replied there and I agree with his conclusion.

Note: this is a dup of T2742

I tend to agree with Werner: if we discover a conflict, it needs to be resolved
and deleting a key is not a sufficient resolution.

Another user reported the same problem on IRC. It seems it is Arch Linux
specific but we don't known for sure. The latest test with re-building
Libgcrypt w/o any special options didn't changed anything.

I need top be able to replicate the problem before I can come up with a solution.

That is consistent with the WoT behaviour. Deleting a key is no solution to a
faked key. It might be re-imported as any time.

What you should do instead is to disable the key so that it won't be used again.

While testing with tofu enabled I sometimes see that some actions take very
long. (>1minute)

Like importing a key in Kleopatra where Kleopatra does an import and starts a
keylist afterwards / in parallel.

I'll try to reproduce this on the command line. Just doing a simple import on
the command line is quick.

Do you have any hint what can take so long?
Like a trigger that would cause a rechecks for cross signatures?

On Thu, Dec 1, 2016 at 10:33 AM, Werner Koch via BTS <gnupg@bugs.g10code.com

wrote:

Thanks for your fast reply.
Sadly I have not much time these days... but I have done what you suggested.
Honestly the log files dont tell me much. One thing I recognised is sometimes the logfiles end with "Fatal: libgcrypt
problem: out of core in secure memory" and sometimes they dont (I have not copied every log file here, this might
be to much).

This is what I got with debug memstat and the recent version of gnupg in Arch.

2016-11-30 21:18:35 gpg-agent[5516] listening on socket '/run/user/1000/gnupg/S.gpg-agent'
2016-11-30 21:18:35 gpg-agent[5517] gpg-agent (GnuPG) 2.1.15 started
2016-11-30 21:18:45 gpg-agent[5517] handler 0x7efe1d0e5700 for fd 5 started
2016-11-30 21:18:45 gpg-agent[5517] handler 0x7efe1c8e4700 for fd 6 started
2016-11-30 21:18:45 gpg-agent[5517] starting a new PIN Entry
2016-11-30 21:18:45 gpg-agent[5517] handler 0x7efe17fff700 for fd 8 started
2016-11-30 21:18:45 gpg-agent[5517] handler 0x7efe177fe700 for fd 9 started
2016-11-30 21:18:51 gpg-agent[5517] handler 0x7efe1d0e5700 for fd 5 terminated
2016-11-30 21:18:52 gpg-agent[5517] Fatal: out of core in secure memory while allocating 512 bytes

2016-11-30 21:18:52 gpg-agent[5517] Fatal: libgcrypt problem: out of core in secure memory
2016-11-30 21:18:35 gpg-agent[5516] listening on socket '/run/user/1000/gnupg/S.gpg-agent'
2016-11-30 21:18:35 gpg-agent[5517] gpg-agent (GnuPG) 2.1.15 started
2016-11-30 21:18:45 gpg-agent[5517] handler 0x7efe1d0e5700 for fd 5 started
2016-11-30 21:18:45 gpg-agent[5517] handler 0x7efe1c8e4700 for fd 6 started
2016-11-30 21:18:45 gpg-agent[5517] starting a new PIN Entry
2016-11-30 21:18:45 gpg-agent[5517] handler 0x7efe17fff700 for fd 8 started
2016-11-30 21:18:45 gpg-agent[5517] handler 0x7efe177fe700 for fd 9 started
2016-11-30 21:18:51 gpg-agent[5517] handler 0x7efe1d0e5700 for fd 5 terminated
2016-11-30 21:18:52 gpg-agent[5517] Fatal: out of core in secure memory while allocating 512 bytes

2016-11-30 21:18:52 gpg-agent[5517] Fatal: libgcrypt problem: out of core in secure memory
v2016-11-30 21:18:35 gpg-agent[5516] listening on socket '/run/user/1000/gnupg/S.gpg-agent'
2016-11-30 21:18:35 gpg-agent[5517] gpg-agent (GnuPG) 2.1.15 started
2016-11-30 21:18:45 gpg-agent[5517] handler 0x7efe1d0e5700 for fd 5 started
2016-11-30 21:18:45 gpg-agent[5517] handler 0x7efe1c8e4700 for fd 6 started
2016-11-30 21:18:45 gpg-agent[5517] starting a new PIN Entry
2016-11-30 21:18:45 gpg-agent[5517] handler 0x7efe17fff700 for fd 8 started
2016-11-30 21:18:45 gpg-agent[5517] handler 0x7efe177fe700 for fd 9 started
2016-11-30 21:18:51 gpg-agent[5517] handler 0x7efe1d0e5700 for fd 5 terminated
2016-11-30 21:18:52 gpg-agent[5517] Fatal: out of core in secure memory while allocating 512 bytes

2016-11-30 21:18:52 gpg-agent[5517] Fatal: libgcrypt problem: out of core in secure memory

This is what I got with debuging memstat and downgraded gnupg to gnupg-2.1.15-2-x86_64.pkg.tar.xz

2016-11-30 21:28:30 gpg-agent[5953] listening on socket '/run/user/1000/gnupg/S.gpg-agent'
2016-11-30 21:28:30 gpg-agent[5954] gpg-agent (GnuPG) 2.1.15 started
2016-11-30 21:28:37 gpg-agent[5954] handler 0x7fd6fedca700 for fd 5 started
2016-11-30 21:28:37 gpg-agent[5954] starting a new PIN Entry
2016-11-30 21:28:42 gpg-agent[5954] handler 0x7fd6fedca700 for fd 5 terminated
2016-11-30 21:28:47 gpg-agent[5954] handler 0x7fd6fedca700 for fd 5 started
2016-11-30 21:28:47 gpg-agent[5954] handler 0x7fd6fe5c9700 for fd 7 started
2016-11-30 21:28:47 gpg-agent[5954] handler 0x7fd6fddc8700 for fd 8 started
2016-11-30 21:28:47 gpg-agent[5954] handler 0x7fd6fd5c7700 for fd 9 started
2016-11-30 21:28:47 gpg-agent[5954] handler 0x7fd6fedca700 for fd 5 terminated
2016-11-30 21:28:47 gpg-agent[5954] handler 0x7fd6fddc8700 for fd 8 terminated
2016-11-30 21:28:47 gpg-agent[5954] handler 0x7fd6fe5c9700 for fd 7 terminated
2016-11-30 21:28:47 gpg-agent[5954] handler 0x7fd6fd5c7700 for fd 9 terminated
2016-11-30 21:29:32 gpg-agent[5954] handler 0x7fd6fe5c9700 for fd 7 started
2016-11-30 21:29:32 gpg-agent[5954] handler 0x7fd6fe5c9700 for fd 7 terminated
2016-11-30 21:30:10 gpg-agent[5954] handler 0x7fd6fe5c9700 for fd 5 started
2016-11-30 21:30:10 gpg-agent[5954] handler 0x7fd6fd5c7700 for fd 7 started
2016-11-30 21:30:10 gpg-agent[5954] handler 0x7fd6fddc8700 for fd 8 started
2016-11-30 21:30:10 gpg-agent[5954] handler 0x7fd6fedca700 for fd 9 started
2016-11-30 21:30:10 gpg-agent[5954] Fatal: out of core in secure memory while allocating 512 bytes

2016-11-30 21:30:10 gpg-agent[5954] Fatal: libgcrypt problem: out of core in secure memory

And this is what I got from gdb (I am quite unfamiliar with gdb, so maybe i done something wrong)

Reading symbols from gpg-agent...(no debugging symbols found)...done.
Attaching to program: /usr/bin/gpg-agent, process 3492
Reading symbols from /usr/lib/libgcrypt.so.20...(no debugging symbols found)...done.
Reading symbols from /usr/lib/libgpg-error.so.0...(no debugging symbols found)...done.
Reading symbols from /usr/lib/libassuan.so.0...(no debugging symbols found)...done.
Reading symbols from /usr/lib/libnpth.so.0...(no debugging symbols found)...done.
Reading symbols from /usr/lib/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Reading symbols from /usr/lib/libc.so.6...(no debugging symbols found)...done.
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
0x00007f05452cd18c in pselect () from /usr/lib/libc.so.6
(gdb) break log_fatal
Function "log_fatal" not defined.
Make breakpoint pending on future shared library load? (y or [n])
(gdb) c
Continuing.
[New Thread 0x7f053ffff700 (LWP 4687)]
[New Thread 0x7f05449ea700 (LWP 4698)]
[New Thread 0x7f053f7fe700 (LWP 4699)]
[New Thread 0x7f05451eb700 (LWP 4700)]
[Thread 0x7f053ffff700 (LWP 4687) exited]
[Thread 0x7f053f7fe700 (LWP 4699) exited]
[Thread 0x7f05451eb700 (LWP 4700) exited]
[Thread 0x7f05449ea700 (LWP 4698) exited]
[New Thread 0x7f05449ea700 (LWP 4733)]
[New Thread 0x7f05451eb700 (LWP 4745)]
[New Thread 0x7f053f7fe700 (LWP 4746)]
[New Thread 0x7f053ffff700 (LWP 4747)]
[Thread 0x7f053f7fe700 (LWP 4746) exited]
[Thread 0x7f05449ea700 (LWP 4733) exited]
[Thread 0x7f05451eb700 (LWP 4745) exited]
[Thread 0x7f053ffff700 (LWP 4747) exited]
[New Thread 0x7f053ffff700 (LWP 4775)]
[New Thread 0x7f05451eb700 (LWP 4776)]
[Thread 0x7f053ffff700 (LWP 4775) exited]

Thread 11 "gpg-agent" received signal SIGPIPE, Broken pipe.
[Switching to Thread 0x7f05451eb700 (LWP 4776)]

0x00007f054559a16d in write () from /usr/lib/libpthread.so.0

This should be fixed in: 2f27cb12e30c9f6e780354eecc3ff0039ed52c63 .

Applied to 2.0, too. Will be in 2.0.31.

Fixed in 2.1.16.