Thank you for your work on the proposal. I have two comments:

• Do we have some test vector, which can be used in the testsute to test the new API?
• We need to mention the new API in the documentation.

For the record, the changeset in the attached merge request is final and waiting for reviews.

Experimental branches:
https://dev.gnupg.org/source/libgcrypt/history/t4873/
https://dev.gnupg.org/source/ntbtls/history/t4873/

Should go into 1.10 too

It will be in 1.10.2.

It will be in 1.10.2.

It will be in 1.10.2.

Thanks. Applied. Also, fixed about a warning for ChaCha20.

Here's another one related to this: https://lists.gnupg.org/pipermail/gcrypt-devel/2022-July/005344.html

Applied and pushed.

The last patch is related to FIPS, so, I add the FIPS tag.

Key length requirements for KDFs are specified in SP 800-131Ar2 (https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-131Ar2.pdf), which is linked from SP 800-140Dr1 (https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-140Dr1.pdf) in section "6.2.1 Transitions".

FIPS 140-3 (https://csrc.nist.gov/Projects/cryptographic-module-validation-program/fips-140-3-standards) points to SP 800-140Dr1 (https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-140Dr1.pdf) to list acceptable "Security Parameter Generation and Establishment Methods". From this document, RFC 5869 (i.e., HKDF with the counter at the end) can be reached via two paths:

The change allows internal use of HMAC with shorter key.

Considering again, I concluded the patch above should be applied.
The use of SALT in HKDF may be not secret and there are valid use cases with no last or shorter salt. It's different to the use case of HMAC, where KEY is secret.

I pushed the change needed for GnuPG to t5964 branch.
See: https://dev.gnupg.org/rGc281bd94349e4f7997a89927aaa2c2f45004b902

Added HKDF implementation to master.

I can only find this one: https://github.com/patrickfav/singlestep-kdf/wiki/NIST-SP-800-56C-Rev1:-Non-Official-Test-Vectors

I learned that it's now called "OneStep KDF" in SP 800-56Cr2.
It's "SSKDF" in OpenSSL (Single Step KDF, perhaps).

In T5975#158113, @werner wrote:

I can imagine thar there are use cases for this. Thus I see no problems for the first part.

The second part is imho not a good idea. Libgcrypt is a building block for all kind of software and there are for sure legitimate reasons to use rsa512 (MCUs, short living keys, etc). Thus I think that the decision on the key size should be done by the software using libgcrypt.

Pushed the change (master and 1.10).

At first, we need to add/enhance new API for KDF in libgcrypt. Currently, the term "KDF" in libgcrypt is used with narrower focus, that is, only for password->key KDF.

Lets implement it for 2.3

Ok. Thank you for the clarification. I will drop the second part and keep only the FIPS change in the patch. Merge request already updated.

I can imagine thar there are use cases for this. Thus I see no problems for the first part.

When we implemented this first, Libgcrypt had no appropriate KDF support. I recall that I considered to change this but it turned out the for 2.2 the changes are too large. For 2.3 we will consider such a change.

Fixed in GnuPG 2.3.5.

Feedback from the lab is that they'd recommend returning a specific error code that indicates that the prime search failed and then relying on the caller to decide whether to loop or bubble up the error. I'm not sure who we would consider to be the "caller" of the relevant generation function in this case, though.

Full ack.

Here is my proposal patch:

diff --git a/random/random-drbg.c b/random/random-drbg.c
index 5a46fd92..f1cfe286 100644
--- a/random/random-drbg.c
+++ b/random/random-drbg.c
@@ -341,6 +341,9 @@ enum drbg_prefixes
  * Global variables
  ***************************************************************/

That sounds reasonable. The FIPS 186-5 draft (https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-5-draft.pdf) covers this in section A.1.3, although I'm not quite sure why a lower bound for p was chosen compared to q. The comment that seems to have triggered this change is published on page 68 of https://csrc.nist.gov/CSRC/media/Publications/fips/186/4/final/documents/comments-received-fips186-4-december-2015.pdf by Allen Roginsky. It only contains a suggestion of 20, presumably for both numbers.

I checked FIPS 186-4 (and FIPS 186-5-draft). It is Appendix A 1.3.

Passing fds etc adds complex extra code to gpg-agent. This was not the original design goal, although we violated this anyway by have some OpenPGP specific code there. This needs more thinking. Due to our internal use of OCB we can't make it FIPS compliant without large changes.

Patches applied and pushed. For the common/t-ssh-utils, I applied my fix for the use case with key on command line when FIPS mode is enabled (MD5 error is OK, in this case).

I was pointed by Daiki to the following patch in Fedora binutils, which allows listing the fdo packaging metadata, but it does not list any other unknown objects and unfortunately fails hard:

I just copied the value of 0xcafe2a8e and the name .note.fdo.integrity from Daiki's implementation. No other reason.

I have one follow-up is that the readelf chokes on the integrity note for some reason:

$ readelf -n /usr/lib64/libgcrypt.so.20.4.1
Displaying notes found in: .note.fdo.integrity
  Owner                Data size 	Description
  FDO                  0x00000020	Unknown note type: (0x8e2afeca)

I assume this is just because the readelf does not know this type. I see this type was initially proposed by Daiki, but I did not find any other sources for this magic number so before filling bugs for readelf, do we have some doc why the 0xcafe2a8e is used?

The set_bit is obvious but we should cross check with the specs. In the non-fips mode we also try w/o a limit.

I think that it is OK to loop forever until we find a prime.

Done in 1.10.1.

I had thought that we need to combine hkdf so that key and iv can generate within libgcrypt internally.
Probably, this assumption of mine may be wrong.

You are combining two concepts here -- the KDF and the AEAD cipher itself (at least from the FIPS terminology). I would like to avoid mixing these two together in the new API. If you would like to implement the SSH/TLS KDF, I would suggest to use the kdf API you already have. Then we are here left only with a new geniv API to implement. In the T4873 I mentioned example how it is now used in libssh using libgcrypt, which implements the iv increment outside of the libgcrypt:

Is large change to cipher API really needed (new open/encrypt with less flexibility)? How that would affect performance? Would following new interfaces to gcry_cipher API work instead?

• gcry_cipher_setup_geniv(hd, int ivlen, int method): for setting up IV generator with parameters such as IV length, method id (RFC5116, TLS 1.3, SSH, etc), (other parameters?)
• gcry_cipher_geniv(hd, byte *outiv): for generating new iv: generate IV using select method, set IV internally and output generated IV to 'ivout'.
• gcry_cipher_genkey(hd, byte *outkey, int keylen, int method): for generating keys, generate key internally with parameters (method id, other?), setup key internally and output generated key to 'outkey'. (how keys from key exchange protocol be handled? using existing setkey?)

I think this is not urgent as we are able to FIPS certify libgcrypt without that, but the modern protocols and algorithm use this and if we want to use libgcrypt to implement these in FIPS compliant way, we certainly need something like that.

I don't think it is justified to tag this as "unbreak now" - which we use for severe bugs inhibiting the use of a deployed version.

I simplified the script not to use cmp: rC3c8b6c4a9cad: fips: Fix gen-note-integrity.sh script not to use cmp utility.
And I clarified the semantics of the integrity check.

Ah, right, I can get that added to the containers tomorrow.

I located the cause:

../../src/gen-note-integrity.sh: line 78: cmp: command not found

That only seems to work in some configurations: https://gitlab.com/redhat-crypto/libgcrypt/libgcrypt-mirror/-/pipelines/472626834