Here are changes for gcry_md_open and its friends.

My idea in https://dev.gnupg.org/T7338#195529 doesn't work well when a function call is done multiple times.
Assuming SUCCESS, and marking all non-compliant places in the code works, and it would be good because libgcrypt so far maintains non-compliant path with rejection.

Pushed the change for adding hash tests in rC7faf542f1573: fips,tests: Add t-digest.

It seems that the internal API (as of 2024-12-06) is not enough.
Now, we have _gcry_md_hash_buffer function with the new FIPS service indicator.
It's used for public key crypto, too.
The compliance for hash function is a part of public key crypto, but not all.

A change for gcry_md_hash_* functions are pushed by rC3478caac62c7: fips,md: Implement new FIPS service indicator for gcry_md_hash_*..
It doesn't have tests with FIPS service indicator yet.

New external API is by GCRYCTL_FIPS_SERVICE_INDICATOR and/or the new macro gcry_get_fips_service_indicator.
This change is pushed by rCf51f4e98930e: fips: Introduce GCRYCTL_FIPS_SERVICE_INDICATOR and the macro.

New internal API is introduced with T7340 by the commit rCe1cf31232825: fips: Introduce an internal API for FIPS service indicator.

Change is pushed by rCe1cf31232825: fips: Introduce an internal API for FIPS service indicator.

I created a branch: https://dev.gnupg.org/source/libgcrypt/history/gniibe%252Ft7340/

Autoconf archive has AX_TLS: https://www.gnu.org/software/autoconf-archive/ax_tls.html
Also, AX_GCC_VAR_ATTRIBUTE(tls_model) could be used: https://www.gnu.org/software/autoconf-archive/ax_gcc_var_attribute.html

I think so. We did not submit a modules for recertification with these changes, but we do not plan this in close future so you can consider it completed.

Can we close this?

Applied the change. I write the ChangeLog entry by commit message.

The fix is in 1.10.3.

Fix is in 1.10.3.

Add the check of digest algorithm for EdDSA in: rCd15fe6aac10b: cipher:ecc:fips: Only allow defined digest algo for EdDSA.

No, there are use cases in GnuPG, where we specify the hash algo for signing, and our own tests/benchmark.c.

For the first issue, I added a check in: rCf65c30d470f5: cipher:ecc:fips: Reject use of SHAKE when it's ECDSA with RFC6979.

From the FIPS 186-5 there are some limitations to use the SHAKE in FIPS Mode that we will have to reflect:

Pushed a change in master.

I found the case of X.509, which also uses fixed length output for RSA-PSS and ECDSA: https://www.rfc-editor.org/rfc/rfc8692.html

Thank you for having a look into that! The proposed patch looks good. Should we have this change also in master?

Here is a possible change (... to master, assuming it's good to support use case of RFC 8702):

diff --git a/cipher/keccak.c b/cipher/keccak.c
index 22c40302..76e08cb5 100644
--- a/cipher/keccak.c
+++ b/cipher/keccak.c
@@ -1630,8 +1630,8 @@ const gcry_md_spec_t _gcry_digest_spec_sha3_512 =
 const gcry_md_spec_t _gcry_digest_spec_shake128 =
   {
     GCRY_MD_SHAKE128, {0, 1},
-    "SHAKE128", shake128_asn, DIM (shake128_asn), oid_spec_shake128, 0,
-    shake128_init, keccak_write, keccak_final, NULL, keccak_extract,
+    "SHAKE128", shake128_asn, DIM (shake128_asn), oid_spec_shake128, 32,
+    shake128_init, keccak_write, keccak_final, keccak_read, keccak_extract,
     _gcry_shake128_hash_buffers,
     sizeof (KECCAK_CONTEXT),
     run_selftests
@@ -1639,8 +1639,8 @@ const gcry_md_spec_t _gcry_digest_spec_shake128 =
 const gcry_md_spec_t _gcry_digest_spec_shake256 =
   {
     GCRY_MD_SHAKE256, {0, 1},
-    "SHAKE256", shake256_asn, DIM (shake256_asn), oid_spec_shake256, 0,
-    shake256_init, keccak_write, keccak_final, NULL, keccak_extract,
+    "SHAKE256", shake256_asn, DIM (shake256_asn), oid_spec_shake256, 64,
+    shake256_init, keccak_write, keccak_final, keccak_read, keccak_extract,
     _gcry_shake256_hash_buffers,
     sizeof (KECCAK_CONTEXT),
     run_selftests

Reading RFC 8702, I realized that it defines the hash size in the use of CMS as: SHAKE128 : 32-byte SHAKE256 : 64-byte.

Applied rC8cdd0d353e19: cipher:pubkey: Check digest size which should not be zero. for 1.10.