Tested the master on (faked) FIPS and non-FIPS Fedora and I created couple of more changes for master to work in FIPS mode:

Ah, I understand the point (at least, partially); My understanding is: With FIPS mode, at the module boundary (== libgcrypt), it ensures that all cipher/digest/etc. operations are done under the standard compliance, and it is considered wrong (violation) when non-FIPS mode operation (such as SHA-1) and FIPS mode operation are mixed.

In RHEL, we do not have anything about PCT so the PCT requirement is not completely clear to me: https://git.centos.org/rpms/libgcrypt/blob/c8s/f/SOURCES

SUSE has patches and version 3235 of cavs_driver.pl, bud it seems that it doesn't support DSA with Q+HASHALGO yet.

As a start, I applied your patches.

And... as long as I read the PCT patches, it is not needed to export those API to users.
It is only needed internally for PCT tests (at most).

I am considering API enhancement, for this task.

I went through the patches above + what I suggested in previous comments, tested everything against both upstream and libgcrypt in Fedora in FIPS mode. There were slight differences, some cases were already fixed in master, some needed to upstream some of our changes, but the result is 10 patches working in both FIPS and non-fips mode, hopefully enough annotated. If not, please, ask for clarifications.

I went through the OpenSSL drafts. The module boundary in OpenSSL will be separate fips.so object and only non-deprecated functions of OpenSSL 3.0 will be FIPS compliant. There is a global state, that will allow only approved algorithms and modes and there will be API to query the FIPS mode status using OSSL_PARAM_get* functions, but we still have some unknowns so I hope we will know more on the next meeting.

Just FYI, NSS offers following API:

I was so far testing with changes on top of our patches.

With `/etc/gcrypt/fips_enabled/', make check fails by:

Update: still ./basic --fips fails (for me), because of GCM (18 errors).
Need to fix T4873: Enable AES GCM in FIPS mode.

That crcalgo can be any digest algorithm and SHA256 seems best option to me.

Thank you for checking and for revised patch. I tested your patch and it works fine for the basic test up until this failure with the crcalgo:

basic: algo 316, crcalgo: 3, gcry_md_open failed: Invalid digest algorithm
basic: algo 317, crcalgo: 3, gcry_md_open failed: Invalid digest algorithm

These are GCRY_MD_SHAKE128 and GCRY_MD_SHAKE256, but the md used here is actually GCRY_MD_RMD160 which is hardcoded and not allowed in FIPS.

That reminds me that we we should replace libgcrypt's internal debug functions by those from gpgrt. We have a dependency for gpgrt anyway and thus we should avoid code duplication. Sure we will keep the existsing public functions but that is easy given that gpgrt comes with gpgrt_logv since 1.28 which we can make mandatory (currently libgcrypt requires 1.27 (from 2017, with 1.28 is from 2018)

I applied rC297d31294333: tests: Fix messages to STDERR when FIPS mode is enabled.. Please note that your intention to change check_digests is right, but your patch actually didn't; When a MD algo is not supported, gcry_md_test_algo returns != 0 (an error code), and it "continues" to next entry (before the change).

Thank you for your report.

Implementation Guidance for FIPS 140-3 and the Cryptographic Module Validation Program:
https://csrc.nist.gov/CSRC/media/Projects/cryptographic-module-validation-program/documents/fips%20140-3/FIPS%20140-3%20IG.pdf

Some ideas:

• the someflags thing will probably just be a reserved parameter
• If DATA is not NULL but an MD is set the sign function should fail
• Should ownership of MD be moved to the CTX?

In an email from @werner couple days back, I got a suggestion that we could use hashing tied to the context, rather than this one-shot call tied only to digests. I circled back this suggestion to Stephan and he confirmed that it should be fine from the FIPS point of view so I am posting the suggested API here too:

ctx = gcry_pk_new (someflags)
md = gcry_md_open (...)
gcry_ctx_set_md (md);
gcry_pk_sign_ext (ctx, result, data, skey)
[...]
gcry_ctx_release (ctx);

OK. I think that the patch at SUSE is updated one which works.
As I understand correctly, this is a kind of very old patch, which intended to work around old libgcrypt limitation of RSA PSS.

I think that {D1476} is still a sketch (not real code which works). I would guess an intended use, but it's good to have concrete example program which uses the feature being added.

FWIW, there is also this newer patch: https://dev.gnupg.org/differential/diff/1476/
and SUSE seems to already use a modified API:
https://sources.suse.com/SUSE:Maintenance:15118/libgcrypt.SUSE_SLE-15_Update/26a8df5f96d27d6abca7bd7ba9b0def0/libgcrypt-FIPS-RSA-DSA-ECDSA-hashing-operation.patch

Our public key functions are stateless. For several reasons it would be good to have an option to keep some state (think pre-computations). Our gcry_ctx_t would be a perfect fit for this and it will allow us to join a pubkey function with for example a hash function.

Does the patch really work, or is it a sketch to describe the intended use?

I recall that I talked with Stephan about it but things got lost.