I created an experimental branch:
https://dev.gnupg.org/source/libgcrypt/history/gniibe%252Fnew-pk-api/

Based on GCC bugzilla, affected released GCC versions are 11.1 and 11.2.

(ab | ba) >= 0 is used to make optimization analysis for compiler more difficult. I see that with (ab | ba) == 0, it would be much easier for compiler to conclude than loop could exit early as soon as first a[i] != b[i] is seen.

I filed a bug report to GCC, with modified test case.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=102151

Closing, as downstream ticket has been closed.

Fixed in libgcrypt 1.9.4.

For GCC 11, modified version of struct arg_and_data_s has an issue for x86_64.

We should update jitterentropy to 3.0.2 or newer, which should be easier to get through certification, if we will go this way. From FIPS perspective, we should be fine with either going through getrandom only or with jitter entropy, but the bottom-line was that we should probably keep both as we do now.

From Stephan I got the following response to the allocation handler use case

I think the last user of random-fips was removed with rCed57fed6de1465e02ec5e3bc0affeabdd35e2eb7

Yes, it makes sense to remove it.

For the use case of struct arg_and_data_s in gpgme, which may allocate zero-sized ARG[], it seems that GCC 11 interprets it as an invalid use.

iirc Uli Drepper added a hack to dladdr which we made use of. Seems to be integrated into dladdr1 now.

While I don't know if runtime integrity check is required or not by FIPS 140,
I checked OpenSSL, and it has such a check in openssl/providers/fips. The FIPS module configuration file which has the module checksum by HMAC is generated by openssl fipsinstall command.

Ah... I realized that HMAC integrity check with dladdr (using address of constant string) might work (at some point) to determine the filename of libgcrypt.so, when/if glibc implementation allows searching with address of constant string. So, my claim "never worked" was wrong.

We have the same patch (including the hmac key and we use the switch. The reasoning on our side was to be compatible with fipscheck, but it is no longer used since last year and we use just the hmac256 tool:

Just for the records, the whole HMAC thing including the special dlopen trick used to work fine when we did the original FIPS support.

Right. The clarification is that SHA1 itself (for non-security and non-signature use) is still allowed in FIPS mode. But it is not allowed to be used as part of signature schemes of the new API in FIPS mode. The old API, which allows raw signatures without digests, should just fail in FIPS mode too. And the FIPS-compatible gnupg should use the new API too (it would be good to think about this when putting it together).

For Linux and FIPS, we should be actually fine with using /dev/random or getrandom().

The CAVS driver can be safely removed. The certification goes through the ACVP these days so it does not make sense to keep this.

For use of SHA-1:

(can't access that bug with my account)

For tests with FIPS mode enabled, I manually create the file .libgcrypt.so.20.hmac under src/.libs.

I pushed my further change.
Also, applied and pushed your changes.

Sorry, I didn't test for non-FIPS mode when I committed rC347817438990: fips: Fix tests in fips mode..
Tweaking the value for memory allocation is needed for FIPS mode, because it uses some secure memory by DRBG.

I went a bit back to the history to figure out what is the enforced and soft fips mode as it was initially not completely clear to me. For the record, I used the following bug from 9 years ago:

Tested the master on (faked) FIPS and non-FIPS Fedora and I created couple of more changes for master to work in FIPS mode:

Since I think there is no reason why checking _gcry_enforced_fips_mode () here, I remove the check.

Applied by rC0f118c2dfb8e: cipher: Do not use of non-approved digests in FIPS mode.

Ah, I understand the point (at least, partially); My understanding is: With FIPS mode, at the module boundary (== libgcrypt), it ensures that all cipher/digest/etc. operations are done under the standard compliance, and it is considered wrong (violation) when non-FIPS mode operation (such as SHA-1) and FIPS mode operation are mixed.

In RHEL, we do not have anything about PCT so the PCT requirement is not completely clear to me: https://git.centos.org/rpms/libgcrypt/blob/c8s/f/SOURCES

SUSE has patches and version 3235 of cavs_driver.pl, bud it seems that it doesn't support DSA with Q+HASHALGO yet.

Hmm, do we need a backport?

As a start, I applied your patches.

And... as long as I read the PCT patches, it is not needed to export those API to users.
It is only needed internally for PCT tests (at most).

I am considering API enhancement, for this task.

I went through the patches above + what I suggested in previous comments, tested everything against both upstream and libgcrypt in Fedora in FIPS mode. There were slight differences, some cases were already fixed in master, some needed to upstream some of our changes, but the result is 10 patches working in both FIPS and non-fips mode, hopefully enough annotated. If not, please, ask for clarifications.

I went through the OpenSSL drafts. The module boundary in OpenSSL will be separate fips.so object and only non-deprecated functions of OpenSSL 3.0 will be FIPS compliant. There is a global state, that will allow only approved algorithms and modes and there will be API to query the FIPS mode status using OSSL_PARAM_get* functions, but we still have some unknowns so I hope we will know more on the next meeting.

(OpenSSL for FIPS support is a bit tricky, which is described in README-FIPS.md in their distribution. It offers OpenSSL FIPS provider as shared library fips.so.)

Just FYI, NSS offers following API:

I have couple of references from libssh:

gniibe: Can you please check what openssl does exactly. The problem is that we currently have no permanent state for Libgcrypt (i.e. something stored on disk per user or even better global)

FWIW: Unfortunately everyone is moving to GCM, even Outlook. While GnuPG was evaluated by the German BSI we had discussions about this and their evaluators were wary about GCM due to its brittleness thus our use of OCB was very welcomed. OTOH, another approved product meanwhile comes with GCM for S/MIME and thus it seems thatGCM is accepted.

There is no point in questioning whether a couple of words change racism or any other human problems of these days. It will not.