in rsa.c:generate_fips(), the function always returns 0, so failures cannot be caught by the calling function. The following code seems to override the error condition set as default for the return value:

gpg_err_code_t ec = GPG_ERR_NO_PRIME;

if (nbits < 1024 || (nbits & 0x1FF))
  return GPG_ERR_INV_VALUE;
ec = rsa_check_keysize (nbits);
if (ec)
  return ec;

This code has been reworked recently in rC40d63d09.

Notice that, ec is set to 0 at the end only in case no error was found during key generation, but the invocation to rsa_check_keysize() nullifies the error condition, so GPG_ERR_NO_PRIME is never returned.

The original error code could be easily reinstated by setting it to ec = GPG_ERR_NO_PRIME after the rsa_check_keysize() call or using a different error code variable to receive the error code of rsa_check_keysize().