Executing this in a loop, fails after a while, never exceeds 10k runs
./build/sm/gpgsm -o somefile.sig --include-certs -1 -sb somefile openssl cms -verify -in somefile.sig -inform DER -content somefile \ -CAfile sample.x509.pem -binary -noout
It's openssl that refuses the signature
int_rsa_verify:wrong signature length:../crypto/rsa/rsa_sign.c:132:
Problem does not occur with this hack:
--- a/src/cms.c +++ b/src/cms.c @@ -1675,12 +1675,15 @@ ksba_cms_set_sig_val (ksba_cms_t cms, int idx, ksba_const_sexp_t sigval) return gpg_error (GPG_ERR_INV_SEXP); } +#if 0 if (n > 1 && !*s) { /* We might have a leading zero due to the way we encode MPIs - this zero should not go into the OCTECT STRING. */ s++; n--; } +#endif
I reported this problem a while back
https://lists.gnupg.org/pipermail/gnupg-users/2018-April/060277.html
I also posted the issue on openssl-users,
https://mta.openssl.org/pipermail/openssl-users/2018-April/007916.html
They refer me to the PKCS#1 RFC, that the leading '0' should be preserved
https://tools.ietf.org/html/rfc8017#section-8.2.1
Output: S signature, an octet string of length k, where k is the length in octets of the RSA modulus n
I'm not familiar with the internal presentation of data (s-exp) or the multi-precision-library(MPI). But I expected that gpg-agent would strip excess '0' if needed and that libksba just write the number into ASN.1 format without transforming them.
The commit introducing the zero removal above, doesn't tell much why those zeroes have to be removed:
https://git.gnupg.org/cgi-bin/gitweb.cgi?p=libksba.git;a=commit;h=d93a478116dfcfc813b72340dff9429b8f8859fc
commit d93a478116dfcfc813b72340dff9429b8f8859fc Refs: debian/V0-2-1-3-gd93a478 Author: Werner Koch <wk@gnupg.org> AuthorDate: Wed Mar 13 10:18:54 2002 +0000 Commit: Werner Koch <wk@gnupg.org> CommitDate: Wed Mar 13 10:18:54 2002 +0000 tests/ * t-cms-parser.c (one_file): Distinguish between signed and eneveloped CMS objects. src/ * cms.h (value_tree_s): New and use it for recp_info. * cms.c (release_value_tree): New. (ksba_cms_release): And use it here. (ksba_cms_get_issuer_serial): Use the new recp_list structure and take the IDX into account. (ksba_cms_get_enc_val): Ditto. (ksba_cms_set_sig_val,ksba_cms_set_enc_val): Don't store a leading zero. * cms-parser.c (_ksba_cms_parse_enveloped_data_part_1): Handle multiple recipients. * cms.c (build_signed_data_rest): Write 3 end tags.
I use self-compiled versions when reproducing this problem
$ ./build/sm/gpgsm --version gpgsm (GnuPG) 2.3.0 libgcrypt 1.9.0-beta86 libksba 1.3.6-beta14