gpg --clearsign does not sign the actual input message if the message ends in a newline. Rather, it signs the message without the final trailing newline. gpg --verify's --output option does not emit the actual signed data, if the data does not end in a newline. Rather, it emits the signed data plus a trailing newline.
Clearsigned messages and whitespace have quite a bit of legacy trickiness, as noted in T1419 and T1692.
In particular, RFC 4880 observes that whitespace at the end of newlines will be stripped, and that:
An implementation SHOULD add a line break after the cleartext, but
MAY omit it if the cleartext ends with a line break. This is for
visual clarity.
Presumably this suggestion is meant to add a trailing line break to the message before signing, as it will lend more visual clarity to the final clearsigned message.
And, of course, whitespace at the end of a line in a clearsigned message is all stripped:
Also, any trailing whitespace -- spaces (0x20) and tabs (0x09) -- at
the end of any line is removed when the cleartext signature is
generated.
The spec is also clear that the final newline before the BEGIN PGP SIGNATURE is *not* part of the message being signed:
The line ending (i.e., the <CR><LF>) before the '-----BEGIN PGP
SIGNATURE-----' line that terminates the signed text is not
considered part of the signed text.
However, gpg --clearsign seems to *strip* the trailing newline from input before signing. By contrast, gpg --textmode --detach-sign does not do that.
Stripping the trailing newline is not the same as trimming the whitespace at the end of a line, or adding a line break if no such
linebreak exists.
Instead, the message signed with gpg --clearsign is actually the input message with one *fewer* newline, as long as the original message ended in a newline.
Here is a demonstration of this (mis)behavior, where six deterministic signatures are made in the same second, and then compared to each other. We sign each of three messages (with 0, 1, or 2 trailing newlines) using both --textmode --detach-sign and --clearsign. Then we compare the results:
$ for newlines in 0 1 2; do python3 -c 'print("test" + "\n"*'$newlines', end="")' > $newlines.txt; done
$ for nl in 0 1 2; do gpg -u $PGPID --clearsign < $nl.txt > $nl.signed & gpg -u $PGPID --detach-sign --textmode --armor < $nl.txt > $nl.sig & done; wait
[1] 1165590
[2] 1165591
[3] 1165592
[4] 1165593
[5] 1165594
[6] 1165601
[1] Done gpg -u $PGPID --clearsign < $nl.txt > $nl.signed
[2] Done gpg -u $PGPID --detach-sign --textmode --armor < $nl.txt > $nl.sig
[3] Done gpg -u $PGPID --clearsign < $nl.txt > $nl.signed
[4] Done gpg -u $PGPID --detach-sign --textmode --armor < $nl.txt > $nl.sig
[5]- Done gpg -u $PGPID --clearsign < $nl.txt > $nl.signed
[6]+ Done gpg -u $PGPID --detach-sign --textmode --armor < $nl.txt > $nl.sig
$ diff -u 0.sig 0.signed
--- 0.sig 2024-04-26 10:25:39.238425091 -0400
+++ 0.signed 2024-04-26 10:25:39.374431359 -0400
@@ -1,3 +1,7 @@
+-----BEGIN PGP SIGNED MESSAGE-----
+Hash: SHA256
+
+test
-----BEGIN PGP SIGNATURE-----
iHUEARYIAB0WIQR0vATEPYYIS+hnLAZ3LRYeNc1LgQUCZiu5YwAKCRB3LRYeNc1L
$ diff -u 1.sig 1.signed
--- 1.sig 2024-04-26 10:25:39.226424538 -0400
+++ 1.signed 2024-04-26 10:25:39.234424907 -0400
@@ -1,7 +1,11 @@
+-----BEGIN PGP SIGNED MESSAGE-----
+Hash: SHA256
+
+test
-----BEGIN PGP SIGNATURE-----
iHUEARYIAB0WIQR0vATEPYYIS+hnLAZ3LRYeNc1LgQUCZiu5YwAKCRB3LRYeNc1L
-gZJPAP9TioWz+1YpILMV6EDwGT5aiyWKU96IZdOedgkCvWU8oAEAvfUs8d44otco
-OJ2QKD0jxQoDCzZ3fc1Vr4rLTvGN7QA=
-=d3IB
+gT7ZAQCerr6oFMzanfgLEfH5phZ6Rpxb7e6GRi2XZTCGfsKItQEA9BTnJ4Jzjq00
+URFjZryC0V6SR0YFFWaBDN6mI/yTHQY=
+=cpUs
-----END PGP SIGNATURE-----
$ diff -u 0.signed 1.signed
$ diff -u 0.sig 1.signed
--- 0.sig 2024-04-26 10:25:39.238425091 -0400
+++ 1.signed 2024-04-26 10:25:39.234424907 -0400
@@ -1,3 +1,7 @@
+-----BEGIN PGP SIGNED MESSAGE-----
+Hash: SHA256
+
+test
-----BEGIN PGP SIGNATURE-----
iHUEARYIAB0WIQR0vATEPYYIS+hnLAZ3LRYeNc1LgQUCZiu5YwAKCRB3LRYeNc1L
$ diff -u 1.sig 2.signed
--- 1.sig 2024-04-26 10:25:39.226424538 -0400
+++ 2.signed 2024-04-26 10:25:39.222424353 -0400
@@ -1,3 +1,8 @@
+-----BEGIN PGP SIGNED MESSAGE-----
+Hash: SHA256
+
+test
+
-----BEGIN PGP SIGNATURE-----
iHUEARYIAB0WIQR0vATEPYYIS+hnLAZ3LRYeNc1LgQUCZiu5YwAKCRB3LRYeNc1L
$ gpg --version
gpg (GnuPG) 2.2.40
libgcrypt 1.10.3
Copyright (C) 2022 g10 Code GmbH
License GNU GPL-3.0-or-later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Home: /home/dkg/.gnupg
Supported algorithms:
Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA
Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
CAMELLIA128, CAMELLIA192, CAMELLIA256
Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2
$As you can see from above, clearsigning either of test\n and test results in the same signature, which is the same signature that is made via a --textmode --detach-sign over test (with no trailing newline).
Each --clearsigned message's signature corresponds to the signature made a --textmode --detach-sign of a message with one fewer trailing newline.
I suspect this has never been noticed because gpg --verify's --output option always emits a trailing newline, adding it if the signed data did *not* have a trailing newline:
$ for x in 0 1 2; do gpg --verify --output $x.output < $x.signed ; done gpg: Signature made Fri 26 Apr 2024 10:25:39 AM EDT gpg: using EDDSA key 74BC04C43D86084BE8672C06772D161E35CD4B81 gpg: Good signature from "Daniel Kahn Gillmor" [ultimate] gpg: Signature made Fri 26 Apr 2024 10:25:39 AM EDT gpg: using EDDSA key 74BC04C43D86084BE8672C06772D161E35CD4B81 gpg: Good signature from "Daniel Kahn Gillmor" [ultimate] gpg: Signature made Fri 26 Apr 2024 10:25:39 AM EDT gpg: using EDDSA key 74BC04C43D86084BE8672C06772D161E35CD4B81 gpg: Good signature from "Daniel Kahn Gillmor" [ultimate] $ for x in 0 1 2; do hd < $x.output; done 00000000 74 65 73 74 0a |test.| 00000005 00000000 74 65 73 74 0a |test.| 00000005 00000000 74 65 73 74 0a 0a |test..| 00000006 $
If i'm understanding RFC 4880 correctly, gpg --clearsign could become compliant by either:
- not stripping the trailing newline before signing, or
- not stripping the trailing newline before signing, and *adding* a trailing newline if none exists
gpg --verify's --output option should stop emitting a trailing newline when none exists, so that it accurately represents the signed data.