That is not required by the specs. Another way is to provide a tool to compare keys. That seems to be easier to me. Also consider the cases that there are new new packets or signature subpackets with unknown properties to the current implementations. What about different encodings in signed key material?

I thoroughly agree that this is not required by the specs.

While a specialized tool to compare keys might be useful, being able to have a reproducible, predictable binary output format would be useful in its own right, for example, when trying to make a byte-for-byte identical blob as part of some other system that needs bytewise precision.

A canonicalization that covers all existing known packets would be a good start, even if it doesn't know how to handle some packets. Signature subpackets seem like they'd have to be explicitly canonicalized already (assuming they're hashed subpackets) otherwise the signature wouldn't compute correctly, right?

Right we can't change the order of signature subpackets after they have been created. Given that we create subpackets by directly appending them to a memory buffer instead of keeping a list of subpackets to create, the least invasive method would be a function to shuffle that memory buffer right before the signature is computed.

But wait. Does my idea really help with comparing? I doubt it because a signature also includes a date and other variable stuff and thus they are already binary identical or it is a different signature.

So, what needs to be done is to define a order of the packets and not of parts of the packets. That can indeed be done in the export code and should be easy after we have agreed on order criteria (e.g. newest last or first).

I am not proposing changing the order of the *hashed* subpackets in a signature. I'm proposing removing/changing/canonicalizing the *unhashed* subpackets in a signature. Sorry if i didn't make that clear enough in the initial message.

As a first implementation, we could just ignore the fact that the unhashed subpackets may vary and still have the thing be the same certificate though, and reproduce them verbatim the way we reproduce the hashed subpackets verbatim.

Here is a proposed answer:

Ordering of multiple certificates

• A series of OpenPGP certificates are sorted by their fingerprint, starting at 00000000000000000000000000000000000000000 and ending in FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.

Ordering of signatures

There are several places where a series of signature packets appear.

• A series of OpenPGP signature packets (marked as (sigs)) are always sorted by their creation date, oldest first, most recent last. If two signatures in a series have identical creation dates, then they are sorted by their binary content.

Ordering within a single OpenPGP certificate

• Primary public key packet
• Direct key signatures on the Primary Key (sigs)
• User IDs (sorted as binary strings to avoid locale-specific collation and Unicode normalization forms); after each User ID:
  • signatures binding the User ID to the primary key (sigs)
• User Attributes (sorted as binary blobs); after each User Attribute:
  • signatures binding the User Attribute to the primary key (sigs)
• Subkeys (sorted by creation time, oldest first). If two subkeys have identical creation times, they are sorted as binary blobs); after each subkey:
  • Subkey binding signatures that link the specific subkey to the primary key (sigs)

unhashed sub-packets in signature packets

The simplest would be to drop them all, but there are a few that are concretely useful. I propose dropping all of them, with the following exceptions:

• Issuer key ID (subpacket type 16):
  • if the hashed subpackets already contain an Issuer key ID subpacket, do not include one in the unhashed subpackets.
  • otherwise, if the hashed subpackets already contain an Issuer key fingerprint subpacket (type 33), generate an Issuer key ID subpacket from it, and include it in the unhashed subpackets.
  • otherwise, if the issuer's key ID is known (e.g. from an unhashed subpacket during import, or from some other means), generate an Issuer key ID subpacket and include it in the unhashed subpackets.
  • otherwise, do not include an Issuer key ID subpacket in the unhashed subpackets.
• embedded signature (subpacket type 32):
  • If the signature is a subkey binding signature (signature type 0x18), and the hashed subpackets have a usage flag subpackets (subpacket type 27) that includes any of: sign, certify, or authenticate, and:
  • If the generator of the canonicalized packet knows of any Primary Binding signatures (signature type 0x19) that are issued by the subkey over the primary key, and:
  • any of those signatures are not already included in the hashed subpackets, then:
  • generate a list of all such signatures that are not already included in the hashed subpackets, wrapping each of them in the embedded signature subpacket. Order them as signatures are ordered (sigs) (signatures with matching creation dates are sorted by signature binary blob, without considering the subpacket wrapper), and inclue them as unhashed subpackets.

The result of this is that there are only two types of subpackets that show up in canonicalized unhashed subpackets, and they show up in this order:

• 0 or 1 Issuer key ID subpacket
• 0 or more embedded signature subpackets (sigs)

wdyt?

fwiw, a comment over on T4422 contains a bash script that tries to force GnuPG to do its certificate/signature re-ordering. this doesn't produce anything canonical yet, but it's the closest i've come so far to getting GnuPG to do something repeatable with a certificate after merging (but even that is not quite stable).

then they are sorted by their binary content.

@dkg Can we avoid this or take it as a last resort, e.g. by using a second sorting field (issuer)?

@stm it kind of is a last-resort already, given that it's only in the event where the signature creation dates are equal, but sure, i wouldn't mind adjusting the proposal to say that (sigs) means "sort by date, then issuer, then binary content" -- but what do we think "sort by issuer" means?

Consider that signatures can have 64-bit issuers (hashed "issuer" subpackets), 160-bit or 256-bit issuers (hashed "issuer fingerprint" subpackets for v4 or v5 keys), or no hashed issuers (e.g. signatures that only have unhashed "issuer" or "issuer fingerprint" subpackets, or even no issuer designations at all?

Perhaps we could just say that (sigs) means sorting by these three things in order:

• signature creation date
• the binary contents of the hashed "issuer fingerprint" subpacket (signatures with no hashed "issuer fingerprint" subpacket sort last)
• binary content

That also provides some incentive to provide a hashed "issuer fingerprint" subpacket in every signature.

@dkg You are right. The term "issuer" was too ambiguous.
I like your proposal and would try to implement it. However, "export" of dkg-keycheck and other programs from DKGPG are very limited.

@dkg First step toward the canonical OpenPGP certificate export: http://git.savannah.nongnu.org/cgit/libtmcg.git/commit/?id=75372cac01501ae427dec1ae18805449bf28d087