Exactly same problem is there in libgcrypt.
In the definitions of curves, it uses negative constant internally in some specific places, but for other parts, we have same problems.

Change of gpg-agent for ECC-SOS

Change: when parsing private key data, use /d (interpret as an opaque
octed sequence), so that no removal of leading zero bytes occurs.

But this change will expose hidden issue(s) of the inconsistency of
signed/unsigned MPI representation handling in GnuPG and libgcrypt,
which currently does: adding a leading zero byte at write as signed
data and ignoring a leading zero byte at read (with no check of this
additional zero-byte or not).

So, another change of fixup when reading keys:

Existing cleartext keys may have 'd' with leading zero byte (a single
leading zero byte, because of the lax signedness problem). This
leading zero byte should be removed before the parsing private key
data.

It would be also good to fixup removed zeros for 'd' of Ed25519/EdDSA.

(Existing keys with encrypted 'd' never has 'd' with leading zero
bytes when decrypted, because the leading zero bytes were already
ignored when encrypted.)

a test case:

Prepare cleartext private keys with MSB=1, export them into OpenPGP
format (by gpg --export-secret-key). Make sure no leading zero-byte
in the result.

rG47c1c329ed82: agent,ecc: Use of opaque MPI for ECC, fixup 'd'. does the fixup when reading keys.