A crafted CMS (S/MIME) EnvelopedData message carrying an oversized wrapped session key (AES Key Wrap ciphertext) can cause a stack buffer overflow in gpg-agent during PKDECRYPT --kem=CMS handling.

Security Advisory: gpg-agent stack buffer overflow via PKDECRYPT --kem=CMS
(ECC KEM)
Vulnerability Type: Stack-based Buffer Overflow
Affected Software: GnuPG
Severity: High
Date: 18 Jan 2026
Discoverer: OpenAI Security Research (OutboundDisclosures@openai.com)

Summary
A crafted CMS (S/MIME) EnvelopedData message carrying an oversized wrapped
session key (AES Key Wrap ciphertext) can cause a stack buffer overflow in
gpg-agent during PKDECRYPT --kem=CMS handling. The root cause is an
unbounded session key length derived from attacker-controlled ciphertext
and used as the AES Key Wrap output length when decrypting into a
fixed-size stack buffer. libgcrypt’s AES-KW unwrap copies
attacker-controlled bytes into the caller-provided output buffer before
performing integrity checks, so the stack overflow occurs even if later CMS
or key-wrap integrity validation fails.

Affected Versions

• Software Name: GnuPG
• Confirmed Version(s): 2.5.17-beta2, commit:

d97e52cc7fc5cc7bd8629643fdf37105891ae201, date: 2025-12-30T17:52:43+01:00

• Likely Affected: gpg-agent, gpgsm
• Introduced Commit (If relevant): N/A

Vulnerability Details:
Component: gpg-agent (agent) / S/MIME (gpgsm)

Affected code:

• agent/pkdecrypt.c in the PKDECRYPT --kem=CMS path (agent_kem_decrypt →

ECC KEM unwrap)

• Reachable via sm/call-agent.c sending PKDECRYPT --kem=CMS

Root cause:
agent_kem_decrypt() trusts an attacker-supplied CMS wrapped-key length to
size an AES Key Wrap unwrap into a fixed 256-byte stack buffer.

Execution flow:

1. gpgsm parses the CMS encryptedKey and forwards it to gpg-agent via

Assuan (PKDECRYPT --kem=CMS).

2. gpg-agent derives encrypted_sessionkey_len = (nbits + 7) / 8 from an

opaque MPI.

3. It computes sessionkey_len = encrypted_sessionkey_len - 8 (AES-KW

removes one 64-bit block).

4. It calls gcry_cipher_decrypt with sessionkey as the output buffer, where

sessionkey is a fixed-size stack array (unsigned char sessionkey[256]).

5. libgcrypt’s AES-KW unwrap begins by copying inbuflen - 8 bytes from the

ciphertext into the output buffer using memmove, overflowing the stack when
sessionkey_len exceeds 256.

Relevant code snippet (agent/pkdecrypt.c):
gcry_cipher_hd_t hd;
unsigned char sessionkey[256];
size_t sessionkey_len;

encrypted_sessionkey = gcry_mpi_get_opaque (encrypted_sessionkey_mpi,
&nbits);
encrypted_sessionkey_len = (nbits+7)/8;

err = gcry_cipher_open (&hd, algo, GCRY_CIPHER_MODE_AESWRAP, 0);

if (is_pgp && encrypted_sessionkey[0] != encrypted_sessionkey_len - 1)

{
  err = gpg_error (GPG_ERR_INV_DATA);
  goto leave;
}

err = gcry_cipher_setkey (hd, kek, kek_len);
sessionkey_len = encrypted_sessionkey_len - 8 - !!is_pgp;
if (!err)

err = gcry_cipher_decrypt (hd,
                           sessionkey, sessionkey_len,
                           encrypted_sessionkey + !!is_pgp,
                           encrypted_sessionkey_len - !!is_pgp);

gcry_cipher_close (hd);

Relevant libgcrypt behavior (cipher/cipher-aeswrap.c):
If outbuflen != inbuflen - 8, return error. Otherwise:
memmove(outbuf, inbuf + 8, inbuflen - 8);

This initial memmove copies attacker-controlled bytes into the output
buffer before any integrity checks.

Trigger conditions

• Recipient uses ECC KEM for CMS (for example

dhSinglePass-stdDH-sha1kdf-scheme, sha256, sha384, or sha512 variants with
AES-KW).

• The CMS encryptedKey (wrapped CEK) is attacker-controlled, a multiple of

8 bytes, and large enough that encrypted_sessionkey_len - 8 > 256 (for
example encrypted_sessionkey_len >= 272; 400 bytes in the proof of concept).

• gpgsm forwards the ciphertext to gpg-agent via PKDECRYPT --kem=CMS with

no length gate on the CMS path.

• This issue is not reachable in the KEM_PGP path because a 1-byte length

prefix check effectively caps encrypted_sessionkey_len to 256 bytes.

Proof-of-Concept (PoC)
A Docker-based reproducer and AddressSanitizer crash bundle is available at:
https://drive.google.com/file/d/1eC-VOCmfeqCFq26tGm6SXD3YUgIRD9lK/view

PoC behavior:

• Decrypting a CMS message with a 400-byte wrapped key reliably triggers a

stack buffer overflow in gpg-agent.

• gpgsm forwards the crafted CMS message to gpg-agent, which crashes during

PKDECRYPT --kem=CMS handling.

Observed output excerpt:
gpgsm: encrypted to nistp256 key 9A0D49D91EBEA6C59E9CE32C053CC090B01500BE
gpgsm: error decrypting session key: End of file
gpgsm: decrypting session key failed: End of file
gpgsm: message decryption failed: End of file <GpgSM>
[GNUPG:] FAILURE gpgsm-exit 50331649

ASan or Debug Output
AddressSanitizer reports a stack-buffer-overflow originating from memmove
in libgcrypt during AES-KW unwrap, writing beyond the 256-byte sessionkey
stack buffer in ecc_kem_decrypt. The overflow occurs before CMS or AES-KW
integrity checks complete.

Impact

• Reliable denial of service via gpg-agent crash when decrypting crafted

CMS messages.

• gpg-agent is a long-running, high-value process performing private-key

operations, so memory corruption represents a plausible code-execution
primitive depending on platform and hardening.

• Malicious senders who encrypt CMS messages themselves can fully control

the bytes written past the stack buffer because they derive the same KEK as
the victim.

• Passive or man-in-the-middle attackers can also trigger the overflow and

partially influence overwritten data due to the initial ciphertext copy
performed before integrity checks.

Recommendation

• Add strict bounds checks before computing or using sessionkey_len.
• Reject CMS inputs where sessionkey_len exceeds the size of the sessionkey

buffer, or allocate the buffer dynamically after validating length.

• Validate CEK sizes against CMS expectations (legitimate CEKs are 16, 24,

or 32 bytes for AES-128/192/256).

• Require AES-KW ciphertext lengths consistent with those CEK sizes.
• Additional hardening: check encrypted_sessionkey for NULL, ensure

early-error paths close cipher handles, and prefer secure memory for key
material.

Timeline

• January 2026: Discovered
• January 2026: Reproduced and confirmed
• January 2026: Advisory drafted

This information is being shared by OpenAI solely for the purpose of
improving security and reducing potential harm. This information is
presented as-is. We make no representations or warranties, express or
implied, as to the completeness, accuracy, or fitness for any particular
purpose of the information. This includes, without limitation, any
suggestions or ideas presented on how to remedy or mitigate an identified
vulnerability, including whether such suggestions or ideas would be
effective and/or could have other negative impacts.
OpenAI disclaims any liability for direct or indirect damages arising from
the reliance on, or use, misuse, or interpretation of this information. Any
references to third-party systems, services, or entities are included
solely for identification purposes and do not imply endorsement,
responsibility, or attribution.