Reported by mail to gnupg-devel on Oct 28 and earlier at the gpg4win forum:
On Windows machines whose user locale / regional format is Chinese (Simplified, China), GpgOL 2.7.2 (Gpg4win 5.0.2, GnuPG 2.5.18) hangs Outlook the first time it sanitizes a file name, e.g. when clicking “Secure” in a new mail. Outlook’s UI thread spins and the process memory oscillates between about 100 MB and 8 GB.
The trigger is not the system code page and not UTF-8 vs. non-UTF-8 system locale. The trigger is the C runtime locale that GpgOL itself sets in main.c (setlocale (LC_ALL, "")), which takes the user locale (here Chinese (Simplified)_China.936).
[...]
Windows system locale: Chinese (PRC), ACP 936 — but the result is the same with the system locale set to English (US) (ACP 1252) and with “Beta: Use Unicode UTF-8” (ACP 65001), as long as the regional format (user locale) is Chinese (China)
The suggested fix is to replace the regex by straight checks.
The report by yiguei lin via Gnupg-devel gives more details:
Facts
- Clicking Reply freezes Outlook: main thread 100% CPU in msvcrt!memmove, private memory ~4.2 GB.
- The log stops at "mail.cpp:updateOOMData_o: Called", before "Attachment candidate is ..." in get_tmp_outfile (common.cpp). In between only sanitizeFileName() does real work: std::regex.
- main.c i18n_init() calls setlocale (LC_ALL, "") (LC_MESSAGES is not defined on mingw-w64).
- Under cp950, msvcrt strxfrm() returns 0x7fffffff (errno EILSEQ) for every byte 0x80..0xFF. libstdc++ transforms all 256 bytes when compiling a bracket range, so it allocates ~2 GiB per byte.
[...]
Proposed fix (the regex hang disappears with this change in the
program above): after
setlocale (LC_ALL, ""), call setlocale (LC_COLLATE, "C").
Alternative: avoid std::regex in sanitizeFileName().
--- a/src/main.c +++ b/src/main.c @@ -50,6 +50,10 @@ setlocale (LC_MESSAGES, ""); # else setlocale (LC_ALL, "" ); + /* With a DBCS ANSI code page (e.g. cp950) msvcrt's strxfrm returns + * INT_MAX for bytes 0x80..0xFF; libstdc++'s regex then tries to + * allocate ~2 GiB per byte while compiling any "[x-y]" expression. */ + setlocale (LC_COLLATE, "C"); # endif #endif
(AI based analysis and patch)