Yes I do understand Windows XP is not supported. Just in case it is a minor problem that is easy to fix and will not cost you much effort. I'd like to add more information: I do not change
%LOCALAPPDATA%. There is no such environment variable. A similar environment variable is:
APPDATA=C:\Documents and Settings\myname\Application Data
I do set GNUPGHOME=E:\key, which I think should be allowed because I do not want my personal info be stored in system drive.

E:\key>gpgconf --list-dirs
sysconfdir:C%3a\Documents and Settings\All Users\Application Data\GNU\etc\gnupg
bindir:C%3a\Program Files\gnupg\bin
libexecdir:C%3a\Program Files\gnupg\bin
libdir:C%3a\Program Files\gnupg\lib\gnupg
datadir:C%3a\Program Files\gnupg\share\gnupg
localedir:C%3a\Program Files\gnupg\share\locale
socketdir:E%3a\key
dirmngr-socket:E%3a\key\S.dirmngr
agent-ssh-socket:E%3a\key\S.gpg-agent.ssh
agent-extra-socket:E%3a\key\S.gpg-agent.extra
agent-browser-socket:E%3a\key\S.gpg-agent.browser
agent-socket:E%3a\key\S.gpg-agent
homedir:E%3a\key

The "sysconfdir" "C:\Documents and Settings\All Users\Application Data\GNU" does not exist actually. Does it matter?

Tried and no change -- cmd window still flashes away.

Thanks for the explanation. I understand gnupg-w32 is mainly for installing the command line component, yet adding a context menu for a specific file type is just as simple as importing a reg file like:

Thanks to jaclaz@msfn.org, the workaround is to use pipe operation like:
pause|"C:\Program Files\GnuPG\bin\gpg.exe" --verify "%1"
He also confirmed that gpg.exe does interrupt batch processing, regardless what command is followed.
And I have tested in Windows 7, batch processing is not interrupted. Since this bug is WindowsXp specific, "won't fix" should be more proper.

What I need is exactly ikloecker described on Linux. The point is NSIS installer gnupg-w32-2.2.27_20210111.exe (and versions above, I am sure) do not create context menu shortcut. Windows XP is not the point. Same on another Windows 7 machine. Do you need I find another windows 10 machine to test? I think it's easier to check whether the installer has that feature or not.

Sorry, I should clarify that I am using the windows installer
gnupg-w32-2.2.27_20210111.exe on WindowsXpSp3. The installer do not create
any context menu since I use it. I use Gnupg with Enigmail in Thunderbird,
so Gpg4win is not preferred.

So it is related to code page. Screenshots may be more informative: