No, it's not the config files that are a problem. And maybe I'm using
imprecise terminology. But, the gpg-agent process maintains an open
handle on the current working directory in which the process is started,
until it is killed. Here's an example:

When initially started from the command line (either directly or secondarily via
gpg --sign ..., the gpg-agent creates a lock on the current directory and
doesn't release it until gpg-agent exits.

This makes the directory un-deleteable until the process is killed. If it needs
a lock, it should lock an essentially unremovable directory (such as
%USERPROFILE%, if it needs to be user-owned).

This is a especially a problem when signing files which are in a temporary
directory (such as packages prepped for distribution) which would then be
deleted (with the enclosing directory).

I've noticed this is a problem ever since the 2.0 release using a graphical user
agent.

Thanks for taking the time to look into the problem.