Page MenuHome GnuPG

gnupg --encrypt-files uses file handles inefficiently
Closed, ResolvedPublic

Description

Release: gpg (GnuPG) 1.2.6

Environment

Linux, Fedora Core 3

Description

If you encrypt a ton of files at once with gpg --encrypt-files $LONG_FILE_LIST you will get a "too many open files" error.

This makes me think the file handles aren't being closed properly during processing

How To Repeat

  1. Run this in a large directory. Change "KEY" first. #!/bin/bash KEY=foo@example.com for FILE in find . do
    1. Make sure its a file, not a directory if [ -f $FILE ]; then
      1. Encrypt the new file with all the keys echo will encrypt $FILE ... ENCRYPTS="$ENCRYPTS $FILE"
      2. Add the file to the list of files to commit echo "$FILE.asc \\" >> commit.sh COMMITS="true" fi done
  2. Do the encrypt echo gpg -a -r $KEY --encrypt-files $ENCRYPTS echo -n "Encrypting..." gpg -a $KEYS --encrypt-files $ENCRYPTS

Fix

Unknown

Release Note

gpg 1.2.6 is old and not maintained. Please update to 1.4.1. IIRC, we have fixed this bug.

Fixed in CVS. Fix will go into 1.4.2.

Event Timeline

werner added a subscriber: werner.

version too old

From: Mike Hardy <mike@mikehardy.net>
To: bug-any@bugs.gnupg.org
Cc: gnupg-hackers@gnupg.org, gnats-admin@trithemius.gnupg.org, wk@gnupg.org,

gnupg@mikehardy.net

Subject: Re: gnupg/479
Date: Wed, 20 Jul 2005 08:48:06 -0700

Well, I guess you get what you pay for :-), but I even attached a
script, you could have tried it!

No matter, I have an FC4 machine laying around with gpg 1.4.1 on it, and
I tested it.

My script has a bug in that its missing a '-r' on the actual encrypt
line (I got it right in the echo at least...), but once you fix that,
you see that it does indeed still have the problem.

So this is not fixed.

-Mike

wk@gnupg.org wrote:

Synopsis: gnupg --encrypt-files uses file handles inefficiently

State-Changed-From-To: open->closed
State-Changed-By: werner
State-Changed-When: Wed, 20 Jul 2005 17:10:08 +0200
State-Changed-Why:
version too old

  • Comment added by werner on Wed, 20 Jul 2005 17:10:08 +0200 ****

Sorry for closing the bug. I was pretty sure that we once
fixed that bug. I guess I should send a bug report to my
brain. I can reproduce it easily.

werner claimed this task.
werner added a project: In Progress.
werner added a subscriber: gnupg-hackers.

Just fixed in CVS. Also for --verify-files and --decrypt-files.

werner removed a project: In Progress.

I can still reproduce the bug with 1.4.2.2 and SVN of today. After 1013 files,
I get file open errors.

marcus removed a project: Restricted Project.Jul 29 2006, 1:27 AM
werner raised the priority of this task from Normal to High.Jul 29 2006, 12:11 PM
werner added a project: IO-Buffer.

That whole iobuf caching is a mess and we should remove it. Whoever this
requires major changes in the packet parsing code and thus we leave this for >
gnupg 2.0.

I added a fix which invalidates the entire fd caches after processing of each
file. This goes into 1.4.5.

Well, that bug is actually resolved and there is no way to do it much better.