Page MenuHome GnuPG
Create Task
Maniphest T6351

Kleopatra: Performance problems when encrypting large files
Testing, NormalPublic
Actions

Description

When encrypting large files (without compression) the difference between GnuPG and Kleopatra becomes significant because Kleopatra needs around 5-6x more time than GnuPG. That's why it would be great to increase the performance of Kleopatra.

Edit 2024-01-11 (ikloecker): The problem exists for all crypto operations (sign, encrypt, decrypt, verify). For OpenPGP the problem is tracked by this task. For S/MIME it's tracked by T6928: Kleopatra: Speed up S/MIME crypto operations for large files (on Windows).

Revisions and Commits

rKLEOPATRA Kleopatra
rKLEOPATRA37ee18829622 Fix sign/encrypt/decrypt/verify of notepad
rKLEOPATRA4fc4188ee785 Fix sign/encrypt for S/MIME
rKLEOPATRAc9528f3113ae Use direct file I/O for verifying detached OpenPGP signatures
rKLEOPATRA1750f673088a Make decrypt/verify jobs directly read/write the input/output file
rKLEOPATRAb3129c3e7c2d Make sign/encrypt jobs directly read/write the input/output file

Related Objects
Search...

Event Timeline

cklassen created this task.Jan 20 2023, 11:07 AM
cklassen created this object in space S1 Public.

There is another issue about the performance of Kleopatra when encrypting large archive files

cklassen renamed this task from Kleopatra: Performance problems when encrypting large files to Kleopatra: Performance problems when encrypting large files without compression.Jan 20 2023, 11:09 AM
cklassen updated the task description. (Show Details)
• werner triaged this task as Normal priority.Jan 20 2023, 1:31 PM
• werner added a subscriber: • werner.

There are two issues here:

  1. The overhead due to QT based processing.
  2. The compression takes long and gpg used to had no way to detected already cmpressed data when the data was piped to gpg (as Kleopatra) does. See T6332.

The second issue has been fixed for the majority of compressed documents and some image formats. This will be released with gnupg 2.4.1 and 2.2.43.

• aheinecke renamed this task from Kleopatra: Performance problems when encrypting large files without compression to Kleopatra: Performance problems when encrypting large files.Jan 23 2023, 10:56 AM
• aheinecke added a subtask: T6332: GPG: Extend / rework "is_file_compressed".
• aheinecke added projects: Restricted Project, gpgme.
• aheinecke added a comment.Jan 23 2023, 11:10 AM

I edited this task a bit: For compression we have T6332
For Archives passed to Gpgtar we have: T6342

I would like this issue to be for the case https://lists.gnupg.org/pipermail/gnupg-users/2023-January/066397.html that:

Encryption:
Size | GnuPG | Gpg4win
1GB | 38 sec. | 1 min. 8 sec.
1GB | 37 sec. | 1 min. 7 sec.
2GB | 1 min. 14 sec. | 2 min. 15 sec.
2GB | 1 min. 14 sec. | 2 min. 14 sec.
5GB | 3 min. 10 sec. | 6 min. 10 sec.
5GB | 3 min. 6 sec. | 5 min. 34 sec.
10GB | 6 min. 28 sec. | 11 min. 21 sec.
10GB | 6 min. 21 sec. | 11 min. 6 sec.

Results of decryption:

Size | GnuPG | Gpg4win
1GB | 3 sec. | 36 sec.
1GB | 3 sec. | 34 sec.
2GB | 10 sec. | 1 min. 13 sec.
2GB | 7 sec. | 1 min. 12 sec.
5GB | 22 sec. | 3 min. 1 sec.
5GB | 19 sec. | 3 min. 2 sec.
10GB | 1 min. 3 sec. | 5 min. 52 sec.
10GB | 1 min. 7 sec. | 6 min. 5 sec.

Which would be solved in my opinion if we would have the filenames for encryption and decryption passed directly to GnuPG and not go through the GPGME callback layers. This is very similar to what T6342 is about. But different enough that it warrants its own issue.

• aheinecke mentioned this in T6530: GPGME / QGpgME Extend Archivejobs to accept input / output from a filename.Jun 9 2023, 4:24 PM
• ikloecker changed the status of subtask T6530: GPGME / QGpgME Extend Archivejobs to accept input / output from a filename from Open to Testing.Jun 21 2023, 4:59 PM
• ebo closed subtask T6530: GPGME / QGpgME Extend Archivejobs to accept input / output from a filename as Resolved.Jul 6 2023, 2:45 PM
• aheinecke closed subtask T6332: GPG: Extend / rework "is_file_compressed" as Resolved.Jul 26 2023, 11:41 AM
• ikloecker added a commit: rKLEOPATRAb3129c3e7c2d: Make sign/encrypt jobs directly read/write the input/output file.Dec 22 2023, 2:02 PM
• ikloecker claimed this task.Jan 4 2024, 9:51 AM
• ikloecker moved this task from Restricted Project Column to Restricted Project Column on the Restricted Project board.
• ikloecker added a commit: rKLEOPATRA1750f673088a: Make decrypt/verify jobs directly read/write the input/output file.Jan 4 2024, 9:54 AM
• ikloecker closed subtask T6550: GpgME / QGpgME Extend non-archive jobs to accept input / output from a filename as Resolved.Jan 5 2024, 1:59 PM
• ikloecker mentioned this in T6550: GpgME / QGpgME Extend non-archive jobs to accept input / output from a filename.
• ikloecker added a commit: rKLEOPATRAc9528f3113ae: Use direct file I/O for verifying detached OpenPGP signatures.Jan 8 2024, 10:47 AM
• ikloecker added a commit: rKLEOPATRA4fc4188ee785: Fix sign/encrypt for S/MIME.
• ikloecker mentioned this in T6928: Kleopatra: Speed up S/MIME crypto operations for large files (on Windows).Jan 11 2024, 9:30 AM
• ikloecker removed a subtask: T6922: GpgME: Extend sign/encrypt/decrypt/verify S/MIME jobs to accept input / output from a filename.
• ikloecker changed the task status from Open to Testing.Jan 11 2024, 9:34 AM
• ikloecker updated the task description. (Show Details)

The problem has been addressed for all OpenPGP crypto operations on files.

• ikloecker added a commit: rKLEOPATRA37ee18829622: Fix sign/encrypt/decrypt/verify of notepad.Jan 19 2024, 5:13 PM
• ikloecker added a project: vsd33.May 7 2024, 10:40 AM
• ikloecker moved this task from Backlog to WiP on the vsd33 board.
• ikloecker removed a project: gpgme.
• ebo moved this task from Restricted Project Column to Restricted Project Column on the Restricted Project board.Oct 1 2024, 3:55 PM
• ikloecker mentioned this in T7395: Prepare Release Notes for Gpg4win 4.4.0 for Kleopatra.Nov 13 2024, 5:04 PM
• ebo moved this task from WiP to QA on the vsd33 board.Dec 16 2024, 11:19 AM
• ebo moved this task from Restricted Project Column to Restricted Project Column on the Restricted Project board.EditedMon, Feb 3, 11:30 AM
• ebo added a subscriber: • ebo.

Tested encryption (+ signature) via Kleopatra with a 3 GB file (random data) with:

  • VSD 3.2.4: ~ 5:20
  • VSD 3.3.0: ~ 4:25
  • command line; ~4:25

So there was a speedup for this case an it's now as fast via Kleopatra as via the cli. Mission accomplished ;-)

• ebo moved this task from Restricted Project Column to Restricted Project Column on the Restricted Project board.Mon, Feb 3, 11:30 AM
• ebo moved this task from QA to vsd-3.3.0 on the vsd33 board.
• ebo edited projects, added vsd33 (vsd-3.3.0); removed vsd33.
• werner awarded a token.Mon, Feb 3, 11:57 AM
• ebo added a comment.EditedMon, Feb 3, 12:32 PM

To make the test complete for VSD, here the times for decryption (and verification) of the 3G file from above:

  • VSD 3.2.4: ~3:36
  • VSD 3.3.0: ~2:20
  • command line: ~2:30 (maybe the bit slower than in Kleo is because this was done in a shared folder of the host system instead of on the c: partition)

The cli was done in both tests with the gpg version from VSD 3.3.0

• ebo added a comment.Mon, Feb 3, 2:26 PM

For Gpg4win 4.4.0:
Encryption: ~2:35
Decryption: ~0:44

cli, encryption: ~2:10
cli, decryption: ~ 0:44

For comparison Gpg4win 4.3.1:
Encryption: ~3:30
Decryption: ~2:10

-> definitiv an improvement for encryption + decryption but the encryption has not become as fast as on the command line.

• ebo added a comment.Thu, Feb 13, 9:23 AM

Again for Gpg4win 4.4.0, this time with better attention (and definitively encryption only):

Encryption only in Kleo, same 3GB random data file: ~ 2:23, 2:22, 2:25
(for Ted, same as below)

Encryption cli 4.4.0:
gpg -e -r ted -3G.txt
~ 2:15, 2:14, 2:15

I'd say that's close enough.