Page MenuHome GnuPG

GPG: Extend / rework "is_file_compressed"
Closed, ResolvedPublic

Description

The current code in common/miscellaneous.c to detect whether or not it makes sense to compress only checks for bzip, gzip, zip and pgp compression. And as far as I can see it it also only does this when a filename is given directly to GnuPG. This would cause GPGME using callers never to have the advantage of disabled compression.
From a GPGME clients standpoint I wold probably prefer if this would be handled in GPGMEs data_identify as this can then be handled with a pipe IO for GnuPG when the underlying data is seekable.

Otherwise maybe just checking the file extension which we provide with --set-filename might already be a huge improvement, even without data /magic inspection.

Event Timeline

aheinecke created this task.

The compression check currently detects bzip2, gzip, zip, pkzip, and PDF. This also covers common document formats like odt and docx. We may add some more detection in the future. However, for large files you usually know their type and thus you better use "-z0" for already compressed data or "-z-1" if you want to force compression (may be for PDFs which often can be a shrinked to 80% or so).

werner changed the task status from Open to Testing.Jan 19 2023, 10:54 AM
werner edited projects, added gnupg24, gnupg22; removed gnupg.
werner moved this task from Backlog to WiP on the gnupg22 board.
werner moved this task from Backlog to QA on the gnupg24 board.

Great! But as mentioned I would like to have a setting in Kleo to explicitly disable compression, GPGME_ENCRYPT_NO_COMPRESS. But that is a different task.

werner moved this task from Restricted Project Column to Restricted Project Column on the Restricted Project board.

On the mailing list I wrote down my test results for en- and decrypting files (1-10 GB) with GnuPG and Gpg4win/Kleopatra. The encryption always ran with compression. Kleopatra needed more than 11 minutes to encrypt a file that is 10 GB big. Today I tested the encryption again but this time I added

compress-level 0

to gpg.conf (I also tried to add bzip2-compress-level 0 and then only compress-algo uncompressed because Bernhard was suggesting that in the mailing list but it made no difference).

With that option Kleopatra needed only around 5,5 minutes for a 10 GB file.

werner edited projects, added gnupg24 (gnupg-2.4.1); removed gnupg24.
aheinecke claimed this task.

From my side this can be closed. In Kleopatra we can maybe check for some more MIME types and then use GPGME_ENCRYPT_NO_COMPRESS but that is unreleated.

werner edited projects, added gnupg22 (gnupg-2.2.42); removed gnupg22.

This has been well tested during development and is thus ready for a release.

ebo moved this task from Restricted Project Column to Restricted Project Column on the Restricted Project board.Nov 15 2023, 11:08 AM