Page MenuHome GnuPG

Add progress status output to gpgtar
Open, NormalPublic

Description

gpgtar does not yet output progess indication. If can be partly done using

gpgtar --gpg-args --enable-progress-filer --status-fd N ....

but that is limited because gpg does not known the total size of data. However, gpgtar knows this because it first scans all files. The proposal is to emit progress status lines at the gpgtar site with information on the number of files processed and the their size. Frontends may then pick whatever they like.

Event Timeline

werner triaged this task as Normal priority.Jan 30 2023, 2:59 PM
werner created this task.

How with --status-fd passed to gpgtar we will get these progress lines:

[GNUPG:] PROGRESS gpgtar c 5000 0
[GNUPG:] PROGRESS gpgtar c 10000 0
[...]
[GNUPG:] PROGRESS gpgtar c 50000 0
[GNUPG:] PROGRESS gpgtar c 0 50223
[GNUPG:] PROGRESS gpgtar s 0 6543 MiB
[GNUPG:] PROGRESS gpgtar c 100 50223
[...]
[GNUPG:] PROGRESS gpgtar s 6300 6543 MiB
[GNUPG:] PROGRESS gpgtar s 6400 6543 MiB
[GNUPG:] PROGRESS gpgtar c 44800 50223
[...]
[GNUPG:] PROGRESS gpgtar c 50210 50223
[GNUPG:] PROGRESS gpgtar s 6543 6543 MiB

The first lines with zero as last value are emitted during the initial scan and thus before starting to write. Followed by this are two lines with the expected numbers (50223 files with a total size of 6543 MiB). The writing takes place and the current values are updated. As of now in increments of 100 for the file count and in 100MiB for the total size. The last two lines give the actual written numbers. The file count is a bit less because a few files were not suitable for archiving.

When also enabling the progress filter of gpg, the status line from gpg are mixed in like here:

[GNUPG:] PROGRESS gpgtar s 100 6543 MiB
[GNUPG:] PROGRESS stdin ? 108224 0 KiB
[GNUPG:] PROGRESS stdin ? 127488 0 KiB

Due to compression and the tar format overhead gpg still does not known the final size of the file. gpg uses the name of the input file as first word; this will always be "stdin" here. Thus grepping for gpgatr and 'c' or 's' is sufficient to distinguish the two outputs.

werner moved this task from Backlog to WiP on the gnupg24 board.

I guess we need some gpgme support as well.

How to handle the extraction progress depends on how we can pass the size of the filename etc. Also gpgme related.

werner edited projects, added gnupg24 (gnupg-2.4.1); removed gnupg24.