Thanks for the reply. Not sure about GPGME/gpgme-json. Anyway, it still ends up in gpgme code, isn't it?
I used to modify gpgme sources to receive more information about the issue.
Looks like the next step would be to modify gpg-conf and see what's going on there, or leave it to the Thunderbird developers.

Reading the bugzilla report it seems that TB is loading gpgme at runtime. In particular the hints on using externally build stuff (Homebrew) is worrying. Someone(tm) needs to check how gpgme is used by TB and that it is properly initialized. GPGME is actually not designed to be loaded at runtime but should be used as standard shared object or static library.

In fact, Thunderbird does not use gpgme-json, but loads the gpgme shared library at runtime. The interesting thing is that Thunderbird works fine if gpgOSX is used.

I'm not sure if you can open a shared library created with homebrew in an application that is created using the "standard" macOS toolchain (XCode/clang etc).

In T5250#141705, @werner wrote:

Sure that TB uses GPGME - they claimed they won't use it due to license incompatibility (LGPL). I assumed they use gpgme-json via naticve messaging.

TB can and does ship LGPL.
TB cannot ship GPL.

Thunderbird cannot use anything requiring GPL in its default configuration, because Thunderbird wants to distribute a single MPL licensed package that includes all components that are required for OpenPGP.
The scenario discussed here is about an optional configuration, not the default configuration.

Because we currently cannot offer a solution supporting smartcards out of the box, we're offering the optional use of GPGME as explained here:
https://wiki.mozilla.org/Thunderbird:OpenPGP:Smartcards

In this scenario, it's the user's responsibility to install and configure the optional components, like GPGME and GnuPG.
Thunderbird doesn't control which optional components are used.

I understand that in theory, distributing GPGME with Thunderbird is permissible.
However, I also understand that is a bad idea, because GPGME and GnuPG are supposed to be combined in compatible versions.
That's why we don't bundle it, but require that the user installs it as part of the advanced configuration.

From what has been said in this ticket, it sounds that users may be using an incompatible combination of GPGME and GnuPG.
If that's true, then the above Thunderbird HOWTO page could be edited to more clearly warn about this detail,
and if certain combinations are known not to work, we can mention them

At the time I started to add an optional binding from Thunderbird to GPGME, I wasn't aware of gpgme-json.

It seems gpgme-json is intended to execute in the Web JavaScript sandbox of a browser.

While Thunderbird sometimes runs JavaScript code in a sandbox, for example when opening a web page to use oauth, or when loading an RSS article, or if an Add-on wants to display a web page, this is a different scope.

Thunderbird's OpenPGP user interface and messaging integration code runs as privileged code. It is implemented in JavaScript, and isn't sandboxed, in the same way all Thunderbird internal JavaScript that implements the user interface isn't sandboxed.

So, gpgme-json seems to have certain expectations, like running in a browser context, being able to use webpack, bundling modules, that may not be suitable for the way we use internal JS code in TB. At least I'd expect adjustments to be necessary.

Furthermore, we have the specific requirement that GPGME is used for decryption and signing, but specifically not for encryption and verification - because TB wants to use GPGME for secret key operations, only - but always wants to use its bundled OpenPGP engine for public key operations - because it always wants to use its internal trust management for public keys. That means we require GPGME_DECRYPT_UNWRAP API, which apparently isn't offered by gpgme-json

My current impression is, calling directly into GPGME using the C API calls seems more appropriate and easier for our needs.

I said "we're offering the optional use of GPGME

I should have said:

"we're offering the optional use of GPGME for sceret key operations"

In T5250#143872, @kaie wrote:

It seems gpgme-json is intended to execute in the Web JavaScript sandbox of a browser.

I don't that it requires a Web JavaScript Sandbox. In my eyes, gpgme-json is intended to be executed in any environment. The result is a JSON object, which can be parsed using var x = JSON.parse(data), and then processed safely.

Using GPGME is probably the best way, even if gpgme-json might also work for some operations.

GPGME is actually not designed to be loaded at runtime but should be used as standard shared object

@werner can you elaborate on why loading a shared object at runtime is different from loading it at startup and how this is affected by gpgme's design?

(Here are some explanations of the general differences I've found: https://stackoverflow.com/questions/2055840/difference-between-load-time-dynamic-linking-and-run-time-dynamic-linking )

BTW @kaie

Thunderbird cannot use anything requiring GPL in its default configuration, because Thunderbird wants to distribute a single MPL licensed package that includes all components that are required for OpenPGP.

Any pointer why, they have made that choice, though? A bundle of MPL and GNU GPL components is fully allowed by the licenses as far as I know.

In T5250#148131, @bernhard wrote:

BTW @kaie

Thunderbird cannot use anything requiring GPL in its default configuration, because Thunderbird wants to distribute a single MPL licensed package that includes all components that are required for OpenPGP.

Any pointer why, they have made that choice, though? A bundle of MPL and GNU GPL components is fully allowed by the licenses as far as I know.

https://blog.gerv.net/2012/01/mozilla-projects-and-gpled-code/

https://blog.gerv.net/2012/01/mozilla-projects-and-gpled-code/

@kaie, thanks for the pointer!

Reading it, I'll find it inconclusive. It is 9 years old and the licensing policy has
since allowed the use of GNU LGPL libraries. So a use of GPGME which is under GNU LGPL would be meanwhile okay. If this is okay, a bundle shipping with GnuPG is also fine from the licensing point of view as it has less dependencies than a linked in library as it can be more easily exchanged.

(Feel free to mail me directly bernhard@intevation.de for followups or to find a better place for the discussion, it seems we are getting more offtopic in this special issue.)

Back to the MacOS case, I read that in the Mozilla report (see external link at top),
it works now. So it could have been a problem of incompatible build properties.

Of course it would be cooll to identify the problem, especially if it comes up again.
And potentially fix it or find a detection method so it can be checked.

Reading the mozilla entry more carefully, there still seems to be an issue.

An there is another defect with GPGME and Thunderbird on Windows, strangely it seems to change behaviour if GPGME_DEBUG=7 is used.
https://bugzilla.mozilla.org/show_bug.cgi?id=1681060

To extend on this: dlopen'ing of gpgme is NOT SUPPORTED. It is in general not a good idea to do this on standard Unix systems. On Windows we could make it work because DLLs on that platform are well designed and not a hack like the Unix shared objects.

dlopen'ing of gpgme is NOT SUPPORTED. It is in general not a good idea to do this on standard Unix systems.

Can you expand or point to an explanation why this is a bad idea in general? (The use case is: only load a library/module if the user (configuration) actually wants its functionality. If not, it is okay that the library is not there.)

If this is the case, what is the the recommendation for MacOS and GNU/Linux users on how to integrate with GnuPG in that use case? (It is not calling "gpg" by themselves I presume. >;) )

On Windows we could make it work because DLLs on that platform are well designed and not a hack like the Unix shared objects.

(I read: on Linux/MacOS like systems the .so are missing important structure or information?)