GPGME error: "invalid crypto engine" in the MSYS2 version
Closed, InvalidPublic
Actions

Assigned To

Authored By

	lovetox
	Feb 27 2018, 12:23 AM

Description

Hi,

Im on Windows 10 x64

I installed gpgme from MSYS2, mingw-w64-i686-gpgme which in turn installs automatically
mingw-w64-i686-gnupg (2.2.4)

afterwards i run one of the python lang bindings examples

def print_engine_infos():
    print("gpgme version:", gpg.core.check_version(None))
    print("engines:")

    for engine in gpg.core.get_engine_info():
        print(engine.file_name, engine.version)

    for proto in [gpg.constants.protocol.OpenPGP, gpg.constants.protocol.CMS]:
        print("Have {}? {}".format(gpg.core.get_protocol_name(proto),
                                   gpg.core.engine_check_version(proto)))

this results in

gpgme version: 1.10.0
engines:
C:\msys64\mingw32\bin\gpgconf.exe 1.0.0
/nonexistent 1.0.0
Have OpenPGP? False
Have CMS? False

every other example that uses OpenPGP runs into the invalid engine error

the directory is correct, gpg.exe is inside C:\msys64\mingw32\bin, but it does not seem to find it.

im at a loss as what to do here

Details

Version: 1.10.0

Related Objects

Mentioned Here: T3505: Port GPGME's Python bindings to Windows
T4086: GPGME: sharing C compiler configuration with SWIG for generated bindings
T3515: Gpg4win: Gpgconf used to open "windows" and slows down kleo startup

Event Timeline

lovetox created this task.Feb 27 2018, 12:23 AM

This may be related to T3515: Gpg4win: Gpgconf used to open "windows" and slows down kleo startup since it depends on data from gpgconf.

The two lines following the engines: line above are actually supposed to be six lines and the one with gpgconf normally displays the same version number as GPG. For example, here is the output for that same couple of lines of code on a FreeBSD system with GPG 2.2.5 and GPGME 1.10.0:

>>> for x in gpg.core.get_engine_info():
...     print(x.file_name, x.version)
/usr/local/bin/gpg2 2.2.5
/usr/local/bin/gpgsm 2.2.5
/usr/local/bin/gpgconf 2.2.5
/home/ben/.gnupg/S.gpg-agent 1.0.0
/home/ben/.gnupg/S.uiserver 1.0.0
/nonexistent 1.0.0
>>>

The first three lines are the paths to the executables, the fourth and fifth are the sockets which component programs connect to and the last one is a dummy entry which tells GPGME there are no more engines to search for.

BenM claimed this task.Apr 11 2018, 3:37 AM

• werner renamed this task from GPGME error: invalid crypto engine to GPGME error: "invalid crypto engine" in the MSYS2 version.Apr 17 2018, 10:34 AM

We never tried to build gpgme with MSYS2 and I would also say this is not supported. A wild guess is that this mixes platform specific code.

Not only is this not supported, but I've now confirmed that MSYS2 isn't even supported by its own project and they direct all downloaders to their MinGW-get installer.

That said, MSYS2 hasn't been updated since 2011 and MinGW does not appear to have been updated since 2013. Though I believe the latter is deceptive and the installer simply points to servers where they update the underlying packages. Nevertheless, MSYS2 support is far too ancient to deal with since it's almost too old to have even known python 3 existed at all.

Slight addendum: MSYS2 isn't even part of MinGW at all, see the commentary in this bug report. Nor is it a part of Cygwin, but apparently it is a part of a fork of Cygwin.

What a mess, this illustrates once again why forks ought to be renamed and make it clear what they are.

You seem to have reached the wrong page when searching for msys2

This is under very active development, see https://github.com/msys2/msys2 or http://www.msys2.org/

please reconsider looking into this, this is probably a small problem with finding the right path in this enviroment, GPGME is already packaged there and builds and installs fine as does GNUPG

if that is the case config.{guess,sub} needs to support this and we should be able to handle this the same way as other Unix platforms.

If that is indeed the case and the theory regarding runtime conflicts, currently under investigation in T3505 and T4086, also proves to be true; then MSys2 users and developers will need to cease using the precompiled versions of Python available from python.org and compile their own version of Python copy with MSys2.

This very likely also means that all C Python modules, bindings and other wrappers will also need to be compiled using the same runtime.

I mean, @lovetox, we are correct in inferring that you installed Python from one of the Windows installers on python.org, right? Which one was it?

Alternatively, if they wish to keep using the Python installer from python.org then they would need to drop MSys2 in favour of the same version of Microsoft Visual Studio used to compile the that specific version of Python with and use it to compile every part of the GnuPG stack, up to and including GPGME.

Though this is only the case if the current theory regarding runtime conflicts causing these problems is accurate. While that has not yet been confirmed, there is a strong consensus as to it being very likely and the SWIG documentation indicates that compilers used with the C code being bound and the languages the bindings are being generated for must match.

Note that on POSIX systems, including OS X, it is common for both Python and the GnuPG stack to either be manually compiled by the end user or prepared by the same package management build system; in each scenario both components (Python and GPGME) would be compiled using the same compilers and runtimes. Hence these sorts of issues are generally avoided by default. Since all OS X build environments ultimately depend on XCode and Apple's Developer Tools, everything there uses the same build environment, regardless of whether or not Python is obtained from python.org or not.

BenM, msys2 uses pacman as packagemanager, all packages are build from source

I dont see a reason why it would use different compilers.

see here the PGKBUILD files and also the patches applied

https://github.com/Alexpux/MINGW-packages/tree/master/mingw-w64-gpgme
https://github.com/Alexpux/MINGW-packages/tree/master/mingw-w64-gnupg

there is one patch that caught my interest 0004-gpgme-find-gnupg.patch

here python3 but its quite massive
https://github.com/Alexpux/MINGW-packages/tree/master/mingw-w64-python3

i fear i can be of not much help at this point because im just a user of msys2, but maybe someone with experience with building can see some obvious problems with the PKGBUILD files

Or with both packages installed, could i maybe debug somehow where it searches?

I close this for now, this seems a problem of the mingw packages in msys2

Actually i have now more debug output and i think i found the issue

_gpgme_mkstemp seems always to fail.

Any pointers how i can further debug this?

Before you ask, i cleared the tmp folder, so the tmp file didnt exist, and after this was run, 3 tmp files were created, i guess for each call to gpgconf one

GPGME 2018-08-08 23:24:55 <0x1e30>  gpgme_debug: level=9
GPGME 2018-08-08 23:24:55 <0x1e30>  gpgme_debug: gpgme='C:\msys64\mingw32\bin'
GPGME 2018-08-08 23:24:55 <0x1e30>  gpgme_check_version: call: 0=00000000, req_version=(null), VERSION=1.11.1
GPGME 2018-08-08 23:24:55 <0x1e30>  gpgme_check_version_internal: call: 0=00000000, req_version=(null), offset_sig_validity=32
GPGME 2018-08-08 23:24:55 <0x1e30>  gpgme_check_version: call: 0=00000000, req_version=(null), VERSION=1.11.1
GPGME 2018-08-08 23:24:55 <0x1e30>  gpgme_check_version_internal: call: 0=00000000, req_version=(null), offset_sig_validity=32
gpgme version: 1.11.1
engines:
GPGME 2018-08-08 23:24:55 <0x1e30>  gpgme-dinfo: gpgconf='C:\msys64\mingw32\bin\gpgconf.exe'
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_pipe: enter: filedes=0028f220, inherit_idx=1 (GPGME uses it for reading)
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_pipe: leave: read=0x0 (00000114/0x0), write=0x1 (00000124/0x0)
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_spawn: enter: path=0050cab8, path=C:\msys64\mingw32\bin\gpgconf.exe
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_spawn: check: path=0050cab8, argv[ 0] = C:\msys64\mingw32\bin\gpgconf.exe
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_spawn: check: path=0050cab8, argv[ 1] = --list-dirs
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_spawn: check: path=0050cab8, _gpgme_mkstemp failed: File exists
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_spawn: error: File exists
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: enter: fd=00000000
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: check: fd=00000000, fd=0 -> handle=00000114 socket=-1 dupfrom=-1
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: leave: result=0
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: enter: fd=00000001
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: check: fd=00000001, fd=1 -> handle=00000124 socket=-1 dupfrom=-1
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: leave: result=0
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_pipe: enter: filedes=0028f220, inherit_idx=1 (GPGME uses it for reading)
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_pipe: leave: read=0x0 (00000124/0x0), write=0x1 (00000118/0x0)
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_spawn: enter: path=0050cab8, path=C:\msys64\mingw32\bin\gpgconf.exe
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_spawn: check: path=0050cab8, argv[ 0] = C:\msys64\mingw32\bin\gpgconf.exe
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_spawn: check: path=0050cab8, argv[ 1] = --list-components
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_spawn: check: path=0050cab8, _gpgme_mkstemp failed: File exists
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_spawn: error: File exists
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: enter: fd=00000000
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: check: fd=00000000, fd=0 -> handle=00000124 socket=-1 dupfrom=-1
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: leave: result=0
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: enter: fd=00000001
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: check: fd=00000001, fd=1 -> handle=00000118 socket=-1 dupfrom=-1
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: leave: result=0
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_pipe: enter: filedes=0028f5fc, inherit_idx=1 (GPGME uses it for reading)
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_pipe: leave: read=0x0 (00000118/0x0), write=0x1 (00000114/0x0)
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_spawn: enter: path=0050cab8, path=C:\msys64\mingw32\bin\gpgconf.exe
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_spawn: check: path=0050cab8, argv[ 0] = C:\msys64\mingw32\bin\gpgconf.exe
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_spawn: check: path=0050cab8, argv[ 1] = --version
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_spawn: check: path=0050cab8, _gpgme_mkstemp failed: File exists
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_spawn: error: File exists
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: enter: fd=00000000
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: check: fd=00000000, fd=0 -> handle=00000118 socket=-1 dupfrom=-1
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: leave: result=0
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: enter: fd=00000001
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: check: fd=00000001, fd=1 -> handle=00000114 socket=-1 dupfrom=-1
GPGME 2018-08-08 23:24:55 <0x1e30>  _gpgme_io_close: leave: result=0
C:\msys64\mingw32\bin\gpgconf.exe 1.0.0
/nonexistent 1.0.0
GPGME 2018-08-08 23:24:55 <0x1e30>  engine.c:161: returning error: Ung▒ltige Krypto-Engine
Have OpenPGP? False
GPGME 2018-08-08 23:24:55 <0x1e30>  engine.c:161: returning error: Ung▒ltige Krypto-Engine
Have CMS? False

Ok i saw they apply custom patches to _gpgme_mkstemp which are outdated and should be revisited, sorry for the noise

GPGME error: "invalid crypto engine" in the MSYS2 versionClosed, InvalidPublicActions

Description

Details

Related Objects

Event Timeline

GPGME error: "invalid crypto engine" in the MSYS2 version
Closed, InvalidPublic
Actions