@tab @uref{https://gnupg.org/people/index.html#sec-1-5, Ben McGinnes} <ben@@gnupg.org>
@item Author GPG Key:
@tab DB4724E6FA4286C92B4E55C4321E4E2373590E5D
@item Language:
@tab Australian English, British English
@item xml:lang:
@tab en-AU, en-GB, en
@end multitable
This document provides basic instruction in how to use the GPGME
Python bindings to programmatically leverage the GPGME library.
@menu
* Python 2 versus Python 3::
* Examples::
@end menu
@node Python 2 versus Python 3
@section Python 2 versus Python 3
Though the GPGME Python bindings themselves provide support for both
Python 2 and 3, the focus is unequivocally on Python 3 and
specifically from Python 3.4 and above. As a consequence all the
examples and instructions in this guide use Python 3 code.
Much of it will work with Python 2, but much of it also deals with
Python 3 byte literals, particularly when reading and writing data.
Developers concentrating on Python 2.7, and possibly even 2.6, will
need to make the appropriate modifications to support the older string
and unicode types as opposed to bytes.
There are multiple reasons for concentrating on Python 3; some of
which relate to the immediate integration of these bindings, some of
which relate to longer term plans for both GPGME and the python
bindings and some of which relate to the impending EOL period for
Python 2.7. Essentially, though, there is little value in tying the
bindings to a version of the language which is a dead end and the
advantages offered by Python 3 over Python 2 make handling the data
types with which GPGME deals considerably easier.
@node Examples
@section Examples
All of the examples found in this document can be found as Python 3
scripts in the @samp{lang/python/examples/howto} directory.
@node GPGME Concepts
@chapter GPGME Concepts
@menu
* A C API::
* Python bindings::
* Difference between the Python bindings and other GnuPG Python packages::
@end menu
@node A C API
@section A C API
Unlike many modern APIs with which programmers will be more familiar
with these days, the GPGME API is a C API. The API is intended for
use by C coders who would be able to access its features by including
the @samp{gpgme.h} header file with their own C source code and then access
its functions just as they would any other C headers.
This is a very effective method of gaining complete access to the API
and in the most efficient manner possible. It does, however, have the
drawback that it cannot be directly used by other languages without
some means of providing an interface to those languages. This is
where the need for bindings in various languages stems.
@node Python bindings
@section Python bindings
The Python bindings for GPGME provide a higher level means of
accessing the complete feature set of GPGME itself. It also provides
a more pythonic means of calling these API functions.
The bindings are generated dynamically with SWIG and the copy of
@samp{gpgme.h} generated when GPGME is compiled.
This means that a version of the Python bindings is fundamentally tied
to the exact same version of GPGME used to generate that copy of
@samp{gpgme.h}.
@node Difference between the Python bindings and other GnuPG Python packages
@section Difference between the Python bindings and other GnuPG Python packages
There have been numerous attempts to add GnuPG support to Python over
the years. Some of the most well known are listed here, along with
what differentiates them.
@menu
* The python-gnupg package maintained by Vinay Sajip::
* The gnupg package created and maintained by Isis Lovecruft::
* The PyME package maintained by Martin Albrecht::
@end menu
@node The python-gnupg package maintained by Vinay Sajip
@subsection The python-gnupg package maintained by Vinay Sajip
This is arguably the most popular means of integrating GPG with
Python. The package utilises the @samp{subprocess} module to implement
wrappers for the @samp{gpg} and @samp{gpg2} executables normally invoked on the
command line (@samp{gpg.exe} and @samp{gpg2.exe} on Windows).
The popularity of this package stemmed from its ease of use and
capability in providing the most commonly required features.
Unfortunately it has been beset by a number of security issues in the
past; most of which stemmed from using unsafe methods of accessing the
command line via the @samp{subprocess} calls. While some effort has been
made over the last two to three years (as of 2018) to mitigate this,
particularly by no longer providing shell access through those
subprocess calls, the wrapper is still somewhat limited in the scope
of its GnuPG features coverage.
The python-gnupg package is available under the MIT license.
@node The gnupg package created and maintained by Isis Lovecruft
@subsection The gnupg package created and maintained by Isis Lovecruft
In 2015 Isis Lovecruft from the Tor Project forked and then
re-implemented the python-gnupg package as just gnupg. This new
package also relied on subprocess to call the @samp{gpg} or @samp{gpg2}
binaries, but did so somewhat more securely.
The naming and version numbering selected for this package, however,
resulted in conflicts with the original python-gnupg and since its
functions were called in a different manner to python-gnupg, the
release of this package also resulted in a great deal of consternation
when people installed what they thought was an upgrade that
subsequently broke the code relying on it.
The gnupg package is available under the GNU General Public License
version 3.0 (or any later version).
@node The PyME package maintained by Martin Albrecht
@subsection The PyME package maintained by Martin Albrecht
This package is the origin of these bindings, though they are somewhat
different now. For details of when and how the PyME package was
folded back into GPGME itself see the @emph{Short History} document@footnote{@samp{Short_History.org} and/or @samp{Short_History.html}.}
in the Python bindings @samp{docs} directory.@footnote{The @samp{lang/python/docs/} directory in the GPGME source.}
The PyME package was first released in 2002 and was also the first
attempt to implement a low level binding to GPGME. In doing so it
provided access to considerably more functionality than either the
@samp{python-gnupg} or @samp{gnupg} packages.
The PyME package is only available for Python 2.6 and 2.7.
Porting the PyME package to Python 3.4 in 2015 is what resulted in it
being folded into the GPGME project and the current bindings are the
end result of that effort.
The PyME package is available under the same dual licensing as GPGME
itself: the GNU General Public License version 2.0 (or any later
version) and the GNU Lesser General Public License version 2.1 (or any
later version).
@node GPGME Python bindings installation
@chapter GPGME Python bindings installation
@menu
* No PyPI::
* Requirements::
* Installation::
* Known Issues::
@end menu
@node No PyPI
@section No PyPI
Most third-party Python packages and modules are available and
distributed through the Python Package Installer, known as PyPI.
Due to the nature of what these bindings are and how they work, it is
infeasible to install the GPGME Python bindings in the same way.
This is because the bindings use SWIG to dynamically generate C
bindings against @samp{gpgme.h} and @samp{gpgme.h} is generated from
@samp{gpgme.h.in} at compile time when GPGME is built from source. Thus to
include a package in PyPI which actually built correctly would require
either statically built libraries for every architecture bundled with
it or a full implementation of C for each architecture.
See the additional notes regarding @ref{CFFI is the Best™ and GPGME should use it instead of SWIG, , CFFI and SWIG} at the end of this
section for further details.
@node Requirements
@section Requirements
The GPGME Python bindings only have three requirements:
@enumerate
@item
A suitable version of Python 2 or Python 3. With Python 2 that
means Python 2.7 and with Python 3 that means Python 3.4 or higher.
@item
@uref{https://www.swig.org, SWIG}.
@item
GPGME itself. Which also means that all of GPGME's dependencies
must be installed too.
@end enumerate
@node Installation
@section Installation
Installing the Python bindings is effectively achieved by compiling
and installing GPGME itself.
Once SWIG is installed with Python and all the dependencies for GPGME
are installed you only need to confirm that the version(s) of Python
you want the bindings installed for are in your @samp{$PATH}.
By default GPGME will attempt to install the bindings for the most
recent or highest version number of Python 2 and Python 3 it detects
in @samp{$PATH}. It specifically checks for the @samp{python} and @samp{python3}
executables first and then checks for specific version numbers.
For Python 2 it checks for these executables in this order: @samp{python},
@samp{python2} and @samp{python2.7}.
For Python 3 it checks for these executables in this order: @samp{python3},
@samp{python3.6}, @samp{python3.5}, @samp{python3.4} and @samp{python3.7}.@footnote{As Python 3.7 is a very recent release, it is not given
priority over 3.6 yet, but will probably be prioritised by the release
of Python 3.7.2.}
@menu
* Installing GPGME::
@end menu
@node Installing GPGME
@subsection Installing GPGME
See the GPGME @samp{README} file for details of how to install GPGME from
source.
@node Known Issues
@section Known Issues
There are a few known issues with the current build process and the
Python bindings. For the most part these are easily addressed should
they be encountered.
@menu
* Breaking Builds::
* Multiple installations::
* Won't Work With Windows::
* CFFI is the Best™ and GPGME should use it instead of SWIG::
@end menu
@node Breaking Builds
@subsection Breaking Builds
Occasionally when installing GPGME with the Python bindings included
it may be observed that the @samp{make} portion of that process induces a
large very number of warnings and, eventually errors which end that
part of the build process. Yet following that with @samp{make check} and
@samp{make install} appears to work seamlessly.
The cause of this is related to the way SWIG needs to be called to
dynamically generate the C bindings for GPGME in the first place. So
the entire process will always produce @samp{lang/python/python2-gpg/} and
@samp{lang/python/python3-gpg/} directories. These should contain the
build output generated during compilation, including the complete
bindings and module installed into @samp{site-packages}.
Occasionally the errors in the early part or some other conflict
(e.g. not installing as @strong{@emph{root}} or @strong{@emph{su}}) may result in nothing
being installed to the relevant @samp{site-packages} directory and the
build directory missing a lot of expected files. Even when this
occurs, the solution is actually quite simple and will always work.
That solution is simply to run the following commands as either the
@strong{root} user or prepended with @samp{sudo -H}@footnote{Yes, even if you use virtualenv with everything you do in
Python. If you want to install this module as just your user account
then you will need to manually configure, compile and install the
@emph{entire} GnuPG stack as that user as well. This includes libraries
which are not often installed that way. It can be done and there are
circumstances under which it is worthwhile, but generally only on
POSIX systems which utilise single user mode (some even require it).} in the @samp{lang/python/}
directory:
@example
/path/to/pythonX.Y setup.py build
/path/to/pythonX.Y setup.py build
/path/to/pythonX.Y setup.py install
@end example
Yes, the build command does need to be run twice. Yes, you still need
to run the potentially failing or incomplete steps during the
@samp{configure}, @samp{make} and @samp{make install} steps with installing GPGME.
This is because those steps generate a lot of essential files needed,
both by and in order to create, the bindings (including both the
@samp{setup.py} and @samp{gpgme.h} files).
@enumerate
@item
IMPORTANT Note
If specifying a selected number of languages to create bindings for,
try to leave Python last. Currently the majority of the other
language bindings are also preceding Python of either version when
listed alphabetically and so that just happens by default currently.
If Python is set to precede one of the other languages then it is
possible that the errors described here may interrupt the build
process before generating bindings for those other languages. In
these cases it may be preferable to configure all preferred language
bindings separately with alternative @samp{configure} steps for GPGME using
the @samp{--enable-languages=$LANGUAGE} option.
@end enumerate
@node Multiple installations
@subsection Multiple installations
For a veriety of reasons it may be either necessary or just preferable
to install the bindings to alternative installed Python versions which
meet the requirements of these bindings.
On POSIX systems this will generally be most simply achieved by
running the manual installation commands (build, build, install) as
described in the previous section for each Python installation the
bindings need to be installed to.
As per the SWIG documentation: the compilers, libraries and runtime
used to build GPGME and the Python Bindings @strong{must} match those used to
compile Python itself, including the version number(s) (at least going
by major version numbers and probably minor numbers too).
On most POSIX systems, including OS X, this will very likely be the
case in most, if not all, cases.
@node Won't Work With Windows
@subsection Won't Work With Windows
There are semi-regular reports of Windows users having considerable
difficulty in installing and using the Python bindings at all. Very
often, possibly even always, these reports come from Cygwin users
and/or MinGW users and/or Msys2 users. Though not all of them have
been confirmed, it appears that these reports have also come from
people who installed Python using the Windows installer files from the
@uref{https://python.org, Python website} (i.e. mostly MSI installers, sometimes self-extracting
@samp{.exe} files).
The Windows versions of Python are not built using Cygwin, MinGW or
Msys2; they're built using Microsoft Visual Studio. Furthermore the
version used is @emph{considerably} more advanced than the version which
MinGW obtained a small number of files from many years ago in order to
be able to compile anything at all. Not only that, but there are
changes to the version of Visual Studio between some micro releases,
though that is is particularly the case with Python 2.7, since it has
been kept around far longer than it should have been.
There are two theoretical solutions to this issue:
@enumerate
@item
Compile and install the GnuPG stack, including GPGME and the
Python bibdings using the same version of Microsoft Visual Studio
used by the Python Foundation to compile the version of Python
installed.
If there are multiple versions of Python then this will need to be
done with each different version of Visual Studio used.
@item
Compile and install Python using the same tools used by choice,
such as MinGW or Msys2.
@end enumerate
Do @strong{not} use the official Windows installer for Python unless
following the first method.
In this type of situation it may even be for the best to accept that
there are less limitations on permissive software than free software
and simply opt to use a recent version of the Community Edition of
Microsoft Visual Studio to compile and build all of it, no matter
what.
Investigations into the extent or the limitations of this issue are
ongoing.
@node CFFI is the Best™ and GPGME should use it instead of SWIG
@subsection CFFI is the Best™ and GPGME should use it instead of SWIG
There are many reasons for favouring @uref{https://cffi.readthedocs.io/en/latest/overview.html, CFFI} and proponents of it are
quite happy to repeat these things as if all it would take to switch
from SWIG to CFFI is repeating that list as if it were a new concept.
The fact is that there are things which Python's CFFI implementation
cannot handle in the GPGME C code. Beyond that there are features of
SWIG which are simply not available with CFFI at all. SWIG generates
the bindings to Python using the @samp{gpgme.h} file, but that file is not
a single version shipped with each release, it too is generated when
GPGME is compiled.
CFFI is currently unable to adapt to such a potentially mutable
codebase. If there were some means of applying SWIG's dynamic code
generation to produce the Python/CFFI API modes of accessing the GPGME
libraries (or the source source code directly), but such a thing does
not exist yet either and it currently appears that work is needed in
at least one of CFFI's dependencies before any of this can be
addressed.
So if you're a massive fan of CFFI; that's great, but if you want this
project to switch to CFFI then rather than just insisting that it
should, I'd suggest you volunteer to bring CFFI up to the level this
project needs.
If you're actually seriously considering doing so, then I'd suggest
taking the @samp{gpgme-tool.c} file in the GPGME @samp{src/} directory and
getting that to work with any of the CFFI API methods (not the ABI
methods, they'll work with pretty much anything). When you start
running into trouble with "ifdefs" then you'll know what sort of
things are lacking. That doesn't even take into account the amount of
work saved via SWIG's code generation techniques either.
There's a greater likelihood of turning to Cython to add something to
these bindings at this point than there is of turning to CFFI. Except
that there's little reason to do so when the C code in question either
already exists or could be produced fairly quickly by the far more
expedient means of asking either @uref{https://gnupg.org/people/index.html#sec-1-1, Werner}, @uref{https://gnupg.org/people/index.html#sec-1-3, Niibe} or @uref{https://gnupg.org/people/index.html#sec-1-6, Andre} to write
whatever was missing. Perhaps even @uref{https://gnupg.org/people/index.html#sec-1-4, Jussi}, but there shouldn't be any
need for these bindings to interface directly with his work and for
much the same reasons as we recommend people not call libgcrypt
directly unless they know what they're doing.
@node Fundamentals
@chapter Fundamentals
Before we can get to the fun stuff, there are a few matters regarding
GPGME's design which hold true whether you're dealing with the C code
directly or these Python bindings.
@menu
* No REST::
* Context::
@end menu
@node No REST
@section No REST
The first part of which is or will be fairly blatantly obvious upon
viewing the first example, but it's worth reiterating anyway. That
being that this API is @emph{@strong{not}} a REST API. Nor indeed could it ever
be one.
Most, if not all, Python programmers (and not just Python programmers)
know how easy it is to work with a RESTful API. In fact they've
become so popular that many other APIs attempt to emulate REST-like
behaviour as much as they are able. Right down to the use of JSON
formatted output to facilitate the use of their API without having to
retrain developers.
This API does not do that. It would not be able to do that and also
provide access to the entire C API on which it's built. It does,
however, provide a very pythonic interface on top of the direct
bindings and it's this pythonic layer that this HOWTO deals with.
@node Context
@section Context
One of the reasons which prevents this API from being RESTful is that
most operations require more than one instruction to the API to
perform the task. Sure, there are certain functions which can be
performed simultaneously, particularly if the result known or strongly
anticipated (e.g. selecting and encrypting to a key known to be in the
public keybox).
There are many more, however, which cannot be manipulated so readily:
they must be performed in a specific sequence and the result of one
operation has a direct bearing on the outcome of subsequent
operations. Not merely by generating an error either.
When dealing with this type of persistent state on the web, full of
both the RESTful and REST-like, it's most commonly referred to as a
session. In GPGME, however, it is called a context and every
operation type has one.
@node Working with keys
@chapter Working with keys
@menu
* Key selection::
* Get key::
* Importing keys::
* Exporting keys::
@end menu
@node Key selection
@section Key selection
Selecting keys to encrypt to or to sign with will be a common
occurrence when working with GPGMe and the means available for doing
so are quite simple.
They do depend on utilising a Context; however once the data is
recorded in another variable, that Context does not need to be the
same one which subsequent operations are performed.
The easiest way to select a specific key is by searching for that
key's key ID or fingerprint, preferably the full fingerprint without
any spaces in it. A long key ID will probably be okay, but is not
advised and short key IDs are already a problem with some being
generated to match specific patterns. It does not matter whether the
pattern is upper or lower case.
So this is the best method:
@example
import gpg
k = gpg.Context().keylist(pattern="258E88DCBD3CD44D8E7AB43F6ECB6AF0DEADBEEF")
keys = list(k)
@end example
This is passable and very likely to be common:
@example
import gpg
k = gpg.Context().keylist(pattern="0x6ECB6AF0DEADBEEF")
keys = list(k)
@end example
And this is a really bad idea:
@example
import gpg
k = gpg.Context().keylist(pattern="0xDEADBEEF")
keys = list(k)
@end example
Alternatively it may be that the intention is to create a list of keys
which all match a particular search string. For instance all the
addresses at a particular domain, like this:
@example
import gpg
ncsc = gpg.Context().keylist(pattern="ncsc.mil")
nsa = list(ncsc)
@end example
@menu
* Counting keys::
@end menu
@node Counting keys
@subsection Counting keys
Counting the number of keys in your public keybox (@samp{pubring.kbx}), the
format which has superseded the old keyring format (@samp{pubring.gpg} and
@samp{secring.gpg}), or the number of secret keys is a very simple task.
@example
import gpg
c = gpg.Context()
seckeys = c.keylist(pattern=None, secret=True)
pubkeys = c.keylist(pattern=None, secret=False)
seclist = list(seckeys)
secnum = len(seclist)
publist = list(pubkeys)
pubnum = len(publist)
print("""
Number of secret keys: @{0@}
Number of public keys: @{1@}
""".format(secnum, pubnum))
@end example
@node Get key
@section Get key
An alternative method of getting a single key via its fingerprint is
available directly within a Context with @samp{Context().get_key}. This is
the preferred method of selecting a key in order to modify it, sign or
certify it and for obtaining relevant data about a single key as a
part of other functions; when verifying a signature made by that key,
for instance.
By default this method will select public keys, but it can select
secret keys as well.
This first example demonstrates selecting the current key of Werner
All of these draft versions are generated from this document via Emacs
@uref{https://orgmode.org/, Org mode} and @uref{https://www.gnu.org/software/texinfo/, GNU Texinfo}. Though it is likely that the specific @uref{https://files.au.adversary.org/crypto/gpgme-python-howto.org, file}
@uref{http://files.au.adversary.org/crypto/gpgme-python-howto.org, version} used will be on the same server with the generated output
formats.
In addition to these there is a significantly less frequently updated
version as a HTML @uref{https://files.au.adversary.org/crypto/gpgme-python-howto/webhelp/index.html, WebHelp site} (AWS S3 SSL); generated from DITA XML
source files, which can be found in @uref{https://dev.gnupg.org/source/gpgme/browse/ben%252Fhowto-dita/, an alternative branch} of the GPGME
git repository.
These draft editions are not official documents and the version of
documentation in the master branch or which ships with released
versions is the only official documentation. Nevertheless, these
draft editions may occasionally be of use by providing more accessible
web versions which are updated between releases. They are provided on
the understanding that they may contain errors or may contain content
subject to change prior to an official release.
@node License GPL compatible
@section License GPL compatible
This file is free software; as a special exception the author gives
unlimited permission to copy and/or distribute it, with or without
modifications, as long as this notice is preserved.
This file is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY, to the extent permitted by law; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
| Language: | Australian English, British English |
| xml:lang: | en-AU, en-GB, en |
This document provides basic instruction in how to use the GPGME
Python bindings to programmatically leverage the GPGME library.
** Python 2 versus Python 3
:PROPERTIES:
:CUSTOM_ID: py2-vs-py3
:END:
Though the GPGME Python bindings themselves provide support for both
Python 2 and 3, the focus is unequivocally on Python 3 and
specifically from Python 3.4 and above. As a consequence all the
examples and instructions in this guide use Python 3 code.
Much of it will work with Python 2, but much of it also deals with
Python 3 byte literals, particularly when reading and writing data.
Developers concentrating on Python 2.7, and possibly even 2.6, will
need to make the appropriate modifications to support the older string
and unicode types as opposed to bytes.
There are multiple reasons for concentrating on Python 3; some of
which relate to the immediate integration of these bindings, some of
which relate to longer term plans for both GPGME and the python
bindings and some of which relate to the impending EOL period for
Python 2.7. Essentially, though, there is little value in tying the
bindings to a version of the language which is a dead end and the
advantages offered by Python 3 over Python 2 make handling the data
types with which GPGME deals considerably easier.
** Examples
:PROPERTIES:
:CUSTOM_ID: howto-python3-examples
:END:
All of the examples found in this document can be found as Python 3
scripts in the =lang/python/examples/howto= directory.
* GPGME Concepts
:PROPERTIES:
:CUSTOM_ID: gpgme-concepts
:END:
** A C API
:PROPERTIES:
:CUSTOM_ID: gpgme-c-api
:END:
Unlike many modern APIs with which programmers will be more familiar
with these days, the GPGME API is a C API. The API is intended for
use by C coders who would be able to access its features by including
the =gpgme.h= header file with their own C source code and then access
its functions just as they would any other C headers.
This is a very effective method of gaining complete access to the API
and in the most efficient manner possible. It does, however, have the
drawback that it cannot be directly used by other languages without
some means of providing an interface to those languages. This is
where the need for bindings in various languages stems.
** Python bindings
:PROPERTIES:
:CUSTOM_ID: gpgme-python-bindings
:END:
The Python bindings for GPGME provide a higher level means of
accessing the complete feature set of GPGME itself. It also provides
a more pythonic means of calling these API functions.
The bindings are generated dynamically with SWIG and the copy of
=gpgme.h= generated when GPGME is compiled.
This means that a version of the Python bindings is fundamentally tied
to the exact same version of GPGME used to generate that copy of
=gpgme.h=.
** Difference between the Python bindings and other GnuPG Python packages
:PROPERTIES:
:CUSTOM_ID: gpgme-python-bindings-diffs
:END:
There have been numerous attempts to add GnuPG support to Python over
the years. Some of the most well known are listed here, along with
what differentiates them.
*** The python-gnupg package maintained by Vinay Sajip
:PROPERTIES:
:CUSTOM_ID: diffs-python-gnupg
:END:
This is arguably the most popular means of integrating GPG with
Python. The package utilises the =subprocess= module to implement
wrappers for the =gpg= and =gpg2= executables normally invoked on the
command line (=gpg.exe= and =gpg2.exe= on Windows).
The popularity of this package stemmed from its ease of use and
capability in providing the most commonly required features.
Unfortunately it has been beset by a number of security issues in the
past; most of which stemmed from using unsafe methods of accessing the
command line via the =subprocess= calls. While some effort has been
made over the last two to three years (as of 2018) to mitigate this,
particularly by no longer providing shell access through those
subprocess calls, the wrapper is still somewhat limited in the scope
of its GnuPG features coverage.
The python-gnupg package is available under the MIT license.
*** The gnupg package created and maintained by Isis Lovecruft
:PROPERTIES:
:CUSTOM_ID: diffs-isis-gnupg
:END:
In 2015 Isis Lovecruft from the Tor Project forked and then
re-implemented the python-gnupg package as just gnupg. This new
package also relied on subprocess to call the =gpg= or =gpg2=
binaries, but did so somewhat more securely.
The naming and version numbering selected for this package, however,
resulted in conflicts with the original python-gnupg and since its
functions were called in a different manner to python-gnupg, the
release of this package also resulted in a great deal of consternation
when people installed what they thought was an upgrade that
subsequently broke the code relying on it.
The gnupg package is available under the GNU General Public License
version 3.0 (or any later version).
*** The PyME package maintained by Martin Albrecht
:PROPERTIES:
:CUSTOM_ID: diffs-pyme
:END:
This package is the origin of these bindings, though they are somewhat
different now. For details of when and how the PyME package was
folded back into GPGME itself see the /Short History/ document[fn:1]
in the Python bindings =docs= directory.[fn:2]
The PyME package was first released in 2002 and was also the first
attempt to implement a low level binding to GPGME. In doing so it
provided access to considerably more functionality than either the
=python-gnupg= or =gnupg= packages.
The PyME package is only available for Python 2.6 and 2.7.
Porting the PyME package to Python 3.4 in 2015 is what resulted in it
being folded into the GPGME project and the current bindings are the
end result of that effort.
The PyME package is available under the same dual licensing as GPGME
itself: the GNU General Public License version 2.0 (or any later
version) and the GNU Lesser General Public License version 2.1 (or any
later version).
* GPGME Python bindings installation
:PROPERTIES:
:CUSTOM_ID: gpgme-python-install
:END:
** No PyPI
:PROPERTIES:
:CUSTOM_ID: do-not-use-pypi
:END:
Most third-party Python packages and modules are available and
distributed through the Python Package Installer, known as PyPI.
Due to the nature of what these bindings are and how they work, it is
infeasible to install the GPGME Python bindings in the same way.
This is because the bindings use SWIG to dynamically generate C
bindings against =gpgme.h= and =gpgme.h= is generated from
=gpgme.h.in= at compile time when GPGME is built from source. Thus to
include a package in PyPI which actually built correctly would require
either statically built libraries for every architecture bundled with
it or a full implementation of C for each architecture.
See the additional notes regarding [[#snafu-cffi][CFFI and SWIG]] at the end of this
section for further details.
** Requirements
:PROPERTIES:
:CUSTOM_ID: gpgme-python-requirements
:END:
The GPGME Python bindings only have three requirements:
1. A suitable version of Python 2 or Python 3. With Python 2 that
means Python 2.7 and with Python 3 that means Python 3.4 or higher.
2. [[https://www.swig.org][SWIG]].
3. GPGME itself. Which also means that all of GPGME's dependencies
must be installed too.
** Installation
:PROPERTIES:
:CUSTOM_ID: installation
:END:
Installing the Python bindings is effectively achieved by compiling
and installing GPGME itself.
Once SWIG is installed with Python and all the dependencies for GPGME
are installed you only need to confirm that the version(s) of Python
you want the bindings installed for are in your =$PATH=.
By default GPGME will attempt to install the bindings for the most
recent or highest version number of Python 2 and Python 3 it detects
in =$PATH=. It specifically checks for the =python= and =python3=
executables first and then checks for specific version numbers.
For Python 2 it checks for these executables in this order: =python=,
=python2= and =python2.7=.
For Python 3 it checks for these executables in this order: =python3=,
=python3.6=, =python3.5=, =python3.4= and =python3.7=.[fn:3]
*** Installing GPGME
:PROPERTIES:
:CUSTOM_ID: install-gpgme
:END:
See the GPGME =README= file for details of how to install GPGME from
source.
** Known Issues
:PROPERTIES:
:CUSTOM_ID: snafu
:END:
There are a few known issues with the current build process and the
Python bindings. For the most part these are easily addressed should
they be encountered.
*** Breaking Builds
:PROPERTIES:
:CUSTOM_ID: snafu-a-swig-of-this-builds-character
:END:
Occasionally when installing GPGME with the Python bindings included
it may be observed that the =make= portion of that process induces a
large very number of warnings and, eventually errors which end that
part of the build process. Yet following that with =make check= and
=make install= appears to work seamlessly.
The cause of this is related to the way SWIG needs to be called to
dynamically generate the C bindings for GPGME in the first place. So
the entire process will always produce =lang/python/python2-gpg/= and
=lang/python/python3-gpg/= directories. These should contain the
build output generated during compilation, including the complete
bindings and module installed into =site-packages=.
Occasionally the errors in the early part or some other conflict
(e.g. not installing as */root/* or */su/*) may result in nothing
being installed to the relevant =site-packages= directory and the
build directory missing a lot of expected files. Even when this
occurs, the solution is actually quite simple and will always work.
That solution is simply to run the following commands as either the
*root* user or prepended with =sudo -H=[fn:4] in the =lang/python/=
directory:
#+BEGIN_SRC shell
/path/to/pythonX.Y setup.py build
/path/to/pythonX.Y setup.py build
/path/to/pythonX.Y setup.py install
#+END_SRC
Yes, the build command does need to be run twice. Yes, you still need
to run the potentially failing or incomplete steps during the
=configure=, =make= and =make install= steps with installing GPGME.
This is because those steps generate a lot of essential files needed,
both by and in order to create, the bindings (including both the
=setup.py= and =gpgme.h= files).
**** IMPORTANT Note
:PROPERTIES:
:CUSTOM_ID: snafu-swig-build-note
:END:
If specifying a selected number of languages to create bindings for,
try to leave Python last. Currently the majority of the other
language bindings are also preceding Python of either version when
listed alphabetically and so that just happens by default currently.
If Python is set to precede one of the other languages then it is
possible that the errors described here may interrupt the build
process before generating bindings for those other languages. In
these cases it may be preferable to configure all preferred language
bindings separately with alternative =configure= steps for GPGME using
the =--enable-languages=$LANGUAGE= option.
*** Multiple installations
:PROPERTIES:
:CUSTOM_ID: snafu-the-full-monty
:END:
For a veriety of reasons it may be either necessary or just preferable
to install the bindings to alternative installed Python versions which
meet the requirements of these bindings.
On POSIX systems this will generally be most simply achieved by
running the manual installation commands (build, build, install) as
described in the previous section for each Python installation the
bindings need to be installed to.
As per the SWIG documentation: the compilers, libraries and runtime
used to build GPGME and the Python Bindings *must* match those used to
compile Python itself, including the version number(s) (at least going
by major version numbers and probably minor numbers too).
On most POSIX systems, including OS X, this will very likely be the
case in most, if not all, cases.
*** Won't Work With Windows
:PROPERTIES:
:CUSTOM_ID: snafu-runtime-not-funtime
:END:
There are semi-regular reports of Windows users having considerable
difficulty in installing and using the Python bindings at all. Very
often, possibly even always, these reports come from Cygwin users
and/or MinGW users and/or Msys2 users. Though not all of them have
been confirmed, it appears that these reports have also come from
people who installed Python using the Windows installer files from the
[[https://python.org][Python website]] (i.e. mostly MSI installers, sometimes self-extracting
=.exe= files).
The Windows versions of Python are not built using Cygwin, MinGW or
Msys2; they're built using Microsoft Visual Studio. Furthermore the
version used is /considerably/ more advanced than the version which
MinGW obtained a small number of files from many years ago in order to
be able to compile anything at all. Not only that, but there are
changes to the version of Visual Studio between some micro releases,
though that is is particularly the case with Python 2.7, since it has
been kept around far longer than it should have been.
There are two theoretical solutions to this issue:
1. Compile and install the GnuPG stack, including GPGME and the
Python bibdings using the same version of Microsoft Visual Studio
used by the Python Foundation to compile the version of Python
installed.
If there are multiple versions of Python then this will need to be
done with each different version of Visual Studio used.
2. Compile and install Python using the same tools used by choice,
such as MinGW or Msys2.
Do *not* use the official Windows installer for Python unless
following the first method.
In this type of situation it may even be for the best to accept that
there are less limitations on permissive software than free software
and simply opt to use a recent version of the Community Edition of
Microsoft Visual Studio to compile and build all of it, no matter
what.
Investigations into the extent or the limitations of this issue are
ongoing.
*** CFFI is the Best™ and GPGME should use it instead of SWIG
:PROPERTIES:
:CUSTOM_ID: snafu-cffi
:END:
There are many reasons for favouring [[https://cffi.readthedocs.io/en/latest/overview.html][CFFI]] and proponents of it are
quite happy to repeat these things as if all it would take to switch
from SWIG to CFFI is repeating that list as if it were a new concept.
The fact is that there are things which Python's CFFI implementation
cannot handle in the GPGME C code. Beyond that there are features of
SWIG which are simply not available with CFFI at all. SWIG generates
the bindings to Python using the =gpgme.h= file, but that file is not
a single version shipped with each release, it too is generated when
GPGME is compiled.
CFFI is currently unable to adapt to such a potentially mutable
codebase. If there were some means of applying SWIG's dynamic code
generation to produce the Python/CFFI API modes of accessing the GPGME
libraries (or the source source code directly), but such a thing does
not exist yet either and it currently appears that work is needed in
at least one of CFFI's dependencies before any of this can be
addressed.
So if you're a massive fan of CFFI; that's great, but if you want this
project to switch to CFFI then rather than just insisting that it
should, I'd suggest you volunteer to bring CFFI up to the level this
project needs.
If you're actually seriously considering doing so, then I'd suggest
taking the =gpgme-tool.c= file in the GPGME =src/= directory and
getting that to work with any of the CFFI API methods (not the ABI
methods, they'll work with pretty much anything). When you start
running into trouble with "ifdefs" then you'll know what sort of
things are lacking. That doesn't even take into account the amount of
work saved via SWIG's code generation techniques either.
There's a greater likelihood of turning to Cython to add something to
these bindings at this point than there is of turning to CFFI. Except
that there's little reason to do so when the C code in question either
already exists or could be produced fairly quickly by the far more
expedient means of asking either [[https://gnupg.org/people/index.html#sec-1-1][Werner]], [[https://gnupg.org/people/index.html#sec-1-3][Niibe]] or [[https://gnupg.org/people/index.html#sec-1-6][Andre]] to write
whatever was missing. Perhaps even [[https://gnupg.org/people/index.html#sec-1-4][Jussi]], but there shouldn't be any
need for these bindings to interface directly with his work and for
much the same reasons as we recommend people not call libgcrypt
directly unless they know what they're doing.
* Fundamentals
:PROPERTIES:
:CUSTOM_ID: howto-fund-a-mental
:END:
Before we can get to the fun stuff, there are a few matters regarding
GPGME's design which hold true whether you're dealing with the C code
directly or these Python bindings.
** No REST
:PROPERTIES:
:CUSTOM_ID: no-rest-for-the-wicked
:END:
The first part of which is or will be fairly blatantly obvious upon
viewing the first example, but it's worth reiterating anyway. That
being that this API is /*not*/ a REST API. Nor indeed could it ever
be one.
Most, if not all, Python programmers (and not just Python programmers)
know how easy it is to work with a RESTful API. In fact they've
become so popular that many other APIs attempt to emulate REST-like
behaviour as much as they are able. Right down to the use of JSON
formatted output to facilitate the use of their API without having to
retrain developers.
This API does not do that. It would not be able to do that and also
provide access to the entire C API on which it's built. It does,
however, provide a very pythonic interface on top of the direct
bindings and it's this pythonic layer that this HOWTO deals with.
** Context
:PROPERTIES:
:CUSTOM_ID: howto-get-context
:END:
One of the reasons which prevents this API from being RESTful is that
most operations require more than one instruction to the API to
perform the task. Sure, there are certain functions which can be
performed simultaneously, particularly if the result known or strongly
anticipated (e.g. selecting and encrypting to a key known to be in the
public keybox).
There are many more, however, which cannot be manipulated so readily:
they must be performed in a specific sequence and the result of one
operation has a direct bearing on the outcome of subsequent
operations. Not merely by generating an error either.
When dealing with this type of persistent state on the web, full of
both the RESTful and REST-like, it's most commonly referred to as a
session. In GPGME, however, it is called a context and every
operation type has one.
* Working with keys
:PROPERTIES:
:CUSTOM_ID: howto-keys
:END:
** Key selection
:PROPERTIES:
:CUSTOM_ID: howto-keys-selection
:END:
Selecting keys to encrypt to or to sign with will be a common
occurrence when working with GPGMe and the means available for doing
so are quite simple.
They do depend on utilising a Context; however once the data is
recorded in another variable, that Context does not need to be the
same one which subsequent operations are performed.
The easiest way to select a specific key is by searching for that
key's key ID or fingerprint, preferably the full fingerprint without
any spaces in it. A long key ID will probably be okay, but is not
advised and short key IDs are already a problem with some being
generated to match specific patterns. It does not matter whether the
pattern is upper or lower case.
So this is the best method:
#+BEGIN_SRC python -i
import gpg
k = gpg.Context().keylist(pattern="258E88DCBD3CD44D8E7AB43F6ECB6AF0DEADBEEF")
keys = list(k)
#+END_SRC
This is passable and very likely to be common:
#+BEGIN_SRC python -i
import gpg
k = gpg.Context().keylist(pattern="0x6ECB6AF0DEADBEEF")
keys = list(k)
#+END_SRC
And this is a really bad idea:
#+BEGIN_SRC python -i
import gpg
k = gpg.Context().keylist(pattern="0xDEADBEEF")
keys = list(k)
#+END_SRC
Alternatively it may be that the intention is to create a list of keys
which all match a particular search string. For instance all the
addresses at a particular domain, like this:
#+BEGIN_SRC python -i
import gpg
ncsc = gpg.Context().keylist(pattern="ncsc.mil")
nsa = list(ncsc)
#+END_SRC
*** Counting keys
:PROPERTIES:
:CUSTOM_ID: howto-keys-counting
:END:
Counting the number of keys in your public keybox (=pubring.kbx=), the
format which has superseded the old keyring format (=pubring.gpg= and
=secring.gpg=), or the number of secret keys is a very simple task.
#+BEGIN_SRC python -i
import gpg
c = gpg.Context()
seckeys = c.keylist(pattern=None, secret=True)
pubkeys = c.keylist(pattern=None, secret=False)
seclist = list(seckeys)
secnum = len(seclist)
publist = list(pubkeys)
pubnum = len(publist)
print("""
Number of secret keys: {0}
Number of public keys: {1}
""".format(secnum, pubnum))
#+END_SRC
** Get key
:PROPERTIES:
:CUSTOM_ID: howto-get-key
:END:
An alternative method of getting a single key via its fingerprint is
available directly within a Context with =Context().get_key=. This is
the preferred method of selecting a key in order to modify it, sign or
certify it and for obtaining relevant data about a single key as a
part of other functions; when verifying a signature made by that key,
for instance.
By default this method will select public keys, but it can select
secret keys as well.
This first example demonstrates selecting the current key of Werner
All of these draft versions are generated from this document via Emacs
[[https://orgmode.org/][Org mode]] and [[https://www.gnu.org/software/texinfo/][GNU Texinfo]]. Though it is likely that the specific [[https://files.au.adversary.org/crypto/gpgme-python-howto.org][file]]
[[http://files.au.adversary.org/crypto/gpgme-python-howto.org][version]] used will be on the same server with the generated output
formats.
In addition to these there is a significantly less frequently updated
version as a HTML [[https://files.au.adversary.org/crypto/gpgme-python-howto/webhelp/index.html][WebHelp site]] (AWS S3 SSL); generated from DITA XML
source files, which can be found in [[https://dev.gnupg.org/source/gpgme/browse/ben%252Fhowto-dita/][an alternative branch]] of the GPGME
git repository.
These draft editions are not official documents and the version of
documentation in the master branch or which ships with released
versions is the only official documentation. Nevertheless, these
draft editions may occasionally be of use by providing more accessible
web versions which are updated between releases. They are provided on
the understanding that they may contain errors or may contain content
subject to change prior to an official release.
** License GPL compatible
:PROPERTIES:
:CUSTOM_ID: license
:END:
This file is free software; as a special exception the author gives
unlimited permission to copy and/or distribute it, with or without
modifications, as long as this notice is preserved.
This file is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY, to the extent permitted by law; without even the
implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR