GPGME Python Bindings HOWTO org-babel examples losing indenting
Closed, ResolvedPublic

Description

Python source code included in the org-mode HOWTO displays correctly for very simple functions, such as where the Python indenting is only one level deep (i.e. four characters of whitespace as per PEP8 compliance).

However when an example includes nested indented blocks, such as in some of the encryption and signature verification examples), though the source code displays correctly in the org file and, of course, in the source block itself (which uses the correct python-mode in Emacs), it does not export to HTML or XHTML correctly and retaining the nested indenting.

It is not yet clear whether this is a configuration issue at my end, a known issue with certain export/publishing formats or a bug in org-mode and/or org-babel.

Currently using Emacs 25.3 and the latest release of Org-Mode (9.1.13); further testing required.

BenM created this task.May 13 2018, 8:37 PM
BenM created this object with visibility "All Users".
BenM added a comment.EditedMay 14 2018, 5:31 AM

Org-Mode was updated to today's release and further testing was conducted.

A correction of previous observation: the org-mode python source block does not in fact honour the indentation of the code. So this is not an export/publishing issue and is most likely a bug in either org-mode itself or babel.

BenM changed the visibility from "All Users" to "Public (No Login Required)".May 14 2018, 5:33 AM
BenM added a comment.May 15 2018, 5:25 AM

As a work-around for this bug I've ported the HOWTO from org-mode to DITA XML and will generate a webhelp-responsive (i.e. searchable) version to put on another website (an Amazon S3 bucket since it will be reliable and cheap) in the interim.

BenM added a comment.May 15 2018, 7:33 AM

Webhelp version of the Python bindings HOWTO is currently available here:

http://files.au.adversary.org/crypto/gpgme-python-howto/webhelp/index.html

BenM added a comment.Jul 9 2018, 2:28 PM

Massive overhaul of the entire document performed in this commit.

If this doesn't fix it then the problem really must be with babel in which case we either lose the colour enhancements by not telling org/babel that this is python code and just make it text to retain the indenting or we switch to using reStructured Text for Python documentation (perhaps this is why reST exists instead of just adopting org-mode in python-land) or we switch to something else capable of documenting everything properly.

While reST does python documentation brilliantly, there are no guarantees when it comes to the remainder of the GnuPG Project documentation requirements. Those requirements include documenting all the non-Python stuff, producing validating output (where applicable) and producing Info pages which integrate into Emacs.

So if org can't cope here we need to either mix use the org files to generate reST files for just this bit, but that still leaves broken info files for the python bindings. So that's out.
Or do it all with reST and generate org to ... no, that still leaves broken info files for the Python bindings ...

The source format would need to be able to produce an org file and produce output with correctly formatted python (reST optional, but would be nice to appease the pythonistas) and able to produce Info files even for the python bits ...

I can only think of one thing, but it might require hiring an XSLT/XPath/XQuery specialist to produce DITA to Texinfo XML and DITA to Docutils XML transformations. Although once that's done, all that is needed is a means of feeding org-mode into DITA directly (markdown to DITA already exists).

BenM added a comment.Jul 9 2018, 3:20 PM

Even after running additional tests like converting the file to reStructuredText, fixing any remaining possible errors and then converting back, the result is that org-mode still appears to save the source correctly, but cannot export it in a correct format with babel. Certainly not to XHTML and likely not to any other format either. So that's definitely a bug.

Whereas the work-around draft format using DITA XML has absolutely no trouble at all, includes syntax highlighting, correct indentation, is easily selectable to copy and is searchable. I'm fast running out of reasons to not use XML here other than "but org-mode is what we've always used" and "org-mode is really nice to write documentation in with Emacs" of which only the second one has any validity at all. The first argument is merely fear of change.

Anyway, if DITA transformations (essentially XSLTs) to both the Texinfo XML and Docutils XML can be obtained we can get around the bulk of the authoring issues by using org-mode and sending it to DITA via either Markdown or XHTML and then automate everything from there using the DITA Open Toolkit. Ideally getting the DITA-OT to handle the Texinfo XML to .texi transformation and the Docutils XML to .rst, as well, but if not then ant should handle it (or libxml). That should appease everyone ... with the added bonus that it will also provide reliable information (and be capable of future scaling issues when used with multiple translations, but no need to worry about that now).

BenM closed this task as Resolved.EditedJul 22 2018, 7:18 PM

Since first observing this … annoyance … the following updates have been made: Emacs has been upgraded to version 26.1, Org-Mode has been updated multiple times, including significant changes to Babel and the XHTML export, python-mode has been updated, multiple variations on the source blocks have been attempted, the document has had any and all tabs stripped out and replaced, plus each code block has been refactored and re-entered multiple times.

Somewhere in all that, it finally started behaving itself. It's unclear how delicate that state is, though. Still, for now it seems to be okay, so closing this as fixed (for now).

Fix effective from commit 4d1642b11ea64b8009a8720d9b59e89636d691d2 (master).