I missed to properly encode a comma and ran into this:
$ echo 'keyserver:0:"ldap%3Aldap%3A389%3A%3A%3Adc=gnupg,dc=test%3A' | gpgconf --change-options gpgsm gpgconf: Ohhhh jeeee: Assertion "*arg == '"'" in change_options_program failed (../../tools/gpgconf-comp.c:2597) Aborted (core dumped)