For ECC, it is better to use KEM API, so that the implementation is more coherent.
I looked into the code and figured out that we don't need to change the protocol between gpgsm and gpg-agent much (the KDF parameters are sent to gpg-agent too as well as the encrypted session key).
Simply it is enough to move the work of ECDH KDF things from gpgsm to gpg-agent; The return value will be decrypted session key.