Great spotting! This was it. Quite embarrassing that I've looked at this code so many time yet it didn't cross my mind to double check arguments order.

Note that exactly same data and length computed by ssh_signature_encoder_rsa, including additional 0, reach:
https://github.com/openssh/openssh-portable/blob/V_10_2_P1/sshkey.c#L517-L537

Added to some debug logging and whenever login issue occurs new logic is applied:
https://github.com/gpg/gnupg/blob/bc7c91bee521e4adf3506ca32bf34177b84ce1c5/agent/command-ssh.c#L1482

Looks like indeed related to T7882. After reverting c7e0ec12609b401ea81c4851522d86eb5ec27170 I was able to make 2000 connections without any issue. Bringing the change back and retrying issue appeared within first 300.

I've already tried with verbose which gave no errors. That's why I moved to debug logging. With double verbose I don't see anything wrong either. Excerpt from log for relevant 100 connections among which 1 failed:

$ cat gpg.log | 
    sed 's/.*gpg-agent\[[0-9]*\] //'  | # remove date, time and process id                            
    grep -v 'ssh handler .* \(started\|terminated\)' | # appears to be mostly noise wit hex address
    sort|uniq -c
     80 new connection to /usr/libexec/gnupg2/scdaemon daemon established
     20 new connection to /usr/libexec/gnupg2/scdaemon daemon established (reusing)
    100 received ssh request of length 1
    100 received ssh request of length 208
    100 received ssh request of length 748
    100 sending ssh response of length 1
    100 sending ssh response of length 281
    100 sending ssh response of length 626
    100 ssh request handler for extension (27) ready
    100 ssh request handler for extension (27) started
    100 ssh request handler for request_identities (11) ready
    100 ssh request handler for request_identities (11) started
    100 ssh request handler for sign_request (13) ready
    100 ssh request handler for sign_request (13) started
    100 ssh-agent extension 'session-bind@openssh.com' not supported
    100 ssh-agent extension 'session-bind@openssh.com' received

Looks like libksba 1.6.1 is available for download at: https://gnupg.org/download/ , however tag is missing at: https://git.gnupg.org/cgi-bin/gitweb.cgi?p=libksba.git;a=summary

Package maintainer from PLD here. We still ship Qt4 and therefore provide pinentry qt4 for as long as it's supported. I have no problem with dropping it if it's no longer support, but last release still supported Qt4, there's no mention of dropping such support in NEWS and both code as well as configure.ac appear to still carry Qt4 support which is a bit confusing.