As recently mentioned here https://lists.gt.net/gnupg/users/81309 by me and by a Debian user here https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=701062 the scdaemon option "card-timeout" seems to do nothing. At least for openpgp smartcards.
Please check if this behavior can be fixed. Maybe the finding of the debian user helps (second link).
This option could be important for this feature request as well: https://dev.gnupg.org/T3362