I'm encountering a problem with gcry_mpi_powm() as used by libssh2. On a 32 bit
x86 Linux system with both libgcrypt-1.5.4 and libgcrypt-1.6.4 I discovered that
this function occasionally doesn't return (or, at least it doesn't after a day
of waiting) during the calculations performed on an ssh connection. I tracked
the problem to a loop within _gcry_mpi_powm that is initialized < 0 and
therefore never hits the == 0 condition to exit without the counter variable
The loop is at line 661 of mpi/mpi-pow.c:
for (j += W - c0; j; j--)
On one example of this condition being hit, on entry j=-139312672, W=3 and c0=0.
I tried creating a standalone program to reproduce this by hard-coding a set of
four arguments to gcry_mpi_powm that caused this condition to be hit, but didn't
succeed in getting it to end up in this loop. It only takes a few dozen ssh
connections to get it into this state, so it's easy to reproduce. How can I
track this down further?