Home GnuPG
Diffusion GnuPG 7a634e48b13c

g10: Avoid gratuitious SQLite aborts and starving writers.
7a634e48b13cUnpublished

Unpublished Commit ยท Learn More

Not On Permanent Ref: This commit is not an ancestor of any permanent ref.

Description

g10: Avoid gratuitious SQLite aborts and starving writers.

* g10/tofu.c: Include <time.h>, <utime.h>, <fcntl.h> and <unistd.h>.
(tofu_dbs_s): Add fields want_lock_file and want_lock_file_ctime.
(begin_transaction): Only yield if DBS->WANT_LOCK_FILE_CTIME has
changed since we took the lock.  Don't use gpgrt_yield to yield, but
sleep for 100ms.  After taking the batch lock, update
DBS->WANT_LOCK_FILE_CTIME.  Also take the batch lock the first time we
take the real lock.  When taking the real lock, use immediate not
deferred mode to avoid gratuitious aborts.
(end_transaction): When dropping the outermost real lock, drop the
batch lock.
(busy_handler): New function.
(opendbs): Set the busy handler to it when opening the DB.  Initialize
CTRL->TOFU.DBS->WANT_LOCK_FILE.
(tofu_closedbs): Free DBS->WANT_LOCK_FILE.

By default, SQLite defers transactions until they are actually needed.
A consequence of this is that if we have two readers and both decide
to do a write, then one has to abort. To avoid this problem, we can
make the outermost transaction an immediate transaction. This has the
disadvantage that we only allow a single reader at a time, but at
least we don't have gratuitous aborts anymore.

A second problem is that SQLite apparently doesn't actually create a
queue of waiters. The result is that doing a sched_yield between
dropping and retaking the batch transaction is not enough to allow the
other process to make progress. Instead, we need to wait a
while (emperically: 100ms seems reasonable). To avoid waiting when
there is no contention, we use a new file's timestamp to signal that
there is a waiter.

  • Signed-off-by: Neal H. Walfield <neal@g10code.com>

Details

Provenance
Neal H. Walfield <neal@g10code.com>Authored on Oct 31 2016, 3:02 AM
Parents
rGeec365a02bd3: g10: Avoid reading in keys when possible.
Branches
Unknown
Tags
Unknown

Event Timeline

Neal H. Walfield <neal@g10code.com> committed rG7a634e48b13c: g10: Avoid gratuitious SQLite aborts and starving writers. (authored by Neal H. Walfield <neal@g10code.com>).Oct 31 2016, 3:10 AM
neal mentioned this in Unknown Object (Maniphest Task).Mar 27 2017, 5:10 PM
neal mentioned this in Unknown Object (Maniphest Task).Mar 27 2017, 5:33 PM