5. Thread safety¶
The module relies on the global interpreter lock (GIL) to maintain the consistency of internal data structures. Internally, the building blocks are mostly normal Python objects, although they’re accessed and modified via the C API. The GIL is expected to be held by the method caller, so that accesses are ordered as a result of the GIL’s functioning. When a thread is using a method, it can be certain that no other thread is modifying the data at the same time.
However, there are four places where the protection is not perfect. These are
the callback, the key’s
__hash__()
/__eq__()
methods (used by the internal
dict
), and the __del__()
method on the key or value. In
the notes on compatibility we have described the precaution
about these code paths that can in principle do anything.
Here, we should mention the qualities of these callable objects that makes the programme overall safer. We will refer to them as “external code”.
First, avoid releasing the GIL in the external code. The GIL is released, for
example, when input/output (I/O) is performed. Some code may drop the GIL when
computing the hash by a C extension on an internal buffer for speed. Even if __hash__()
may be made to execute
before entering the critical section (relying on not-so-public Python C API),
__eq__()
currently cannot be. When a thread-switch happens in the
middle of a method call, another thread may try to call into a method, too,
causing contention. There’s limited built-in ability to detect this at
run-time, but currently,
Warning
No additional locking is performed,
because locking tend to degrade single-thread performance and necessarily make the code more complex. Decisions about contention-handling (such as blocked waiting, timed waiting, or other tactics) is best made by client code.
Second, avoid unwanted side effects in the external code. Modifying the
LRUDict
object itself is one such effect. Again, this can be detected
at runtime, but it is not preferable to rely on this.
Overall, callbacks and __del__()
are mostly safe. However, the
timing and order of the callback execution cannot be guaranteed in general in a
threaded environment.
5.1. Summary¶
Note
As long as the key’s
__hash__()
and__eq__()
do not drop the GIL or modify theLRUDict
, usage in a threaded environment is generally safe.The callback and the
__del__()
method of keys or values are generally safe, but it is not preferable to modify theLRUDict
in there or clog up the purge queue by taking long computations.Limited ability to detect contention at runtime is provided (see
LRUDictBusyError
), and the user can implement their own synchronization.For single-thread programme it’s “life as usual”, and the GIL provides adequate protection.
5.2. Example: contention¶
In the following Python script, we simulate a (highly contrived) situation
where individual threads simply try to insert keys into a cache without
synchronization. The GIL is dropped in the __hash__
and __eq__
methods of the key by taking some time to read from a file.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | import threading
from lru_ng import LRUDict
class Key:
def __init__(self, value, read_blocks=128):
self._v = value
self.nbytes = read_blocks * 512
def __hash__(self):
h = hash(self._v)
remaining = self.nbytes
with open("/dev/urandom", "rb") as f:
while remaining > 0:
remaining -= len(f.read(remaining)) # Release GIL
return h
def __eq__(self, other):
try:
v = other._v
except AttributeError:
v = other
remaining = self.nbytes
with open("/dev/urandom", "rb") as f:
while remaining > 0:
remaining -= len(f.read(remaining)) # Release GIL
return self._v == v
cache = LRUDict(5)
def runner(n):
for i in range(n):
k = Key(i)
cache[k] = i # Insertion without synchronization
num_threads = 10
num_keys = 1000
threads = [threading.Thread(target=runner, args=(num_keys,))
for i in range(num_threads)]
for th in threads:
th.start()
for th in threads:
th.join()
|
The program will output error messages like this:
Traceback (most recent call last):
...
File "run_insertion.py", line 36, in runner
cache[k] = i # Insertion without synchronization
lru_ng.LRUDictBusyError: attempted entry into LRUDict critical section while busy
(The actual error messages themselves from all the threads may be out of order too!) The threads are failing because of the unhandled error.
If the __hash__
or __eq__
methods had not released the GIL,
these errors would have been prevented.
By adding a lock around the insertion operation, we can make it “work” in the
sense of ensuring sequential access to the global cache. The following snippet
replacing the definition of runner()
should make the error messages
disappear:
32 33 34 35 36 37 38 | cond = threading.Condition()
def runner(n):
for i in range(n):
k = Key(i)
with cond: # Wait on the underlying lock
cache[k] = i
cond.notify() # Wake up one thread waiting on the lock
|