weakref_Weak References.txt

The weakref module allows the Python programmer to create weak references to
objects.

In the following, the term referent means the object which is referred to by a
weak reference.

A weak reference to an object is not enough to keep the object alive: when the
only remaining references to a referent are weak references, garbage
collection is free to destroy the referent and reuse its memory for something
else. A primary use for weak references is to implement caches or mappings
holding large objects, where it’s desired that a large object not be kept
alive solely because it appears in a cache or mapping.

For example, if you have a number of large binary image objects, you may wish
to associate a name with each. If you used a Python dictionary to map names to
images, or images to names, the image objects would remain alive just because
they appeared as values or keys in the dictionaries. The WeakKeyDictionary and
WeakValueDictionary classes supplied by the weakref module are an alternative,
using weak references to construct mappings that don’t keep objects alive
solely because they appear in the mapping objects. If, for example, an image
object is a value in a WeakValueDictionary, then when the last remaining
references to that image object are the weak references held by weak mappings,
garbage collection can reclaim the object, and its corresponding entries in
weak mappings are simply deleted.

WeakKeyDictionary and WeakValueDictionary use weak references in their
implementation, setting up callback functions on the weak references that
notify the weak dictionaries when a key or value has been reclaimed by garbage
collection. Most programs should find that using one of these weak dictionary
types is all they need – it’s not usually necessary to create your own weak
references directly. The low-level machinery used by the weak dictionary
implementations is exposed by the weakref module for the benefit of advanced
uses.

Not all objects can be weakly referenced; those objects which can include
class instances, functions written in Python (but not in C), methods (both
bound and unbound), sets, frozensets, file objects, generators, type objects,
DBcursor objects from the bsddb module, sockets, arrays, deques, regular
expression pattern objects, and code objects.

Changed in version 2.4: Added support for files, sockets, arrays, and
patterns.

Changed in version 2.7: Added support for thread.lock, threading.Lock, and
code objects.

Several built-in types such as list and dict do not directly support weak
references but can add support through subclassing:

    class Dict(dict):
        pass

    obj = Dict(red=1, green=2, blue=3)   # this object is weak referenceable

CPython implementation detail: Other built-in types such as tuple and long do
not support weak references even when subclassed.

Extension types can easily be made to support weak references; see Weak
Reference Support.

class weakref.ref(object[, callback])

    Return a weak reference to object. The original object can be retrieved by
    calling the reference object if the referent is still alive; if the referent
    is no longer alive, calling the reference object will cause None to be
    returned. If callback is provided and not None, and the returned weakref
    object is still alive, the callback will be called when the object is about to
    be finalized; the weak reference object will be passed as the only parameter
    to the callback; the referent will no longer be available.

    It is allowable for many weak references to be constructed for the same
    object. Callbacks registered for each weak reference will be called from the
    most recently registered callback to the oldest registered callback.

    Exceptions raised by the callback will be noted on the standard error output,
    but cannot be propagated; they are handled in exactly the same way as
    exceptions raised from an object’s __del__() method.

    Weak references are hashable if the object is hashable. They will maintain
    their hash value even after the object was deleted. If hash() is called the
    first time only after the object was deleted, the call will raise TypeError.

    Weak references support tests for equality, but not ordering. If the referents
    are still alive, two references have the same equality relationship as their
    referents (regardless of the callback). If either referent has been deleted,
    the references are equal only if the reference objects are the same object.

    Changed in version 2.4: This is now a subclassable type rather than a factory
    function; it derives from object.

weakref.proxy(object[, callback])

    Return a proxy to object which uses a weak reference. This supports use of the
    proxy in most contexts instead of requiring the explicit dereferencing used
    with weak reference objects. The returned object will have a type of either
    ProxyType or CallableProxyType, depending on whether object is callable. Proxy
    objects are not hashable regardless of the referent; this avoids a number of
    problems related to their fundamentally mutable nature, and prevent their use
    as dictionary keys. callback is the same as the parameter of the same name to
    the ref() function.

weakref.getweakrefcount(object)

    Return the number of weak references and proxies which refer to object.

weakref.getweakrefs(object)

    Return a list of all weak reference and proxy objects which refer to object.

class weakref.WeakKeyDictionary([dict])

    Mapping class that references keys weakly. Entries in the dictionary will be
    discarded when there is no longer a strong reference to the key. This can be
    used to associate additional data with an object owned by other parts of an
    application without adding attributes to those objects. This can be especially
    useful with objects that override attribute accesses.

    Note Caution: Because a WeakKeyDictionary is built on top of a Python
        dictionary, it must not change size when iterating over it. This can
        be difficult to ensure for a WeakKeyDictionary because actions performed by
        the program during iteration may cause items in the dictionary to vanish “by
        magic” (as a side effect of garbage collection).  WeakKeyDictionary objects
        have the following additional methods. These expose the internal references
        directly. The references are not guaranteed to be “live” at the time they are
        used, so the result of calling the references needs to be checked before being
        used. This can be used to avoid creating references that will cause the
        garbage collector to keep the keys around longer than needed.

WeakKeyDictionary.iterkeyrefs()

    Return an iterable of the weak references to the keys.
    New in version 2.5.

WeakKeyDictionary.keyrefs()

    Return a list of weak references to the keys.
    New in version 2.5.

class weakref.WeakValueDictionary([dict])

    Mapping class that references values weakly. Entries in the dictionary will be
    discarded when no strong reference to the value exists any more.

    Note Caution: Because a WeakValueDictionary is built on top of a Python
    dictionary, it must not change size when iterating over it. This can be
    difficult to ensure for a WeakValueDictionary because actions performed by the
    program during iteration may cause items in the dictionary to vanish “by
    magic” (as a side effect of garbage collection).  WeakValueDictionary objects
    have the following additional methods. These method have the same issues as
    the iterkeyrefs() and keyrefs() methods of WeakKeyDictionary objects.

WeakValueDictionary.itervaluerefs()

    Return an iterable of the weak references to the values.
    New in version 2.5.

WeakValueDictionary.valuerefs()

    Return a list of weak references to the values.
    New in version 2.5.

class weakref.WeakSet([elements])

    Set class that keeps weak references to its elements. An element will be
    discarded when no strong reference to it exists any more.

    New in version 2.7.

weakref.ReferenceType

    The type object for weak references objects.

weakref.ProxyType

    The type object for proxies of objects which are not callable.

weakref.CallableProxyType

    The type object for proxies of callable objects.

weakref.ProxyTypes

    Sequence containing all the type objects for proxies. This can make it simpler
    to test if an object is a proxy without being dependent on naming both proxy
    types.

exception weakref.ReferenceError

    Exception raised when a proxy object is used but the underlying object has
    been collected. This is the same as the standard ReferenceError exception.

See also
PEP 0205 - Weak References

The proposal and rationale for this feature, including links to earlier
implementations and information about similar features in other languages.

Weak Reference Objects

Weak reference objects have no attributes or methods, but do allow the
referent to be obtained, if it still exists, by calling it:

    >>>
    >>> import weakref
    >>> class Object:
    ...     pass
    ...
    >>> o = Object()
    >>> r = weakref.ref(o)
    >>> o2 = r()
    >>> o is o2
    True

If the referent no longer exists, calling the reference object returns None:

    >>>
    >>> del o, o2
    >>> print r()
    None

Testing that a weak reference object is still live should be done using the
expression ref() is not None. Normally, application code that needs to use a
reference object should follow this pattern:

    # r is a weak reference object
    o = r()
    if o is None:
        # referent has been garbage collected
        print "Object has been deallocated; can't frobnicate."
    else:
        print "Object is still live!"
        o.do_something_useful()

Using a separate test for “liveness” creates race conditions in threaded
applications; another thread can cause a weak reference to become invalidated
before the weak reference is called; the idiom shown above is safe in threaded
applications as well as single-threaded applications.

Specialized versions of ref objects can be created through subclassing. This
is used in the implementation of the WeakValueDictionary to reduce the memory
overhead for each entry in the mapping. This may be most useful to associate
additional information with a reference, but could also be used to insert
additional processing on calls to retrieve the referent.

This example shows how a subclass of ref can be used to store additional
information about an object and affect the value that’s returned when the
referent is accessed:

    import weakref

    class ExtendedRef(weakref.ref):
        def __init__(self, ob, callback=None, **annotations):
            super(ExtendedRef, self).__init__(ob, callback)
            self.__counter = 0
            for k, v in annotations.iteritems():
                setattr(self, k, v)

        def __call__(self):
            """Return a pair containing the referent and the number of
            times the reference has been called.
            """
            ob = super(ExtendedRef, self).__call__()
            if ob is not None:
                self.__counter += 1
                ob = (ob, self.__counter)
            return ob

Example

This simple example shows how an application can use objects IDs to retrieve
objects that it has seen before. The IDs of the objects can then be used in
other data structures without forcing the objects to remain alive, but the
objects can still be retrieved by ID if they do.

    import weakref

    _id2obj_dict = weakref.WeakValueDictionary()

    def remember(obj):
        oid = id(obj)
        _id2obj_dict[oid] = obj
        return oid

    def id2obj(oid):
        return _id2obj_dict[oid]