Tuesday, July 23, 2013

Delaying attribute initialization for Python class/object

I've found myself in need to delay initialization of a certain class/object attribute until I access for the first time.

For example, I have db module that has Connections class and this class in turn has two attributes - read_conn and write_conn. I did not want to create those these connections in advance - because the code may not even use them. Also I did not want each time I access db.Connection.read_conn to check whether I've already created this connection or not.

Now for simplicity of solution explanation, lets abstract from real-life example above to this: I have nursery module containing Mother class with several Baby attributes. Each baby is a instance of Baby. The trick is that I do not want babies to be "born" until they are accessed for the first time.

So we start with the following:

class Baby(object):
    def __init__(self, name):
        print "%s is born" % name
        self.name = name

class Mother(object):
    john = Baby("John")
The above of course does not do what I want - when I load this module, the john value is initialized right away:
>>> import nursery
John is born
To achieve what I want I need somehow to interfere in the process of accessing the john argument. Python has something called descriptors that can do just that. Lets use them:
class BabyDescriptor(object):
    def __init__(self, baby_name):
        self.baby_name = baby_name
        self.instance = None

    def __get__(self, obj, cls):
        if not self.instance:
            self.instance = Baby(self.baby_name)
        return self.instance
        
class Mother(object):
    john = BabyDescriptor("John")
>>> import nursery   # no babies are born!
>>> nursery.Mother.john
John is born
<nursery.Baby object at 0x7ffe909c4050>
>>> nursery.Mother.john
<nursery.Baby object at 0x7ffe909c4050>
>>> 
This is better! Babies are born on demand :) The only downside are:
  • nursery.Mother.john points to descriptor instance and not to the Baby instance
  • The descriptor checks every time if it has already created the object
To solve this, I need to reassign john attribute to point to the real Baby instance once I create it:
class BabyDescriptor(object):
    def __init__(self, baby_name, prop_name):
        self.baby_name = baby_name
        self.prop_name = prop_name

    def __get__(self, obj, cls):
        to_augment = obj or cls # To insure it works from both Mother.john and Mother().john
        instance = Baby(self.baby_name)
        setattr(to_augment, self.prop_name, instance)
        return instance
        
class Mother(object):
    john = BabyDescriptor("John", "john") # attribute name is specified twice :(
>>> import nursery
>>> nursery.Mother.john
John is born
<nursery.Baby object at 0x7fe7ab115cd0>
>>> nursery.Mother.__dict__['john']
<nursery.Baby object at 0x7fe7ab115cd0> # Yeehaw!! - Baby instance not BabyDescriptor!
>>>
This almost perfect! It does the job but missing some elegance - notably, I do not like the fact that I have to specify attribute name twice. Fortunately Python is flexible enough to figure it out "automagically". To achieve this I have to add some decorator functionality to BabyDescriptor:
class BabyDescDecotrator(object):
    def __init__(self, baby_name):
        self.baby_name = baby_name

    def __call__(self, prop):
        self.prop = prop
        return self # Important to return ourselves to keep functioning as descriptor

    def __get__(self, obj, cls):
        to_augment = obj or cls 
        instance = Baby(self.baby_name)
        setattr(to_augment, self.prop.__name__, instance) # extracting name form the prop
        return instance
        
class Mother(object):
    @BabyDescDecotrator("John")
    def john(self): pass
Lets see now how the john is constructed:
  • When loading module, Python sees that we have a decorated attribute. It also notes that this decorator takes parameters
  • So Python invokes BabyDescDecotrator("John")(john) and assigns the result to Mother's john attribute:
    • First __init__ is called recording desired baby name in the instance
    • Then resulting BabyDescDecotrator instance is __call__-ed recording the property it decorates
  • Then when we access john, Baby instance is created and Mother is augmented

Finally

Finally, below is the generic code that can proxy any class with any arguments:
class ProxyProperty(object):
    def __init__(self, *args, **kwargs):
        if not args:
            raise ValueError("Need at least to supply class object")
        self.klass = args[0]
        if not isinstance(self.klass, type):
            raise ValueError("Supplied argument does not look like a class")

        self.klass_args = args[1:]
        self.klass_kwargs = kwargs
        print self.klass, self.klass_args, self.klass_kwargs

    def __call__(self, prop):
        self.prop = prop
        return self

    def __get__(self, obj, cls):
        to_augment = obj or cls
        instance = self.klass(*self.klass_args, **self.klass_kwargs)
        setattr(to_augment, self.prop.__name__, instance)
        return instance
And here is how you use it:
    class Mother(object):
        @ProxyProperty(Baby, "Scott")
        def scott(self): pass

        @ProxyProperty(Baby, "John")
        def john(self): pass