Decorator Pattern

CHAPTER 7 Decorator Pattern

Time abides long enough for those who make use of it.

Leonardo da Vinci

As you become more experienced, you will find yourself moving beyond the type of problems that can easily be solved with the most general programming constructs. At such a time, you will realize you need a different tool set. This chapter focuses on one such tool.

When you are trying to squeeze every last drop of performance out of your machine, you need to know exactly how long a specific piece of code takes to execute. You must save the system time before you initiate the code to be profiled, then execute the code; once it has concluded, you must save the second system timestamp. Finally, subtract the first time from the second to obtain the time elapsed during the execution. Look at this example for calculating Fibonacci numbers.

fib. py

import time n = 77

start_time = time.time() fibPrev = 1

fib = 1

for num in range(2, n):

fibPrev, fib = fib, fib + fibPrev

end_time = time.time()

print("[Time elapsed for n = {}] {}".format(n, end_time - start_time)) print("Fibonacci number for n = {}: {}".format(n, fibIter(n)))

105

' Wessel Badenhorst 2017

W. Badenhorst, Practical Python Design Patterns, https://doi.org/10.1007/978-1-4842-2680-3_7
CHAPTER 7 DECORATOR PATTERN

Every system is different, but you should get a result shaped like the one here:

[Time elapsed for n = 77] 8.344650268554688e-06 Fibonacci number for n = 77: 5527939700884757

We now extend our Fibonacci code so we can request the number of elements in the Fibonacci sequence we wish to retrieve. We do this by encapsulating the Fibonacci calculation in a function.

fib_ function.py import time

def fib(n):

start_time = time.time() if n < 2:

return

fibPrev = 1 fib = 1

for num in range(2, n):

fibPrev, fib = fib, fib + fibPrev

end_time = time.time()

print("[Time elapsed for n = {}] {}".format(n, end_time - start_time))

return fib

if __name__ == "__main__":

n = 77

print("Fibonacci number for n = {}: {}".format(n, fib(n)))

In the preceding code, the profiling takes place outside of the function itself. This is fine when you only want to look at a single function, but this is not usually the case when you are optimizing a program. Luckily, functions are first-class citizens in Python, and

as such we can pass the function as a parameter to another function. So, let s create a function to time the execution of another function.

106
CHAPTER 7 DECORATOR PATTERN

fib_ func_profile_me.py import time

def fib(n):

if n < 2:

return

fibPrev = 1 fib = 1

for num in range(2, n):

fibPrev, fib = fib, fib + fibPrev

return fib

def profile_me(f, n):

start_time = time.time()

result = f(n)

end_time = time.time()

print("[Time elapsed for n = {}] {}".format(n, end_time - start_time))

return result

if __name__ == "__main__":

n = 77

print("Fibonacci number for n = {}: {}".format(n, profile_me(fib, n)))

Whenever we want to get the time it takes to execute a function, we can simply pass the function into the profiling function and then run it as usual. This approach does have some limitations, since you must pre-define the parameters you want applied to the timed function. Yet again, Python comes to the rescue by allowing us to also return functions as a result. Instead of calling the function, we now return the function with the profiling added on.

base_profiled_ fib.py import time

def fib(n):

if n < 2:

return

107
CHAPTER 7 DECORATOR PATTERN

fibPrev = 1 fib = 1

for num in range(2, n):

fibPrev, fib = fib, fib + fibPrev

return fib

def profile_me(f, n):

start_time = time.time()

result = f(n)

end_time = time.time()

print("[Time elapsed for n = {}] {}".format(n, end_time - start_time))

return result

def get_profiled_function(f):

return lambda n: profile_me(f, n)

if __name__ == "__main__":

n = 77

fib_profiled = get_profiled_function(fib)

print("Fibonacci number for n = {}: {}".format(n, fib_profiled(n)))

This method works a lot better but still requires some effort when trying to profile several functions, as this causes interference with the code executing the function. There has to be a better way. Ideally, we want a way to tag a specific function as one to be profiled, and then not worry about the call that initiates the function call. We will do this using the decorator pattern.

The Decorator Pattern

To decorate a function, we need to return an object that can be used as a function.

The classic implementation of the decorator pattern uses the fact that the way Python implements regular procedural functions these functions can be seen as classes with some kind of execution method. In Python, everything is an object; functions are objects with a special __call__() method. If our decorator returns an object with a __call__() method, the result can be used as a function.

108
CHAPTER 7 DECORATOR PATTERN

The classic implementation of the decorator pattern does not use decorators in the sense that they are available in Python. Once again, we will opt for the more pythonic implementation of the decorator pattern, and as such we will leverage the power of the built-in syntax for decorators in Python, using the @ symbol.

class_decorated_profiled_ fib.py import time

class ProfilingDecorator(object):

def __init__(self, f):

print("Profiling decorator initiated") self.f = f

def __call__(self, *args):

start_time = time.time()

result = self.f(*args)

end_time = time.time()

print("[Time elapsed for n = {}] {}".format(n, end_time - start_time))

return result

@ProfilingDecorator def fib(n):

print("Inside fib") if n < 2:

return

fibPrev = 1 fib = 1

for num in range(2, n):

fibPrev, fib = fib, fib + fibPrev

return fib

if __name__ == "__main__":

n = 77

print("Fibonacci number for n = {}: {}".format(n, fib(n)))

109
CHAPTER 7 DECORATOR PATTERN

The print statement inside the fib function is just there to show where the

execution fits relative to the profiling decorator. Remove it once you see it in action so as to not influence the actual time taken for the calculation.

Profiling decorator initiated

Inside fib

[Time elapsed for n = 77] 1.1682510375976562e-05 Fibonacci number for n = 77: 5527939700884757

In the class definition, we see that the decorated function is saved as an attribute

of the object during initialization. Then, in the call function, the actual running of

the function being decorated is wrapped in time requests. Just before returning, the profile value is printed to the console. When the compiler comes across the _@ProfilingDecorator_, it initiates an object and passes in the function being wrapped as an argument to the constructor. The returned object has the __call__() function as

a method, and as such duck typing will allow this object to be used as a function. Also, note how we used *args in the __call__() method s parameters to pass in arguments. *args allows us to handle multiple arguments coming in. This is called packing, as all the parameters coming in are packed into the args variable. Then, when we call the stored function in the f attribute of the decorating object, we once again use *args. This is called unpacking, and it turns all the elements of the collection in args into individual arguments that are passed on to the function in question.

The decorator is a unary function (a function that takes a single argument) that takes a function to be decorated as its argument. As you saw earlier, it then returns a function that is the same as the original function with some added functionality. This means that all the code that interacts with the decorated function can remain the same as when the function was undecorated. This allows us to stack decorators. So, in a silly example, we could profile our Fibonacci function and then output the result string as HTML.

stacked_ fib.py

import time

class ProfilingDecorator(object):

def __init__(self, f):

print("Profiling decorator initiated") self.f = f

110
CHAPTER 7 DECORATOR PATTERN

def __call__(self, *args):

print("ProfilingDecorator called")

start_time = time.time()

result = self.f(*args)

end_time = time.time()

print("[Time elapsed for n = {}] {}".format(n, end_time - start_time))

return result

class ToHTMLDecorator(object):

def __init__(self, f):

print("HTML wrapper initiated") self.f = f

def __call__(self, *args):

print("ToHTMLDecorator called")

return "{}

".format(self.f(*args))

@ToHTMLDecorator @ProfilingDecorator def fib(n):

print("Inside fib") if n < 2:

return

fibPrev = 1 fib = 1

for num in range(2, n):

fibPrev, fib = fib, fib + fibPrev

return fib

if __name__ == "__main__":

n = 77

print("Fibonacci number for n = {}: {}".format(n, fib(n)))

111
CHAPTER 7 DECORATOR PATTERN

This results in the following output. When decorating a function, be careful to take note what the output type of the function you wrap is and also what the output type of the result of the decorator is. More often than not, you want to keep the type consistent so functions using the undecorated function do not need to be changed. In this example, we change the type from a number to an HTML string as an example, but it is not something you usually want to do.

Profiling decorator initiated

HTML wrapper initiated

ToHTMLDecorator called

ProfilingDecorator called

Inside fib

[Time elapsed for n = 77] 1.52587890625e-05

Fibonacci number for n = 77: 5527939700884757

Remember that we were able to use a class to decorate a function because in Python everything is an object as long as the object has a __call__() method, so it can be used like a function. Now, let s explore using this property of Python in reverse. Instead of using a class to decorate a function, let s use a function directly to decorate our fib function.

function_decorated_ fib.py import time

def profiling_decorator(f):

def wrapped_f(n):

start_time = time.time()

result = f(n)

end_time = time.time()

print("[Time elapsed for n = {}] {}".format(n, end_time -

start_time))

return result return wrapped_f

@profiling_decorator def fib(n):

112
CHAPTER 7 DECORATOR PATTERN

print("Inside fib") if n < 2:

return

fibPrev = 1 fib = 1

for num in range(2, n):

fibPrev, fib = fib, fib + fibPrev

return fib

if __name__ == "__main__":

n = 77

print("Fibonacci number for n = {}: {}".format(n, fib(n)))

Get ready for a bit of a mind bender. The decorator function must return a function to be used when the decorated function is called. The returned function is used to wrap the decorated function. The wrapped_f() function is created when the function is called, so it has access to all the arguments passed to the profiling_decorator() function.

This technique by which some data (in this case, the function passed to the profiling_ decorator() function) gets attached to code is called a closure in Python.

Closures

Let s look at the wrapped_f() function that accesses the non-local variable f that

is passed in to the profiling_decorator() function as a parameter. When the wrapped_f() function is created, the value of the non-local variable f is stored and returned as part of the function, even though the original variable moves out of scope and is removed from the namespace once the program exits the profiling_decorator() function.

In short, to have a closure, you have to have one function returning another function nested within itself, with the nested function referring to a variable within the scope of the enclosing function.

113
CHAPTER 7 DECORATOR PATTERN

Retaining Function __name__ and __doc__ Attributes

As noted before, you ideally do not want any function using your decorator to be changed in any way, but the way we implemented the wrapper function in the previous section caused the __name__ and __doc__ attributes of the function to change to that of the wrapped_f() function. Look at the output of the following script to see how the __name__ attribute changes.

func_attrs.py

def dummy_decorator(f):

def wrap_f():

print("Function to be decorated: ", f.__name__) print("Nested wrapping function: ", wrap_f.__name__) return f()

return wrap_f

@dummy_decorator

def do_nothing():

print("Inside do_nothing")

if __name__ == "__main__":

print("Wrapped function: ",do_nothing.__name__)

do_nothing()

Check the following results; the wrapped function took on the name of the wrap function.

Wrapped function: wrap_f

Function to be decorated: do_nothing Nested wrapping function: wrap_f Inside do_nothing

In order to keep the __name__ and __doc__ attributes of the function being wrapped, we must set them to be equal to the function that was passed in before leaving the scope of the decorator function.

114
CHAPTER 7 DECORATOR PATTERN

def dummy_decorator(f):

def wrap_f():

print("Function to be decorated: ", f.__name__) print("Nested wrapping function: ", wrap_f.__name__) return f()

wrap_f.__name__ = f.__name__ wrap_f.__doc__ = wrap_f.__doc__ return wrap_f

@dummy_decorator

def do_nothing():

print("Inside do_nothing")

if __name__ == "__main__":

print("Wrapped function: ",do_nothing.__name__)

do_nothing()

Now there is no longer a difference between the do_nothing() function and the decorated do_nothing() function.

The Python standard library includes a module that allows retaining the __name__ and __doc__ attributes without setting it ourselves. What makes it even better in

the context of this chapter is that the module does so by applying a decorator to the wrapping function.

from functools import wraps def dummy_decorator(f):

@wraps(f)

def wrap_f():

print("Function to be decorated: ", f.__name__) print("Nested wrapping function: ", wrap_f.__name__) return f()

return wrap_f

@dummy_decorator

def do_nothing():

print("Inside do_nothing")

115
CHAPTER 7 DECORATOR PATTERN

if __name__ == "__main__":

print("Wrapped function: ",do_nothing.__name__)

do_nothing()

What if we wanted to select the unit that the time should be printed in, as in seconds rather than milliseconds? We would have to find a way to pass arguments to the decorator. To do this, we will use a decorator factory. First, we create a decorator function. Then, we expand it into a decorator factory that modifies or replaces the wrapper. The factory then returns the updated decorator.

Let s look at our Fibonacci code with the functools wrapper included.

import time

from functools import wraps

def profiling_decorator(f):

@wraps(f)

def wrap_f(n):

start_time = time.time() result = f(n)

end_time = time.time()

elapsed_time = (end_time - start_time)

print("[Time elapsed for n = {}] {}".format(n, elapsed_time))

return result return wrap_f

@profiling_decorator def fib(n):

print("Inside fib") if n < 2:

return

fibPrev = 1 fib = 1

116
CHAPTER 7 DECORATOR PATTERN

for num in range(2, n):

fibPrev, fib = fib, fib + fibPrev

return fib

if __name__ == "__main__":

n = 77

print("Fibonacci number for n = {}: {}".format(n, fib(n)))

Now, let s extend the code so we can pass in an option to display the elapsed time in either milliseconds or seconds.

import time

from functools import wraps

def profiling_decorator_with_unit(unit):

def profiling_decorator(f):

@wraps(f)

def wrap_f(n):

start_time = time.time()

result = f(n)

end_time = time.time()

if unit == "seconds":

elapsed_time = (end_time - start_time) / 1000 else:

elapsed_time = (end_time - start_time)

print("[Time elapsed for n = {}] {}".format(n, elapsed_time)) return result

return wrap_f

return profiling_decorator

@profiling_decorator_with_unit("seconds") def fib(n):

print("Inside fib")

if n < 2:

return

117
CHAPTER 7 DECORATOR PATTERN

fibPrev = 1 fib = 1

for num in range(2, n):

fibPrev, fib = fib, fib + fibPrev

return fib

if __name__ == "__main__":

n = 77

print("Fibonacci number for n = {}: {}".format(n, fib(n)))

As before, when the compiler reaches the @profiling_decorator_with_unit it calls the function, which returns the decorator, which is then applied to the function being decorated. You should also note that once again we used the closure concept to handle parameters passed to the decorator.

Decorating Classes

If we wanted to profile every method call in a specific class, our code would look something like this:

class DoMathStuff(object):

@profiling_decorator def fib(self):

... @profiling_decorator def factorial(self):

...

Applying the same method to each method in the class would work perfectly fine, but it violates the DRY principle (Don t Repeat Yourself). What would happen if we decided we were happy with the performance and needed to take the profiling code out of a number of sprawling classes? What if we added methods to the class and forgot to decorate these newly added methods? There has to be a better way, and there is.

What we want to be able to do is decorate the class and have Python know to apply the decoration to each method in the class.

118
CHAPTER 7 DECORATOR PATTERN

The code we want should look something like this:

@profile_all_class_methods class DoMathStuff(object):

def fib(self):

... @profiling_decorator def factorial(self):

...

In essence, what we want is a class that looks exactly like the DoMathStuff class from the outside, with the only difference being that every method call should be profiled.

The profiling code we wrote in the previous section should still be of use, so let s

take that and make the wrapping function more general so it will work for any function passed to it.

import time

def profiling_wrapper(f):

@wraps(f)

def wrap_f(*args, **kwargs):

start_time = time.time()

result = f(*args, **kwargs)

end_time = time.time()

elapsed_time = end_time - start_time

print("[Time elapsed for n = {}] {}".format(n, elapsed_time))

return result return wrap_f

We received two packed arguments: a list of regular arguments and a list of arguments mapped to keywords. Between them, these two arguments will capture any form of argument that can be passed to a Python function. We also unpack both as we pass them on to the function to be wrapped.

Now, we want to create a class that would wrap a class and apply the decorating function to every method in that class and return a class that looks no different from the class it received. This is achieved through the __getattribute__() magic method. Python

119
CHAPTER 7 DECORATOR PATTERN

uses this method to retrieve methods and attributes for an object, and by overriding this method we can add the decorator as we wanted. Since __getattribute__() returns methods and values, we also need to check that the attribute requested is a method.

class_profiler.py import time

def profiling_wrapper(f):

@wraps(f)

def wrap_f(*args, **kwargs):

start_time = time.time()

result = f(*args, **kwargs)

end_time = time.time()

elapsed_time = end_time - start_time

print("[Time elapsed for n = {}] {}".format(n, elapsed_time))

return result return wrap_f

def profile_all_class_methods(Cls):

class ProfiledClass(object):

def __init__(self, *args, **kwargs):

self.inst = Cls(*args, **kwargs)

def __getattribute__(self, s):

try:

x = super(ProfiledClass, self).__getattribute__(s) except AttributeError:

pass

else:

x = self.inst.__getattribute__(s)

if type(x) == type(self.__init__):

return profiling_wrapper(x)

else:

return x

return ProfiledClass

120
CHAPTER 7 DECORATOR PATTERN

@profile_all_class_methods class DoMathStuff(object):

def fib(self):

... @profiling_decorator def factorial(self):

...

There you have it! You can now decorate both classes and functions. It would be helpful to you if you download and read the Flask source code (download form the official website: http://flask.pocoo.org/), specifically noting how they make use of decorators to handle routing. Another package that you may find interesting is the Django Rest Framework (form the official website: http://www.django-rest- framework.org/), which uses decorators to alter return values.

Parting Shots

The decorator pattern differs from both the adapter and facade pattern in that the interface does not change, but new functionality is somehow added. So, if you have a specific piece of functionality that you want to attach to a number of functions to alter these functions without changing their interface, a decorator may be the way to go.

Exercises

Extend the Fibonacci function wrapper so it also returns the time

elapsed in minutes and hours, printed hrs:minutes:seconds. milliseconds .

Make a list of ten common cases you can think of in which you could

implement decorators.

Read the Flask source code and write down every interesting use of

decorators you come across.

121