This post is all about the generic functions capability in Python 3. This was initially a bit mystifying to me — “Generics and function overloading in a dynamically typed language? You wot?”
Closer inspection reveals a very elegant and powerful tool which many programmers who are more familiar with Python 2 (as I was) might not be aware of.
How languages evolve and the capabilities that their designers add to enhance, simply or clarify their use is a source of much fascination to me. This falls right into that category, so I figured why not share my findings!
Python
Python is a very interesting programming language when considered from the viewpoint of language evolution.
It is a multi-paradigm language.
There are three major programming paradigms, each of which “take something away from us” — as Uncle Bob puts it in his latest book on Clean Architecture.
Those are structured, object-oriented and functional programming. Multi-paradigm is a somewhat hand-wavey way of saying that a language allows its users to participate in all three. In this way Python belongs in the same category as C++.
Structured programming took away our ability to mangle our programs flow of control with GOTO statements, object oriented programming imposed discipline on the use of function pointers and functional programming imposed immutability and thus stricter control of assignment.
Another fact which makes Python stand out is that it is a language with a very well known split personality. If you choose to use Python you also have the further complication of choosing whether to use Python 2 or Python 3, with version 3 breaking backward compatibility with its parent.
You can read up on the whens and whys of this history to get a sense of what motivated the breaking changes.
For various reasons I have mostly used Python 2 in my run ins with Python in a professional context. If you’re like me then you may not be fully aware of many of the newer features that have been added in version 3 which enhance its qualities as a language.
This post is concerned with one such enhancement. Specifically the Single Dispatch mechanism which was introduced in version 3.4.
Function Overloading
At first glance this feature appears to have simply added method overloading to Python.
To quote from PEP-443 -https://www.python.org/dev/peps/pep-0443/:
A generic function is composed of multiple functions implementing the same operation for different types. Which implementation should be used during a call is determined by the dispatch algorithm. When the implementation is chosen based on the type of a single argument, this is known as single dispatch.
Actions are dispatched in the sense that you would dispatch someone to deliver a message rather than y’know… dispatching an informant.
Because Python is a dynamically typed language, it does not have the classic method overloading approach were a function signature varies to handle different types which may be passed to it, as illustrated in the simple C++ example below:
In C++ the function signature can differ by both the type of the arguments or by the number of arguments in the list.
A common approach in Python prior to the introduction of the single dispatch mechanism would have been to use conditional statements with type inspection:
This is a “bad” way to go with regards to maintainability. Or more pragmatically speaking, it adds some level of overhead whenever we want to extend the codebase with as yet unknown type handling capability.
Open-Closed
The ‘O’ in the oft-quoted SOLID principles stands of ‘Open-Closed’.
The one-liner for this principle is that
software components should be open for extension, but closed for modification
Software archirekts love to talk about extensibility and things being extensible.
Put simply, if I have a class that is responsible for some behaviour within my application then it is extensible if I can change that behaviour without having to crack it open and re-write some or all of it.
I want to be able to add new code which represents new behaviour whilst avoiding changing code which is already written.
Although the code in the Python sample above is a particularly bad example of violating Open-Closed, the C++ version is not without it’s own problems. In particular, it breaks the ‘I’ and the ‘D’ in SOLID As well. That is the ‘Interface Segregation’ and the ‘Dependency Inversion’ principles.
What it boils down to is this: If I want to include that class in any of my other components, those other components are now coupled to the implementations of myFunction — including all the overloaded versions that it may not actually use.
Changes to the Overloaderer will necessitate re-compilation of the components which depend on it.
In a statically typed language like C++ the answer is to depend on abstractions (such as interfaces) rather than concretions as shown in the UML diagram below.
Python being dynamically typed means that it does not suffer from this problem in the same way.
As we shall see, singledispatch also gets us away from the need to use brittle type inspection to determine behaviour. However it goes further than that.
Can it bring us closer to the promised land of closed modification and open extensibility? Let’s find out!
The singledispatch Decorator
The singledispatch mechanism is implemented as a function decorator in the functools module.
In Python, decorators are added to methods in a manner which should be familiar to users of other languages which provide this type of aspect-oriented syntax. Here’s something similar to the previous example re-imagined using this syntax:
Annotating a function with @singledispatch is just syntactic sugar for writing: singledispatch(myfunction).
The `_` name for the registered functions for `int` and `float` is to indicate that the overloaded names are not to be used. Only myfunction is ever externally referenced and the singledispatch mechanism will determine the correct registered function to use at run-time, based on the type which is passed to it.
To used this function we can simply call it with different parameter values as per the C++ example:
In [10]: myfunction(‘What about ye’)
Do something with ‘What about ye’
In [11]: myfunction(8)
Do something octal 10
In [12]: myfunction(3.14159)
Do something float: 3.14
The cool thing about this approach is that it does not just give us method-overloading style dispatch behaviour.
Once the base `myfunction` handler has been created, new variants can be registered in any module. New classes can include their own implementations of `myfunction` which provide custom handling for their own types.
This is super cool and moves us further towards our SOLID goals of avoiding tight coupling, avoiding the inclusion of more dependencies than necessary and allowing us to add new behaviour through extension rather than modification of existing code.
References / Further Reading
- PEP 443 — Single dispatch generic functions : Open design proposal for the implementation of `singledispatch`
- Pyhon Decorators: General reference on Python decorators
- Clean Architecture — Robert Martin: A new book from Uncle Bob! Contains a clear and concise description of the SOLID principles and thoughts on how programming paradigms affect architecture.
- Fluent Python — Luciano Ramalho: Chapter 7 Decorators and Closures is a fine reference using `singledispatch` as one of the main examples. Also just an awesome book if you are interested in Python.
- C++ function overloading: C++ is also an awesome language and the tutorialspoint page is a great reference.
