Mar 14 2020
Monkey Patching Modules
It is easy to get in the isolated TDD groove and implement a bunch of well-tested methods. Once it comes to utilizing these methods, we get excited and just go for it. We forget the bigger picture and proper decoupling principles.
Let’s say that we have to fetch a bunch of resources from an API.
Let’s also assume that all of these methods are well tested with isolated
TDD. We are also abiding by good testing practice by providing a conn
object through dependency injection. This way there are not any secret
variables being used – such as through singletons.
Awesome! We were able to get 100% code coverage on these three methods and we even ran the methods in the python REPL to verify that they work in a more production-like environment. Now it’s time to use them!
Sweet. We just verified that get_resources()
works as intended
since we just ran it in the python REPL. Maybe we quick throw in some tests
to prevent against possible regression issues in the future.
Uh oh. This really does not feel right. For starters, two glaring issues are:
- There is more than one assertion in the test
- There are way too many mocks
Well, we can fix the first issue by separating them out:
Great… by getting rid of one glaring issue, we just multiplied the other
3-fold. It feels really disgusting to have this many mocks. We should not
need a total of 9 mocks across these tests. The bad news is that the number
of mocks needed is n * m
where n
is the number of fetch_thing_*
methods, while m
is the number of unit tests.
In the case that this grows to fetching 7 resources, we will have at least 7 unit tests verifying the 7 fetches. 49 mocks is unfathomable. It’s probably at this point that we just delete all of this progress and commit an untested method? Wait! There’s one more thing we did not consider yet.
Decoupling.
But we aren’t doing function decoupling. We already did that. Our functions
do one job, and they do them well. Let’s move our attention to module-level
decoupling. Let’s see what happens when we split get_resources
into a
separate file. Let’s move it to a file called lib/myresources.py
.
Now instead of mocking the individual fetch functions, we can mock the
entire lib/myreqs.py
module.
Holy cow, this is so clean now. Now we are only increasing our mocks by a
factor of m
– the number of unit tests. It no longer matters how many
different fetch_thing_*
functions there are.
Fast forward 6 months…
Your client says that there are customers who are begging for a way to
fetch the resources themselves. You can provide a well-documented REST API,
but the customers want minimal friction with an already implemented
solution. You decide that releasing a pip dependency with your
lib/myreqs.py
code is the best way to do this.
Since the modules are already decoupled, there is little-to-no effort in releasing these non-sensitive lines of source code.
It’s at this point that you realize the saving grace of that gross, non-isolated looking unit test. Having ignored that red flag would have lead to a bad pattern, technical debt, and a lot more effort in cleaning up.
Good tests write good code! Whenever a test looks gross, it’s usually because the implementation is gross – even if it doesn’t initially appear so.