Mar 14 2020

Monkey Patching Modules

It is easy to get in the isolated TDD groove and implement a bunch of well-tested methods. Once it comes to utilizing these methods, we get excited and just go for it. We forget the bigger picture and proper decoupling principles.

Let’s say that we have to fetch a bunch of resources from an API.

Let’s also assume that all of these methods are well tested with isolated TDD. We are also abiding by good testing practice by providing a conn object through dependency injection. This way there are not any secret variables being used – such as through singletons.

Awesome! We were able to get 100% code coverage on these three methods and we even ran the methods in the python REPL to verify that they work in a more production-like environment. Now it’s time to use them!

Sweet. We just verified that get_resources() works as intended since we just ran it in the python REPL. Maybe we quick throw in some tests to prevent against possible regression issues in the future.

Uh oh. This really does not feel right. For starters, two glaring issues are:

Well, we can fix the first issue by separating them out:

Great… by getting rid of one glaring issue, we just multiplied the other 3-fold. It feels really disgusting to have this many mocks. We should not need a total of 9 mocks across these tests. The bad news is that the number of mocks needed is n * m where n is the number of fetch_thing_* methods, while m is the number of unit tests.

In the case that this grows to fetching 7 resources, we will have at least 7 unit tests verifying the 7 fetches. 49 mocks is unfathomable. It’s probably at this point that we just delete all of this progress and commit an untested method? Wait! There’s one more thing we did not consider yet.

Decoupling.

But we aren’t doing function decoupling. We already did that. Our functions do one job, and they do them well. Let’s move our attention to module-level decoupling. Let’s see what happens when we split get_resources into a separate file. Let’s move it to a file called lib/myresources.py.

Now instead of mocking the individual fetch functions, we can mock the entire lib/myreqs.py module.

Holy cow, this is so clean now. Now we are only increasing our mocks by a factor of m – the number of unit tests. It no longer matters how many different fetch_thing_* functions there are.

Fast forward 6 months…

Your client says that there are customers who are begging for a way to fetch the resources themselves. You can provide a well-documented REST API, but the customers want minimal friction with an already implemented solution. You decide that releasing a pip dependency with your lib/myreqs.py code is the best way to do this.

Since the modules are already decoupled, there is little-to-no effort in releasing these non-sensitive lines of source code.

It’s at this point that you realize the saving grace of that gross, non-isolated looking unit test. Having ignored that red flag would have lead to a bad pattern, technical debt, and a lot more effort in cleaning up.

Good tests write good code! Whenever a test looks gross, it’s usually because the implementation is gross – even if it doesn’t initially appear so.