Everybody knows that unit tests are valuable. So it's a lot of fun playing Devil's advocate as I did some time ago when I raved about the
value of compiler checks for keeping up the quality of your program. So here we go: unit tests are expensive. You never know when or if they pay off. Thing is, my experience shows that unit tests are not always good. Sometimes they even backfire. I've seen this in at least two projects. This time, I don't want to talk you out of using unit tests (maybe another day), but I want to make you aware of the risks.
Act 1: Tedium
If you're like me, your first contact with unit testing is underwhelming. You've spent a lot of time implementing your algorithm. Now you're asked to write unit tests for it. Essentially, that means you have to repeat every thought. Which are legal parameters? What should happen when an illegal argument is passed? What should happen when correct parameters are passed? Boring. You've already implemented these constraints when you've implemented the algorithm. Writing automated tests feels like doing everything twice. So how to test the correct behavior of the algorithm effectively and efficiently? That's an art taught way too seldom. Most articles about unit tests focus on how to implement tests. They flood you with information about Mockito, JUnit5, Arquillian, and many more useful stuff. But nobody tells you which tests to write - and which tests not to write. The general theory seems to be that every developer gets the idea sooner or later. If you're lucky, you find a flaw in your algorithm while rethinking it a second time. That's a good start to enter act 2.
Act 2: Euphoria
The real value of unit tests shows one or two versions later. You modify or refactor an algorithm, and one of the unit tests break. They shouldn't break. The unit tests are like a contract. They describe how the algorithm behaves. If it doesn't, it's your fault. The unit test is an early-warning system. You haven't even committed the code, nobody else has seen it, and you're already told your code is wrong. So you can silently fix it before anybody else notices. That's a big plus. It takes the scare out of difficult tasks like refactoring. By definition, refactoring is modifying the code without changing the functionality. Unit tests tell you you're on the right track. If you have such an experience - and almost every developer has - you've bought it. You'll write unit tests for everything. You're on your way to be a test addict.
Act 3: Exaggeration
There are many things people write tests for without thinking whether it'll ever pay off. A popular example is the reactive forms of Angular. There's nothing wrong with reactive forms. But when I asked, I was told reactive forms are better than template driven forms "because they can be tested." Pressing further, I learned that unit tests could test the validation rules of reactive forms. They can't check the validation rules of template driven forms because those rules are hidden in the HTML code of the component. Here's the catch: there's no need to test those validation rules that can be defined declaratively. There are only three reasons why the test might fail. At least as long as we're talking about standard validation rules provided by a third-party framework or your base framework. Custom validators may be a different story.
Excursion: what's wrong with testing validation rules?
First, it may fail at implementation time. That's easy to fix. Either the validation rule is incorrect, or you've written a wrong test. In both cases, you could easily run the same test as a manual UI test. Remember my previous claim: unit tests only pay in the long run. There are always cheaper alternatives for the initial implementation. Second, it may fail because the validation framework is broken. Those things happen, but they are a rare exception. In any case, you don't need a unit test to detect such a failure. If the validation framework is broken, you'll notice pretty soon, even without unit tests. If it's a widely used validation framework, you'll even read about it in the news (or Twitter, or Reddit). Third, the validation test may break because someone has modified the validation declarations. This kind of things hardly ever happens by accident. If it does, you've proven the value of unit testing. But at least my experience is different. In almost every case, my team changes the validation rule because the business requirements have changed. Unit tests fail to protect you from this source of change. Even worse, they add to the cost of these changes. Each time the business requirements change, you have to change both the implementation and the test. Or the tests, if there's more than one.
Act 4: Getting sloppy
That's where unit tests start to get in your way. Of course, that's a slow process. During the first year, you won't have many problems with unit tests. But if you're serious about unit testing, you'll end with many unit tests after a while. Every aspect of your program is tested, possibly even more than once. Testing the same thing multiple times is a good thing if each test tackles the problem from a different angle. On the other hand, writing tests redundantly is one of the things that just happen in large teams. More likely than not, you've grown familiar with a culture demanding a unit test for every algorithm you've written. At this point, interesting things start to happen. For instance, I've observed tests testing the wrong thing. Instead of testing the API - i.e. the contract between the caller and the implementor - they check the implementation. Refactoring the algorithm becomes difficult because it breaks the test. That's not a nuisance. If this kind of tests become the norm, they'll bring the progress of your project to a standstill. Luckily, that's the sort of thing you can easily address once you're aware of them. Just keep the danger in mind, and act if it becomes imminent. Another funny side effect of unit tests is that developers rely on them. They believe they've tested the program exhaustively if the unit tests are all green. They forget to run other tests. For instance, they may not see the need to open the application in the browser and test their program using the UI.
Act 5: Unit tests as a survival strategy
Now things become dangerous. If everybody's happy running the unit test, they stop focusing on the global picture. They don't care about the quality of the user interface. They never see it, so why bother? They stop caring about how difficult it is to install the application. They don't have to. All they need is the unit test. They stop worrying about how long it takes to start the application. The unit test is up and running in a few seconds, and that's all it takes to implement an efficient development process. And we don't even have to talk about UX design. By definition, unit tests run without UI, so a team depending primarily on unit tests probably won't think a lot about UX design. I've joined several projects suffering from development round-trips between one minute and ten minutes. The worst case I've ever heard of was 90 minutes. It's obvious that no developer never starts the application server. Unit tests become a survival strategy. If you can't use the real system to check whether you've written the right program, you need an alternative. Unit tests to the rescue. The problem is that you stop caring about the target system. For instance, you may add a useful library to your project. The library slows down the start of the application server a couple of seconds, but you don't care. All you care about is the unit test, and it's still fast enough. Add enough useful-but-slow libraries, and your application server's performance goes south. But that's the operations department's problem. Maybe it's also the customer's problem. The developer team won't notice. Their unit tests are both fast and green.
Quitting the boards
No doubt about it, unit tests are a valuable tool to keep up the quality of your product. But they are only part of the story. I've observed several times developers becoming so obsessed with unit tests that they forgot about the product they deliver. If you have a say in your project, see to it that this kind of things doesn't happen. Make sure that the real product starts in a few seconds. Don't buy tools like JRebel just to patch the faults of your development process. Mind you: JRebel is a great and useful product, but more often than not it's used to fix errors that shouldn't have occurred in the first place. See to it that every developer runs a UI test every once in a while. Also, see to it that they don't run it on the integration test environment. They should run it locally. Try to ensure that your team tests everything that needs to be tested - but only those things that can sensibly be tested. If the team gets to euphoric about unit testing, tell them about the other approaches to test software. And here's the bonus section. There's one challenge every single project I've joined or led during the last ten years fails: Make sure that every developer can install the entire project within half an hour. In theory, installing an application server and importing a Maven
pom.xml, a Gradle
build.gradle, or a Node.js
package.json is all you need to get started. If these steps are followed by many instructions how to configure things, you know something's gone wrong.
Wrapping it up
Developers love complexity. It's fun, and it shows their skill. However, in the long run, complexity is going to boomerang you. Unit tests are a great tool to keep things simple. They break the program into small, manageable chunks that are easy to swallow. The catch is that this only makes the developers' lives easier. The program as a whole is a collection of many small improvements adding to the complexity. Like so often, the emerging complexity is more than simply the sum of its parts. So beware of unit tests. They are great tools, but if you notice they become the primary development tool, you know you have to do something about it.