Beyond Java

Published under an CC-BY-3.0 licence by Oliver Widder.

Be that as it may, framework developers and language designers do worry a lot about the pitfalls of accidental NullPointerExceptions. Almost every modern programming language has some protection against NPEs. Think of Kotlin, Ceylon, TypeScript, C#, just to name a few. And even Java programmers can do something about it. Let's examine the state of the art.

Elvis operator to the rescue

As to my experience - an experience not every Java programmer shares! But we'll come back to this in a minute - well, as to my experience, a typical Java source code looks like so:

You know the drill. You can never rely on a variable having a reasonable value. So you end up adding null checks every second line. Other programming languages have elegant solutions for that. For example, Groovy allows you to use the Elvis operator:

The Elvis operator calls the method if the call site isn't null, and returns null if the call site is null. That allows you to get rid of the zillions of null checks.

Null-safety in other programming languages

It's not only Groovy. Kotlin, C# 8+, Swift, TypeScript 3.7+, Ceylon, you name it: most programming languages protect you against NullPointerExceptions. The Elvis operator is even going to land in JavaScript soon. Ceylon, TypeScript, and Kotlin offer an even more useful approach. By default, null isn't a legal value. You simply can't assign null to a variable. If you really need null values, you have to declare them:

Unfortunately, there's no such thing in the Java language, and it's unlikely that's ever going to change. That's Java for you. It's a rock-solid language you can build a business on. But it's a bit slow to pick up new ideas. In a way, that's a good thing: if the Java core team picks up a new feature, we know they didn't do the prematurely. Even so, there are nasty surprises. Even Lambdas, which have been added after many years of discussion, proved to be used in unexpected ways. The Java language architects have learned to be very careful, so they slowly adopt new features.

So we have to deal with NullPointerExceptions differently.

What about Optional?

At this point, things get interesting. Most developers believe that Optional is the solution the Java language team offers.

I never bought that. A variable of type Optional can still be null. So you've won little. Instead of getting rid of the null check, you now have to check for null and for Option.isEmpty(). When you see an Optional, you never expect it to be null. But that's just a convention that's easy to ignore.

As it turns out, Optional was never intended to get rid of the NullPointerExceptions. It has been invented to solve a completely different problem: dealing gracefully with null values in streams. When a stream operator returns a null value, the fluent API is broken, so it's better not to use null to indicate there's no value. Watch the talk of Stuart Marks to learn about all the details. It's both an interesting and fun talk. You won't mind it takes an hour. Highly recommended!

But wait - are NullPointerExceptions a problem at all?

When I asked my team if we should add annotations like @NonNull or @Nullable, some of them told me they've got no problems with the NullPointerException.

There might be a lesson in this experience. More often than not, modern software development is test-driven development. Or at least some approximation to that, depending on your team's culture. Truth to tell, most projects I've worked in don't depend on automated tests. But if you do, it changes a lot of things.

Published under a Pixabay licence by Gerd Altmann.

But it doesn't have to be that way. Apples and oranges belong to different categories of fruit. If your compiler is aware of the difference, it can tell you you're making a non-sensical comparison.

Languages with a dynamic or weak type system tend to report the error at runtime. What could be a squiggly line in the IDE becomes a JIRA ticket reported by an angry customer.

It could be worse: If you're a C programmer, the equivalent of a NullPointerException usually is a bluescreen.

Strong tests vs. strong types

How can you survive in such an environment without going nuts?

Well, maybe you're the adventurous one. Ten years ago, the occasional bluescreen was perfectly normal. So why bother?

But it's much more likely you've picked up the habit of writing tests. Chances are you're writing tests to passing null parameters. The other effect of writing tests is it changes your mindset. You spend more time thinking about your algorithm. Plus, well-designed tests define a contract on how to use your code. TDD converts a hurried hacker like me to a careful, pensive programmer. You stop passing null pointers accidentally. Careless mistakes like this become an extinct species in a test-driven world.

Just look at the average JavaScript or Ruby project. The weaker the type system of your pet language is, the more the developers insist on writing tests.

When you're approaching 100% test coverage, you're covering many corner cases your type system also covers. You can compensate for the weaknesses of your language of choice by writing more tests. That's great because a loose type system means you can do many elegant tricks you couldn't with a strict type system.

Ain't that great? We've solved the billion-dollar problem. Just increase your test coverage, as you should, and you're fine!

Compiler checks are cheaper

Here's the catch: writing a test to protect you against null pointers is tedious and boring. Even if you're fast, it takes at least five to ten minutes. That's where the approach of TypeScript, Ceylon, and Kotlin really excels: null-safety is baked into the language. You have to write more code to be able to suffer from a NullPointerException.

Testing the null safety annotations

When I presented the idea to my team, a team member insisted on testing the null-safety data types. We have to protect ourselves from a program who accidentally converts the string to a string | null.

I don't think so. In my eyes declaring a method parameter never to be null is the same as writing a test. It's just you delegate writing and executing the test to the compiler. Plus, the compiler offers faster feedback. Modern IDEs evaluate the null-safety data types, so you see the error messages as you type.

How to add null-safety to Java

Several years ago, there was a discussion about adding null-safety to the Java language, but it turned out to be complicated. It'd break a lot of existing code, and backward compatibility is the holy grail of the Java world. Other programming languages were more courageous, and it proved to be a long way to null-safety. For example, it took at least one year until you could activate the null-safety in Angular. The main problem is usually libraries written without null-safety in mind.

The Java approach to declare "must never be null" is using an annotation. The annotation can be anything: just documentation, it can be evaluated by your IDE, it can be checked by your build pipeline, and it can even be evaluated at runtime. Annotations are a very flexible and powerful concept.

Annotations have become so popular that they've become a programming language of itself. That's why Lukas Eder considers nullability annotations a bad idea. He's working a lot with OR-mappers, so his programs already have many annotations. Adding more annotations drowns the payload code in annotations. I don't think this much of a problem with the nullability annotations. They are mostly used for method parameters and return types, which are hardly ever annotated.

So annotations are the way to go. During our research, we've found a surprising number of nullability annotations:

To my surprise, there's no annotation with @Retention(SOURCE). Your IDE and Sonarqube can analyze the source code, so it's a bit surprising that the annotation makes it into the class file. As a consequence, you have to ship the annotation library in production. That's not much - in most cases, it's really just two annotations - so the performance impact is negligible.

Nullness checks with IntelliJ

IntelliJ may not be my favorite tool, but I have to admit the team at JetBrain does a great job catching NullPointerException in the IDE. IntelliJ ships with its own set of nullness annotations. However, you can configure IntelliJ to use other annotations.

I've promised you squiggly lines, but for some reason, IntelliJ simply highlights the conspicuous line. That may come naturally for somebody working with IntelliJ frequently, but from a UX design perspective, that's wrong on many levels. In real life, marking a text with a yellow text marker simply means the line is important. If you want to mark it as wrong, you'd probably use another color. Second, the yellow mark represents a warning in IntelliJ, but that nonsense, too: it's obviously an error, so it should be displayed as an error. Luckily, you can also configure nullness violations to be reported as errors (second screenshot). If you prefer to stick with the warning, but want to display them differently, the third screenshot is your friend.

Be that as it may, the screenshot showing the source code is pretty exciting. This article is about marking parameters non-nullable, but the source code doesn't do such a thing. Instead, IntelliJ does that under the hood. That's excellent news. For one, IntelliJ already cares about nullness. We don't have to modify our code to achieve that goal. The other good news is that we can have IntelliJ generate us all the nullness annotations automatically. I've run this for an entire project. The feature works like a charm. Alternatively, IntelliJ can display the inferred annotations after configuring it accordingly.

I want my project to be future-proof, so I prefer adding the annotations to the source code. We want to write fewer tests by delegating the hard work to the compiler. The inferred annotation changes immediately when you modify the annotation. That's the equivalent of automatically adjusting the test, opening Pandora's box for all kinds of accidental mistakes.

Nullchecks with Eclipse

By default, Eclipse uses a different set of annotations than IntelliJ. Apart from that, the null-safety annotations work pretty much the same. Here's a presentation covering Eclipse in some more detail. Eclipse allows you to declare a class-wide default using the annotation @NonNullByDefault.

Nullchecks with Visual Studio Code

At the time of writing, the Java plugin of VS Code doesn't support the nullness annotations. That's a bit disappointing. However, the Java plugin is under heavy development, so there's hope. But maybe my initial theory is correct: modern Java programming is less chaotic and more test-driven than it used to be. Nobody's missing nullness annotations.

However, you can install the SonarLint plugin. So Sonarqube is responsible for checking the annotations, and the error messages are displayed in Visual Studio Code. As far as I can see, you don't even need a Sonarqube installation.

Nullchecks with static code analyzers

There used to be many static code analyzers. As to my experience, Checkstyle, Findbugs, and Sonarqube are the most popular of them. For some reason, the Findbugs project died due to open source developer burnout. There's a follow-up project called Spotbugs, but it has yet to become as popular as it deserves. So I focus on Sonarqube.

That works pretty well. Sonarqube reliably detects every potential NPE we hid in our test code. My co-worker Malte has prepared a GitHub project you can clone and run yourself. It's both a Gradle project and a shell script running Sonarqube in a Docker container.

Currently, I don't have a Sonarqube screenshot at hand, but the Sonarqube messages are also displayed in IntelliJ:

Breaking your build pipeline

If you're using a CI or CD pipeline like Jenkins, you can install the build breaker plugin in Sonarqube to prevent nullness violations from slipping into production.

Lombok

Lombok also adds nullness annotations. However, they are only evaluated at runtime. When you call a method with a null value, you're rewarded with a helpful error message.

I consider this a bit disappointing. If you're lucky, Lombok detects the possible NPE several method calls before it happens. Shifting runtime error detection up the call stack is always useful. But it's a far cry from what the compiler or the IDE can offer. So I don't recommend using the nullness annotations of Lombok.

What about JSR305?

There's a Java standard request for @NonNull and @Nullable. It was accepted unanimously in 2006. For some reason, it lays dormant since then. So most type-checker projects have deprecated these libraries. It's one of the rare cases you should avoid a Java standard.

Which annotations do I recommend?

At the end of the day, it's up to you. Currently, I tend to recommend using the annotations shipped with your IDE. Most IDEs and Sonarqube seem to be widely configurable, so it doesn't make much of a difference. If you need more advice, have a look at the answer of Milan Dhameliya who's collected an impressive number of facts about different nullness annotations.

If you're adopting the JSR305 annotations, think twice. There's a legal pitfall. The license terms of Java do not permit anyone outside Oracle to implement the JSR305 annotations, but the only implementation available has been implemented and published by Findbugs. That causes some legal uncertainty, so many projects like SpotBugs and Guava have stopped using the JSR305 annotations. You can find more details in a SpotBugs ticket.

Is it fair to call it the billion dollar mistake?

The trademark of BeyondJava.net is it doesn't just explain how things are. It tries to give you some background, too. This time, the obvious question is why we got into this mess. How come the NullPointerException is such a loyal friend (or fiend) of every Java developer?

As far as I remember, the guy who's introduced null to the Java language has even apologized for doing so. But that's a bit unfair.

What's true is that almost every Java program is scattered with guards and checks against the NPE. From this point of view, the invention of the null reference has been a - pardon the language! - pain in the arse to countless programmers. It's cost a lot of effort to deal with. It's possible this cost companies millions or even billions of bucks.

When Java was conceived, memory was a scarce resource, and CPU were one hundred times slower than they are today. Null values are remarkably compact, and CPUs have optimizations for them. Ignoring such a performance optimization in favor of maintainability would've sounded weird. No language risking such a gamble could gain a significant market share.

Actually, there were innovative languages like Smalltalk taking the risk. However, it's not Smalltalk, which is ubiquitous today. It's Java.

It seems the billion-dollar mistake was the key to success.

Wrapping it up

The NullPointerException haunts the Java world, so it's good you can do something about it. Probably the best option is to use the nullness annotation your IDE and your static code checker use.

However, it seems the Java world feels far less haunted than I expected. In part, that's because the IDEs have become pretty good and detect many potential NullPointerExceptions out-of-the-box. In part, it may be due to the popularity of automated tests.

However, I still believe that annotating a method parameter never to be null is as good as writing a test. It's just a lot cheaper and more effective because you see your mistake immediately while writing the code. An interesting result is you can't write a nullness test after introducing the @NonNull annotation. Your IDE immediately marks your test passing Null as an error.