Identify Problems with Testing Multithreaded Code
Recognize the issues that create the need for unit testing in a multithreaded Java environment
by Robert W. Nettleton

Posted November 26, 2003

Thread safety—it's a term we use to describe Java code that can be used safely by multiple concurrent threads of control in a program. There are many techniques that must be used in a concurrent application to ensure correctness. One of the most challenging tasks a programmer can face is to fix threading bugs in already existing code. Making someone else's code thread-safe is quite a daunting task. Deadlocks and race conditions provide truly challenging work for any developer. There are many problems that developers face when working on concurrent code. These problems deal with issues that stem from the facts of concurrent code itself, and the difficulties associated with reproducing bugs and errors. Another set of problems then occurs: how does a team of developers keep concurrency bugs from reappearing in their code?

I faced these very issues on a recent project in which I was tasked with fixing some serious threading problems in a given component that I had inherited from another developer. Being a believer in the power of test-first coding, I would develop tests alongside each new feature, which enabled me to modify code quickly while assuring a certain base level of quality. For simple functionality tests, these tests were reasonably easy to write. I would instantiate a given class, feed it simulated data, and vary the correctness of the outputs. For concurrency problems, however, I needed something more than that. I needed a small framework for running given tasks concurrently, and support for checking the pass/fail condition of a given set of concurrent tasks. Let's begin looking at that framework here.

If you're currently developing or maintaining concurrent Java applications, then read on. In this first installment we'll begin by discussing the inherent problems with developing and testing multithreaded code, including the challenges of duplicating bugs that occur only when there is more than one thread of control executing. Then in the next installment we'll look at a method that is very useful in the development and maintenance of concurrent Java software: test-first coding. I'll provide some examples of how to use test-first in a multithreaded scenario, including helpful strategies on using these techniques in your own projects. Then we'll conclude in the third installment with a set of best practices for testing concurrent code as well as some limitations. These best practices can be helpful when developers are implementing unit tests during the development cycle. When test-first coding is used, it is possible to guarantee a degree of thread safety that ordinary development methodologies don't allow for.

Confront the Hurdles
If you're the developer of a library shared by multiple applications, or if you're tasked with maintaining one, you probably already have an inkling of the complexity involved in creating and maintaining thread-safe code. Many of the problems arise because developers are taking educated guesses at the correctness of their code. Developers might not envision their component being used in a multithreaded environment. Another problem that could arise is what I call the hidden thread bug: a developer properly maintains a thread-safe object, synchronizing access to all shared mutable data, but the object maintains references that are not thread-safe. Using a static hashmap to hold references to objects that are expensive to create serves as an example. Many times it will not be obvious to developers that this situation creates a possible threading nightmare that can allow deadlocks, race conditions, and other bugs that can be very difficult to duplicate.

Why are these problems so difficult to solve? With all the advances in the state of the art in software, one would think that developers should be able to easily write thread-safe code, especially with the help of modern integrated development environments (IDEs). The sad truth of the matter is that concurrent development in Java or any other language is difficult, and there are no easy answers.

One problem is the nondeterministic nature of a thread scheduler. When a set of threads runs in a given Java Virtual Machine (JVM) instance, there is no way for the developer to determine which thread will execute, for how long it will be allowed to execute, and where each thread is allowed to start and stop running. In fact, there is no guarantee that a given thread will be given time to run at all. Since the JVM needs to support threading on multiple platforms, it needs to map abstractly to several operating systems' threading models.

Forcing developers to remain scheduler-neutral, the JVM is free to implement this algorithm in a variety of ways. For the most part, this is a good thing. When developers are freed from the problem of scheduling concurrent threads, they can focus on the creation of application-level threads and only worry about synchronizing access at the appropriate times. The big problem with this approach is that it can be hard to visualize. Since the scheduler can pick any thread to run for any length of time, there is no way to know where each thread will be interrupted to make way for another thread. This approach can cause statement execution to interleave between threads. It is for this very reason that we must protect access to any mutable data that is shared between threads.

This restriction is a fact of life for Java developers working on concurrent solutions to problems, which can cause larger headaches in the sense that concurrency bugs (deadlocks, race conditions, and the like) might not always be caught in normal testing because the amount of load, number of threads, and execution time may differ in applications using a given component concurrently. A large problem occurs because most multithreaded development is done on single-processor machines, where it is never possible for more than one given thread to execute at a time. Code that is considered thread safe may run correctly for years on a single-processor Windows machine, and then break the instant it is installed on a multiprocessor Solaris or Linux server. The interweaving of instructions that can cause improperly synchronized code to fail is more likely to happen on a multiprocessor machine. For most of us this is not an option, so we need some tool to simulate this.

The Complexity of It All
A second problem facing developers is code complexity. In large-scale, object-oriented systems, the task of understanding the threading semantics of applications with hundreds or thousands of classes can be almost impossible to grasp. There are approaches that allow a system to maintain locking semantics in a systematic and orderly way. Doug Lea's Concurrent Programming in Java: Design Principles and Patterns (Addison-Wesley, 1999) mentions several approaches that are very helpful. If you're building a system from scratch, then these approaches can be very useful in building code that can be thread safe. Since most of us don't code in a complete vacuum, writing everything from scratch (although this would be nice), developers are faced with the complexity of understanding the locking semantics of code written by other developers, as well as interact with other components in a system that has evolved over time. This environment may mean that the system does not define a universal locking mechanism to avoid deadlocks and liveness problems. If a lock is acquired in one method, it can be very difficult to envision the other locks acquired in the call stack and how each lock may affect the other. This kind of problem is what leads to deadlocks, or hanging problems in code. Many times I've fixed synchronization bugs that occur in classes that at first seem remotely related, until you can picture the resource request inversion.

The third most daunting problem to face developers in this type of code follows from the inherent complexity in large-scale software systems—regression. On medium- to large-sized projects there can be many developers making changes at any given time. A rapidly evolving code base can be viewed as fertile ground for bugs of many different types to sprout up. As a team races to meet demanding deadlines, the functional correctness of the code can decrease over time, as a result of unintended effects of last-minute changes. In my experience, concurrent code is most easily broken during this period. As a product is fully integrated from its component parts, issues like race conditions and deadlocks become particularly noticeable.

These issues can stand in the way of any developer who wants to write concurrent Java code correctly. The difficulties in duplicating use cases as well as the behavior of the JVM can make verifying multithreaded code a very challenging task. As bugs start to appear, and the team enters a hectic mode of bug fixing at the eleventh hour, a developers may ask themselves, "What can be done to make this easier?"

In the next installment, we'll look at test-first coding, how it can aid developers in rapid development environment, and then look at developing a simple unit test library for concurrent testing.

About the Author
Robert W. Nettleton is a Java developer working for Oracle's J2EE application server team. Contact Robert at .