We don’t need a ‘pre-crime division’ for code writers

Remember the Tom Cruise flick Minority Report?

If you somehow missed it (you should go back and watch), the futuristic thriller is based on the development of prescient beings who could see the near future and detect crimes before they happened. Then, a division of “pre-crime” cops led by Cruise’s character would swoop in and arrest the perpetrator before he could act. Like this:

Why do I bring this up in the context of software development and testing? Because there’s talk of a pre-crime capability to be used on (against?), the code writer.

A pair of Swiss researchers, Sebastian C. Müller and Thomas Fritz, presented a scary argument recently at the 38th International Conference on Software Engineering in Austin. They say it’s possible to predict code quality by monitoring your biometrics. You should find their paper, “Using (bio)metrics to predict code quality online,” more than a little chilling.

Here’s their summary:

“Our results show that biometrics are indeed able to predict quality concerns of parts of the code while a developer is working on, improving upon a naive classifier by more than 26 percent and outperforming classifiers based on more traditional metrics. … Overall, the results from the presented studies suggest that biometrics have the potential to predict code quality concerns online and thus lower development and evolution costs.”

The researchers’ field study involved attaching biometric monitoring devices — one was a wristband that captured skin- and heart-related measurements and the other a chest band that picked up other heart and breathing metrics — on 10 professional developers over two weeks. Those measurements were correlated to a peer review of their work during the two weeks. Reviewers looked for actual bugs, inadequate documentation or violations of coding style.

The result? Indeed they found they could measure your body and predict when you were most likely to be screwing up the code you’re writing. Here are their words:

“The results of our study are promising, suggesting that developers’ biometrics can indeed be used to determine the perceived difficulty of code elements and furthermore to identify places in the code that end up with code quality concerns, such as bugs. A second smaller replication study we conducted also confirmed some of our findings on the automatic determination of difficult parts in the code. These results open up new opportunities to support developers when they are experiencing difficulties in the code and to fix quality concerns as early as possible, even right when they are being created. With the recent advances in biometric sensing technologies, and their decrease in invasiveness, we might soon be able to collect biometric data on each developer just like we are now already able to collect interaction data. However, this also opens up a discussion on privacy concerns and more research is needed to investigate a feasible solution.”

Indeed, the tail end of that conclusion, where they barely hint at the elephantine problem in their proposed method, cuts to the heart of why you should pay attention. Biometric screening of workers might well indicate when a worker of any kind might be lapsing into error, but it’s hard to see how the privacy concerns from such a method could ever be overcome.

With all due respect to these researchers, their study misses the point when it comes to achieving the worthy goal of better, cleaner code. The fact is, better methods already exist that don’t involve making employees wear invasive biometric devices.

Continuous, automated testing methods — including the use of Service Virtualization to enable testing throughout the SDLC — are growing in use across the enterprise spectrum. The analysts at Forrester found, empirically, that release automation powered by Service Virtualization and test-data management had substantial return on investment.

So, if your enterprise is looking at ways to root out errors quicker, before they cost you millions, my advice is to consider reliable technology that already exists rather than wait on the development of a “pre-crime division” for dev/test.  That stuff works only in the movies. Let’s leave it there.