If your company is dragging its feet on software test automation, a recent report from a pair of academic computer scientists might provide the ammunition you need to confront the C-suite and get things moving.
The academic paper, by Divya Kumar and K.K. Mishra of the Motilal Nehru National Institute of Technology in India, was published in the journal Procedia. The researchers say their experiments “clearly show the positive effects of test automation on cost, quality and time to market of the software” they studied.
As they note, software testing soaks up half or more of total resources devoted to development, which we can all agree constitutes a major obstacle to profitability. It follows that removing such an obstacle would be a huge opportunity to drive up software’s value and give companies an upper hand in the marketplace.
That’s pretty obvious, right? But C-level executives can be reluctant to approve test automation because of another C-word: Cost. That’s where Kumar and Mishra went to work — to establish a precise ROI when balancing costs against improved quality, reduced human effort and speedier time to market.
They set up an experiment, using three different enterprise software projects, all of which were produced in an iterative enhancement model. That is, a version is released while work continues to add new features.
Kumar and Mishra worked to evaluate the impacts of test automation in terms of five types of effort expended, from manual testing effort to automated testing effort to total test-team effort. They also defined six main quality characteristics: functionality, reliability, usability, efficiency, maintainability and portability.
A complex equation boiled all the work down to a simple question: What’s the total cost or effort impact of automating?
In each of the three software cases, the results were significant (see figure at left). The resulting products all had fewer errors and were produced at far less effort. Most notable were significant gains across the board in both cost and time. (That’s not to mention how test automation reduces your stress!) As the table shows, all three projects required slightly more effort on the first try at automation, but subsequent iterations saw effort reductions of up to 75 percent.
“It is a common observation in all the projects that there [are] positive cost and time impacts of test automation, and quality is also improved in most of the cases as program [are] found incorrect fewer numbers of times with automated test cases than with manual testing,” the researchers wrote. “The availability increases in all the cases, and relative time in testing is also fairly decreased because of test automation.”
A reminder here: This is not your run-of-the-mill marketing message. Kumar and Mishra are unaffiliated computer scientists, which ought to give their findings more velocity in the broader discussion about dev/test best practices.
And, they’re emphatic in their conclusion. If your company is among those reluctant to take on the up-front cost of automation, this report is a clip-and-save moment. Have a look for yourself, and share this post with those who are still dragging their feet.