Talk about a nightmare before Christmas. According to Computerworld, online shoppers went bananas on Black Friday, topping $1 billion in purchases for the first time ever. Cyber Monday sales were even bigger. But not all retailers cashed in, and not because consumers didn’t want what they sell. To the contrary, customer demand deluged those retailers, who responded by floating the big holiday fail whale because their apps couldn’t handle the traffic.
Panopta, which monitors website performance as part of its business, reports in its 2012 Availability Index Holiday Edition that 77 retail sites have reported website outages. Several, including kmart.com and sears.com have been down for nearly 11 hours. It’s hard to imagine the sales lost during that downtime.
Kohl’s had the great idea of giving customers a jump on Black Friday sales by offering 500 early-bird specials starting the day before Thanksgiving. Then came Thanksgiving evening, when eager shoppers were met with an outage message every time they tried to click on their shopping carts.
You have to think a lot of these problems go back to inadequate functional and performance testing. How do you test apps under loads hundreds of times heavier than normal?
Testing Isn't An After-The-Fact Activity
Too many people – almost everyone – in the software community still think of performance testing as an “after the fact” activity. You run a finished or nearly finished app under a certain load, record the app’s response time and ensure that it meets specifications. If it does, great. But what if it doesn’t? If testing comes at the end of the process, when you have the least ability to actually fix problems, then you have no option but to start over.
Agile techniques, where smaller, independent teams define, test and develop small units of functionality in shorter cycles, help some. But even then, some hard-to-catch structural code errors and bugs will often slip through.
And that’s to say nothing about load tests under the extreme conditions of holiday shopping season. What if the loads are 10x larger than anticipated? Or 100x?
Will Your Company Be On the Sideline When It Counts?
With Service Virtualization techniques, we can capture the performance and response rate of constrained systems and measure a profile of the requests that an app would typically make of it – load patterns, scenarios, types of data, etc.
The result is a more reliable outcome, with lower costs for development, repair and adjustment, so when online shopping is hitting new records, you’re not sitting on the sideline wondering where you went wrong.
Service Virtualization should be ready and available between every layer where dependencies exist between systems to ensure that apps are developed in an environment that is as realistic and “live-like” as possible. We’d hazard a guess that several dozen retailers wish now that they’d checked out SV.