Take a second and think about your favorite businesses. What do they do well, and where do they fall short? Good companies have successfully created a brand that resonates with their customer base. Great companies have found a way to evolve their reach beyond brand loyalty to excellent customer service. The best organizations know that Technology isn’t a ‘nice to have’, it’s a channel to your customers. Translation: if you want to be the next Amazon, Netflix, or Uber, you need to find a way to scale your brand to a global audience.
But how do you scale Quality in the eyes of your users especially when the end-user has more options than ever before? Organizations have been given the power to quickly build and customize technology iteratively. Within relatively constrained timeframes, organizations can deliver their new capabilities to the hands of their users. But without the right approach, that kind of velocity is a danger proposition (more on that later).
Now, don’t be confused, I still believe that organizations need to develop new ideas and put those ideas into the hands of their customers as quickly as possible, but it’s important for those ideas to functionally work, scale to the size of your potential customer-base, integrate with what you already have, and are acceptable in the eyes of your loyal customers. Sound like a lot, huh? Well, it is. Businesses need Digital Transformation to survive. What does that Digital Transformation need in order for it to survive? Continuous Testing.
What does Continuous Testing even mean?
Good question. Continuous Testing does not mean automated testing. It doesn’t mean picking Service Level Agreements that are extremely aggressive for when Testing will be complete.
Continuous Testing means that you’re able (at any given point in time) to execute valuable parts of your validation process to ensure that you’re flowing at the same pace that you are delivering ideas to your customers.
Why is Continuous Testing even necessary?
Well, let’s roll the tape back and understand how organizations delivered quality to their customers in the past. Most organizations delivered technology in what was referred to as a Waterfall Software Development methodology. ‘Waterfall’ implies that organizations would flow from stage to stage of their Application Development process sequentially and the overall time budgeted for a project would be extended over a period of months, or in some cases even years. The problem with that mindset today is that technology changes more rapidly than ever before. 6 months can be an eternity for your clients to wait. Fast forward the tape, businesses are now developing and delivering software within an Agile methodology which basically means smaller chunks more rapidly. This means that ideas are your lifeblood and you’re only as relevant to those customers if you can deliver them at the pace that they demand. But be careful, because if those ideas are delivered, but don’t execute as intended or scale to the size of your total audience, then you risk losing your customers as well.
Okay, I’m ready to deliver ideas rapidly, but how do I ensure quality?
Great, you’ve come to the right place. Let me help you understand what you have to verify:
- Is your application functioning the way you intended it to?
- Does every piece of working technology that you had in previous versions of your software still work (as intended)?
- Does your product scale to your customer base today?
- Does it scale to your total potential customer base?
- Will it scale during buying events that cause more customers to come to you? (Black Friday, Cyber Monday, the Super Bowl, President’s Day, Christmas Eve, Tax Season, or whatever season causes your most loyal and new clients to come visit you)
- How well does that new functionality integrate with what you had in previous versions?
- Will your loyal customers accept the new product?
Now, you have to validate that in an on-going 2 to 4 week interval, because that’s the rate at which the business has chosen to roll out features to your customer base. If that’s not enough to make you quiver, you also have to deliver it on more platforms, devices, and form factors than ever before.
But I have already spent a lot of money on Automated Testing – will that help?
Short answer, yes. But you don’t want to use a hammer to do a screwdriver’s job. Automated testing is the process of automatically executing multiple tests in a fraction of the time it takes to execute a single test manually. Automated tests is extremely effective and useful, but it requires on-going maintenance. As your business changes, so does the relevance of the tests that you’re executing. You don’t have time to figure out and do the analysis on what tests in your suite are still relevant and which ones need to be updated – as I mentioned before, you have less time than ever before to get this entire process done.
Okay, I’m a bit behind, but my team has done the analysis, I know exactly which tests to execute and I have traced that back to the requirements set by the business, but… I still have to wait?!?!
Now that you’ve fallen behind, you’re margin for error is extremely small. You’re validating that the last version of your product is working as intended. You are also doing this process in a race against your customers as they use your product in production, and they outnumber you significantly. The challenge is that you have some constraints that are slowing you down from executing that elusive optimal test set:
- Your Test Environment doesn’t have access to the critical endpoints/systems you need to test against.
- You don’t have access to the data to power your automated testing process.
As we continue to develop and deploy more features to our customers, the gap between where we want to Test and where we actually are Testing continues to widen – is that normal?
In a word, absolutely. Here’s the challenge. You’ve started rolling out capabilities to your audience, but you never balanced the Testing Time budget. This means that you never truly found out how much time it takes to perform your testing process from beginning to end. This means more technical debt. Technical Debt is the gap between what your team delivered and what your team validated works. It’s a slippery slope. Much like Financial Debt, if you continue to take on more of it without paying it down, there can be significant consequences to you. Think of a car getting repossessed. In the business world, technical debt could mean losing a customer… forever!
So crunch the numbers and figure out where you currently stand so that you understand how to ‘trim the fat’.
Enter Continuous Testing.
The good news is that everything above is possible to overcome no matter the level of maturity. I know right about now, I’ve made it seem like something to be worried about. But, don’t be. I’m here to offer a solution:
- Start to understand your choke points. The firsts step in any process is to accept reality and start measuring the gap. This typically means to model out your entire delivery process from Idea to Outcome and leverage that visibility to build realistic improvement goals for your process.
- Shorten the gap between requirements and testing. Once you understand how the business intends for something to work, and how it’ll be delivered, you want to make sure that you have that flow & process diagrammed for what you’re going to test and when it’ll happen. This is your blueprint to get everybody on the same page.
- Now, take that diagram and leverage it to automatically generate the least amount of tests required to give you 100% test coverage.
- Whether you’re leveraging automation or manual testing processes (or both), reach your fastest point by ensuring that you have access to every possible thing you need to execute those test.
- Help your business users better understand what happens when you need the ability to scale on events like Black Friday, Cyber Monday, or whatever event is causing your customers to come to you. And do this as early as you possibly can.
- Rinse, repeat, and orchestrate for success.
Continuous Testing isn’t a myth. It’s a core competency with significant benefits for those that adapt and a liability for those that forego. When done right, it’s what makes IT a trusted partner to the Business. When done incorrectly, here’s what’s at stake:
- Revenue & Profit
- Brand/Customer Loyalty
- Regulatory Compliance & Risk
- Your Entire Business
Everyone in your company owns quality. And every journey starts somewhere. If there’s one piece of advice to take, it’s that Testing at the Speed of Business is achievable, but to experience success, Testing has to be taken seriously.
Shall we begin?