Why Continuous Testing Is Critical, But Not Enough

Shoeb Javed

The ability to innovate and quickly adapt to changes (both business- and IT-driven) is key to an organization’s success. Companies are under pressure to release innovation faster across their packaged applications. While innovation creates great opportunity, it also introduces great risk. The implementation and ongoing management of enterprise applications is complex, and a crucial component is making sure that end-to-end processes work—every time, every day.

A new vision for test automation extends to enabling business users and organizations to adopt, accept, and utilize new applications faster. When processes across enterprise applications are executed seamlessly and as designed, companies can leverage the potential of their enterprise apps to achieve strategic goals like entering new markets, increasing revenue, and maintaining a competitive advantage. Continuous testing and quality are the foundation of successful business outcomes.

Here’s a look at continuous testing vs. continuous quality. 

What is continuous testing?

Let’s start with the basics. The simplest definition of continuous testing is “the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.”

What does DevOps have to do with continuous testing?

In 2009, DevOps was introduced as an extension of the Agile development methodology, enabling higher efficiency and collaboration, so companies can react more swiftly to the constant changes in today’s digital economy. Although increased speed and velocity are positive outgrowths of Agile and DevOps, customers may be delivering software that hasn’t been properly tested or tested at all. This can expose companies to risk like defects in production.  This drives the need for a new approach to testing that aligns with the new 24/7 release cycle.

Why isn’t continuous testing enough?

Continuous testing isn’t enough because it doesn’t equate to continuous quality. Continuous testing is just the tip of the iceberg. I see it as a fundamental necessity because of the complexity of enterprise landscapes. Continuous testing is a critical phase of the continuous delivery process, and it helps us address the demand for superior customer experiences. We gain immediate and real-time feedback on business risks and maintain continuity across the business. Despite the valuable insights we gain from continuous testing, I strongly believe no amount of testing alone can ensure quality.

So, what is continuous quality in DevOps?

The problem is, there’s not an industry standard for continuous quality in DevOps. Even though DevOps was originally designed to produce high-quality, updated software that gets to the user faster, “quality” is not well-defined. I believe our ability to develop an industry standard is an essential next step, especially if we expect to move from aspiring to actually delivering highly functional systems that consider the user experience.     

How do we get to a higher standard?

I believe there are two paths to a higher standard. The first path is for us to better define continuous testing, or raise our game and make a collective shift to continuous quality. I see this issue as one of the most important factors in the next phase of the DevOps evolution.

The scope of continuous quality is higher-reaching than continuous testing. As we look to develop an industry definition of continuous quality, I believe we must consider the following factors:

  • A deep understanding of business processes in production
  • Upfront promotion of the need for objective data from the business
  • Recognition of internal and external customers for quality metrics
  • Prioritization of continuous quality from day one
  • Incorporating quality into the Agile development methodology (don’t “throw everything over the wall”)
  • Tracking project health to include quality at each phase of the SDLC
  • End-to-end business process testing, not just single service/component quality
  • Performance monitoring and optimization throughout development and production for feedback

How does your DevOps team currently define continuous quality?

For more on software development trends, see How Open Source Is Changing Software Innovation.


Shoeb Javed

About Shoeb Javed

Shoeb Javed is chief technology officer at Worksoft, a leading global provider of automation software for high-velocity business process discovery and testing. Shoeb is responsible for the technology strategy, software development, quality assurance and customer support for all Worksoft solutions. As CTO, Shoeb works with quality assurance and business leaders of some of the largest global Fortune 1000 corporations to help automate testing of complex packaged enterprise applications to speed up project timelines and improve operational efficiencies.