An Analytical Look at the Disagreeable Business of Gatekeeping


“The coding is done, we are just waiting on testing to release.”

Not every project is held up in QA, but any experienced manager (and too often new managers) will have faced this dreaded, most visceral example, of where quality and schedules collide. We will look at how we can make the most of this situation.


The benefit of a new product or feature is that it will visibly add near-term value. The future and often the hidden benefit of proper testing is the intangible and often unmeasured quality. Testing products ensure that they maintain value for your customers. Longevity keeps customers satisfied over the long term and improves retention. The challenge is that in a world driven by schedules, quarterly revenue projections, and new subscriber numbers, quality does not align very well with these goals. Too often the quality assurance “gatekeeper” is pushed aside to meet deadlines and products are delivered with fingers crossed. As they say, ignorance is bliss…until you check your email the next morning.


Rather than digging into old arguments and best practices that justify pausing for quality and pushing instead for more accurate scheduling, let us instead look at skipping quality assurance (in a kind of, sort of, responsible fashion). You have a skilled development team, so why not ask them the simple question of “Can we release?” And if they respond “Yes, we can release.” Is their response good enough? How does bypassing the gatekeeper have an effect on the outcome? Certainly, there is the actual risk of features going out that have never been seen by a tester. Less obviously, there is the change in perspective. If you skip traditional QA, the role of the tester is no longer synonymous with “gatekeeper.” Effectively, the decision of whether or not to release has now become the responsibility of a broader array of team members. The development team can no longer rely on the “tester(s)” to be responsible for over-eagerness in release schedules, and managers lose out on the option to blame schedule delays on the gatekeepers.

Changing the type of information and the method of delivery by QA to the development team will fundamentally change the value of quality assurance. All stakeholders prefer to have confidence in making a correct decision. This paradigm shift nullifies the developer vs. the gatekeeper conflict and instead creates a team of stakeholders, invested in achieving a common goal, a release that delivers improved value to customers at a high level of confidence.

Meaningful tests


What would a relationship look like with a “confidence team?” A confidence team would place less emphasis on overall testing and more emphasis on important tests that deliver meaningful data. As a process improvement consultant, what I would like to see is the output of clear and concise user experience level feedback. The feedback that presents meaningful scenarios based upon releasing the current version of the code and that demonstrates confidence. This means that the output cannot simply be presented as “95% of regression tests passed,” because really, what does that even mean?

For the product owners, the software never does what you want it to do, partially due to bugs, and somewhat due features that are not yet implemented. Even though we think of bugs as different than missing elements, in the end, they both are deviations from the desired state of the software. What the product owner tried to do, is to achieve as much of the desired state, for the least cost.

With that in mind, we want to communicate feedback to the product owner about how the software lines up to the desired picture. We want to present meaningful information so that decisions can be made with confidence. Let us look at two different statements presented to convey meaningful results:

“When a user cancels their account and then tries to reactivate their account, they are unable to log back in” and “New users don’t receive the email to activate their account.”

These are meaningful statements of how the capabilities of the software compare to the desired state. This is feedback that can be utilized by product owners to decide if this release is more valuable to customers than the last one.

This is very different information than some tests merely passing or failing. In effect, you want to get the patch notes of known defects as the output from your tests. As you can see from the examples, these two failures are not equivalent. But, if you look only at the number of cases that fail, rather than the result of the fails, there is not even enough information to demonstrate the unequal nature of the results and or to make a decision about what is acceptable and what is not.

Confidence leads to speed-2CONFIDENCE LEADS TO SPEED

People only want to go as fast as they can see, consider driving on a clear day versus a foggy night. By creating better visibility in the shortcomings of the software and offering this information to the decision makers they can bring features to market faster. People are used to working with solutions that don’t work perfectly. A life-saving heart drug won’t be sidelined for causing patients a dry mouth, but it might be if it also causes severe nausea. And people don’t stop driving their car because a blinker doesn’t work. People are accustomed to making these decisions, and the better the information is, the more confident people will be in their decisions.


In the fight between schedule and quality, schedule historically wins in a world in which time is money, and new subscriber numbers make headlines. What I propose is that if we shift our focus to delivering confidence, we reduce the struggle between scheduling, budget, and the QA gatekeeper. Delivering confidence means that we are releasing products with our eyes wide open. The schedule becomes less critical, and the value of tests is better understood. We start to approach quality assurance from a position of confidence: “what would it mean if we went live today?” If we ask this question regularly and consistently, the information gathered will give our businesses the confidence needed to optimize the trade-off between the delivery of value versus the cost of quality. Ultimately, functional products will be delivered, and long-term profits will be enhanced.

For QA and Release Managers:
Do you have an excellent way to communicate the impact of a release?
Join the discussion and share your keys to success.

Leave a Reply