Preventing Major Bugs Late in Development


This is one of a number of articles I am writing to help identify common problems related to Quality Assurance (QA) the impacts the release of  products to market.  While I have been a product manager for most of my career I have helped develop QA test plans and spent 6 years developing a QA test management tool.  The guidelines I outline here are based on the QA best practices that we helped our customers implement in the field.

When possible I will provide real world examples from QA experts to illustrate the root cause of the challenges, and the real world techniques that are used that can help you avoid these pitfalls in your day to day efforts. Hopefully if you can employ these suggestions you’ll reduce the churn between teams and demonstrate a more collaborative path forward is possible with the right implementation.

The Challenge: Preventing Major Bugs Late in Development

Moving away from Waterfall to Agile

A majority of testing organizations have traditionally followed the Waterfall method, thus allowing engineering to bring the project to a “testable state” at which point all available test resources would move from test authoring and other activities to performing a full pass or regression effort against a very complex set of features and changes that were un-tested (by QA) for weeks or possibly months. While most development team have transitioned to a more Agile methodology, many test teams are still using Waterfall type processes.

We don’t believe this is due to a lack of skill or experience from QA professionals, as there are many valid arguments that can be made in support of validation type testing.  This is especially true for third party testing when products are ready to be “certified” for submission into a program or marketplace. Another reasons for waterfall persisiting in QA is that many QA teams are outsourced.  Outsourced QA usually follows a Scope of Work (SOW) with defined requirements and milestones.  It is hard to teach Agile when the team is told to wait for, and then work towards, a SOW.

No matter the reason, testing efforts that are being undertaken as part of an ongoing development effort can receive many benefits in becoming more agile themselves.  Whenever possible they should work in parallel with engineering at as early a point in time as possible.

Striking a Balance Between Documentation, Coverage and Agility

When should your team begin testing a work-in-progress development effort? It’s not an easy decision for any QA Stakeholder to make. If your team has been provided with limited documentation on a new product or new feature set that you will be testing in the coming weeks the old mentality from Waterfall-style testing was to spend weeks or months writing test cases in a vacuum. Sure, you can share your progress with Product Management or Engineering to ensure coverage is adequate once you get your hands on a build to begin testing, but you’re not actively testing anything. Before we go any further, let’s be clear about our intentions – documentation is critical.

User Stories as a Framework for Testing

If your engineering team is working off of User Stories to define new features and there is adequate documentation for engineering to begin their work, then it’s likely that there will soon be an opportunity for your test engineers to begin testing early code integrations in support of the new stories. In fact, writing test cases in an agile environment reduces the common problem where QA has to start testing new features in “ad hoc” mode with little to no documentation. The stories themselves should provide enough information as a jumping off point and short, collaborative contact with engineering can help flesh out the test engineer’s authoring process so writing a valid test of a new feature doesn’t take weeks to complete.

We’re not drawing a line in the sand stating that spending time on writing formal test cases has no value, far from it. Consider the longer your resources are not engaged in providing incremental test support for engineering then there’s a good chance that major bugs are being written into the product’s systems and engineering is building on top of an unstable foundation. Depending on what flavor of Agile you’re working in you may even have assigned a 1:1 resource for QA and Engineering, in which case this engagement can be started earlier in the sprint than you might think.

Shift Left and Why It Matters

If QA is working in parallel with development than they should be able to test earlier in the cycle than waiting for a miracle build and then starting their regression testing. The Shift Left movement is helping Quality Assurance transition to Quality Engineering and allowing test efforts to begin much earlier in development, which should help to alleviate many of these last minute roadblocks. Many organizations are moving aggressively to automation testing with the goals of enhancing efficiency and expanding scope, but many are being driven by basic market forces. Their competition is working faster and the market disruptors can turn out more innovation through nimble product release schedules across many device platforms, most of which would not be possible and still carry a reasonable level of quality and functionality without the support of automation tools and new processes. For a different perspective, look to your own past of product testing and you must have experienced a disastrous show stopping issue that impacted the business and your team that might have been avoided if found earlier in the cycle. I certainly have and surely if you haven’t yet stay vigilant and keep improving your skills and maybe you’ll be able to avoid the worst case scenarios.

Thanks for reading and keep an eye out for more best practices.