QA Testing 101
MAR 6, 2015

QA Testing 101

By Delmar Howard, Program Manager, Intertek

Requirements Analysis

Requirements are the foundation of creating accurate and successful tests for your products. The first step is to perform a requirements analysis on your software.

This will help guide the project and is the start of documenting, validating and managing your test process.

The core principles of requirement analysis are:

  1. Collecting
  2. Analyzing
  3. Creation

Collecting requirements is a group effort that needs to take into account all stakeholders involved in the project to ensure the most coverage. It's ideal to use an outline method to identify specific test groups and sub-groups to make the next steps easier.

Analyzing: Once the collection phase is completed there must be an agreement by all stakeholders that the requirements created are in line with expectations.

Creation: After you have gathered all stakeholder feedback the creation of the test criteria should begin. This needs to take into account all feedback and information provided by stakeholders and put into a test executable format. There are many ways to perform this; see an example here: https://start.apptestnow.com/docs/FullTestSuite.xlsx.

What to automate

There is a delicate balance between manual and automated testing; they both have benefits but there are clear cases of tests that can and should be automated and those that should not.

A basic automation principle is to automate the smoke test portion of the test requirements and stress testing. Smoke tests are generally short (15 - 30 minutes) and cover only the most essential areas of testing. For example you may want to automate the launching, close and uninstallation of an application. While these are fairly short tests they can help you avoid allocating test cycles to testers for builds that are not quite ready.

Stress testing is key for applications that rely on a server or external resource for application functionality. While your application may work extremely well with your internal team it may fail when 30,000 users try to logon at once. This is where stress testing is needed to identify any weak points or data intensive functions your application has.

Execute your plan

At this point you should have your split of automated and manual test criteria as well as your test plans. The execution phase is where you will provide the steps to your test team. It's important to review the test criteria with your test team before they begin. While test cases should never be ambiguous sometimes they need to be refined. Having a group meeting to review the test criteria before execution will ensure the least amount of wasted time from improper test execution.

Track your bugs

The major principles in bug-tracking are bug report formatting and priority designation.

During the execution phase there will likely be many issues both small and large that your testers identify. One of the most important processes is to standardize the format of your bugs. Having all team members following the same template for bug entries will allow for easier export to bug-tracking systems and reconciliation with future regression tests.

Some basic examples priority designation are Minor, Major, Critical. When creating priority lists it is essential to be realistic; every bug that is found is not a critical bug and you need to make sure that when your testers classify a bug as minor it truly is minor and impacts the end user in an insignificant way. On the other end of the spectrum you want to have all critical bugs moved to the highest priority for your development team.

Regression Testing

If using the proper process of tracking bugs regression testing should be a natural continuation of the test cycle. Once a test pass is completed the bugs that are found need to be sent to your developers for fixes. Any bugs that come back as fixed by your development team need to have a regression pass completed to ensure the functionality is fixed and that it didn't break other areas of the application.

Completion

While testing is never truly "done" there will come a point where all critical bugs have been fixed and a determination is made to release a build to the public. This is where your last mile testing really needs to be evaluated. While your internal team performed admirably they are also the most familiar with your product and often that means there are areas that they may have been overlooked.

Many successful QA teams utilize a 3rd party to review the final builds before releasing them to the general public. This provides piece of mind and can validate your own internal team’s efforts or highlight areas that may need some refinement. Even with all of the effort you put in thus far the public launch can be an entirely different experience; in the final stretch your team should test the public build against your test criteria to ensure no changes occurred while publishing your application.


comments powered by Disqus