“What…? You Say That QA and Testing Are Not the Same Thing…???”

Yes. That’s exactly what I’m saying.

In the software testing world and in the quality assurance industry, this is a sticky point. It irks us that these terms are oft used interchangeably.  How many times do you hear someone say, “Has that gone through QA yet?” when the person is talking about testing?  It’s actually an incorrect statement.  Personally, I have worked at numerous Fortune 100 companies where my title included “Quality Assurance” or “QA” when my role had absolutely nothing to do with QA.  At all.

Are you confused?

Well, that’s what I want to do—help explain the difference between Quality Assurance (QA) and software testing.

So, first let’s talk about testing.

Testing consists of many activities, but most people use it to describe the activity of executing tests, also known as test cases.  A test case has explicit steps that have predefined (key!!!) expected results as well as known inputs and outputs.  When these activities and steps are followed—or “executed”— the results of the activities (‘actual results’) are compared to the expected results. The test passes if the actual results match the expected results.  The test fails if the actual results differ from the expected results.

So, the distinctive thing about the above is that there are expected results identified and documented before the tests are executed.  That means one thing. The tests are planned and someone must figure out how the application under test should respond when the tests are run.  So, there has to be enough information in the requirements and design documentation in order to figure this out.  If you can’t figure out the expected results beforehand, then you have nothing to compare your actual test results to after you execute the tests. This makes it extremely difficult to measure if your tests pass or fail.

You might be thinking this: You don’t have actual test cases documented but your application is supposed to add 1 + 1 and you know that this equals 2, so why do you need a written test for this? You know that if you get a result other than 2, the system fails.

Yes, that was in fact a test—an undocumented test case.  You compared the results to a known, expected answer, albeit in your head, and then determined if the actual result matched your expected result.

The important thing to note about the example above is that the test case only resides in your head.  No one else’s.  What if you need someone else to execute this test? How do you know that person knows that the expected result is 2?  How do you know that person has the same expected result as you?  (This is starting to sound like a Dr. Seuss book with the rhyming sentences.)

The 1+1 problem is an easy example, but software testing is not typically that simple.  If it were, we would not need requirements, design, architects, developers, or testers.

So, written tests are needed because the person(s) executing the tests may not be as familiar with every nuance of the system.

That’s testing in a nutshell.  A VERY high-level explanation, at least.

So what is quality assurance or QA, you ask?

According to IEEE, QA is a planned and systematic pattern of all actions necessary to provide adequate confidence that an item or product conforms to technical requirements.

Per SEI’s CMMI, Process and Product Quality Assurance (PPQA) is responsible for providing staff and management with objective insight into the processes and associated work products.

The role of Quality Assurance is to audit compliance, improve processes, gather metrics, and communicate findings.  The QA role helps to implement quality throughout an organization by evaluating and improving processes and templates, typically using industry-recognized bodies of knowledge as baseline standards.  These standards can be from IEEE, SEI’s CMMI, ISO 9001, COBIT, or ITIL.  All of these standards offer different levels of guidance—some are higher level frameworks while others offer detailed templates for use.

Is there a policy, processes, and communicated templates in place for Requirements Management?  Are the team members adhering to the policy? To the Configuration Management process?  To the Change Management process?  How are these processes being monitored and/or audited for compliance?  Can these processes be improved and become more mature?

Quality Assurance is really about evaluating, monitoring, assessing, and improving an organization’s IT processes in order to help build quality checkpoints into the development process.  It’s not testing.

So why the misnomer?  Why do so many people misuse these terms?  That’s a good question.  Your task is to spread the word to the masses.  Get the HR Department to rename job roles so that testing positions have the word ‘test’ and not ‘QA’.  Go forth and conquer, I say!

Question for you: Does your organization properly use the terms “quality assurance,” “QA,” and “testing?”  Does your job title correspond to your function?

Let us know.  We want to hear from you!

This entry was posted in blog, featured, quality assurance, software testing. Bookmark the permalink.

8 Responses to “What…? You Say That QA and Testing Are Not the Same Thing…???”

  1. Dave Geller says:

    I too have often found myself in the situation of being the “Quality Assurance” lead on a project that was more ad hoc testing than anything else. At one recent employer I was literally the Director of Quality Assurance, but all I did was gorilla test off of a ticketing system. Real Quality Assurance is all of that and much, much more.

    A QA effort could be about documentation and organization, whether we are talking about creating a reusable set of test cases or picking up the slack of a project manager who is better suited to marketing tasks than keeping the train on the tracks during the technology phase of the project.

    Other times QA finds itself making significant contributions by working with the product group to define new requirements or to perform gap analysis on the ones they have (if they have any, that is.) We’re not only talking about the actual functionality, but also about how said functionality is going to be tested. That there is a credit card validation in the system under test is fairly straightforward from a functionality perspective, but how to test that in a non-production environment is not.

    Sometimes QA is thrust into the role of communicator. Pointing out the deficiencies in someone’s work can be delicate business, but there’s really more to it than that. Frequently, I find myself acting as a facilitator to translate between the technology and the business. Not only do they speak a different language, but they often approach things from very different perspectives, whether that’s because they have different goals or their brains are just wired differently.

    All told…QA is much more than just test execution.

    • vareynolds says:

      Dave, thanks so much for sharing your thoughts and your experiences about quality assurance and testing! It seems that your title didn’t reflect your role, similar to my story.

  2. Bill Roske says:

    It seems to me that BOTH titles are out of sync with what we are and/or should be doing. They both come from manufacturing. QA is the process police: If you just follow the process, you can’t go wrong. Testing (like Quality Control) is about inspecting the final product: report measurements and deviations. We are NOT software manufacturers. Little of what we do is governed by statistical sampling and analysis.

    Quality Assurance is a process monitoring set of activities. If you re-read the initial post, and replace QA with Project Management, it kind of works, don’t you think. Many of the tasks described fall into the realm of what we now call the Software Program (or Project) Manager. So clearly, Testing is NOT QA/PM.

    Testing, as described above, is the act of executing test cases… THAT’S what I don’t like about: the “Testing” label. It encourages “over-the-wall” thinking. Where do the test cases come from? “It ‘s not ready for testing, we are still writing code and trying to get it to build..” We all know that the sooner we find (or better yet, prevent) bugs, the cheaper it is to fix them. Well, if testing waits until the code is in place, what do we call the other activities we should be doing? You know; critically analyzing requirements (acceptance criteria, if you are of the agile ilk), assessing design for weaknesses, suggesting additions changes for testability, designing test scaffolding and programs (we don’t just manually test, do we?), developing and debugging test programs… This smells of an engineering process, not “just execute tests”.

    So, I have to agree with you: QA and Testing are NOT the same thing. Neither term is sufficient.

    • vareynolds says:

      Bill, thanks so much for commenting. Yes, I agree—so much of what we do extends beyond the traditional definition of the term “testing.” Yes, engineering! I’ve actually had the title “Lead Test Engineer” at one point, which sounds more applicable. What other word or phrase do you think would be a better fit to describe what we do?

  3. Bill Roske says:

    I have become an evangelist for the term Quality Engineering.

    My current department is called QED: Quality Engineering Development (a bit of a play on words from my days doing mathematical proofs: quod erat demonstrandum – what was to be demonstrated). The idea is; yes, we do testing. But that is only a small (hopefully automated) part of what we do. We partner with development teams, providing a different perspective on EVERYTHING they do, to help engineer quality into the product from the beginning.

    It creates a difference in mind-set in the whole organization. People think of testing as exercising the result of coding to find bugs. Quality Engineering reminds them that we are all about preventing bugs as much, and as early, as possible.

    • vareynolds says:

      Bill–I love the phrase “Quality Engineering Development”! It’s fantastic because it’s QED (nice!) as well as changing the mind-set of the rest of the organization in terms of what it is that we do. I’d love to help you evangelize the phrase.

  4. Stan Wocial says:

    Great post. My present company seems to be an exception in that testing and quality management are treated as different and with job titles to match. I have implemented an ISO 13485 (based on ISO 9001) quality management system for a medical device software company. Your description of QA fits my role very well. Unfortunately it feels like a very niche role a the market which seems to fall into two groups: one set are IT companies that equate QA with testing only and the other set are manufacturers which rightly expect QA to have hardware manufacturing knowledge and experience. Genuine software QA feels like a solution looking for someone who recognises the problem.

    • vareynolds says:

      Thanks, Stan! Great job in implementing ISO 13485. It sounds like you are working on a critical application that I’m assuming must be compliant with regulatory standards within the medical field. In my opinion, I think it’s crucial to have true QA as as well as true testing to ensure the quality of the overall processes as well as the product. There are different levels of formality that I think are needed, depending on the product (rockets and medical devices needing more formal processes while non-mission critical products need less). It’s good to know that there are organizations out there that do, in fact, understand the differences and value in having both. Unfortunately, there is a lot of software or products that do not need to meet any regulatory standards, therefore, there is no real push from the top-down in these organizations to voluntarily add quality management —what they see as “overhead’ to their already costly budgets.

      I think it really comes down to having a proactive mindset verses a reactive mindset. If the leaders in an organization are proactive, then the organization will likely be quality-driven and budgets will be approved for planning and upfront preventative costs. If the leaders are reactive, then the organization is fine with the status quo and would rather spend money later to fix things, with the mindset that the “fixing later” is not a guaranteed cost because it might be needed or might not be needed. Thus, saving money, in the reactive leader’s mindset. What I have a difficult time understanding is how these reactive leaders choose not to see, ignore, or just don’t notice the recurring pattern and cost to the products that are consistently delivered late, defective, and having poor quality.

      Please subscribe to our RSS and comment on our next post! We’d love to hear your thoughts!

Leave a Reply

Your email address will not be published. Required fields are marked *


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>