Value of QA

Back to my very first blog, Root Cause Analysis thoughts, I brought up my value diagram for QA.

QA Value

This Porter competitive advantage graph is how I truly see how QA should, not only use to promote the discipline to stakeholders, it should also be used within our very discipline.

Since I achieved my MBA years ago I have viewed our discipline in a different set of eyes. Speaking about the value QA will provide and remove the stereotypes that people have regarding it. I have lost count on how many times I need to correct people when discussing my career.

“What do you do?”

“I am a Director of QA”


“Quality Assurance”

“Oh, you test things?”

Then I would go into a long conversation of what it is I actually do. Now don’t get me wrong I do talk about how I am responsible for “Testing”, it is just I go further down the rabbit hole in what that all involves.

Now when this conversation started about early in my career I would have just agreed that I “test things” and left it at that. To say that now, with what I have learned and practiced over the years,  lowers the value of what it is that is actually done.

As a discipline and community I feel that are some areas that don’t help the cause to better legitimize what it is that we do.  Just as there are great books, speakers, articles and classes on the subject matter there are more that lowers peoples views of what it really means to be QA.

Just recently I did a search for courses on QA and Test planning for some peers that were looking for something to do outside of going to a physical class. A few were based on certification standards,  my views on that Thoughts on certification, while other were just sites that offered a course or a series of courses. Viewing some of the overviews I provided some recommendations and sent them off.

Do say I was disappointed with the feedback coming back on what they went through is an understatement. The courses did not even go into QA, they were focused on Quality Control. Physical testing of software not trying to avoid issues early. I get it, it is something that we do within the discipline. What gets me upset is they talked about learning everything they need to know about QA. In the overview it looked like they were going to go over some of that, unfortunately it was wrong.

I have asked this for years is “why can’t there be a diploma program on software QA?”. There are ones for software engineering and development. To help with that I am trying to get into post secondary institutions to see what can be done to get that going or to enhance what they already have. I feel getting something a little more solid than a printed piece of paper with some third party logo in the corner for attending a 3 or 5 day course would help legitimize the QA discipline more.

We as a community have to not only show the value of QA to those on the outside, we need to also keep up the value realization within ourselves. Help each other build and grow.



Last month I had the privilege to share my ideas at the QAI QUEST conference. It is something that you all know I enjoy doing and even had the opportunity to fill in for another presenter, who unfortunately fell ill and was not able to present.

I love attending conferences it is a great situation to not only meet others within your own community, it is also where everybody has the ability to take ideas and lessons learned by others into their organizations. The networking that goes on is unbelievable and understanding how others see and approach QA.  Every year I have been at QUEST I catch up with people that remember something I spoke about years ago and mention that some of my ideas have been introduced in their teams successfully. Now I did have some not so successful stories come in, and it was great to discuss it and see what could have been done differently.

Not all ideas will work for all organizations and that is one of the key things that anyone that goes to them should understand. When attending courses, workshops or track sessions it must be viewed with an open mind while keeping in mind how their own organizations work. There are limitations that could stop anything from happening and having the right way to frame ideas to work is key. To go in there to take someone’s idea and directly try to apply it right away will have a high probability of failure.

In my talk at QUEST 2017 about Root Cause Analysis; one of the things I brought up was that when initially trying what I was discussing was going to be hard at first and that people would need to be persistent in getting the right information. I didn’t want to sugar coat it or give anyone false hope that what I was presenting was going to be the silver bullet that everyone wants. In the end it was a well received discussion where people came up to me afterwards to discuss what they were going through and asked for ideas on how to implement points to help them out.

I also had the opportunity to have one of the senior development leaders from my place of employment come with me. It was a good learning experience for him to see the different aspects of QA and how what I am doing with the organization’s QA team fits into what the discipline is doing. It provided a better appreciation for what is done and how closely Development and QA are tied and how close the two are tied to efficiency in development of a product.

I think this is something that should be done with different discipline’s conferences. The opportunity to see how the “other side” works would be such a great benefit It is a great opportunity for leaders in QA Development, BA and Project Managers to fully step into the others shoes for few days and gain heightened awareness of impacts they have with each other. It will also provide opportunities to find areas of improvement and effectively work together.

I look forward to future conferences, as a speaker or attendant, because each time the amount of knowledge sharing is at a different level because instead of just reading about something I will have the opportunity to foster discussion.

UAT in Agile

In my previous discussion I talked about Product owners being involved in the testing process. In some way shape or form they are, whether they realize it or not. So with the current topic it is a perfect segue: UAT in Agile testing. Depending who you talk to or read there are different opinions, the yes and no side.

Yes side:

With the user involved within the team it makes sense for them to play with what is being changed so they can provide the feedback sooner.

No side:

Takes time and the feedback loop occurs during the demo at the end of the sprint for future planning.

There are a lot more detailed arguments for both sides and I don’t want to delve too deep into each before I discuss my thoughts on the topic. If you are at the QUEST 2017 conference in Chicago maybe, we can meet up for a discussion over some beverages.

So here it is, what I think: I am on the yes and no side. Makes sense? Probably not, let me walk you through my logic and hopefully it makes sense in the end.

If we look at it from a pure Static Testing view it is most definitely a yes. Reviews and assurances that the stories have sufficient detail or will have enough detail as the sprint moves along is critical to have what you want in the end. At the beginning it is a vague idea of what is wanted:

“As a (someone), I want (something), so that (there is a benefit)”

That at the beginning is pretty vague, although what happens is ideas start to flow and things get into motion. When that happens the vagueness lifts and more details are needed to ensure the endpoint. The key to all this is the fact that although there is no physical program to use in the early stages the fact that the feedback loop early is a form of testing that the user has a lot of involvement.

So the code is done and being test within the team, what does the user do now? Aside from planning what is needed in the next few sprints why can’t they start to look at what is being developed early? The caveat with this is that they need to be aware that it is early and it may not exactly work as planned due to multiple reasons like environment, data availability, or general issues. From there they can get that sneak peek that may help clear that picture up early.

In the end is all about the planning and timing of what can be done and by whom. Regardless of what methodology is followed planning is needed, formal or informal, and as long as that plan allows for flexibility great things can be easily achieved.

Now on to the No side. Sometimes the timing just doesn’t allow it and the demo is the only time users would see what was developed during the sprint and provide the feedback needed. Or there is the preparation for future sprints that take up the time available to look at something early. These scenarios fall within scheduling and planning. Depending on how the team or organization operates it might not be suitable for a true form of UAT to occur. Now they could still do the Static form of testing and that is usually what will occur to help improve the quality, there just won’t be any Dynamic testing.

The other reason it may not occur during a sprint could be that the changes are strictly back end or architecture in nature and there would be really nothing for the user to see or do. They may play with the front end to see nothing has changed yet that would be a quick set of activities.

Something that could occur is taking a sprint for full on UAT when everything is completed, very counter intuitive to what Agile is. That one clean cycle that shows that everything that was requested is there and working as expected. Usually it would be a Beta test in the end.

It is all about planning, comfort level and maturity of a development group that would determine in the end if or when UAT occurs.


A lot of people like surprises: parties, finding that 20 dollar bill in your jacket you forgot about, or finding out some really good news. Yet there are some that don’t like getting scared at a haunted house at the amusement park or getting a flat tire. We live in a world where there is always something that will give you that quick initial reaction of joy or fear. In testing, at any level there are surprises on a daily basis, and for the most part they don’t end up being any fun.

No matter how much planning is done from the start there will always be that one or two things that could cause things to go off the rails. Risk analysis and mitigation is something everyone should be doing when going through any initiative, regardless of the methodology use. The old adage “hope for the best, plan for the worst” is something that I use at anytime I am thinking about how to go about things. Now there are limits to how much someone should plan for the worst. Like if I plan a family trip I don’t plan for earthquakes because they are very rare. Just like in a software delivery project, you don’t plan for a potential flood or power outage the probability for those to happen are low, one would hope.

So the best thing anyone can do is plan for the most probable things to happen with a few not so probable. That is the best thing one can do and the use of a risk analysis process is something that would help identify what issues could come up and how they will be mitigated or handled if they occur.

The best two stories that I have for surprises where at two different stages in my career and helped me understand the best way to handle them. One was from a previous post about taking ownership of your actions. A simple keystroke can do a lot of damage :).

The other story is when I was working on a project with a third party where information needed to me passed back and forth. During a two month execution process of sending files we were about to consider the project closed and ready for deployment. I think you can see where this is going to to. Well about a week before we were going to deploy the third party decided to tell us that the information that was being sent was incorrect and was always incorrect. This not only put the project in jeopardy it potentially put an entire release, with about 4 other projects, at risk of not going in. After some discussions and investigation it was found that a requirement was not detailed enough, yet both sides signed off on the expectations. Luckily it was a quick fix and a full day of validation and risk assessments to ensure the release was ok to go.

Now that surprise wasn’t one of the fun ones and could have been avoided. Asking the right questions, pushing back on requirements, or getting consensus on every detail.  Regardless in the end it is how someone deals with a surprise that will determine success. A person can be one of the ones that can be seen on Youtube completely freaking out over everything of it could be a person that is calm and works through the issue to get things right. It may not have to be a perfect plan on the spot, as long as something is thought through and collaborated with others within the team to get things done.

Product owner in the testing process

Product owners are great to have within a team. They spend time understanding the needs of the product from a client side and disseminate that information to the development teams. With that knowledge, should they be involved in the testing process? I would say yes. In fact I find that they have to be ingrained in the process from the start to have good visibility of what is going on.

Static testing is one of the key things they would need to be cognizant of. They need to have the assurance of the information that they are passing of is detailed enough to get the work done. Now in an Agile environment that level of detail may not be there from the start to get the exact product change they are looking for, yet they should have enough information to allow the team to see the path that they must go down.

Additional understandings of the process, such as issues found, changes to stories and subsequent impacts help the overall process. Clear paths of bidirectional communication allows for effective risk analysis and decision making to ensure the product is what is needed by the clients.

Now onto dynamic testing. Why shouldn’t the Product Owner be involved in doing some testing of the product? In the end they should have some idea of what the product looks like and how it processes the information according to client needs. Now does that mean they test each sprint or during the beginning stages of the test cycle? I don’t think so. There are potentials to find too many issues that could raise unwarranted alarms. Let the people running the tests get those out of the way and come back to the product owners for clarification or advice on what was found. In the end they should be a part of some form of UAT type of cycle prior to completion. Let them get their hands a little dirty using all the information they gathered throughout the process to see if it is what they want.

In the end it is their product and they should have an idea of what it does and how the clients will feel when they start using it after it goes into production. They may have an unusual workflow that they found out a client does and will need to see if it is run through. Whether it is the test team themselves or they try it to find potential issues. This level of understanding along with strong communications will help with up front quality improvement along with closing any gaps with testing.

Thoughts on certification

Doing a Google search on QA certification you will come up with over a million results with the first page having a fair amount of individual certification programs. To me I think that is a bit much. There are the big ones from the Quality Assurance Institute and International Software Testing Qualifications Board to smaller College certificates of private training facilities. Going through each one has the same general premise yet what is being asked for to get the certification seems to be consistent across.

Now don’t get me wrong, I appreciate it when someone takes the time and energy to learn the craft and prove they have the general knowledge of the discipline. What I have issues with is what I find most people do after the certification is achieved. From QA to PM I have seen people come into my career with different certifications yet do not follow the general premises that they learned while getting it. To me what I see is that they are willing to spend the hours to cram for the exam, pass it and collect the prize at the end thinking this is what will get them that sweet job they are looking for. Yet once they get it they do what they want and not help with any improvements for the organization.

Yes this view is a general and does not apply to all those that use what they learned in practice. In discussions with others, not only in the QA discipline, the general feel is that the “certification” is watered down. There is a term in the martial arts world called McDojo where belts are handed out almost at will as long as the person paid membership dues and testing fees. That is not to say all martial arts schools are like that, much like not all certifications are like that as well, it is the fact that there are easy paths to achieve somethings that shouldn’t be so simple.

I have interviewed over a 100 QA analysts, coordinators, and managers over the years and I do look to see if they have a certification, yet it is not a deciding factor on whether they will move forward in the process. I formulate my questions to see what type of understanding the individual has with QA and software development. I also look for the passion of what value QA has for an organization aside from executing testing. To me that is a big part of what I look for, second to experience.

To help I think the QA thought leaders within the discipline should look to post secondary establishments to enhance or start a potential diploma program. No different then electricians or construction, there is a specific skills that QA analysts need to know, understand and know how to apply it.  Most of what is being done is “Learn as you go” , which is great yet there are gaps that can happen.

I think it is time to get more focus on help the discipline show the true value we can provide.

QA Value