Risk management and testing

Risk aversion, that is the name of the game. That is what we do in QA, we help determine risks and work through some mitigation or treatment.

Throughout a project there are different views of risk and the ones that get the biggest notice is for the overall project itself. What is the risk to the organization or customers? What are the financial impacts? The ROI? There are plenty of risks identified and decisions are made.

Now the project has started and things move along until, bam, something happens and a bottleneck or worse a complete stop happens and the project is in danger. Happens a lot. So why? Why is it that with all the risks that are assessed at the beginning of a project that issues during development seem to have been overlooked or improperly assessed?

From my experience most of these issues occur either just before or during the execution of any sort of testing. The environment was not set up right. Test data not there. Partners not ready to test. These are just a few reasons that I have come across where things go bad.

Is it something that someone in QA did not foresee? No, we all have identified risks before in any sort of planning. What I feel the issue is that nobody listened until it was too late. I have had a lot of conversations with senior management on why things are they way they are. Those are not fun conversations because it usually ends up that the risks were identified just not added to the overall risks or that it was too late to do anything. Normally the latter is the case.

So what can we do? Obviously the status quo is not going to work. Senior leadership needs to have a better understanding of the entire picture before decisions are made. Do they need to see all the details? No, they just need to understand that there are some potholes in the path they want to take and they can figure out how to navigate them. In Value of QA I go into detail how knowledge management can improve everything across an organization and how QA has a clear defined role in doing so.

QA: Quality Assurance: Trying to avoid issues from happening. There is no reason why risks to testing initiatives should be identified in a project’s risk assessment. Senior management makes assumptions that testing efforts will be easy once everything is built. Then they start getting anxious when it doesn’t. As a discipline we should be preparing them for those moments.

Now is identifying the risks all that is needed? No, there must be a plan. Well two actually: A mitigation plan and a treatment plan.

Mitigation plans should be relatively easy. What are you going to do to make sure this doesn’t happen? It takes a little forethought and in the end has a good chance of working out.

Where I see things need work is a treatment plan. I have rarely seen a treatment plan outside of what I do. In most of my conversations I have heard that when things go bad it is usually an all hands on deck scenario with meetings all over the place to point blame more then come up with a solution. There is no need for that. There is no reason a treatment plan can be decided upon from the start. What would a treatment plan do? Well first off it would reduce the noise as to who is to blame and should get right to the solution.

Having a plan A and B is great, yet having other scenarios thought of is better. I am not saying that every scenario needs to be thought out. There is not time for that. Going with the more probable situations is the best way then having a high level plan of attack for the improbable scenarios. Just a here are the steps we will follow to get us back on track.

Risk treatment seems to be something that in QA should get more focus. Using that and getting more involvement with stakeholders early will help overall with avoiding the big potholes, or at least having the material with you to keep things moving forward.


Building testing skills

In the over 25 years I have been in the QA discipline I have come across two types of discussions about testing skills.

1 – It is an art and it is inherently it is a trait within an individual that cannot be truly taught.

2 – It is a mindset that can be taught and flourished to get the right outcomes and succeed.

I stand with option 2. Anyone can be taught anything and have them understand what it is they need to do to have the right mindset. Much like developing code or writing requirements it is all about how much you enjoy doing it and the ease it is to understand what is being done.

I have come across individuals that just couldn’t get how to test software. They understood the processes and what was needed they couldn’t understand how to get the tasks done effectively. Does that mean they were bad testers? No, from my view was that it was not framed in a way for them to get it completely.

One of the things that I have a hard time with with the multitude of courses out there regarding QA and testing. Some of them are really good and provide the understanding of true QA and how testing is a small component of a very big thing. There are others that just focus on testing and work on just the execution, not really providing the reason as to why they are doing what they are doing.

I have even come across some colleges that provide courses in QA. The descriptions I have read on a couple don’t give the real indications of what QA actually means and solely focuses on testing. When I researched these course it was a few years ago. A couple of them now have been discontinued with those schools. I can see why, partly because they never really gave the value this discipline has.

I have stated before I would love to see an entire program on QA within a college. Why not? They have them for engineering and development. It only makes sense.

In the meantime, here is what I do to build the skills of anyone that wants to do QA and testing.

1 – Get them to ask questions? They don’t have to get into the full details of what is going on, yet they should ensure they are getting enough details to get the job done effectively. Understand how the code works. When I first started QA and Testing I went overboard on writing cases and doing work. Since I didn’t understand the flow it got out of hand and a lot of duplication of work.

2 – Think efficient. To many times I have seen inefficient was of testing work that took too long to get done. Line up the cases in a flow, be smart about recording results, don’t write cases that are big.

3 – Review, review, review. This is one of the most important things I stress. Everyone within the delivery team should all agree to what the expected outcome a test is or that there is enough detail in requirements/stories to get moving on development.

The interesting thing here is what I just talked about is nothing new. There is plenty of literature out there that says basically the same thing. So if that is the case why is it difficult to build the skill set? Who knows, maybe management lack of understanding, conflicting views of what it means within the discipline, or no time to get that knowledge.

I fell if, at the very least, the three base methods I provided will give guidance to search out that knowledge. Get what is needed to improve professionally and the organization as well.

For me personally I am trying to draw out plans for a proper college program for QA, and I mean true QA. Obviously there will be some components of how to test, where I want to really focus is on Quality Assurance with Software. I want to build the community and help show the value.

Coverage – Pros and Cons

Code coverage, the goal of companies try to attain in software development. To understand how much of the code is exercised and better understand how and what needs to be tested when a change occurs. This is a great thing to have in any software development that will help teams become as efficient as possible to get to a quality product for customers.

As this is something has extremely high value to an organization how could I put “Pros and Cons” in my blog title? You’re right the concept of coverage are highly valuable, when I talk about “Pros and Cons” it is more to do with what is needed to get done to get that understanding of coverage.

The reason I am bringing this up stems from a vendor cold call I got for third party testing services. In my position I get a lot of those calls, and for the most part I do not have an issue to start a conversation to see what they provide. As long as they are able to differentiate from all other third party vendors. In this one call there was one statement that came out that peaked my interests and I pushed back to get more details. That comment was “We can have 100% code coverage through automation.” Sounds too good to be true, right? Well in my experience it is. I have never seen any software shop that had 100% code coverage through automation. There are too many variables out there where to do the work for that statement to be true it would take a lot of work.

Before I continue on with the above comments lets go through what I see as the pros and cons.


  • Automation coverage can target specific components
  • No need for full regression testing with a change
  • Better estimation to changes, more accurate
  • Shortened test timeframes

There are more and there is plenty of literature out there that will go into details. Here I just wanted to list a few and foster some conversation, as always.


  • A lot of effort needed
  • Tedious
  • May not be completely accurate
  • May not have full understanding of code

Again the list of the cons can be a lot longer. The good news is the cons are issues that can be worked on and resolved within the company to get to what is needed. There is no technical reason or roadblock. It is about getting the work culture changed to tackle these cons.

There are software packages out there that will give you the code coverage an organization desires, Like all tools it is all about how the tool is used to get the most out of it.

Do I think an organization can have 100% coverage? Yes. In automation? Maybe. Technology is getting better and better over time that it is something that could be achieved. There are a lot of other variables that need to fall into place that will allow it to happen. Consistent coding practices, full knowledge sharing within the teams, and full cooperation between everyone within the organization to name a few.


Helping others in the discipline

This week I had the opportunity to talk with someone within the QA discipline and provide some career advice. We talked from helpful hints on how to get a foot in the door with the resume to a good discussion of QA as a whole.

This was the first instance where someone took up on my offer to have these types of discussions. It was a great conversation and fun to have this type of discussion with someone outside of someone who directly reports to me.

This individual had some great questions from what changes could be done with the resume, how to approach an interview and what could be done to improve knowledge within the industry. The conversation went well over an hour, and that time flew by.

I think these types of conversations are needed to help the community stick together and improve the value that everyone can bring. As I say in every one of my appearances in SPaM Cast “feel free to reach out to me to further the discussion or start a new one”.

Keeping it short this week. If there are any topics you would like to hear, read or discuss about let me know.

Being self reliant without depending on 3rd parties

Keeping with the recent theme about freedoms I want do discuss testing with 3rd parties,  we can even add different departments within an organization as well. In the past 20 years more and more applications and back end systems have required to be more integrated than ever. Most of this is due to basic business logic of identifying value and whether it is better to buy (or subscribe to service) or to build.

Prior to Y2K (I just twitched as I wrote that) most organizations built what they needed to succeed. Now time to market was fairly long and Agile was not as widespread as it is today so it made sense to build. Also different departments were rarely integrated. Then the internet came into play and things changed. People were getting access to stuff they never thought possible and wanted more. Exponentially the demand for information grew. Along with that the demand to do transactions online as well instead of going to a store. It snowballed to something quite amazing because it forced organizations to adjust to meet those demands and also become innovated to catch a bigger piece of the market and improve client satisfaction.

So here is where things started to get tricky, to build what was needed would take a lot of work and speed to market is everything. What started to happen was companies started to form to only focus on key components to sell to larger organizations. What this did was  make the “Buy vs Build” decision making a key process to help improve value. Why spend time building something when somebody else already did and we can get it at a fraction of the costs? Also we don’t have to support it. Sometimes it makes sense to do that.

This also happens within an organization as well. The best example I can come up with is the banking industry and the use of the internet. Online banking has become such a big part of most peoples lives they take it for granted the amount of work that went into a site that shows your account balance and do almost all transactions you can do at an actual branch and you can do it sitting at the dinning room table with a nice cup of coffee in hand. There are so many departments and divisions that need to work together it is amazing.

Wow that took a little longer than what I wanted, none the less if you were not part of the growing pains myself and a lot of the experienced (that sounds better than old) individuals were we can now move on to the topic at hand.

I have worked and seen other groups focus so much effort on working with other groups or organizations that they become obsessed with testing with them to get work done. Now I do agree that there is a need to ensure that integration between systems are tested it is a must. The question that really needs to be asked is; do we really need to wait for someone to exercise our code? My answer is no, and I am sure that most of you that do any sort of SOA or transaction testing would agree.

For me as long as the Format, Content, and Layout have been defined (doesn’t have to be fully defined at first it can be done in Agile or Iterative fashion) teams can test their code before integration starts. Why not? Does the code know where the information is coming from? No, it is not like asking someone to hand you something from the other end of a table. Code expects to see something in a specific location and in something that it is coded to understand and process. Here is an example:

System flow A

Here is a very high level view of sending information between systems. Now I know most people I talk with understand the simulation of data and sending transactions through, yet this concept is still foreign to others even with a lot of literature out there about it. In the end it is a very simple process to exercise all the code you can without the wait.

System flow B

Now teams are in more control of the data that comes and goes. It is hard to do and not very fun to get started. Once you do the quality of what comes out should improve. Development can start doing some quick integration tests to make sure nothing goes awry. If you look at it from a certification perspective where sometimes you have to pull use specific data to run through the system, now you can.

Taking control of what you do as a team to get code out is important, relying on others outside of your control could cause delays or even frustration if issues arise and now it turns into a blaming session as to who’s fault it is. Stuff like that will put the progress of the product to a complete stop.

I am sure that this blog entry is what some of you and your teams are doing. My hope is that for those that don’t can use this to foster conversations here or other areas to help them get to where they need to go.

Listen to my segments and others on SPaMCast http://spamcast.libsyn.com/

Better late than never

In my End of the year blog – 2017 , I mentioned I was going to do some changes to this site. Life and the holidays definitely got in the way. Regardless I have started to make the changes I want to improve my message going out and to make it a more enjoyable site for you.

Now I will say the changes will be iterative as I get back into the web development mode and blaze a new trail, actually get rid of the brush that covered the trail I went down all those years ago.

One of the changes you will see is the web address. Gone is the generic Word Press name and now it is QAcorner.blog. I am really excited about this as I hope to get more people to come in and join the conversation. As I write this, I have been told the link might be wonky for the next three days as it is getting set up. As a QA person I did do a quick check and it works, there might be intermittent outages.

The other cool thing that is happening now is I have actually put pen to paper, well fingers to keyboard, on my book. This is something I have been planning to do for the past three years, and now I am taking the leap to make it a reality. I intend on using conversations I have during my sections in the SPaMcast podcast, my blogs, my talks at conferences and reading others views. I have not given myself a deadline, yet, on when I will be done. My hope is that by the spring I will have a good amount of work done that I can have discussions with a publisher to get that ball rolling.

I cannot that you for your support as you read my views on issues, concerns and views of the QA. All I can ask is to pass the word and share your views with what I discuss. By all means disagree with me and show another side to what I wrote. I will never call myself and expert or a know it all. My main goal is to foster conversation so that we all improve this great and needed discipline.



Trust in your team without micromanaging

Sort of a follow up from Giving team freedom (within limits)

Trust is something that should not be handed out willy nilly. It is something that is very important and in the workplace is one of the components that improves employee satisfaction. There is plenty of literature out that that will talk about how to be a great leader within a team and most of them will touch on trusting those within the team in some shape or form.

Here is one of the issues that comes with leading teams, if you are new to the team it is difficult to let the team go ahead and continue to work without getting more involved than what you should be. It only makes sense, you will need to know how things work and see where improvements can happen. Now the real trick is, when to stop hovering.

I have yet to encounter someone that likes to be micromanaged. It is not a great feeling and tends to choke off potential innovation within the organization.  People like to be given direction with the allowance of letting them decide how to get there. This leads to the next tricky part, how much freedom do you give them?

One of my mantras is “Trust is a two way street, it is earn not given”. As a leader you have to let the team go and do things while letting them know that you will be paying attention from a far. They also need to understand that decisions made come from multiple sources and that they are not made just to make a decision. Teams should also have trust in what you  as a leader is doing. How do you build that trust? Ask for feedback, and not just at the end, when the decision is made. Go to the team and let them in on what is coming down and what direction you are thinking of going. Now keep in mind I do understand there might be times where, you as a leader cannot do that, I have been in that scenario a few times. In those scenarios I find it best to let the team know what is going on as soon as you are able to do so. Give them the “Why” things happened the way they did, “How” it all came about and “What” you are doing to help the team through it. This is the basic principle of Golden Circle, this is how you will help get that trust while you are also given them trust as well.