After reading the article, I could not differentiate btw the Thread based test management and Scrum activities.It sounds to me more like another name for scrum. Am I missing important details..?Plz let me know.

[James' Reply: Really? How much do you know about Scrum? Describe Scrum for me, please.]

[James' Reply: They aren't reviewing the actual testing, just the descriptions of the testing. Management generally doesn't perform such reviews, however. Or if they do, they generally do a poor job of it.]

Note that a similar approach can/was/is also used for development (at least in a traditional model).

Using SBTM I assume that leads/managers can be much more involved on an on-going basis in the actual testing strategy. Is that essential? If the testers are experienced, do the leads/managers get involved on an on-going basis?

Thanks,
- Nilanjan

[James' Reply: That's up to them. But the SBTM method that I created called for debriefings after every session.]

I really loved the idea. Moreover for me this post has come at a very appropriate time as I am involved in an agile project.
I have been doing things in bits and pieces around similar approach as my day today activity using either an xls or MPP.

At a high level these have a well detailed WBS with all micro level activities nested at each level. The dependencies and pre-requisites are clearly called out. This helps me manage my day today tasks more efficiently and leaves me well planned.

In this post, specially liked the clarity that I got from section “How Do Threads End?”
It makes it so objective.

Regards
Meeta

Some problems I struggle with, or have struggled with, are:
– How to explain to someone that there really is an overall thought process behind what might otherwise seem a little haphazard. (I don’t manage the testing process very tightly, half because I’m lazy and half because I really believe that if I try to manage it more tightly than I do, I will damage it somehow.)

[James' Reply: By weaving your threads into a fabric. Making sense of your threads, in other words. A key part of thread-based test management is having a coherent mental model that explains what threads are needed and how they relate to each other. That's why I have my Heuristic Test Strategy model. Threads must be woven into a compelling story of the product and the testing.]

– How to explain to an Agile organization that there is testing that can and should be done, that is nevertheless outside the scope of the dev-centric sprint heartbeat. (This is often entangled in the discussion of “what does ‘done’ really mean?” in that you try to do some testing in the sprint in order to allow people to call it “done”, but then everyone knows there’s more testing to be done after that.)

[James' Reply: You must explain that there is no particular reason why testing effort should mirror development effort. If you decide to switch to a new set of libraries for doing charts, that transition may take a few minutes or a few hours to carry out. But testing may require several days or more. Just because the programmer says "I'm done" doesn't mean we know the implications of that work.]

– How to manage an overall plan, in a way that doesn’t bury me in management-level artifacts (charts and tables and outlines and more charts and spreadsheets and aaaargh), but that gives me the ability to push back a little when Today’s (or This Week’s) Big Red Alert pops up and tries to steal people and resources that I’m using on a longer-term effort towards a longer-term payoff.

[James' Reply: I prefer to use a one-page test strategy, in mind-map form.]

–JMike

[James' Reply: Good point.]

Testing for a patch release is good example where TBTM view can be helpfull:

Bug Retest Thread:
- List of Bugs to retest, every bug retested represents mini-thread

Regression Thread:
- Regression tests you normally do plus extra regression based on the specific risks associated with the release are mini-threads here

Release Management Thread:
- If the group (like ours) is testing / coordinating installation packages and reviewing the release documentation lots of threads will happen here

Karol

I see SBTM and your thread example from two sides, one is from the tester perspective looking outside (how do I report to the test/project manager what I did) and the other is inside (how can I organise my testing so that I get the best coverage given the time and context of the project).

The thread approach now helps with the former with the question “Why does it take so long”? I think the thread approach helps the tester to organise their thoughts around the threads leading to better presentation of what has actually been done, not just the tiny part that people see as “testing”. This might help with PMs that are not that experienced with testing.

For the internal facing view of threads I can see that it helps me as a tester to be more organised in my thinking and play out the different approaches and heuristics that I can see useful to gain the best coverage in the context of the project.

Where I see problems is with the lack of estimating and, to a lesser degree, evaluation of progress. I’d argue that the thread approach is suited for projects not only with lots of interruptions but also with extreme time pressures where no in-depth testing can be carried out. I finished a project like that last week where I thought I’d use SBTM but ended up with a more thread based approach (even though I wasn’t thinking of it in this term) . It was both possible and useful to use the same session sheet for notes as for SBTM, however it was proportionally bigger.

[James' Reply: There is no lack of estimating or evaluation in TBTM. All I was saying is that I don't yet see how threads influence that process one way or another. You estimate and evaluate in whatever way you see fit, it's just done thread-by-thread.

For session-based management, on the other hand, there is a special approach we use for estimation and evaluation.]

Where I’d disagree is that a thread doesn’t imply a commitment.

Say you have a web application where you can report a broken streetlight. You can also track your case(s) that you submitted, filter by date and see them on a map and table. I could have three threads, one for submitting, one for tracking, one for viewing. Or even more if you add in multiple browsers/Oss. Each thread could be tested by a single tester.

[James' Reply: I might not track threads to that level of detail. But it's up to you.]

What each tester says by picking up the thread is that they’re “committing” to test that thread to the best of their ability in context of the project. They may opt to create individual session out of it if they have the time, or they might not. But the commitment is there either way. A thread could be by functional area as described or by browser.

[James' Reply: But you can also pick up a thread without committing to it. The mere existence of a thread does not imply that a commitment has been made to pursue it and tie it up.

You're right that part of TBTM is to decide who will work on which threads, and that probably means commitment. I'm just saying that commitment isn't built-into the thread itself, whereas in sessions there is commitment.]

In a similar project I opted for the “by browser” approach and changed functional areas multiple times as I learned something in one part that might have an impact on another area – that wouldn’t have easily been possible in SBTM.
From project managers point of view the how or the approach isn’t that important. What they care about is what information the tester can provide, if there are additional risks to the project or if we de-risked a part and what that part is. So the commitment might not be that a particular session is followed but the commitment is that the browser, functional area, performance, etc is investigated and information provided about what has been learned.

I really like the idea about knots and Jon’s description of splicing threads to sub-threads or recognising overarching threads. I can see that as helpful to gain a better understanding of the system when looking out for these possibilities.

Thanks for making me think.

Thomas

PS: Do you want to provide a link to Jon’s blog as well? It compliments this one very well.

[James' Reply: Good point. I'll add one.]

Thankyou.

I have been attempting to introduce something I feel is quite familiar that encompasses every task I would be expected to perform as part of a normal project. In my attempts to explain the meaning of my work to my manager, I identified this as similar to agile (in effect, a test sprint that runs as a project within the project), with milestones related to significant project dates – broken down in a very similar way to how you describe what TBTM looks like. This also includes status reports and a burndown chart of work. Thankyou again for explaining this in a way I can present and work towards much more easily.

Based on my hands on experience in this project, I’ll add my two cents.

You should add preparation phase for TTM method.

Preparation phase should contain:
1. Documentation Tread
Detailed system setup/preparation document, for example:
- Custom Simulators (include calibration process)
- Off the shelf equipment (verify calibration process)
- System setup

2. Test cycles
In current project i planned, as following:
- Sanity or Debugging cycle
- Test protocol design & Test protocol executing
- Formal Validation test cycle (at least one)

May be tomorrow i can add more.

[James' Reply: Okay, folks, Sagi was the test lead on a recent project where I was test architect. It was his status reporting each day and review of priorities each morning that got me thinking about threads.

I didn't intend to write out the real thread list from that project, man. I was just giving people a flavor.]

Threads are much like a ToDo list thing. I can feel it in my bones that it will be a whole lot more too. I just can’t yet put my finger on it. What I’d like to see is someone beating me to the punch in coming up with simple methods and documentation ideas around threads that make it similarly easy as SBT is today.

I am writing a test strategy (or some call it test plan…so a high level testing document) for quite a large project atm. SBT already has a firm place in it and I will use the concept of a thread. The remaining question is if I will go as far as Jon suggested and llist ALL threads or just those that really pertain to testing and project.

So, keen to see where this goes. I’ll be “on” it for the next 3 weeks. By that time I will have to have nailed down how my project will test. I’ll keep an eye out for more info and post stuff if I find something interesting.

Thanks heaps to Jon and James for frying their neurons over this for the greater good!

Oliver

I often model products in terms of their elements and quality criteria based on your Heuristic Test Strategy Model to come up with a high-level list of testing ideas. Collectively, these testing ideas form my test strategy and become the basis of one or more test sessions. Up until now I hadn’t thought of modelling all project related activities (threads) in that manner, thanks!

I can see TBTM and SBTM becoming complimentary approaches where you use TBTM to outline all high-level project activities and SBTM to actually do the work.

One of the things I like best about SBTM is that my sessions leave finger-prints of where I’ve been, and more importantly, what I tested in those sessions. If distracted for periods of time I can quickly review my sessions to determine where I left off last and what I was doing. If sessions weren’t used in TBTM due to time constraints, the fact threads have the potential to run for long periods of time, and the work is very dynamic in that you can drop threads, switch threads, and pick them up again – how would suggest testers evaluate and keep track of what testing they’ve done for a given thread? Notes scribbled in their pads? Comments in a Status Report? This is where SBTM becomes very powerful for me.

[James' Reply: I would recommend using SBTM where possible, and thread-based when it's just not feasible to stop the interruptions. For instance, on my recent project, we did two sessions on the first day of testing, but then we realized that straight-ahead testing had to be suspended, because what was more needed was for certain analysis and documentation tasks to be done first. Meanwhile, developers were coming in and out, asking questions, and I kept popping back to the product to do micro-sessions of 15 minutes or so, to answer those questions.

We found that doing a status report at the end of each day, and going over the mindmap of threads at the beginning of each day, helped us feel on track and allowed us to roughly gauge progress.]

This also validates my thinking that the normal open/closed/delivered/etc. model of normal bug reporting systems is a horrible way to manage testing charters.

[James' Reply: A session is like a bead on the thread. You might have many sessions on that thread.]