Rapid Testing Intensive Report… Report
We recently ran the first Rapid Testing Intensive on Orcas Island. One of the outcomes of this, as I have promised, is a public test report on what happened. This is my report report. I am reporting on my reporting…
What was the “Rapid Testing Intensive”?
Oh, it was just the most realistic and ambitious test training event ever. Sorry if you missed it. We (17 testers onsite and nearly ninety online) tested a part of eBay for four solid days, finding many important bugs. But it was more about training and coaching than bug finding.
Where am I with this report?
I am actually creating several reports. One is a report about our process. This is meant for people curious about Rapid Testing methodology, and in it I will contrast what we did with the “traditional” approach. I will also create a report for our primary client, eBay, whose software we tested.
There’s a lot to do and I am plugging away. I just finished the usability testing roll-up report. I’m going through 320+ bug reports now. Here’s my overall todo list:
Analyze HipChat traffic
- who was talking most?
- who was being responded to most?
- when were people talking most?
- were conversations more social or technical?
- what proportion of the conversation were people seeking information, and what proportion were people providing information?
- was HipChat helpful in coordinating a worldwide testing effort?
Analyze bug list
- Assess bug report quality
- Reproduce bugs
- Close duplicates and non-bugs
- Add labels and categorize according to risk
- What did eBay know about before we started?
- What does eBay consider important that we found?
Produce “official” test coverage outline
- Analyze student TCOs
Produce “official” Risk List
Produce official sequence of events
Describe our test activities
- Analyze student test session reports
Describe what we did not test
Analyze tools used
Identify outstanding student work
Write report narrative and summary
How “Rapid” is this?
Yeah, this seems like a lot of work and yes it will take a while (I’m working evenings on it while teaching during the day). But this is because it’s a seminar and demonstration project. In a real project, our reporting (other than the bug list itself) is either oral, or expressed in brief written statements. On a normal project, once we get organized (after a few days if it’s a new project) reporting gets pretty easy.
August 6th, 2012 at 2:52 am
I am happy I was able to be a part of this, although only online and with limited time due to some personal “issues”. The “issues” part is quoted because they are happy issues
.
.
So, besides what I learned from the seminar, I also learned that I have to isolate myself from “distractions” while working.
I though I did this by taking vacation from my work for this seminar, it seems this was not enough, so perhaps next time I will be present on site
Anyway, I am satisfied (:P) with what I learned from this, I know I will do a better job from now on
.
If a review of this seminar would help you I am happy to provide my feedback.
Keep up the good work,
Victor
September 12th, 2012 at 1:05 pm
James,
Thanks for the update
An idea for the next RTI might be to include (part of) the reporting in the intensive program? That might decrease the amount of work it takes you to do it all.
It was fun to be part of it, although only on-line.
Thanks,
Ray
November 12th, 2012 at 1:00 am
Am sorry i wasn’t a part of this seminar. But i am very interested in the reports and outcome of the test; i would also like to know when future seminars would be happening so i can be a part of it.
Keep up the good work.
January 24th, 2013 at 4:21 am
Hey James,
I would love to see the Report if it’s finished
Chris