Lessons Learned
in Software Testing

A Context-Driven Approach

by Cem Kaner, James Bach & Bret Pettichord

Published by Wiley (2002)

286 pages

ISBN: 0-471-08112-4

~^US$27 from Amazon

Summary

Through a series of nearly 300 “lessons”, the authors share their accumulated wisdom about how to test application systems - not so much which buttons to press but more how to establish and manage a test team, plan the work and dynamically adjust the testing process according to what is found and how much time is left.

Scope

The chapter titles ably illustrate the book’s scope:

Role of the tester
Thinking like a tester
Testing techniques
Bug advocacy
Automating testing
Documenting testing
Interacting with programmers
Managing the testing project
Managing the testing group
Your career in software testing
Planning the testing strategy

[Appendix] The context driven approach to software testing.

I would definitely encourage anyone who thinks ‘test automation’ is a great idea and is perhaps contemplating the purchase and use of automation tools, to read chapter 5 before they commit the budget and finalize the project plans. The authors eloquently explain the advantages and disadvantages of common automation techniques such as user input replay tools, providing a real-world counter to the tool vendors’ optimistic sales pitches. They don’t say “Forget it”, rather “If you can live with these significant drawbacks, automated testing may be useful for a certain subset of testing activities”. This is a good example of the pragmatism and wisdom found throughout the book.

The book is not an academic treatise full of theoretical constructs/models and testing methodologies. Nor is it a step-by-step manual on how to test a system. It is an excellent read for testing practitioners who are seeking or at least open to advice on how to do their jobs more effectively and efficiently.

Audience

“This book is for anyone who test software, anyone who manages testers, and anyone who has to deal with testers in their software development projects. That includes project managers and executives.” The hints and tips plus career development advice are valuable for testers, especially if they have a few years testing under their belts already. The technical content is minimal and should be readily understood by any IT professional while the management advice should be appreciated by those with management experience or who aspire to become managers.

The book strongly encourages testers to work with developers and project managers, becoming an integral and valuable part of the team rather than an impediment to progress and a threat to delivery deadlines (lesson 12 is typically direct: “Never be the gatekeeper”!). The subtitle’s reference to being ‘context driven’ introduces a dynamic approach to testing, relating test activities to the development lifecycle and promoting those that will be of most help to the project at any point. The approach is described further in the appendix but is only subtly referenced elsewhere, unlike certain other books that insist on pushing their One Big Idea down the reader’s throat at every possible opportunity.

Authors

All three authors clearly have solid testing experience, some 60 work-years between them. There are also numerous (but not intrusive) citations to other helpful resources, further demonstrating the authors’ pedigree.

Cem Kaner, a consultant and IT professor at Florida Institute of Technology, was the lead author of Testing Computer Software (1993), one of this reviewer’s all time favorite IT books. Cem also practices law.

James Bach is the founder of a software testing and QA company with silicon valley experience.

Bret Pettichord is an independent consultant who edits the Software Testing Hotlist and founded the Austin Workshop on Test Automation.

Writing style

The “lessons” format leads to a somewhat disjointed flow in places although overall the book is well -structured. At times, successive lessons are directly contradictory, again emphasizing the need for readers to be both alert and open-minded. This is another example of being ‘context-driven’. Which lesson you choose to follow depends on the circumstances facing you, a form of contingency planning if you will.

Unusually for a published book, several critical comments from reviewers of the draft, as well as occasional differences of opinion or approach between the three authors, are included as footnotes or asides. The authors openly acknowledge the ambiguities and leave the reader to think about them and make the final decision - I like that. This is a book for grown-ups.

There are valiant attempts to describe and promote ‘the tester’s nose’, that seemingly innate ability of experienced and successful testers to sniff-out aspects of the system that are likely to harbor serious bugs and to design targeted tests that will reveal them. The advice on unstructured ‘guerilla testing’ is not quite so useful, in my opinion, but I’m impressed that the book even tackles such ephemeral concepts.

Utility & value

Even if you only learn something new from a few of the lessons, this book is well worth the purchase price. Testers relatively new to the profession will learn more than grey-beards but even they will probably find some of the suggestions make them re-think long-established ways of working (habits) and subconscious assumptions (prejudices). In the main, the lessons are pragmatic. Some are a bit contentious, perhaps deliberately, and most are both thought provoking and helpful.

Bottom line: recommended for any thinking person involved in application testing including development project managers and IT auditors.