Prompts for Posts

Questions from Kaner & Bach (2005) To Help You Interrogate Your Own (and Others') Research

Asking Questions

Here are some key contrasts:

• Hypothetical (what would happen if …) vs. behavioral (what have you done / what has happened in the past in response to …)
• Factual (factual answers can be proved true or false) vs. opinion (what is the author’s—or your– interpretation of these facts?)
• Historical (what happened already) vs. predictive (what the author—or you—expect to happen in the future under these conditions)
• Open (calls for an explanatory or descriptive answer; doesn’t reveal the answer in the question) vs. closed (calls for a specific true answer, often answerable yes or no)
• Context-dependent (the question is based on the specific details of the current situation) vs. context-free (the question is usable in a wide range of situations—it asks about the situation but was written independently of it)

slide 15 Black Box Software Testing Copyright © 2005 Cem Kaner & James Bach

More Questions

• Causal (Why did this happen? Why is the author saying that?)
• Ask for evidence (What proof is provided? Why should you believe this?)
• Evidentiary sufficiency (Is this conclusion adequately justified by these data?)
• Trustworthiness of the data (Were the data collection and analysis methods valid and reliable?)
• Critical awareness (What are the author’s assumptions? What are your assumptions in interpreting this?)
• Clarification (What does this mean? Is it restated elsewhere in a clearer way?)
• Comparison (How is this similar to that?) and Contrast (How is this different from that?)
• Implications (If X is true, does that mean that Y must also be true?)
• Affective (How does the author (or you) feel about that?)
• Relational (How does this concept, theme or idea relate to that one?)
• Problem-solving (How does this solve that problem, or help you solve it?
• Relevance (Why is this here? What place does it have in the message or package of information the author is trying to convey? If it is not obviously relevant, is it a distractor?)
• Author’s comprehension (Does the author understand this? Is the author writing in a way that suggests s/he is inventing a concept without having researched it?)
• Author credibility (What basis do you have for believing the author knows what s/he is talking about?)
• Author perspective / bias (What point of view is the author writing from? What benefit could the author gain from persuading you that X is true or desirable (or false, etc.)?)

slides 16 and 17 Black Box Software Testing Copyright © 2005 Cem Kaner & James Bach

More Questions

• Application (How can you apply what the author is saying? How does the author apply it?)
• Analysis (Can you (does the author) break down an argument or concept into smaller pieces?)
• Synthesis (Does the author (or can you) bring together several facts, ideas, concepts into a coherent larger concept or a pattern?) (More along these lines come from Bloom’s taxonomy…)

slide 18 Black Box Software Testing Copyright © 2005 Cem Kaner & James Bach

The Classic Context-Free Questions

Ask the traditional news reporters’ questions:

• Who?
• What?
• When?
• Where?
• How?
• Why?

For example, Who will use this feature?  What does this user want to do with it?  Who else will use it?  Why? Who will choose not to use it?  What do they lose?  What else does this user want to do in conjunction with this feature?  Who is not allowed to use this product or feature, why, and what security is in place to prevent them?

slide 19 Black Box Software Testing Copyright © 2005 Cem Kaner & James Bach

Using Context-Free Questions to Define A Problem

• Why is it necessary to solve the problem?
• What benefits will you receive by solving the problem?
• What is the unknown?
• What is it that you don’t yet understand?
• What is the information that you have?
• What is the source of this problem? (Specifications? Field experience? An individual stakeholder’s preference?)
• Who are the stakeholders?
• How does it relate to which stakeholders?
• What isn’t the problem?
• Is the information sufficient?  Or is it insufficient?  Or redundant?  Or contradictory?
• Should you draw a diagram of the problem?  A figure?

Based on: The CIA’s Phoenix Checklists (Thinkertoys, p. 140) and Bach’s Evaluation Strategies (Rapid Testing Course notes)

slide 20 Black Box Software Testing Copyright © 2005 Cem Kaner & James Bach

Using Context-Free Questions to Define A Problem

• Where are the boundaries of the problem?
• What product elements does it apply to?
• How does this problem relate to the quality criteria?
• Can you separate the various parts of the problem? Can you write them down? What are the relationships of the parts of the problem?
• What are the constants (things that can’t be changed) of the problem?
• What are your critical assumptions about this problem?
• Have you seen this problem before?
• Have you seen this problem in a slightly different form?
• Do you know a related problem? 
• Can you think of a familiar problem having the same or a similar unknown?
• Suppose you find a problem related to yours that has already been solved.  Can you use it?  Can you use its method?
• Can you restate your problem? How many different ways can you restate it? More general? More specific? Can the rules be changed?
• What are the best, worst, and most probable cases you can imagine?

slide 21 Black Box Software Testing Copyright © 2005 Cem Kaner & James Bach

Source: 

Kaner, C., & Bach, J. (2005). Specification-Based Testing [Workshop Slides]. Melbourne, FL: Black Box Software Testing.