Chat Room Log

eWorkshop Software Inspections and Pair Programming:
Log started at 12/16/03 15:30 MET

Christian | has entered the room.
1 Christian: test
Christian | has left the room.
Christian | has entered the room.
Christian | has left the room.
tran | has entered the room.
2 tran: truong - testing testing
tran | has left the room.
| 12/16/2003 3:45:43 PM | test | has entered the room.
test | has left the room.
test | has entered the room.
test | has left the room.
Marcus | has entered the room.
Marcus | has left the room.
Marcus | has entered the room.
Marcus | has left the room.
Tom_Gilb | has entered the room.
1 Tom_Gilb: OK I am just testing this from another location (London)
Moderator | has entered the room.
Marcus | has entered the room.
Marcus | has left the room.
2 Tom_Gilb: Mikael, are you monitoring this? Tom
Christian_Denger | has entered the room.
Christian_Denger | has left the room.
Christian_Denger | has entered the room.
3 Christian_Denger: /homepage http://www.iese.fhg.de/staff/denger
3 Moderator: 2 Hi Tom, yes I will
Christian_Denger | has left the room.
Christian_Denger | has entered the room.
Marcus | has entered the room.
Tom_Gilb | has entered the room.
lee | has entered the room.
lee | has left the room.
Marcus | has left the room.
Krebs | has entered the room.
4 Moderator: Hello everybody and welcome!
Marcus | has entered the room.
Marcus | has left the room.
5 Krebs: hi! /nick Bill
6 Krebs: /nick Bill
7 Bill: /nick Bill Krebs
8 Moderator: 5 Hi Bill!
Marcus | has entered the room.
9 Bill Krebs: hello!
Marcus | has left the room.
Arisholm | has entered the room.
10 Tom_Gilb: hi bill, pleased to meet you. I am downloading paper -20 right now
Dieter_Rombach | has entered the room.
11 Dieter_Rombach: /homepage http://wwwagse.informatik.uni-kl.de/staff/rombach
11 Bill Krebs: 10 Excellant. You may also want to look at 18
Arisholm | has left the room.
12 Bill Krebs: 11 TR-2003-18 is the words and theory, and -20 is the detailed how to
Arisholm | has entered the room.
13 Moderator: We're preparing the control room here in Germany and are ready to start in 10 minutes.
14 Bill Krebs: 10 Nice to meet you to. If we were in person I'd love to get some autographs
15 Dieter_Rombach: Hi to everyone. How is everyone?
16 Bill Krebs: 15 Ganz gut
Karl_Wiegers | has entered the room.
17 Arisholm: Very good, thanks!
18 Moderator: Hi Karl, welcome to the eWorkshop!
19 Karl_Wiegers: Good morning from Oregon, everyone.
20 Tom_Gilb: hei på deg erik vi må treffes i norge en gang! er du interessert i evolusjonære utviklingsmetoder?
21 Tom_Gilb: hello Karl good to be with you
Frank_Maurer | has entered the room.
22 Arisholm: 20 Jada, jeg har gjort noen små undersøkelser!
23 Bill Krebs: 20 Ska vi talar Englska eller Svenska?
24 Moderator: vi kan ta svenska
Laurie_Williams | has entered the room.
25 Arisholm: 20 norsk??
rumpe | has entered the room.
Ralf_Kalmar | has entered the room.
26 Karl_Wiegers: Hey, you guys, some of us less educated types only know one language!
27 Tom_Gilb: skandinaviska?
28 rumpe: Hello.
29 Bill Krebs: 26

manzo | has entered the room.
Basili | has entered the room.
30 manzo: Hello
tran | has entered the room.
Costa | has entered the room.
31 Laurie_Williams: Good morning.
Scott_Ambler | has entered the room.
32 Scott_Ambler: howdy howdy
Costa | has left the room.
33 tran: good morning all
34 Tom_Gilb: Good afternoon, it is getting dark in London and I assume is darker in Norway
35 Scott_Ambler: 26+
barry | has entered the room.
Patricia_Costa | has entered the room.
36 Patricia_Costa: Hello!
37 Basili: Hello
38 Arisholm: 34 yes, it sure is dark. No snow

39 Karl_Wiegers: 34: Strange, it's dark in Oregon, also. I wonder if it's light anywhere?
40 Moderator: Hello everybody! We're soon ready to start
41 Tom_Gilb: OK, Just this once we won't do the workshop in Viking language!
42 Frank_Maurer: Hi everybody
43 barry: Hi everyone
Filippo_Lanubile | has entered the room.
44 Filippo_Lanubile: hi everyone
45 Tom_Gilb: Hi Barry, welcome to California (my home state)
46 Moderator: Welcome everybody, the meeting is open!
47 Dieter_Rombach: Welcome to the eWorkshop organized by the competence networks ViSEK (Germany) and CeBASE (USA).
Keun | has entered the room.
48 Dieter_Rombach: The goal of ViSEK/CeBASE is to evaluate software technologies in order to understand their strengths and weaknesses, and under what conditions they deliver best results.
Sharman | has entered the room.
49 Dieter_Rombach: Looking at Code Inspections (CI) and Pair Programming (PP), at a first glance, they seem to have the same common aim of collaboration among developers/reviewers to develop quality software with minimal defects. But one takes place during developing software and the other in reviewing already existing software products.
50 Keun: Hello everyone..
Sharman | has left the room.
51 Dieter_Rombach: The goals of this eWorkshop are to 1) elicit commonalities/differences and relative strengths/weaknesses are, 2) under what conditions you might prefer one or the other, and 3) under what conditions you might merge the two practices and to what gain.
Russo | has entered the room.
Gunjan_Sharman | has entered the room.
52 Tom_Gilb: what about non code inspections such as requirements and design inspection that cause 50% of the bugs?
53 Dieter_Rombach: Let’s start with the first topic: What are Commonalities/Differences and Strengths/Weaknesses of CI and PP? From the pre-meeting feedback (PMF) it seems like everyone agrees that:
lanubile | has entered the room.
johnson | has entered the room.
54 Scott_Ambler: 52+
55 Gunjan_Sharman: Hello everyone
56 Dieter_Rombach: PP is less formal than code inspections, and that PP lacks a third party perspective. Further, there is no documentation of defects etc. in PP. Inspections are more objective (because of the third party perspective). With PP, we can achieve a certain quality level but it is very hard to go beyond this level. With CI it is harder to reach this level, but it is possible to go beyond it.
57 manzo: Early in my career I was an eager practitioner of inspections, through years of experience in applying them, I came to conclude that they work much better in theory than in practice. I favor PP
58 Basili: 52+
59 Dieter_Rombach: Do you agree on this summary?
60 Scott_Ambler: 56: Where is the evidence of this?
61 Laurie_Williams: 57+
62 Dieter_Rombach: 52: Currently, we focus on code inspections. We will address other kinds of inspections later
63 Scott_Ambler: 57+
64 Bill Krebs: 56 There is some formality to pairing in that there is a set 'algorithm' for doing it per Laurie's book. Also, we rotate pairs, iterate, and refactor to address the risk that the first two folks will miss a bug
65 Karl_Wiegers: 52 + 59 I agree, but we should perhaps consider that there are multiple types of "peer reviews" besides inspection. Can we agree on terminology before we get too far along? Are we considering that any peer review besides PP is an inspection? I don't think this is the case --
66 Dieter_Rombach: 60: This is summary of the pre-meeting; we are looking for evidence from the participants now.
Winsor_Brown | has entered the room.
67 Basili: 57 What are the problems you have had with inspections
68 Scott_Ambler: Whiteboard: I don't think we can agree to that yet, other than perhaps the third point
69 lanubile: inspection IS A type of peer review
70 Bill Krebs: 57+
alberto_sillitti | has entered the room.
71 Christian_Denger: 68: What are the issues you cannot agree with?
72 Tom_Gilb: I now beleive that Inspection should be used to sample and measure, not to try to clean up. The measurement of major defects is used to decide process exit, and to motivate people to follow the standards used to judge the specification
73 Arisholm: If you want to compare these two techniques I think it is important to determine what type of "pair" we are considering and what type of inspector. For example, it may not be so useful to use a junior navigator, etc...
74 rumpe: PP focusses on quality of result, learning by doing, and construction of an artefact. Inspection on quality only. How to incorporate that into the comparison?
75 Karl_Wiegers: 69 agreed; thre are other types too, like what I call a peer deskcheck -- 1 other person reviewing someone's work on their own. Also walkthroughs, and team reviews less formal than Fagan or Gilb/Graham inspection.
monvorath | has entered the room.
76 Laurie_Williams: 57 When I was in industry, I found that people did not prepare as well as they should prior to an inspection. The inspection seemed more of a technicality -- something that needed to be "checked off." I'm sure our results wouldn't have matched most of the research studies.
77 Basili: 74 Don't you learn how to better construct artifacts from inspection or review?
78 Tom_Gilb: My position is that they are two different and complimentary techniques. We need to understand their costs and benefits quantitatively, and their best bractice modes.
79 lanubile: 75 inspections were proposed because walkthroughs were considered not enough disciplined
80 Bill Krebs: 75 irregardless of the style I suspect we agree pairing and or inspections alway win over solo.
81 Christian_Denger: 57: Can you think of any reason why inspections don't work in practice? What are your experiences there?
82 Karl_Wiegers: 77 You can learn how to do better work any time you look over someone else's shoulder, via PP or peer review.
83 Scott_Ambler: 76: I've seen a lot of that too, although it doesn't have to be the case. However, if it is happening you really need to step back and ask yourself why you're doing the reviews to begin with? To keep bureaucrats happy? If so, drop the reviews.
84 Christian_Denger: 78+
85 Arisholm: 78+
86 Tom_Gilb: Inspections work in practice if done properly, most people fail to use them properly ( example respecting optimum checking speed).
87 Basili: 76 I agree but some organizations require to see review times before having the inspection and I htink this is an important part. You can do inspectinos badly as you can do PP badly
88 rumpe: 77 Partially true, but the learing effect is better when discussing decisions when made instead of seeing results
89 Karl_Wiegers: 83 I've never worked anywhere that people did reviews just to say they did them. Most people have a sincere interest in making work products better.
90 Bill Krebs: 76 It's possible to do an imperfect job of either method and also to have good results with either.
91 Dieter_Rombach: There seems to be a difference between formal and informal inspections. Is the original hypothesis at least true for comparing PP with formal inspections?
92 Bill Krebs: 87+
93 Scott_Ambler: 80: co=operative development seems to work really well for most people, although for a minority of folks it doesn't seem to
94 Basili: 78+
95 Frank_Maurer: 81: I rarely talked to a developer who was leen on doing an iinspection. I met many developers who love pair programming. So, why not simply accept these preferences
96 Karl_Wiegers: 78 +
97 Laurie_Williams: 78 Can we outline how we can go about doing that?
98 Scott_Ambler: whiteboard: is't the code the documentation of the results?
99 barry: Outside of defect reduction, most data so far indicate that pair programming is also a way to trade extra cost for reduced schedule, which is often valuable in itself.
Steve_McConnell | has entered the room.
100 Bill Krebs: 95 or start with paing and Formally Inspect key artifacts
101 johnson: 83: - I have worked for a major electronics manufacturer where developers did exactly that: reviews only in order to say they did them (i.e. satisfied the process.)
102 rumpe: How about using PP in general and inspection in critical cases (complex modules, high quality necessary etc.)
103 manzo: Also, I’ve found Inspections to be too inefficient (defect yield per staff hour) and perhaps even worse, they slow the project’s natural rhythm requiring much staff energy to regain momentum.
104 Karl_Wiegers: 95 Maybe this is because the specific inspection technique was onerous or the culture was in attack mode, not improve mode. Other peer review techniques might be more appropriate besides inspection.
105 lanubile: 102+
106 Karl_Wiegers: 102 +
107 Dieter_Rombach: 102: We will discuss this later.
108 Moderator: We seem to have some consenus around 78: My position is that they are two different and complimentary techniques. We need to understand their costs and benefits quantitatively, and their best bractice modes. We will vote on this.
109 Frank_Maurer: 102+
110 Tom_Gilb: 97 yes. We measure such things as defect finding effectiveness versus time and costs (and more).
111 Bill Krebs: 99 Pairing has other benefits - 'present value of feedback', sharing Cockburn 'micropractices', courage to tackle tough problems
112 Scott_Ambler: 89 I unfortunately have seen it in several locations, at least that's how it seemed from the team's perspective. We suggested many other ways to work together but they insisted on reviews.
113 Laurie_Williams: 110 Can you say more?
114 Moderator: /vote PP and CI are two different and complimentary techniques.
115 Ralf_Kalmar: [voteyes]
115 Karl_Wiegers: [voteyes]
115 Dieter_Rombach: [voteyes]
115 Bill Krebs: [voteyes]
115 monvorath: [votenotsure]
115 Christian_Denger: [voteyes]
115 Scott_Ambler: 104+
116 Laurie_Williams: +
117 barry: Another dimension of advantage for PP is that it helps create the shared tacit knowledge that is critical to many agile methods, particularly if pairs recombine across the team.
118 Patricia_Costa: [voteyes]
118 rumpe: [voteyes]
118 Laurie_Williams: [voteno]
118 barry: [voteyes]
118 tran: [voteyes]
118 alberto_sillitti: [voteyes]
118 lanubile: [voteyes]
118 Bill Krebs: 117+
119 Basili: [voteyes]
119 Scott_Ambler: [votenotsure]
119 manzo: 117+
120 manzo: 117+
121 Frank_Maurer: 117+
122 Scott_Ambler: i question the vote -- there are two issues there
123 Tom_Gilb: 110 yes but this is too hectic a place and there is a lot of literature on measuring inspection and I have some (including yours) measuring PP
124 johnson: [voteyes]
124 manzo: [voteno]
124 Scott_Ambler: PP and CI are different techniques
125 Frank_Maurer: [voteyes]
125 Tom_Gilb: [voteyes]
125 Moderator: /endvote
125 125 | 12/16/2003 5:13:18 PM | Vote question: PP and CI are two different and complimentary techniques.
Vote results:

       Yes  (15/19) 79%
        No  (2/19) 11%
  Not Sure  (2/19) 11%
  Total: 19 votes

126 Scott_Ambler: PP and CI are complementary
127 Arisholm: When comparing the techniques, one should also consider the complexity of the module being developed or inspected. If there is any truth in existing results from group dynamics, complex tasks are best performed in solo (i.e., inspections) whereas simpler tasks are performed efficiently in groups (i.e., PP)
monvorath | has left the room.
monvorath | has entered the room.
128 Basili: For those who voted no, can you explain why?
129 Laurie_Williams: Can we start listing the PP Strengths and Insp Strengths on the whiteboard? Several have been listed, including 117.
130 rumpe: 129+
131 Scott_Ambler: we should also consider the ability to abstain from the vote -- if you don't have experience with both techniques then why are you voting?
132 Christian_Denger: 129+
133 Basili: 117 Don't expection teams create shared tacit knowledge?
134 Moderator: 129 What strenghts are missing?
135 lanubile: 133+
136 Karl_Wiegers: I find it frustrating that in so many agile/traditional debates people seem to act as though they represent orthogonal choices. Teams can elect to do both PP and CI as appropriate, rather than rejecting either.
137 Laurie_Williams: 134 See 117, 111 for now.
138 barry: 129+; also see 99
139 Tom_Gilb: PP gives immediate feedback early and during, Inspection might wait until too much damage is done.
140 lanubile: 133 one of the benefits from inspection is learning
141 Frank_Maurer: 13+
142 Bill Krebs: 134 Pair Programming: 'quicker feedback cycle, or present value of feedback'. Shared 'micropractices'. courage
143 Frank_Maurer: 136+
144 manzo: 133 Yes, but it's more diffuse, less personal
145 rumpe: 133 I think not that much, because inspection teams have a specific objective and not a general viewpoint. So their knowledge is specific as well.
146 Bill Krebs: 136+
147 Basili: I would say that shared tacit knowledge is a commonality
148 Bill Krebs: 134 Good audit trail for inspections
149 lanubile: 148+
150 Scott_Ambler: 136 I agree, we should use each technique where/when appropriate. You need to fill your intellectual toolbox with many tools and use them appropriately
151 Christian_Denger: Whenever we discuss the strength of the different techniques it is important to have some kind of evidance for the claims!
152 barry: PP is also good for mentoring junior people
153 manzo: 152+
154 rumpe: 152+
155 lanubile: 152 you can say the same for inspections
156 Christian_Denger: 150+
157 Basili: 152 So ae inpsections
158 Scott_Ambler: 152 It's good for mentoring "senior" people as well

159 Karl_Wiegers: 139 Only if people wait until a product is "done" to do inspection. Incremental reviews can do a great deal of defect filtering and identifying improvement opportunities for the work yet to be done.
160 Bill Krebs: 152, yes, If you know you're doing that. You have to be careful if you think you're paing and find out you're educating. Ouch!

161 barry: 155 true, but i would say not to the same extent
162 rumpe: 155- inspections need experienced people and young ones don't get the decision making process
163 Tom_Gilb: 134 Inspection provides long and short term statistics to manage the software engineering process.
164 Scott_Ambler: 159 The feedback cycle is still much longer than that with PP
165 Christian_Denger: 152: From our experience in a smal enterprise where we introduced code inspections we learned that this was one major benefit for the company
166 manzo: 157 True, but in the case of PP the mentoring is more personal and can be more specific... not lost in the group process.
167 Scott_Ambler: 159 The PP feedback cycle is on the order of seconds or minutes
168 Tom_Gilb: 164+
169 johnson: Hypothesis: PP is better for "tactical" quality improvement (how do I most efficiently use Eclipse to debug this particular NPE), while inspection is better for "strategic" quality improvement (look for concurrency/synchronization errors in this package).
170 Scott_Ambler: I think that a major issue is how you do PP -- if you're not swapping pairs then I would have to counteract this with inspections
171 lanubile: 166 but for inspections groups are not made up of dozens of people; there can be 3 or four persons in the team
172 Karl_Wiegers: I think we've lost in this discussion the notion of PP lacking the external, 3rd party perspective of a reviewer who isn't immersed in the product, absorbed all its assumptions. An outside perspective (yes, it is slower) often reveals insights that people too close to the work didn't spot.
173 lanubile: 172+
174 Scott_Ambler: If you're swapping pairs regularly you'll find that the value of inspections plumments dramattically.
175 Laurie_Williams: Philip: I think I remember you wrote a paper that said that 80% (?) of surveyed professionals don't do reviews/inspections. Is that correct? I've always considered Frank's point [95] that many professionals don't like doing inspections and do like pair programming (after adjusting) -- so then the defect removal of inspections actually happen.
176 Christian_Denger: 172: I totally agree people who are familiar with their code might miss important issues
177 lanubile: 174 any evidence for it?
178 rumpe: PP is somewhat "binary": You can use it or not. Inspections instead can be scaled to desired level of quality.
179 barry: Outside of defect reduction, most data so far indicate that pair programming is also a way to trade extra cost for reduced schedule, which is often valuable in itself.
180 Scott_Ambler: 172 that's where the swapping issue comes into play, collective ownership is critical too
181 Tom_Gilb: 178+
182 Laurie_Williams: 172+ This should be a listed Inspection Strength on the whiteboard.
183 Scott_Ambler: 177 Only anecdotal, although Laurie?????
184 Bill Krebs: 180+
185 Dieter_Rombach: Let's organize the discussion according to five topics. Effects on quality, productivity, repeatability, certifiability, and acceptance. Who can provide evidence regarding quality?
186 Tom_Gilb: swapping pairs is interesting, but let's get back to software engineering!
187 Frank_Maurer: 178: not true. Many companies I work with are not doing PP for all code - I would argue that is bad - but it allows for the scaling that you mention.
Russo | has entered the room.
188 Karl_Wiegers: 186

189 Laurie_Williams: 183 It would be great to have an experiment of this -- I've asked lots of groups to work with me on this and have never had any takers

190 rumpe: 187 downward, yes, but not beyond level 1
191 Scott_Ambler: 186 Can you elaborate?
192 Arisholm: 185 According to a small pilot experiment we conducted in industry, the pairs did produce solutions that were assesed to have better maintainability than did the solo developers
193 Tom_Gilb: 191 no! (:
194 rumpe: 185 in my experience quality did not come from PP alone, but from additional techniques such as automated tests.
195 manzo: 194++
196 Scott_Ambler: 189 I would really like to see this issue looked into. Is there any way we can "talk up" stuff like this as a community? I'm personally frustrated with several "critical research needs" that I see, this being one of them.
197 Tom_Gilb: The effect on quality for inspection is well documented in extensive literature.
198 lanubile: 194 are we talink about PP effects or the cumulative effects of agile practices?
199 Bill Krebs: We told our small team you must either pair or inspect or Justify why you did solo. We got 48 % of the changes paired, 50% solo, 2% informal multi person review. Little Unit test. 2x improvement in quality as compared to earlier days of lower pairing frequency
200 rumpe: 194 PP was e.g. useful to ensure that test coverage was good
201 manzo: 194 For defect identification, I much prefer automated contract testing and mitigated pair programming–PP only on the more algorithmically or logically complex modules.
202 Scott_Ambler: 186 Swapping pairs is critical, IMHO.
203 Karl_Wiegers: 185 There's ample evidence about the impact of inspections (good ones!) on quality improvements. Good paper in latest issue of Software Quality Professional showing inspections reducing defect leakage to customers from 10.6/KLOC to 0.9/KLOC.
204 Frank_Maurer: 194: yes, that is the problem when you try to isolate individual techniques from holistic approaches.
205 rumpe: Do we have project reports that used PP in an otherwise NON-agile setting?
206 Scott_Ambler: 197 The problem that I see with a lot of the "evidence" is that it analyzes things within non-agile environments. this is great, but when the context shifts to agile we need to reconsider the "evidence"
207 lanubile: 204 when I hear about holistic techniques my hand goes to the gun
208 Karl_Wiegers: 206 - defects are defects!
209 Laurie_Williams: The first PP experiment I did was definitely PP in non-agile. It was PP imbedded in the Personal Software Process -- since PSP was a known baseline. It showed statistically significant higher quality. That would have been PP vs. solo desk checking (not inspections).
210 Tom_Gilb: 206+
211 Christian_Denger: 206: Thats definately true, so we need to come up with hypothesis we could analyze in agile development to get the evidance that the techniques improve quality etc.
212 Scott_Ambler: 208 Yes, but what is the context in which those defects are removed? Change the context and perhaps the technique is no longer as effective
213 barry: One advantage of inspections is that you can work on multiple qualities. Perspective-based inspections enable artifacts to be reviewed by experts in safety, usability, performance, etc.
214 Dieter_Rombach: Can anyone provide tangible evidence on the impact of PP or inspections on quality?
215 lanubile: inspections has tons of reports on improvement of quality
216 Frank_Maurer: 207: Usually, mine doe too. But controlled experiments usually focus on individual aspects - and there is lots of anecdotal evidence that you can optimize one aspect by giving up others
217 Scott_Ambler: 215 Yes, but within context
218 Moderator: 215 Can you provide an example?
219 Tom_Gilb: Keep in mind the recent work published by Cusomano et al in IEEE Software. Conclusion: it is critical which exactly set of techniques that work together and have effects. We cannot judge PP or CI alone, but only in context with the other set of things being done.
220 Basili: 214 There are severl dozen papers about the effects of inspections of quality
221 Scott_Ambler: 219+
222 Scott_Ambler: 220+
223 Moderator: 214 Any evidence regarding PP?
224 Christian_Denger: 213+ And thats one major difference between the techniques as we get other perspectives in.
225 Karl_Wiegers: 212 Maybe the point is that PP results in injecting fewer (code) defects, so inspections wouldn't yields as much. I can buy that. But we aso need to address here the fact that PP is pair PROGRAMMING. How about the many defects injected in earlier life cycle activities, like requirements?
226 Frank_Maurer: 223: sure Laurie's work
227 Scott_Ambler: 225++++++++
228 Bill Krebs: 214 My data is murky, but we improved quality by 2x for our small team by emphasizing pairing and unit test (over less frequet pairing). The improvement came with little Unit test and 48% pairing
229 manzo: 223 My guess is only anecdotal evidence except for ALurie's work
230 Arisholm: 223 I think both Laurie and myself provided some evidence on PP
231 lanubile: 225+
232 Tom_Gilb: 223 yes I believe I have several papers in my collection , but too hectic to dig up here and now.
233 Tom_Gilb: 225+
234 Moderator: 230 Could you examplify with data?
235 Laurie_Williams: 225 Pair PROGRAMMING is a very bad name. Most people who practice pair programming do it in all phases of development (after requirements probably). In my studies, it is used more often in design and analysis and actually the least during actual coding.
236 Scott_Ambler: 225 In my initial response to this meeting I pointed out that the scope of the discussion was too narrow -- we really need to look at collaborative techniques, PP being one of them. Model With Others from AM is the modeling equivalent to PP, and it offers many of the same benefits as PP does except with respect to modeling
237 Scott_Ambler: 235+
238 rumpe: Evidence for PP: Mine again is only anectdotical, but I felt confident and so did the other team members. However, we also strictly used other techniques and I am totally unsure what combination of techniques gave which effects.
239 Frank_Maurer: 235+ - pair programing and test driven development are about design - not code only
240 manzo: 235+
241 Dieter_Rombach: 235: What do you consider to be the difference between your approach and inspections?
242 barry: 220+ Some of the studies indicate that inspections are highly effective for certain classes of defects (e.g., programming blunders, logic errors, interface errors, missing portions) but not so effective for others (e.g., timing, program dynamics, and numerical approximation errors)
243 Tom_Gilb: moderator failed to pickup my comment on Inspection strengths that is give a statistical basis for managing the whole software engineering process.
244 Scott_Ambler: 235: Meta issue -- the community still has itself wrapped around the "XP axle". XP is one of many agile methods, yet it always seems to dominate the discussion. Until we move on I suspect the agile movement is in trouble
245 Dieter_Rombach: 242: Do we have similar evidence for PP?
246 manzo: 220+ I might also say the same for PP
247 Christian_Denger: 242: We should somehow discuss which types of defects can be detected with PP / inspections and where are the differences
248 rumpe: Some collected evidence: We have surve on 45 projects in XP'02 conference where PP was ranked as rather helpful (I don't recall the details know, see my website)
249 Karl_Wiegers: 242 So maybe we need to figure out what kind of problems are best solved with what kind of techniques, and make sure that professional software developers have them all -- and some judgement -- in their tool kits?
250 Bill Krebs: Lots of "Knowledge work" benefits from pairing
251 Moderator: We're going to do a series of votes to see if there's is any consensus on these issues.
252 manzo: 249 I'm a strong proponent of Contract testing. I think CI and PP take a back seat to this.
253 Scott_Ambler: 249+
254 Karl_Wiegers: 243 This is fine, but the fact remains that most development teams do have defects in their work that need to be removed. So inspection still has a place as defect removal, until more teams can use them for statistical quality sampling.
255 barry: 245 Not that I've seen so far. Laurie?
256 Moderator: /vote Inspection benefits are well documented esp. w/r/t reduced defect slippage.
257 Dieter_Rombach: [voteyes]
257 Karl_Wiegers: [voteyes]
257 Arisholm: 234 The presentation I had on ISERN'02 shows the data from the pilot experiment
258 Christian_Denger: [voteyes]
258 rumpe: [voteyes]
258 Scott_Ambler: [voteyes]
258 Ralf_Kalmar: [voteyes]
258 manzo: [voteyes]
258 Bill Krebs: [voteyes]
258 Laurie_Williams: [voteyes]
258 Tom_Gilb: [voteyes]
258 Basili: [voteyes]
258 Steve_McConnell: 256 +
259 barry: [voteyes]
259 lanubile: [voteyes]
259 Winsor_Brown: 256 What do you mean by "slippage"?
260 Steve_McConnell: [voteyes]
260 Patricia_Costa: [voteyes]
260 Arisholm: [voteyes]
260 Frank_Maurer: [voteyes]
260 tran: [voteyes]
260 Moderator: /endvote
260 260 | 12/16/2003 5:37:12 PM | Vote question: Inspection benefits are well documented esp. w/r/t reduced defect slippage.
Vote results:

       Yes  (18/18) 100%
        No  (0/18) 0%
  Not Sure  (0/18) 0%
  Total: 18 votes

261 manzo: 259 Defect containment I think
262 Laurie_Williams: 255 No -- would be good to investigate.
263 Frank_Maurer: 251: Are they also shown that they are more cost effective than PP?
264 Tom_Gilb: 259 I assume schedule delay is meant
265 Moderator: /vote The effectiveness of inspections vary by defect type
266 Karl_Wiegers: Correction in the white board, under Issues for comparison -- I think Laurie said in 235 that PP was done AFTER reqs, in design.
267 Scott_Ambler: 249: What we really need to do is present the technique, discuss it's advantages and disadvantages, and provide advice for when to use it. I didn't this to some degree regarding database development techniques in Agile Database Techniques (www.ambysoft.com/agileDatabaseTechniques.html). It's pretty hard to do, but doable.
268 rumpe: [voteyes]
268 Tom_Gilb: [voteyes]
268 Christian_Denger: [voteyes]
268 Dieter_Rombach: [voteyes]
268 manzo: [voteyes]
268 Steve_McConnell: 265 +
269 Bill Krebs: [votenotsure]
269 Karl_Wiegers: [voteyes]
269 Steve_McConnell: [voteyes]
269 lanubile: [voteyes]
269 Basili: [voteno]
269 Scott_Ambler: [voteyes]
269 tran: [votenotsure]
269 Moderator: /endvote
269 269 | 12/16/2003 5:38:47 PM | Vote question: The effectiveness of inspections vary by defect type
Vote results:

       Yes  (9/12) 75%
        No  (1/12) 8%
  Not Sure  (2/12) 17%
  Total: 12 votes

270 Ralf_Kalmar: [votenotsure]
270 manzo: Bye all... and thank you.
271 Moderator: /vote PP benefits are well documented esp. w/r/t reduced defect slippage.
272 Steve_McConnell: 271 -
273 Basili: [voteno]
273 Bill Krebs: [votenotsure]
273 Ralf_Kalmar: [votenotsure]
273 Dieter_Rombach: [votenotsure]
273 Laurie_Williams: [voteyes]
273 tran: [voteno]
273 Patricia_Costa: [voteno]
273 Christian_Denger: [voteno]
273 Tom_Gilb: [voteno]
273 Steve_McConnell: [voteno]
273 Karl_Wiegers: [votenotsure]
273 lanubile: [voteno]
273 rumpe: [voteno]
Scott_Ambler2 | has entered the room.
273 Moderator: /endvote
273 273 | 12/16/2003 5:41:06 PM | Vote question: PP benefits are well documented esp. w/r/t reduced defect slippage.
Vote results:

       Yes  (1/13) 8%
        No  (8/13) 62%
  Not Sure  (4/13) 31%
  Total: 13 votes

274 Arisholm: I am not getting the vote buttons...
275 Patricia_Costa: 274: hit the send button.
276 Moderator: For those who voted yes, where are the studies where this is documented?
277 rumpe: 269 I hope we also see the question, whether we believe that "PP benefits reduce defect slippage".
278 Scott_Ambler2: I suspect that the software is getting overloaded
279 lanubile: 276 you run an eworkshop about this
280 Ralf_Kalmar: 274 The moderator broke the vote after a certain amount of time, probably that's why.
281 Tom_Gilb: I did not vote yes, but I am willing to share my collection of PP other peoples research with one central oint for analysis. It is mixed with XP research.
282 Moderator: 279 explain....
283 Winsor_Brown: [votenotsure]
283 lanubile: 276 the results of the eworkshop are reported on www.cebase.org, right vic?
284 Winsor_Brown: 269 What is "slippage". Please define!!!
285 Scott_Ambler2: Whiteboard: Issues for comparison -- You need to look beyond PP if you want to compare fairly, e.g. include modeling with others. sigh
alberto_sillitti | has entered the room.
286 Basili: 276 We ran eWorkshops on agile ingeneral and defect reductin using inspectinos but not on pair programming in particular. The results areon cebase.org
287 lanubile: 284 defect slippage refers to how many defects remain undiscovered
288 Bill Krebs: 281+
johnson | has left the room.
Winsor | has entered the room.
289 Moderator: /vote PP benefits reduced defect slippage
290 Bill Krebs: [voteyes]
290 Frank_Maurer: [voteyes]
290 Russo: [voteyes]
290 Arisholm: [voteyes]
290 Steve_McConnell: 289 +
291 monvorath: [voteyes]
291 Steve_McConnell: [voteyes]
291 Laurie_Williams: [voteyes]
291 rumpe: [voteyes]
291 Scott_Ambler2: [voteyes]
291 Christian_Denger: [votenotsure]
291 Ralf_Kalmar: [votenotsure]
291 Patricia_Costa: [voteyes]
291 Basili: [votenotsure]
291 Dieter_Rombach: To change the direction of the discussion: Do you see a difference between PP and inspections with respect to repeatability of quality results across projects?
292 rumpe: 291 could you please elaborate on the question?
293 Dieter_Rombach: [voteno]
293 tran: [votenotsure]
293 Tom_Gilb: 287 Opps 'Defect slippage'! Sorry Winsor, I reacted to slippage. A better more concventional term is defect finding effectiveness, or 'estimated remaining defects'.
294 Dieter_Rombach: 292: Assume that you let the same team do a similar task again, what is the likelihood that they deliver the same quality in the same timeframe/effort using SP & CI vs. PP?
295 Moderator: /endvote
295 295 | 12/16/2003 5:48:16 PM | Vote question: PP benefits reduced defect slippage
Vote results:

       Yes  (10/15) 67%
        No  (1/15) 7%
  Not Sure  (4/15) 27%
  Total: 15 votes

296 Scott_Ambler2: 291: Inspections might be able to bring increased repeatability across projects, if it's done early enough in the process. Leave it too late and nobody is likely to act on the results
297 Tom_Gilb: [voteyes]
297 Basili: 291 I think it would be harder to to build predictive models of defect reduction based upon pair programming
298 Scott_Ambler2: 294 Greater with PP I bet
299 Laurie_Williams: 297 I think PP is more about defect prevention so harder to quanify.
300 rumpe: 297+ agreed, but I think PP is an optimising strategy in that setting, whereas inspections are not
301 Dieter_Rombach: 298: How can you repeat when you exchange people, without having evidence from past projects?
302 Basili: 298 How do you think you would get better repeatability? What kind of model would you build?
303 rumpe: 298- Inspecations ar emore stable I Bet
304 Tom_Gilb: 297 good point. ! PP is more in a prentative mode! Remember that Inspection can contain CMM 5 Defect Prevention Process integrated ( that is the way I teach/write it).
305 Scott_Ambler2: 301 If you completely change the team it will be hard. If you keep many of the team members then they will have built a culture and a way of working that will be much more effective.
Winsor | has entered the room.
306 Scott_Ambler2: 303 Looks like there's a research opportunity there!
307 Basili: 305 That does not allow you to build a predictive model
308 Winsor: 289 Unfortunately, it is not precise since it is "estimated"!
309 Laurie_Williams: 303+
310 Moderator: We're going to vote on which of PP and CI is more repeatable
311 rumpe: It's an interesting question whether predictability is more important than efficiency. It depends on the project context.
312 Bill Krebs: 311+
313 Scott_Ambler2: 311+
314 Steve_McConnell: 299 I don't agree that PP is about defect prevention any differently than inspections are about prevention. PP is about shortening the cycle time in detecting the defect to practically zero because the other person in the pair sees the error quickly. That isn't prevention; it's early detection. Inspections are also about early detection, but the feedback cycle is longer. As far as I can tell, which is more cost-effective to attain a specific level of quality is an open question.
315 Scott_Ambler2: 307 Why is a predictive model important to you?
316 Winsor: 269 I repeat, What is "slippage". Does anyone have a better definition than Toms?
317 Arisholm: 314+
318 Moderator: /vote PP is more repeatable than CI
319 Patricia_Costa: 314+
320 Tom_Gilb: 314 Agreed, the initial reaction is about detection, but i assume it prevents defect injection later in the same and other sessions.
321 Dieter_Rombach: [voteno]
321 Tom_Gilb: [votenotsure]
321 rumpe: [voteyes]
321 Frank_Maurer: [votenotsure]
321 Ralf_Kalmar: [voteno]
321 Laurie_Williams: 314 When a pair works together, they brainstorm and negotiate quite a bit (a few seconds/a few minutes) before the keyboard is touched. I feel that is defect prevention.
322 Scott_Ambler2: [votenotsure]
322 lanubile: [voteno]
322 Basili: [voteno]
322 Bill Krebs: [votenotsure]
322 Laurie_Williams: [voteno]
322 Christian_Denger: [voteno]
322 Patricia_Costa: [votenotsure]
322 monvorath: [votenotsure]
322 Moderator: I'm breaking the vote
323 Moderator: /endvote
323 323 | 12/16/2003 5:55:25 PM | Vote question: PP is more repeatable than CI
Vote results:

       Yes  (1/13) 8%
        No  (6/13) 46%
  Not Sure  (6/13) 46%
  Total: 13 votes

324 Steve_McConnell: 320 Tom, can you share your data about reduced defect injection rates after inspections have been in operation for awhile?
325 Basili: 321 I fell that you and Steve are using different derinitino sof pair programming
326 Bill Krebs: 316 I assume 'slippage' means a defect that escapes past the phase in which you want to catch it
327 Scott_Ambler2: Perhaps we need to discuss what we mean by PP then?
328 Winsor: 318: Why are you limiting the vote to Code Inspections? Laurie already commented that she has seen pair approaches used in Analysis and Design.
329 lanubile: 326+
barry | has entered the room.
330 Dieter_Rombach: 326: Slippage means the percentage of defects not caught by a particular inspection
331 Karl_Wiegers: 314 +
332 Bill Krebs: 328 I using pairing for everything I can - any intellectual work.
333 Basili: 315 One would like to be able to predict cost, quality, scheudle, etc. and that required predictive capbility
Bernhard_Rumpe | has entered the room.
334 Scott_Ambler2: 332+
335 Winsor: 330 Thank you, although I ask again how is that really measured. Don't you mean the percentage of defects found later than the activity-phase in which they were [supposedly] injected?
336 Tom_Gilb: 324 Yes. Our industrial data is the , at the individual person level, once the defect-found feedback has worked ( using 'no more than x major defects/page allowed before exit' type motivation) the individual can systematically inject two orders of magnitude fewer defects in their daily work. (Remember Winsor?).
337 Steve_McConnell: 336 To clarify, that data is from using inspections, right?
338 Karl_Wiegers: This is a fine theoretical discussion, but at some point we might discuss how to persuade ordinary practitioners (not just early adopters) to invest in either PP or CI. I find very little of either happening with my consulting and seminar clients. I'm rather fond of reduction to practice.
339 Bill Krebs: 338+
340 Scott_Ambler2: 338+
341 Tom_Gilb: I am still experiencing this. Recent Banking client, requirements, has about 88 majors/page before motivation and measurement and after a few months is at about 11 majors/page, and we expect this to drop to less than 1 major/page, within months.
342 barry: 338+
343 Bill Krebs: I begged and pleaded for either 100% pairing or inspection and got 48% pairing and 2% inspection
344 Moderator: It's time to review to whiteboard, and especially the first bullet: Providing a 3rd party (outsider) perspective - Inspections allow incorporating different quality foci or perspectives. Do you agree?
Laurie | has entered the room.
345 Dieter_Rombach: 338: We have several industrial studies where inspections have reduced overall effort by up to 40% (by reduction of rework)
346 Tom_Gilb: 336 Yes normal Inspections. My second comment 341 is using sampling inspection ( agile inspections).
347 barry: 344++
348 Bernhard_Rumpe: 344+
349 Christian_Denger: 346: Tom could you please explain what you mean by agile inspections?
350 Karl_Wiegers: 344 +
351 Frank_Maurer: 344: aren't inspectors often from the same team? isn't pair rotation getting the same effect?
352 Scott_Ambler2: 344+, but there are other ways to achieve this
353 Patricia_Costa: 352: what are they?
354 Winsor: 330 A much better term is used at a least one major corporation site here in the US: "escapes" which is the number of defects that escape from one activity or development process "step" to a subsequent activity/step, where they also include the number of steps (and even the injected/removed step, where appropriate.
355 Karl_Wiegers: 351 I've found it helpful to get inspectors from other teams, who *don't* intimately know the product. It's interesting to see what happens when assumptions are questioned and fresh perspectives brought to bear.
356 Scott_Ambler2: 351 I try to get people from outside the team to inspect, although pair rotation and collective ownership does in fact help this immensely
357 Arisholm: 343 In our case study, the most experienced developers more or less refused to use PP. They preferred "partner programming", that is, they shared one office and asked each other questions whenever they got stuck. The more junior developers enjoyed PP.
358 barry: 351+
359 Christian_Denger: 351: But all people are familiar with the code and you don't consider external persons who can add a non-biased view
360 Scott_Ambler2: 353: Modeling with others, collective ownership, proving it with code
361 Bill Krebs: 356+
362 Winsor: Has anyone tried to measure "escapes" in PP?
363 barry: 355+
364 Moderator: Is there anything you want to change or add to the whiteboard before we move on to the next topic?
365 Bill Krebs: 357 It was hard a first to get more experienced people to try new stuff, in this case, pair programming. Once they tried it they liked it.
366 Tom_Gilb: 349 I have developed them as a subset of normal inspections. They rely on sampling ( a page or so) and extrapolation from found defects to total defects. I can send details in form of slides. Tutorial held at EuroSTAR Amsterdam, Dec.
Laurie_Williams | has entered the room.
367 Scott_Ambler2: 364 We really do need to look beyond pair programming
368 Steve_McConnell: Whiteboard: Insp Strengths: Add multiple viewpoints/reviewers brought to bear at the same time.
369 Frank_Maurer: 366: I would like to see the slides
370 barry: 364 Outside of defect reduction, most data so far indicate that pair programming is also a way to trade extra cost for reduced schedule, which is often valuable in itself.
371 Moderator: We're going to move on to the next topic
372 Karl_Wiegers: 364 Whiteboard: Value of an inspection reader describing how they interpret, say, a requirement, so other inspectors can compare this to their own interpretation. Good way to find ambiguities.
373 Dieter_Rombach: Now, we would like to move on to the second topic: Application of Inspection in an Agile context Assume the following scenario: You are applying XP (or some other form of Agile Development) on a project, including pair programming and other practices. Can you imagine a situation, where it might be appropriate to perform a code inspection of some module – in addition to or instead of PP?
374 Scott_Ambler2: 368 You get that when you're modeling with others. With modeling the #people involved can easily be > 2
Laurie | has entered the room.
375 Patricia_Costa: 372: can you give more details?
376 Tom_Gilb: 364 all whiteboad? I suggested twice inspection strengths, statistics
377 Scott_Ambler2: 372 Rapid feedback via delivering working software quickly also achieves this
378 Bernhard_Rumpe: PP usually resides in a process with a rather high level of evolution (refactoring, rework). Only when a module becomes stable, we can do inspection.
379 Bill Krebs: 378+
380 Patricia_Costa: 376: see the 5th bullet on the issues for comparison
381 lanubile: 378+
382 Scott_Ambler2: 378+
383 Dieter_Rombach: 378+
384 Karl_Wiegers: 377 Yes, then you get to fix it.
385 barry: 373 I see this as an economic issue. If the risk exposure due to unfound defects is high, doing the inspection is worthwhile. If the risk exposure is low, then it's not worthwhile.
386 Bill Krebs: 364 Pairing may motivate people to tackle more difficult challenges
387 Scott_Ambler2: 378 but once it stabilizes, how motivated are you REALLY to act on suggested changes?
388 Karl_Wiegers: 378 +
389 Bill Krebs: 385++
390 Moderator: In order to scroll down the white board, mark the text and drag down with the mouse
391 Laurie: 385+
392 Tom_Gilb: 373 Yes, the agile form of inspection, using sampling might be appropriate, at least because it is not time consuming. But that assumes anyone in XP environment can count.
393 Dieter_Rombach: 387: What in case of critical components?
394 lanubile: 385+
395 Dieter_Rombach: 385+
396 Scott_Ambler2: 384 Yes. Plus with active stakeholder participation (I put my users to work and get the to develop requirements, it's not easy being evil) you quickly discover you get a lot of really good feedback
397 Scott_Ambler2: 385+
398 Arisholm: 387+ I think this is a good point. With PP, you can adjust the design before it is too late...
399 Karl_Wiegers: 387 + This is an important point about inspection. If people *think* the work product is done, they can be psychologically resistant to making changes suggested by inspection. It's a good argument for early peer feedback, either by PP or by early incremental peer review.
400 Laurie: 399+
401 Bill Krebs: 398+
402 Bill Krebs: 399+
403 barry: 399+
404 Christian_Denger: 399+
405 lanubile: 399 yes, but inspections are started as soon as the product is available
406 Tom_Gilb: 385 ok but I'll go further, if defects are really critical we must focus on prevention, because cleanup effectiveness % is so poor. 60% is the best IBM MN had as code inspection effectiveness after years of process improvement (citing in my SI Book)
407 Dieter_Rombach: 399: Can you imagine then to apply inspections to critical components without doing PP?
408 Arisholm: 405 ... which perhaps is too late?
409 Scott_Ambler2: 393 As long as the risk>cost of change, it's worth it. Also, the change should have the greatest payback available to you for you investment (e.g. there aren't more pressing needs to address)
410 Tom_Gilb: 398 I thought we were focussing on coding and not design inspections/reviews here !

411 Scott_Ambler2: 407 Why not, people do it all the time
412 Bernhard_Rumpe: 407 I would like to see a process, where the team (leader) is able to decide rather dynamically, where to use extra inspections, based on complexity, criticality, degree of innovation.
413 Karl_Wiegers: 405 ... and they should be, as soon as maybe 10% of the product is available (I think I learned this from Tom!). But peer reviews don't have to be formal inspections! Cheap feedback from a peer deskcheck on a partial component can help a whole lot in future defect prevention.
414 barry: 399 A related phenomenon we found with inspections at TRW is that they caused developers to be more careful in order to avoid looking dimb.
415 Dieter_Rombach: 412+
416 Laurie: 414+
417 Bill Krebs: 414+
418 barry: 414 The last word was supposed to be "dumb"
419 Scott_Ambler2: 410 A fundamental problem with the entire discussion is that we're too far down in the weeds. Why aren't we considering higher level issues such as design insptections?
420 Tom_Gilb: 405+
421 Karl_Wiegers: 414 + so long as they don't expend too much energy perfecting the product before asking for an outside perspective
422 Laurie: 419+ We should because it's hard to limit PP to coding defects only.
423 Tom_Gilb: sorry 413+
424 Karl_Wiegers: 419 +++
425 Scott_Ambler2: 414 but was the additional cost of all that?
426 Winsor: 412 Doesn't a team leader always have that right? Or, more properly, can't a manager always delegate that right?
427 barry: 421+, reinforcing 399
428 Winsor: 414 dimb = dumb?
429 Basili: 422 I agree we shoud not limit the discussion to code
430 Tom_Gilb: 419 + yes! we software engineers need to stop focussing on code itself and focus on the reasons code is bad - like requirmeents and design, and inspections are great tools there
431 Bill Krebs: The winning concept is 'early and often'. Pairing specializes in this this, but inspections can do this too if you hold 'em early and repeat as necessary
432 Karl_Wiegers: 412 + This should be every team member's right and responsibilty
433 Arisholm: 414 I think pair programming might have a similar effect: Pair Pressure.
434 Moderator: Let's vote on 419
435 Bill Krebs: 433++ very much so
436 Tom_Gilb: 412+
437 Scott_Ambler2: 430 Yes, and there are the agile equivalents to PP for modeling -- what I call "Model With Others"
438 Scott_Ambler2: 434+
439 Laurie: 433+
440 Moderator: /vote Do you want to broaden the discussion and also include Design inspections?
441 lanubile: [voteyes]
441 Frank_Maurer: [voteyes]
441 Dieter_Rombach: [voteno]
441 Bill Krebs: [voteyes]
441 Scott_Ambler2: [voteyes]
441 Christian_Denger: [votenotsure]
441 Laurie: [voteyes]
441 Bernhard_Rumpe: [votenotsure]
441 Tom_Gilb: [voteyes]
441 Karl_Wiegers: 440 What about requirements?
442 tran: [voteyes]
442 Patricia_Costa: [voteyes]
442 barry: [voteyes]
442 Arisholm: [voteyes]
442 Karl_Wiegers: [voteyes]
442 Winsor: 422 Wasn't your PP actually Pair Software Development, including requirements, design, coding and testing (i.e., the activity steps of PSP)?
443 Scott_Ambler2: i think the real issue is to go beyond coding
444 Moderator: /endvote
444 444 | 12/16/2003 6:16:51 PM | Vote question: Do you want to broaden the discussion and also include Design inspections?
Vote results:

       Yes  (11/14) 79%
        No  (1/14) 7%
  Not Sure  (2/14) 14%
  Total: 14 votes

445 Karl_Wiegers: 443 + let's not ignore requirements, the highest leverage opportunity for quality improvement
446 lanubile: 445+
447 Tom_Gilb: 444 hurrah for sanity gentlemen, and why not requirements while we are at it?
448 Laurie: 442 yes -- all of it -- the Collaborative Software Process (CSP)
449 Scott_Ambler2: 445+
450 Dieter_Rombach: Ok, let’s broaden the discussion to other types of software documents. From the pre-meeting feedback, we found that most of you seem to agree that PP focuses mostly on code, while inspections can be applied to all kinds of software documentation. However, some mentioned that the concept of PP (“collaborative development”

can be well extended to modelling
451 Basili: 445+
452 Steve_McConnell: 440 -
453 Bernhard_Rumpe: Pairing is of course useful in any stage, however don't we use pairing already quite a lot in modelling workshops etc.
454 Winsor: [votenotsure]
454 Scott_Ambler2: 450 -- DON'T ASSUME THAT MODELS ARE DOCUMENTS!!!!!!
455 barry: 445+ Does anyone have data on the relative effectiveness of inspections for requirements, design, and code?
Steve_McConnell | has left the room.
456 Scott_Ambler2: 450 Yes, but the survey was biased towards PP....

457 Karl_Wiegers: 455 One of my clients did reqs inspections for 5 year. Measured a sustained ROI of 10:1. I'll take it!
458 Dieter_Rombach: 454: What we mean is documented models.
459 Moderator: Do refresh/reload if you don't see all statements
460 Basili: Resources Log
Steve_McConnell | has entered the room.
461 Scott_Ambler2: 453 It's not just pairing, it's modeling in a group
462 Winsor: 450 Please rephrase with "documents" replaced with "artifacts" so models, etc., are included.
463 barry: 457+
464 Scott_Ambler2: 453 you can easily have several people standing up around a whiteboard for example
465 Winsor: 455 Yes, but no time to dig it out now.
466 Karl_Wiegers: 464 How is this different from brainstorming?
467 Patricia_Costa: 455: The defect removal rate seems to be about the same for all phases. WE are looking for data about the effect on the overall life cycle.
468 lanubile: 466+
469 Bernhard_Rumpe: 461 Even more than pairs ...
470 Scott_Ambler2: 466 not much

471 barry: 457 We also use inspections as a way to transform informal stakeholder win-win agreements to more precise requirements.
472 Tom_Gilb: 455 yes IBM developed this early on and Capers jJOnes books are full of data. But in rought terms the effectiveness of Inspections (if properly done at proper rates of checking like one page/hour) are about 75-90% effective for requirements and design but for code are in the 15% to 60% range. I have seen higher numbers from Capers than 60% but I am not sure i trust them. (people don't seem to be good at undersatanding remaining defects downstream).
473 Steve_McConnell: 461 + I think the comparison gets awfully muddy if we start comparing design approaches. We can design as singles, as pairs, as groups of 3-n; we can do informal desk checks, formal desk checks ("Document reading"

, informal reviews, formal inspections. I think the n-way comparison of techniques becomes pretty unwieldy.
474 Winsor: 450 The survey showed that the respondents did not understand what Laurie meant by "Pair Programming" (actually Collaborative Software [development] Process).
475 Karl_Wiegers: 471 +
476 Dieter_Rombach: 455: We do; however, the high effectiveness of requirements effectiveness has to do with the fact that in this environment, most problems were related to requirements.
477 Winsor: 467 That same data is in the table I provided in an earlier eWorkshop.
478 Bernhard_Rumpe: 466 I think Scott uses models for barinstorming mainly. It would be nice to use models for more than that. Then pairing and inspection becomes interesting.
Patricia | has entered the room.
479 Scott_Ambler2: 473 Which gets to Karl's earlier point that people need to be aware of a wide range of techniques, their differences, and advice for when to use them
480 Karl_Wiegers: 476 Precisely the point. This is where the leverage is, probably in most environments. So why not more emphasis on techniques for reducing errors in requirements?
481 Laurie: 473+
482 barry: 472 Our results for inspections at TRW were about the same: in the 60% effectiveness range for both design and code.
483 Scott_Ambler2: 478 I use models for two basic things -- modeling to understand, which has been labelled as brainstorming for now, and modeling to communicate
484 Tom_Gilb: 473+
485 Karl_Wiegers: 473 + especially in the absence of data vs opinions and anecdotes
486 Winsor: How can I re-submit the table summarizing effectiveness?
Pat | has entered the room.
487 lanubile: 473 it is really a mess if we consider all the possible variations
488 Moderator: Let's vote: Do we have have evidence that Pairing is useful for requirements and design as well?
489 Moderator: /vote Do we have have evidence that Pairing is useful for requirements and design as well?
490 Christian_Denger: 487+
491 Bill Krebs: 464+ When folks rely on iterative modelling as a way to reduce the burden of maintaining documents, pairing becomes more attractive.
492 Bernhard_Rumpe: 488 see 473: depends so much
493 Frank_Maurer: [voteyes]
493 Dieter_Rombach: [voteno]
493 Karl_Wiegers: [votenotsure]
493 Steve_McConnell: 489 -
494 Laurie: [voteno]
494 lanubile: 488 pairing means two people working together or n-people?
495 Bernhard_Rumpe: [votenotsure]
495 Bill Krebs: [voteyes]
495 Scott_Ambler2: [voteyes]
495 Steve_McConnell: [voteno]
495 monvorath: [votenotsure]
495 Arisholm: [votenotsure]
495 Christian_Denger: [votenotsure]
495 Bill Krebs: "On site customer" is a way to 'pair review' requirements.
496 Frank_Maurer: 489: anecdotal evidence counts?
497 Dieter_Rombach: Now, we would like to move on to the third topic: Application of PP in a CMM context. Assume you are developing software following a plan-driven (traditional) approach that includes inspections on different documents. The team lead wants to substitute SP&CI with PP.
498 Moderator: 496 yes
499 Moderator: /endvote
499 499 | 12/16/2003 6:25:18 PM | Vote question: Do we have have evidence that Pairing is useful for requirements and design as well?
Vote results:

       Yes  (3/11) 27%
        No  (3/11) 27%
  Not Sure  (5/11) 45%
  Total: 11 votes

500 Karl_Wiegers: 495 Absolutely, if it's the right customer and that one customer can speak accurately for all your user classes.
501 Tom_Gilb: [voteno]
501 Laurie: 488 Can we do a separate vote -- I don't think much has been done on using pairing for requirements but many use pairs for design.
502 Winsor: 488 Is "Pairing" really the same as the well defined CSP?
503 Steve_McConnell: 488 The wording of this question isn't definitive. Pairing is useful to some degree, but so are inspections, reviews, desk checks, personal iteration, etc.
504 lanubile: [votenotsure]
504 lanubile: 501+
505 Tom_Gilb: 503+
506 Moderator: 501 Let's do that
507 Bill Krebs: 503+
508 Moderator: /vote Let's vote: Do we have have evidence that Pairing is useful for design as well?
509 Winsor: 497 what is SP?
510 Karl_Wiegers: [votenotsure]
510 Frank_Maurer: [voteyes]
510 Bill Krebs: [voteyes]
510 Laurie: [voteyes]
510 Tom_Gilb: [voteno]
Laitenberger | has entered the room.
510 Christian_Denger: [votenotsure]
510 Dieter_Rombach: [voteno]
510 Bernhard_Rumpe: [voteyes]
510 Arisholm: [voteyes]
510 lanubile: [votenotsure]
Basili | has entered the room.
510 Steve_McConnell: 508 -
511 Moderator: /endvote
511 511 | 12/16/2003 6:26:57 PM | Vote question: Let's vote: Do we have have evidence that Pairing is useful for design as well?
Vote results:

       Yes  (5/10) 50%
        No  (2/10) 20%
  Not Sure  (3/10) 30%
  Total: 10 votes

512 Laitenberger: [voteno]
512 Steve_McConnell: [voteno]
512 Winsor: [votenotsure]
512 Basili: [votenotsure]
512 Karl_Wiegers: 511 Has evidence along these lines been published somewhere so we can all read it?
513 Tom_Gilb: 511 ? wow please cite some evidence website
514 Dieter_Rombach: 509: SP stands for "solo programming" (see introduction); CI for code inspections
515 Moderator: 513 Who has evidence?
Scott_Ambler3 | has entered the room.
BernhardRumpe | has entered the room.
516 Scott_Ambler3: 515 Just anecdotal
517 Bill Krebs: 516+
518 Moderator: 516 that's good enough, let us know
519 Laurie: In my initial PP study, I have a break out of use by phase. Use of PP was highest in design. However, I don't have any evidence of design isolated -- just by phase for all of development.
520 Frank_Maurer: We have some evidence on the perception of developers/students that they create better designs using pair programming
521 Scott_Ambler3: 518 Two+ heads are basically better than one. By modeling together you get multiple opinions, you build agreement, and frankly you home in on a potential "answer" much faster than with one person struggling alone.
522 Winsor: 514 Pardon my ignorance, but where can I find the "Introduction"?
523 barry: 501 From a stakeholder win-win perspective, just getting two people to determine the correctness of requirements is very risky, as it excludes success-critical stkeholders from the process.
524 Scott_Ambler3: 523+
525 Arisholm: 515 See e.g., Flor & Hutchins: "Analysing Distributed Cognition in Software Teams: A Case Study of Team Programming During Perfective Software Maintenance", proc. 4th workshop on Empirical Studies of Programmers, pp. 36-64, 1991
526 lanubile: 523+
527 Dieter_Rombach: 521: That is not even anecdotal, rather philosophical.
528 Karl_Wiegers: 497 I don't understand what PP has to do with plan-driven or CMM approaches. Isn't PP simply a "good practice" that could and should be selectively applied in an appropriate environment after some thought?
529 Scott_Ambler3: 523 This is why you need 2+, collective ownership, and rotation of people through work pair/groups
530 Karl_Wiegers: 523 +
531 Bill Krebs: 529+
532 BernhardRumpe: 528+
533 Basili: 528+
534 Scott_Ambler3: 527+
535 lanubile: 529 or this is why you need a team inspecting requirements
536 Scott_Ambler3: 528+
537 Basili: 523+
538 Karl_Wiegers: 523 This is why you need to identify your stakeholders and different user classes and understand their needs and alignment with your business objectives. But that's another topic.

539 Scott_Ambler3: 535+, but the feedback cycle is much longer that way
540 Dieter_Rombach: 523+
541 barry: 538+
542 Basili: 523, 528 I think there is aome agrement that multiple heads and perspectives have an advantage over 2 so there are times when multiple reviewers are of value
543 Winsor: 528 Has anyone done a "fittness" test/exercise of Pair Programming to the CMM's "requirements" for "Peer Reviews"?
544 Moderator: Please review the whiteboard
545 Moderator: Is there anything we should add or change?
546 Bill Krebs: I think a key key key point is that pairing may find greater adherence in some small teams than inspections.
547 Scott_Ambler3: 546+
548 Moderator: Please refresh/reload if you don't see the whiteboard. You might need to scroll down.
549 Frank_Maurer: 546+
550 BernhardRumpe: 542+,523+ which brings us back to the point that PP achieves a certain quality, but going beyond is only possible with inspections
551 Tom_Gilb: 546 why can't you add my statistics suggestion for inspections, it is a dominant point!
552 Winsor: 545 Please add the definition of slippage.
553 Tom_Gilb: oops 545 why can't you add my statistics suggestion for inspections, it is a dominant point!
554 Laurie: 528+
555 Moderator: Time is running out. The plan is to switch topic soon, discuss next topic for 10-15 minutes and then wrap-up
556 Scott_Ambler3: 550- Inspections is one way, not the only way
557 Karl_Wiegers: 546 This gets to the point that all "good practices" are culture-dependent. Sometimes, though, you need to change the culture if it isn't getting the results you need!
558 Bill Krebs: 550- I'm not convinced inspections are better than pairing, only that it's hard to measure quality data for pairing
559 Scott_Ambler3: 557+
560 Laurie: 558 To back this up, we need data. Wanna volunteer for another study, Bill

561 Dieter_Rombach: 528: This is exactly the point. How do you suggest to add this practice to regular CMM style processes?
562 BernhardRumpe: 558 I can enfore more inspections, but not more pairing. Inspections scale
563 lanubile: 556+ yes, but actually there are multiple ways to run inspections: e.g., with or without meeting
564 barry: 545 Outside of defect reduction, most data so far indicate that pair programming is also a way to trade extra cost for reduced schedule, which is often valuable in itself.
565 Bill Krebs: 562 true
566 Tom_Gilb: 555 what was next topic?
567 Pat: 551: It is on the 6th bullet in the issues for comparison. You may need to scroll to see it... It's there. It has also been added to the inspections strengths.
568 Scott_Ambler3: 561 what is in the CMM(I) today that is preventing you from adopting PP, model with others, ...
569 Christian_Denger: 550: Could you elaborqate on the level of quality you can achieve?
570 Scott_Ambler3: 564+
571 Tom_Gilb: 553 good, edit Give to gives later!
572 Winsor: 556 This shows there is not a shared definition for "Inspections" in this group, and therefore throws into question a lot of the discussion.
573 Karl_Wiegers: 561 Educate practitioners about PP, inspections, and everything else professional SW developers need to know. Have people choose the right hammer for each kind of nail. But we need data to tell good practices from seems-like-it-should-be-good practices.
574 Bill Krebs: 564++
575 Bill Krebs: 573+
576 Tom_Gilb: 556 + agree there is huge variation in Inspections
577 Karl_Wiegers: 568 Nothing that I know of...
578 Scott_Ambler3: 562 I'm not so sure. Why can't you motivate more pairing? Also, if you try to "enforce" inspections there are many ways to avoid them
579 Bill Krebs: 578 You need to make either inspections, pairing, or hybrid your team's culture. Adherence or 'buy in' is key.
580 Winsor: 567 I do not see a 6th "bullet", even when I scroll the white board. Where are the "issues for comparison"???
581 Scott_Ambler3: 579+
582 BernhardRumpe: 578 The point here is I think that if people pair 100% pairing is at its limit. The resulting defect rate cannot be reduced through further pairing (alone) -- but through additional inspections, as they are done post-construction.
583 Karl_Wiegers: 582 But maybe defect rates could be further reduced by using other quality tools, like static and dynamic code analyzers. I don't see these tools being used much either. Maybe a related topic for another discussion sometime.
584 barry: 573+ CMMI generalized the Peer Revies process area to "Verification," which should include all forms of defect reduction
585 BernhardRumpe: 583+ fully agreed
586 Scott_Ambler3: 582 Yes. But inspections aren't your only option. And it would be interesting to measure the "tolerance" that PPers have for inspections of their code.
587 Arisholm: 564 Good point, and for this reason a comparison of inspections versus pair programming is biased because it does not account for other potential benefits of pair programming.
588 Tom_Gilb: 582 and in fact to further reduce defects you must apply defect prevention process, including p