1 Christian: test
 2 tran: truong - testing testing
 1 Tom_Gilb: OK I am just testing this from another location (London)
 2 Tom_Gilb: Mikael, are you monitoring this? Tom
 
 3 Moderator: 2 Hi Tom, yes I will
 3 Christian_Denger: /homepage http://www.iese.fhg.de/staff/denger
 4 Moderator: Hello everybody and welcome!
 5 Krebs: hi! /nick Bill
 
 8 Moderator: 5 Hi Bill!
 6 Krebs: /nick Bill
 7 Bill: /nick Bill Krebs
 9 Bill Krebs: hello!
 10 Tom_Gilb: hi bill, pleased to meet you. I am downloading paper -20 right now
 
 11 Bill Krebs: 10 Excellant. You may also want to look at 18
 
 12 Bill Krebs: 11 TR-2003-18 is the words and theory, and -20 is the detailed how to
 14 Bill Krebs: 10 Nice to meet you to. If we were in person I'd love to get some autographs
 11 Dieter_Rombach: /homepage http://wwwagse.informatik.uni-kl.de/staff/rombach
 13 Moderator: We're preparing the control room here in Germany and are ready to start in 10 minutes.
 
 141 Frank_Maurer: 13+
 15 Dieter_Rombach: Hi to everyone. How is everyone?
 
 16 Bill Krebs: 15 Ganz gut
 17 Arisholm: Very good, thanks!
 18 Moderator: Hi Karl, welcome to the eWorkshop!
 19 Karl_Wiegers: Good morning from Oregon, everyone.
 20 Tom_Gilb: hei på deg erik vi må treffes i norge en gang! er du interessert i evolusjonære utviklingsmetoder?
 
 22 Arisholm: 20 Jada, jeg har gjort noen små undersøkelser!
 23 Bill Krebs: 20 Ska vi talar Englska eller Svenska?
 25 Arisholm: 20 norsk??
 21 Tom_Gilb: hello Karl good to be with you
 24 Moderator: vi kan ta svenska
 26 Karl_Wiegers: Hey, you guys, some of us less educated types only know one language!
 
 29 Bill Krebs: 26
 35 Scott_Ambler: 26+
 27 Tom_Gilb: skandinaviska?
 28 rumpe: Hello.
 30 manzo: Hello
 31 Laurie_Williams: Good morning.
 32 Scott_Ambler: howdy howdy
 33 tran: good morning all
 34 Tom_Gilb: Good afternoon, it is getting dark in London and I assume is darker in Norway
 
 38 Arisholm: 34 yes, it sure is dark. No snow
 39 Karl_Wiegers: 34: Strange, it's dark in Oregon, also. I wonder if it's light anywhere?
 36 Patricia_Costa: Hello!
 37 Basili: Hello
 40 Moderator: Hello everybody! We're soon ready to start
 41 Tom_Gilb: OK, Just this once we won't do the workshop in Viking language!
 42 Frank_Maurer: Hi everybody
 43 barry: Hi everyone
 44 Filippo_Lanubile: hi everyone
 45 Tom_Gilb: Hi Barry, welcome to California (my home state)
 46 Moderator: Welcome everybody, the meeting is open!
 47 Dieter_Rombach: Welcome to the eWorkshop organized by the competence networks ViSEK (Germany) and CeBASE (USA).
 48 Dieter_Rombach: The goal of ViSEK/CeBASE is to evaluate software technologies in order to understand their strengths and weaknesses, and under what conditions they deliver best results.
 49 Dieter_Rombach: Looking at Code Inspections (CI) and Pair Programming (PP), at a first glance, they seem to have the same common aim of collaboration among developers/reviewers to develop quality software with minimal defects. But one takes place during developing software and the other in reviewing already existing software products.
 50 Keun: Hello everyone..
 51 Dieter_Rombach: The goals of this eWorkshop are to 1) elicit commonalities/differences and relative strengths/weaknesses are, 2) under what conditions you might prefer one or the other, and 3) under what conditions you might merge the two practices and to what gain.
 52 Tom_Gilb: what about non code inspections such as requirements and design inspection that cause 50% of the bugs?
 
 54 Scott_Ambler: 52+
 58 Basili: 52+
 62 Dieter_Rombach: 52: Currently, we focus on code inspections. We will address other kinds of inspections later
 65 Karl_Wiegers: 52 + 59 I agree, but we should perhaps consider that there are multiple types of "peer reviews" besides inspection. Can we agree on terminology before we get too far along? Are we considering that any peer review besides PP is an inspection? I don't think this is the case --
 53 Dieter_Rombach: Let’s start with the first topic: What are Commonalities/Differences and Strengths/Weaknesses of CI and PP? From the pre-meeting feedback (PMF) it seems like everyone agrees that:
 55 Gunjan_Sharman: Hello everyone
 56 Dieter_Rombach: PP is less formal than code inspections, and that PP lacks a third party perspective. Further, there is no documentation of defects etc. in PP. Inspections are more objective (because of the third party perspective). With PP, we can achieve a certain quality level but it is very hard to go beyond this level. With CI it is harder to reach this level, but it is possible to go beyond it.
 
 60 Scott_Ambler: 56: Where is the evidence of this?
 
 66 Dieter_Rombach: 60: This is summary of the pre-meeting; we are looking for evidence from the participants now.
 64 Bill Krebs: 56 There is some formality to pairing in that there is a set 'algorithm' for doing it per Laurie's book. Also, we rotate pairs, iterate, and refactor to address the risk that the first two folks will miss a bug
 57 manzo: Early in my career I was an eager practitioner of inspections, through years of experience in applying them, I came to conclude that they work much better in theory than in practice. I favor PP
 
 61 Laurie_Williams: 57+
 63 Scott_Ambler: 57+
 67 Basili: 57 What are the problems you have had with inspections
 70 Bill Krebs: 57+
 76 Laurie_Williams: 57 When I was in industry, I found that people did not prepare as well as they should prior to an inspection. The inspection seemed more of a technicality -- something that needed to be "checked off." I'm sure our results wouldn't have matched most of the research studies.
 
 83 Scott_Ambler: 76: I've seen a lot of that too, although it doesn't have to be the case. However, if it is happening you really need to step back and ask yourself why you're doing the reviews to begin with? To keep bureaucrats happy? If so, drop the reviews.
 
 89 Karl_Wiegers: 83 I've never worked anywhere that people did reviews just to say they did them. Most people have a sincere interest in making work products better.
 
 112 Scott_Ambler: 89 I unfortunately have seen it in several locations, at least that's how it seemed from the team's perspective. We suggested many other ways to work together but they insisted on reviews.
 101 johnson: 83: - I have worked for a major electronics manufacturer where developers did exactly that: reviews only in order to say they did them (i.e. satisfied the process.)
 87 Basili: 76 I agree but some organizations require to see review times before having the inspection and I htink this is an important part. You can do inspectinos badly as you can do PP badly
 
 92 Bill Krebs: 87+
 90 Bill Krebs: 76 It's possible to do an imperfect job of either method and also to have good results with either.
 81 Christian_Denger: 57: Can you think of any reason why inspections don't work in practice? What are your experiences there?
 
 95 Frank_Maurer: 81: I rarely talked to a developer who was leen on doing an iinspection. I met many developers who love pair programming. So, why not simply accept these preferences
 
 100 Bill Krebs: 95 or start with paing and Formally Inspect key artifacts
 104 Karl_Wiegers: 95 Maybe this is because the specific inspection technique was onerous or the culture was in attack mode, not improve mode. Other peer review techniques might be more appropriate besides inspection.
 
 115 Scott_Ambler: 104+
 59 Dieter_Rombach: Do you agree on this summary?
 68 Scott_Ambler: Whiteboard: I don't think we can agree to that yet, other than perhaps the third point
 
 71 Christian_Denger: 68: What are the issues you cannot agree with?
 69 lanubile: inspection IS A type of peer review
 
 75 Karl_Wiegers: 69 agreed; thre are other types too, like what I call a peer deskcheck -- 1 other person reviewing someone's work on their own. Also walkthroughs, and team reviews less formal than Fagan or Gilb/Graham inspection.
 
 79 lanubile: 75 inspections were proposed because walkthroughs were considered not enough disciplined
 80 Bill Krebs: 75 irregardless of the style I suspect we agree pairing and or inspections alway win over solo.
 
 93 Scott_Ambler: 80: co=operative development seems to work really well for most people, although for a minority of folks it doesn't seem to
 72 Tom_Gilb: I now beleive that Inspection should be used to sample and measure, not to try to clean up. The measurement of major defects is used to decide process exit, and to motivate people to follow the standards used to judge the specification
 73 Arisholm: If you want to compare these two techniques I think it is important to determine what type of "pair" we are considering and what type of inspector. For example, it may not be so useful to use a junior navigator, etc...
 74 rumpe: PP focusses on quality of result, learning by doing, and construction of an artefact. Inspection on quality only. How to incorporate that into the comparison?
 
 77 Basili: 74 Don't you learn how to better construct artifacts from inspection or review?
 
 82 Karl_Wiegers: 77 You can learn how to do better work any time you look over someone else's shoulder, via PP or peer review.
 88 rumpe: 77 Partially true, but the learing effect is better when discussing decisions when made instead of seeing results
 78 Tom_Gilb: My position is that they are two different and complimentary techniques. We need to understand their costs and benefits quantitatively, and their best bractice modes.
 
 84 Christian_Denger: 78+
 85 Arisholm: 78+
 94 Basili: 78+
 96 Karl_Wiegers: 78 +
 97 Laurie_Williams: 78 Can we outline how we can go about doing that?
 
 110 Tom_Gilb: 97 yes. We measure such things as defect finding effectiveness versus time and costs (and more).
 
 113 Laurie_Williams: 110 Can you say more?
 123 Tom_Gilb: 110 yes but this is too hectic a place and there is a lot of literature on measuring inspection and I have some (including yours) measuring PP
 86 Tom_Gilb: Inspections work in practice if done properly, most people fail to use them properly ( example respecting optimum checking speed).
 91 Dieter_Rombach: There seems to be a difference between formal and informal inspections. Is the original hypothesis at least true for comparing PP with formal inspections?
 98 Scott_Ambler: whiteboard: is't the code the documentation of the results?
 99 barry: Outside of defect reduction, most data so far indicate that pair programming is also a way to trade extra cost for reduced schedule, which is often valuable in itself.
 
 111 Bill Krebs: 99 Pairing has other benefits - 'present value of feedback', sharing Cockburn 'micropractices', courage to tackle tough problems
 102 rumpe: How about using PP in general and inspection in critical cases (complex modules, high quality necessary etc.)
 
 105 lanubile: 102+
 106 Karl_Wiegers: 102 +
 107 Dieter_Rombach: 102: We will discuss this later.
 109 Frank_Maurer: 102+
 103 manzo: Also, I’ve found Inspections to be too inefficient (defect yield per staff hour) and perhaps even worse, they slow the project’s natural rhythm requiring much staff energy to regain momentum.
 108 Moderator: We seem to have some consenus around 78: My position is that they are two different and complimentary techniques. We need to understand their costs and benefits quantitatively, and their best bractice modes. We will vote on this.
 114 Moderator: /vote PP and CI are two different and complimentary techniques.
 115 Ralf_Kalmar: [voteyes]
 115 Karl_Wiegers: [voteyes]
 115 Dieter_Rombach: [voteyes]
 115 Bill Krebs: [voteyes]
 115 monvorath: [votenotsure]
 115 Christian_Denger: [voteyes]
 116 Laurie_Williams: +
 117 barry: Another dimension of advantage for PP is that it helps create the shared tacit knowledge that is critical to many agile methods, particularly if pairs recombine across the team.
 
 118 Bill Krebs: 117+
 119 manzo: 117+
 120 manzo: 117+
 121 Frank_Maurer: 117+
 133 Basili: 117 Don't expection teams create shared tacit knowledge?
 
 135 lanubile: 133+
 140 lanubile: 133 one of the benefits from inspection is learning
 144 manzo: 133 Yes, but it's more diffuse, less personal
 145 rumpe: 133 I think not that much, because inspection teams have a specific objective and not a general viewpoint. So their knowledge is specific as well.
 118 Patricia_Costa: [voteyes]
 118 rumpe: [voteyes]
 118 Laurie_Williams: [voteno]
 118 barry: [voteyes]
 118 tran: [voteyes]
 118 alberto_sillitti: [voteyes]
 118 lanubile: [voteyes]
 119 Basili: [voteyes]
 119 Scott_Ambler: [votenotsure]
 122 Scott_Ambler: i question the vote -- there are two issues there
 124 johnson: [voteyes]
 124 manzo: [voteno]
 124 Scott_Ambler: PP and CI are different techniques
 125 Frank_Maurer: [voteyes]
 125 Tom_Gilb: [voteyes]
 125 Moderator: /endvote
 125 125 | 12/16/2003 5:13:18 PM | Vote question: PP and CI are two different and complimentary techniques.
Vote results:
       Yes  (15/19) 79%
        No  (2/19) 11%
  Not Sure  (2/19) 11%
  Total: 19 votes

 126 Scott_Ambler: PP and CI are complementary
 127 Arisholm: When comparing the techniques, one should also consider the complexity of the module being developed or inspected. If there is any truth in existing results from group dynamics, complex tasks are best performed in solo (i.e., inspections) whereas simpler tasks are performed efficiently in groups (i.e., PP)
 128 Basili: For those who voted no, can you explain why?
 129 Laurie_Williams: Can we start listing the PP Strengths and Insp Strengths on the whiteboard? Several have been listed, including 117.
 
 130 rumpe: 129+
 132 Christian_Denger: 129+
 134 Moderator: 129 What strenghts are missing?
 
 137 Laurie_Williams: 134 See 117, 111 for now.
 142 Bill Krebs: 134 Pair Programming: 'quicker feedback cycle, or present value of feedback'. Shared 'micropractices'. courage
 148 Bill Krebs: 134 Good audit trail for inspections
 
 149 lanubile: 148+
 163 Tom_Gilb: 134 Inspection provides long and short term statistics to manage the software engineering process.
 138 barry: 129+; also see 99
 131 Scott_Ambler: we should also consider the ability to abstain from the vote -- if you don't have experience with both techniques then why are you voting?
 136 Karl_Wiegers: I find it frustrating that in so many agile/traditional debates people seem to act as though they represent orthogonal choices. Teams can elect to do both PP and CI as appropriate, rather than rejecting either.
 
 143 Frank_Maurer: 136+
 146 Bill Krebs: 136+
 150 Scott_Ambler: 136 I agree, we should use each technique where/when appropriate. You need to fill your intellectual toolbox with many tools and use them appropriately
 
 156 Christian_Denger: 150+
 139 Tom_Gilb: PP gives immediate feedback early and during, Inspection might wait until too much damage is done.
 
 159 Karl_Wiegers: 139 Only if people wait until a product is "done" to do inspection. Incremental reviews can do a great deal of defect filtering and identifying improvement opportunities for the work yet to be done.
 
 164 Scott_Ambler: 159 The feedback cycle is still much longer than that with PP
 
 168 Tom_Gilb: 164+
 167 Scott_Ambler: 159 The PP feedback cycle is on the order of seconds or minutes
 147 Basili: I would say that shared tacit knowledge is a commonality
 151 Christian_Denger: Whenever we discuss the strength of the different techniques it is important to have some kind of evidance for the claims!
 152 barry: PP is also good for mentoring junior people
 
 153 manzo: 152+
 154 rumpe: 152+
 155 lanubile: 152 you can say the same for inspections
 
 161 barry: 155 true, but i would say not to the same extent
 162 rumpe: 155- inspections need experienced people and young ones don't get the decision making process
 157 Basili: 152 So ae inpsections
 
 166 manzo: 157 True, but in the case of PP the mentoring is more personal and can be more specific... not lost in the group process.
 
 171 lanubile: 166 but for inspections groups are not made up of dozens of people; there can be 3 or four persons in the team
 158 Scott_Ambler: 152 It's good for mentoring "senior" people as well
 160 Bill Krebs: 152, yes, If you know you're doing that. You have to be careful if you think you're paing and find out you're educating. Ouch!
 165 Christian_Denger: 152: From our experience in a smal enterprise where we introduced code inspections we learned that this was one major benefit for the company
 169 johnson: Hypothesis: PP is better for "tactical" quality improvement (how do I most efficiently use Eclipse to debug this particular NPE), while inspection is better for "strategic" quality improvement (look for concurrency/synchronization errors in this package).
 170 Scott_Ambler: I think that a major issue is how you do PP -- if you're not swapping pairs then I would have to counteract this with inspections
 172 Karl_Wiegers: I think we've lost in this discussion the notion of PP lacking the external, 3rd party perspective of a reviewer who isn't immersed in the product, absorbed all its assumptions. An outside perspective (yes, it is slower) often reveals insights that people too close to the work didn't spot.
 
 173 lanubile: 172+
 176 Christian_Denger: 172: I totally agree people who are familiar with their code might miss important issues
 180 Scott_Ambler: 172 that's where the swapping issue comes into play, collective ownership is critical too
 
 184 Bill Krebs: 180+
 182 Laurie_Williams: 172+ This should be a listed Inspection Strength on the whiteboard.
 174 Scott_Ambler: If you're swapping pairs regularly you'll find that the value of inspections plumments dramattically.
 
 177 lanubile: 174 any evidence for it?
 
 183 Scott_Ambler: 177 Only anecdotal, although Laurie?????
 
 189 Laurie_Williams: 183 It would be great to have an experiment of this -- I've asked lots of groups to work with me on this and have never had any takers
 
 196 Scott_Ambler: 189 I would really like to see this issue looked into. Is there any way we can "talk up" stuff like this as a community? I'm personally frustrated with several "critical research needs" that I see, this being one of them.
 175 Laurie_Williams: Philip: I think I remember you wrote a paper that said that 80% (?) of surveyed professionals don't do reviews/inspections. Is that correct? I've always considered Frank's point [95] that many professionals don't like doing inspections and do like pair programming (after adjusting) -- so then the defect removal of inspections actually happen.
 178 rumpe: PP is somewhat "binary": You can use it or not. Inspections instead can be scaled to desired level of quality.
 
 181 Tom_Gilb: 178+
 187 Frank_Maurer: 178: not true. Many companies I work with are not doing PP for all code - I would argue that is bad - but it allows for the scaling that you mention.
 
 190 rumpe: 187 downward, yes, but not beyond level 1
 179 barry: Outside of defect reduction, most data so far indicate that pair programming is also a way to trade extra cost for reduced schedule, which is often valuable in itself.
 185 Dieter_Rombach: Let's organize the discussion according to five topics. Effects on quality, productivity, repeatability, certifiability, and acceptance. Who can provide evidence regarding quality?
 
 192 Arisholm: 185 According to a small pilot experiment we conducted in industry, the pairs did produce solutions that were assesed to have better maintainability than did the solo developers
 194 rumpe: 185 in my experience quality did not come from PP alone, but from additional techniques such as automated tests.
 
 195 manzo: 194++
 198 lanubile: 194 are we talink about PP effects or the cumulative effects of agile practices?
 200 rumpe: 194 PP was e.g. useful to ensure that test coverage was good
 201 manzo: 194 For defect identification, I much prefer automated contract testing and mitigated pair programming–PP only on the more algorithmically or logically complex modules.
 204 Frank_Maurer: 194: yes, that is the problem when you try to isolate individual techniques from holistic approaches.
 
 207 lanubile: 204 when I hear about holistic techniques my hand goes to the gun
 
 216 Frank_Maurer: 207: Usually, mine doe too. But controlled experiments usually focus on individual aspects - and there is lots of anecdotal evidence that you can optimize one aspect by giving up others
 203 Karl_Wiegers: 185 There's ample evidence about the impact of inspections (good ones!) on quality improvements. Good paper in latest issue of Software Quality Professional showing inspections reducing defect leakage to customers from 10.6/KLOC to 0.9/KLOC.
 186 Tom_Gilb: swapping pairs is interesting, but let's get back to software engineering!
 
 188 Karl_Wiegers: 186
 191 Scott_Ambler: 186 Can you elaborate?
 
 193 Tom_Gilb: 191 no! (:
 202 Scott_Ambler: 186 Swapping pairs is critical, IMHO.
 197 Tom_Gilb: The effect on quality for inspection is well documented in extensive literature.
 
 206 Scott_Ambler: 197 The problem that I see with a lot of the "evidence" is that it analyzes things within non-agile environments. this is great, but when the context shifts to agile we need to reconsider the "evidence"
 
 208 Karl_Wiegers: 206 - defects are defects!
 
 212 Scott_Ambler: 208 Yes, but what is the context in which those defects are removed? Change the context and perhaps the technique is no longer as effective
 
 225 Karl_Wiegers: 212 Maybe the point is that PP results in injecting fewer (code) defects, so inspections wouldn't yields as much. I can buy that. But we aso need to address here the fact that PP is pair PROGRAMMING. How about the many defects injected in earlier life cycle activities, like requirements?
 
 227 Scott_Ambler: 225++++++++
 231 lanubile: 225+
 233 Tom_Gilb: 225+
 235 Laurie_Williams: 225 Pair PROGRAMMING is a very bad name. Most people who practice pair programming do it in all phases of development (after requirements probably). In my studies, it is used more often in design and analysis and actually the least during actual coding.
 
 237 Scott_Ambler: 235+
 239 Frank_Maurer: 235+ - pair programing and test driven development are about design - not code only
 240 manzo: 235+
  241 Dieter_Rombach: 235: What do you consider to be the difference between your approach and inspections?
  244 Scott_Ambler: 235: Meta issue -- the community still has itself wrapped around the "XP axle". XP is one of many agile methods, yet it always seems to dominate the discussion. Until we move on I suspect the agile movement is in trouble
  236 Scott_Ambler: 225 In my initial response to this meeting I pointed out that the scope of the discussion was too narrow -- we really need to look at collaborative techniques, PP being one of them. Model With Others from AM is the modeling equivalent to PP, and it offers many of the same benefits as PP does except with respect to modeling
  210 Tom_Gilb: 206+
  211 Christian_Denger: 206: Thats definately true, so we need to come up with hypothesis we could analyze in agile development to get the evidance that the techniques improve quality etc.
  199 Bill Krebs: We told our small team you must either pair or inspect or Justify why you did solo. We got 48 % of the changes paired, 50% solo, 2% informal multi person review. Little Unit test. 2x improvement in quality as compared to earlier days of lower pairing frequency
  205 rumpe: Do we have project reports that used PP in an otherwise NON-agile setting?
  209 Laurie_Williams: The first PP experiment I did was definitely PP in non-agile. It was PP imbedded in the Personal Software Process -- since PSP was a known baseline. It showed statistically significant higher quality. That would have been PP vs. solo desk checking (not inspections).
  213 barry: One advantage of inspections is that you can work on multiple qualities. Perspective-based inspections enable artifacts to be reviewed by experts in safety, usability, performance, etc.
 
  224 Christian_Denger: 213+ And thats one major difference between the techniques as we get other perspectives in.
  214 Dieter_Rombach: Can anyone provide tangible evidence on the impact of PP or inspections on quality?
 
  220 Basili: 214 There are severl dozen papers about the effects of inspections of quality
 
  222 Scott_Ambler: 220+
  242 barry: 220+ Some of the studies indicate that inspections are highly effective for certain classes of defects (e.g., programming blunders, logic errors, interface errors, missing portions) but not so effective for others (e.g., timing, program dynamics, and numerical approximation errors)
 
  245 Dieter_Rombach: 242: Do we have similar evidence for PP?
 
  255 barry: 245 Not that I've seen so far. Laurie?
 
  262 Laurie_Williams: 255 No -- would be good to investigate.
  247 Christian_Denger: 242: We should somehow discuss which types of defects can be detected with PP / inspections and where are the differences
  249 Karl_Wiegers: 242 So maybe we need to figure out what kind of problems are best solved with what kind of techniques, and make sure that professional software developers have them all -- and some judgement -- in their tool kits?
 
  252 manzo: 249 I'm a strong proponent of Contract testing. I think CI and PP take a back seat to this.
  253 Scott_Ambler: 249+
  267 Scott_Ambler: 249: What we really need to do is present the technique, discuss it's advantages and disadvantages, and provide advice for when to use it. I didn't this to some degree regarding database development techniques in Agile Database Techniques (www.ambysoft.com/agileDatabaseTechniques.html). It's pretty hard to do, but doable.
  246 manzo: 220+ I might also say the same for PP
  223 Moderator: 214 Any evidence regarding PP?
 
  226 Frank_Maurer: 223: sure Laurie's work
  229 manzo: 223 My guess is only anecdotal evidence except for ALurie's work
  230 Arisholm: 223 I think both Laurie and myself provided some evidence on PP
 
  234 Moderator: 230 Could you examplify with data?
 
  257 Arisholm: 234 The presentation I had on ISERN'02 shows the data from the pilot experiment
  232 Tom_Gilb: 223 yes I believe I have several papers in my collection , but too hectic to dig up here and now.
  228 Bill Krebs: 214 My data is murky, but we improved quality by 2x for our small team by emphasizing pairing and unit test (over less frequet pairing). The improvement came with little Unit test and 48% pairing
  215 lanubile: inspections has tons of reports on improvement of quality
 
  217 Scott_Ambler: 215 Yes, but within context
  218 Moderator: 215 Can you provide an example?
  219 Tom_Gilb: Keep in mind the recent work published by Cusomano et al in IEEE Software. Conclusion: it is critical which exactly set of techniques that work together and have effects. We cannot judge PP or CI alone, but only in context with the other set of things being done.
 
  221 Scott_Ambler: 219+
  238 rumpe: Evidence for PP: Mine again is only anectdotical, but I felt confident and so did the other team members. However, we also strictly used other techniques and I am totally unsure what combination of techniques gave which effects.
  243 Tom_Gilb: moderator failed to pickup my comment on Inspection strengths that is give a statistical basis for managing the whole software engineering process.
 
  254 Karl_Wiegers: 243 This is fine, but the fact remains that most development teams do have defects in their work that need to be removed. So inspection still has a place as defect removal, until more teams can use them for statistical quality sampling.
  248 rumpe: Some collected evidence: We have surve on 45 projects in XP'02 conference where PP was ranked as rather helpful (I don't recall the details know, see my website)
  250 Bill Krebs: Lots of "Knowledge work" benefits from pairing
  251 Moderator: We're going to do a series of votes to see if there's is any consensus on these issues.
 
  263 Frank_Maurer: 251: Are they also shown that they are more cost effective than PP?
  256 Moderator: /vote Inspection benefits are well documented esp. w/r/t reduced defect slippage.
 
  258 Steve_McConnell: 256 +
  259 Winsor_Brown: 256 What do you mean by "slippage"?
 
  261 manzo: 259 Defect containment I think
  264 Tom_Gilb: 259 I assume schedule delay is meant
  257 Dieter_Rombach: [voteyes]
  257 Karl_Wiegers: [voteyes]
  258 Christian_Denger: [voteyes]
  258 rumpe: [voteyes]
  258 Scott_Ambler: [voteyes]
  258 Ralf_Kalmar: [voteyes]
  258 manzo: [voteyes]
  258 Bill Krebs: [voteyes]
  258 Laurie_Williams: [voteyes]
  258 Tom_Gilb: [voteyes]
  258 Basili: [voteyes]
  259 barry: [voteyes]
  259 lanubile: [voteyes]
  260 Steve_McConnell: [voteyes]
  260 Patricia_Costa: [voteyes]
  260 Arisholm: [voteyes]
  260 Frank_Maurer: [voteyes]
  260 tran: [voteyes]
  260 Moderator: /endvote
  260 260 | 12/16/2003 5:37:12 PM | Vote question: Inspection benefits are well documented esp. w/r/t reduced defect slippage.
Vote results:
       Yes  (18/18) 100%
        No  (0/18) 0%
  Not Sure  (0/18) 0%
  Total: 18 votes

  265 Moderator: /vote The effectiveness of inspections vary by defect type
 
  268 Steve_McConnell: 265 +
  266 Karl_Wiegers: Correction in the white board, under Issues for comparison -- I think Laurie said in 235 that PP was done AFTER reqs, in design.
  268 rumpe: [voteyes]
  268 Tom_Gilb: [voteyes]
  268 Christian_Denger: [voteyes]
  268 Dieter_Rombach: [voteyes]
  268 manzo: [voteyes]
  269 Bill Krebs: [votenotsure]
  269 Karl_Wiegers: [voteyes]
  269 Steve_McConnell: [voteyes]
  269 lanubile: [voteyes]
  269 Basili: [voteno]
  269 Scott_Ambler: [voteyes]
  269 tran: [votenotsure]
  269 Moderator: /endvote
  269 269 | 12/16/2003 5:38:47 PM | Vote question: The effectiveness of inspections vary by defect type
Vote results:
       Yes  (9/12) 75%
        No  (1/12) 8%
  Not Sure  (2/12) 17%
  Total: 12 votes

 
  277 rumpe: 269 I hope we also see the question, whether we believe that "PP benefits reduce defect slippage".
  284 Winsor_Brown: 269 What is "slippage". Please define!!!
 
  287 lanubile: 284 defect slippage refers to how many defects remain undiscovered
 
  293 Tom_Gilb: 287 Opps 'Defect slippage'! Sorry Winsor, I reacted to slippage. A better more concventional term is defect finding effectiveness, or 'estimated remaining defects'.
  316 Winsor: 269 I repeat, What is "slippage". Does anyone have a better definition than Toms?
 
  326 Bill Krebs: 316 I assume 'slippage' means a defect that escapes past the phase in which you want to catch it
 
  329 lanubile: 326+
  330 Dieter_Rombach: 326: Slippage means the percentage of defects not caught by a particular inspection
 
  335 Winsor: 330 Thank you, although I ask again how is that really measured. Don't you mean the percentage of defects found later than the activity-phase in which they were [supposedly] injected?
  354 Winsor: 330 A much better term is used at a least one major corporation site here in the US: "escapes" which is the number of defects that escape from one activity or development process "step" to a subsequent activity/step, where they also include the number of steps (and even the injected/removed step, where appropriate.
  270 Ralf_Kalmar: [votenotsure]
  270 manzo: Bye all... and thank you.
  271 Moderator: /vote PP benefits are well documented esp. w/r/t reduced defect slippage.
 
  272 Steve_McConnell: 271 -
  273 Basili: [voteno]
  273 Bill Krebs: [votenotsure]
  273 Ralf_Kalmar: [votenotsure]
  273 Dieter_Rombach: [votenotsure]
  273 Laurie_Williams: [voteyes]
  273 tran: [voteno]
  273 Patricia_Costa: [voteno]
  273 Christian_Denger: [voteno]
  273 Tom_Gilb: [voteno]
  273 Steve_McConnell: [voteno]
  273 Karl_Wiegers: [votenotsure]
  273 lanubile: [voteno]
  273 rumpe: [voteno]
  273 Moderator: /endvote
  273 273 | 12/16/2003 5:41:06 PM | Vote question: PP benefits are well documented esp. w/r/t reduced defect slippage.
Vote results:
       Yes  (1/13) 8%
        No  (8/13) 62%
  Not Sure  (4/13) 31%
  Total: 13 votes

  274 Arisholm: I am not getting the vote buttons...
 
  275 Patricia_Costa: 274: hit the send button.
  280 Ralf_Kalmar: 274 The moderator broke the vote after a certain amount of time, probably that's why.
  276 Moderator: For those who voted yes, where are the studies where this is documented?
 
  279 lanubile: 276 you run an eworkshop about this
 
  282 Moderator: 279 explain....
  283 lanubile: 276 the results of the eworkshop are reported on www.cebase.org, right vic?
  286 Basili: 276 We ran eWorkshops on agile ingeneral and defect reductin using inspectinos but not on pair programming in particular. The results areon cebase.org
  278 Scott_Ambler2: I suspect that the software is getting overloaded
  281 Tom_Gilb: I did not vote yes, but I am willing to share my collection of PP other peoples research with one central oint for analysis. It is mixed with XP research.
 
  288 Bill Krebs: 281+
  283 Winsor_Brown: [votenotsure]
  285 Scott_Ambler2: Whiteboard: Issues for comparison -- You need to look beyond PP if you want to compare fairly, e.g. include modeling with others. sigh
  289 Moderator: /vote PP benefits reduced defect slippage
 
  290 Steve_McConnell: 289 +
  308 Winsor: 289 Unfortunately, it is not precise since it is "estimated"!
  290 Bill Krebs: [voteyes]
  290 Frank_Maurer: [voteyes]
  290 Russo: [voteyes]
  290 Arisholm: [voteyes]
  291 monvorath: [voteyes]
  291 Steve_McConnell: [voteyes]
  291 Laurie_Williams: [voteyes]
  291 rumpe: [voteyes]
  291 Scott_Ambler2: [voteyes]
  291 Christian_Denger: [votenotsure]
  291 Ralf_Kalmar: [votenotsure]
  291 Patricia_Costa: [voteyes]
  291 Basili: [votenotsure]
  291 Dieter_Rombach: To change the direction of the discussion: Do you see a difference between PP and inspections with respect to repeatability of quality results across projects?
 
  292 rumpe: 291 could you please elaborate on the question?
 
  294 Dieter_Rombach: 292: Assume that you let the same team do a similar task again, what is the likelihood that they deliver the same quality in the same timeframe/effort using SP & CI vs. PP?
 
  298 Scott_Ambler2: 294 Greater with PP I bet
 
  301 Dieter_Rombach: 298: How can you repeat when you exchange people, without having evidence from past projects?
 
  305 Scott_Ambler2: 301 If you completely change the team it will be hard. If you keep many of the team members then they will have built a culture and a way of working that will be much more effective.
 
  307 Basili: 305 That does not allow you to build a predictive model
 
  315 Scott_Ambler2: 307 Why is a predictive model important to you?
 
  333 Basili: 315 One would like to be able to predict cost, quality, scheudle, etc. and that required predictive capbility
  302 Basili: 298 How do you think you would get better repeatability? What kind of model would you build?
  303 rumpe: 298- Inspecations ar emore stable I Bet
 
  306 Scott_Ambler2: 303 Looks like there's a research opportunity there!
  309 Laurie_Williams: 303+
  296 Scott_Ambler2: 291: Inspections might be able to bring increased repeatability across projects, if it's done early enough in the process. Leave it too late and nobody is likely to act on the results
  297 Basili: 291 I think it would be harder to to build predictive models of defect reduction based upon pair programming
 
  299 Laurie_Williams: 297 I think PP is more about defect prevention so harder to quanify.
 
  314 Steve_McConnell: 299 I don't agree that PP is about defect prevention any differently than inspections are about prevention. PP is about shortening the cycle time in detecting the defect to practically zero because the other person in the pair sees the error quickly. That isn't prevention; it's early detection. Inspections are also about early detection, but the feedback cycle is longer. As far as I can tell, which is more cost-effective to attain a specific level of quality is an open question.
 
  317 Arisholm: 314+
  319 Patricia_Costa: 314+
  320 Tom_Gilb: 314 Agreed, the initial reaction is about detection, but i assume it prevents defect injection later in the same and other sessions.
 
  324 Steve_McConnell: 320 Tom, can you share your data about reduced defect injection rates after inspections have been in operation for awhile?
 
  336 Tom_Gilb: 324 Yes. Our industrial data is the , at the individual person level, once the defect-found feedback has worked ( using 'no more than x major defects/page allowed before exit' type motivation) the individual can systematically inject two orders of magnitude fewer defects in their daily work. (Remember Winsor?).
 
  337 Steve_McConnell: 336 To clarify, that data is from using inspections, right?
  346 Tom_Gilb: 336 Yes normal Inspections. My second comment 341 is using sampling inspection ( agile inspections).
 
  349 Christian_Denger: 346: Tom could you please explain what you mean by agile inspections?
 
  366 Tom_Gilb: 349 I have developed them as a subset of normal inspections. They rely on sampling ( a page or so) and extrapolation from found defects to total defects. I can send details in form of slides. Tutorial held at EuroSTAR Amsterdam, Dec.
 
  369 Frank_Maurer: 366: I would like to see the slides
  321 Laurie_Williams: 314 When a pair works together, they brainstorm and negotiate quite a bit (a few seconds/a few minutes) before the keyboard is touched. I feel that is defect prevention.
 
  325 Basili: 321 I fell that you and Steve are using different derinitino sof pair programming
  331 Karl_Wiegers: 314 +
  300 rumpe: 297+ agreed, but I think PP is an optimising strategy in that setting, whereas inspections are not
  304 Tom_Gilb: 297 good point. ! PP is more in a prentative mode! Remember that Inspection can contain CMM 5 Defect Prevention Process integrated ( that is the way I teach/write it).
  293 Dieter_Rombach: [voteno]
  293 tran: [votenotsure]
  295 Moderator: /endvote
  295 295 | 12/16/2003 5:48:16 PM | Vote question: PP benefits reduced defect slippage
Vote results:
       Yes  (10/15) 67%
        No  (1/15) 7%
  Not Sure  (4/15) 27%
  Total: 15 votes

  297 Tom_Gilb: [voteyes]
  310 Moderator: We're going to vote on which of PP and CI is more repeatable
  311 rumpe: It's an interesting question whether predictability is more important than efficiency. It depends on the project context.
 
  312 Bill Krebs: 311+
  313 Scott_Ambler2: 311+
  318 Moderator: /vote PP is more repeatable than CI
 
  328 Winsor: 318: Why are you limiting the vote to Code Inspections? Laurie already commented that she has seen pair approaches used in Analysis and Design.
 
  332 Bill Krebs: 328 I using pairing for everything I can - any intellectual work.
 
  334 Scott_Ambler2: 332+
  321 Dieter_Rombach: [voteno]
  321 Tom_Gilb: [votenotsure]
  321 rumpe: [voteyes]
  321 Frank_Maurer: [votenotsure]
  321 Ralf_Kalmar: [voteno]
  322 Scott_Ambler2: [votenotsure]
  322 lanubile: [voteno]
  322 Basili: [voteno]
  322 Bill Krebs: [votenotsure]
  322 Laurie_Williams: [voteno]
  322 Christian_Denger: [voteno]
  322 Patricia_Costa: [votenotsure]
  322 monvorath: [votenotsure]
  322 Moderator: I'm breaking the vote
  323 Moderator: /endvote
  323 323 | 12/16/2003 5:55:25 PM | Vote question: PP is more repeatable than CI
Vote results:
       Yes  (1/13) 8%
        No  (6/13) 46%
  Not Sure  (6/13) 46%
  Total: 13 votes

  327 Scott_Ambler2: Perhaps we need to discuss what we mean by PP then?
  338 Karl_Wiegers: This is a fine theoretical discussion, but at some point we might discuss how to persuade ordinary practitioners (not just early adopters) to invest in either PP or CI. I find very little of either happening with my consulting and seminar clients. I'm rather fond of reduction to practice.
 
  339 Bill Krebs: 338+
  340 Scott_Ambler2: 338+
 </