Phase 1: Delphi study

James Surowiecki argues that crowds have more wisdom than the individual because, given enough diversity in the members of the crowd, their averaged statements converge closer to the "right answer" than an individual's response [Sur2004]. The Delphi method was developed in the 1940s as a way of finding "the most reliable consensus of opinion of a group of experts" [DH1963]. The method's father, Norman C. Dalkey later formulated the method's predicate as "two heads are better than one, or more generally, n heads are better than one" [Dal1972].

A Delphi study may be characterized as a method for structuring a group communication process so that the process is effective in allowing a group of individuals, as a whole, to deal with a complex problem. It is an instance of moderated communication: a facilitator (or moderator) serves a series of questions to the participants (sometimes referred to as "experts"), who return their answers to the facilitator. The answers are anonymised and collated and returned to all participants, who can then modify their response in the light of the feedback from the previous round. Alternatively, the facilitator may pass out a new set of questions, which have been designed to incorporate the returns from the previous round.

Each of these iterations is also known as the feedback process, which "allows and encourages the selected Delphi participants to reassess their initial judgments about the information provided in previous iterations. Thus, in a Delphi study, the results of previous iterations regarding specific statements and/or items can change or be modified by individual panel members in later iterations based on their ability to review and assess the comments and feedback provided by the other Delphi panelists" [HS2007].

There are three principles inherent in this process [Dal1967]:

Anonymity

a "device to reduce the effect of the socially dominant individual." This features allows participants to voice their opinions without fear of being attacked or ridiculed over them.

Controlled feedback

a "device to reduce noise (among other things)." This feature prevents heated and personal debates among participants and thus helps to keep the study on track.

Statistical "group response"

a number of statistical indices exist which can be used to express the representative group opinion, such as the median, which can be found even without unanimity among the respondents. Thus, this is a "device to reduce group pressure toward conformity."

The Policy Delphi method is a variant of the Delphi technique, whose goal is "not so much to obtain a consensus as it is to establish all the differing positions advocated and the principal pro and con arguments for those positions" [Tur1970]. My research incorporates a Policy Delphi approach.

The use of the Delphi technique is dependent not on the application, but rather on the "particular circumstances surrounding the necessarily associated group communication process" [LT2002]. In summary, the properties of the application which lead to the use of the Delphi method are:

• the problem does not lend itself to precise analytical techniques but can benefit from subjective judgments on a collective basis;
• more individuals are needed than can effectively interact in a face-to-face exchange;
• time and cost make frequent group meetings infeasible;
• the heterogeneity of the participants must be preserved to assure validity of the results, i.e., avoidance of domination by quantity or by strength of personality ("bandwagon effect");
• potential panelists are very familiar with the mode of communication, and the controlled feedback appeals to them, as it gives them the time and opportunity to participate;
• verification cycles to ensure that I correctly understood what each participant meant to communicate boost construct validity;
• the possibility of follow-up interviews, which help gaining a deeper understanding of the issues involved and also add to the validity of the study.

My study seeks to explore complex, interpersonal, and often subjective causes, which call for qualitative analysis and open-ended research. Therefore, I chose the Delphi method, which fits optimally my goal in this context.

The rounds are as follows:

1. collection of factors;
2. definition of a vocabulary of factors;
3. identification of orthogonality in factors.

Phase 2: Application and verification

In the second phase of my research, I seek to test the framework developed thus far. Even though a prescriptive application would be ideal, I leave that for further research as it is outside the scope of my current research. Instead, I intend to descriptively investigate the diffusion of two classes of methods in the Debian Project: package build helpers (debhelper, cdbs, and yada), and patch management and version control systems for package maintenance (dpatch, quilt, SVN, and Git).