A New Approach to Integrating Data From Multiple Informants in Psychiatric Assessment and Research

Mixing and Matching Contexts and Perspectives

Kraemer, Helena C. Ph.D.
Measelle, Jeffrey R. Ph.D.
Ablow, Jennifer C. Ph.D.
Essex, Marilyn J. Ph.D.
Boyce, W. Thomas M.D.
Kupfer, David J. M.D.

Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, Calif.; the Department of Psychology, University of Oregon, Eugene, Ore.; the Department of Psychiatry, University of Wisconsin, Madison, Wisc.; the School of Public Health and the Institute of Human Development, University of California, Berkeley, Calif.; and the Department of Psychiatry, Western Psychiatric Institute, University of Pittsburgh, Pittsburgh.

Address reprint requests to Dr. Kraemer, Department of Psychiatry and Behavioral Sciences, Stanford University, 401 Quarry Rd., MC 5717, Stanford, CA 94305; [email protected] (e-mail).

Supported by the John D. and Catherine T. MacArthur Foundation Research Network on Psychopathology and Development (Dr. Kupfer, Chair) and NIMH grants MH-44340 and MH-52354 (Dr. Essex).

The authors thank Drs. David Offord, Alan Kazdin, Phil Cowan, Carolyn Cowan, and the other members of the John D. and Catherine T. MacArthur Foundation Research Network on Psychopathology and Development for their contributions.

Received Jan. 31, 2002; revision received Jan. 15, 2003; accepted Feb. 2, 2003.

American Journal of Psychiatry 160(9):p 1566-1577, September 2003.

Objective

When there exists no single source of information (informant) to validly measure a characteristic, it is typically recommended that data from multiple informants be used. In psychiatric assessment and research, however, multiple informants often provide discordant data, which further confuse the measurement. Strategies such as arbitrarily choosing one informant or using the data from all informants separately generate further problems. This report proposes a theory to explain observed patterns of interinformant discordance and suggests a new approach to using data from multiple informants to measure characteristics of interest.

Method

Using the example of assessment of developmental psychopathology in children, the authors propose a model in which the choice of informants is based on conceptualizing the contexts and perspectives that influence expression of the characteristic of interest and then identifying informants who represent those contexts and perspectives in such a way as to have the weaknesses of one informant canceled by the strengths of another.

Results

Applications of this approach to several datasets indicate that when these principles are followed, a more reliable and valid consensus measure is obtained, and failure to obtain a reliable, valid measure is indicative of some deviation from the principles.

Conclusions

In obtaining a consensus measure, the issue is not determining how many informants are needed but choosing the right set of informants.