Portfolios in medical education

why do they meet with mixed success? A systematic review

DRIESSEN, ERIK
VAN TARTWIJK, JAN
VAN DER VLEUTEN, CEES
WASS, VAL

¹Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands
²Leiden University Graduate School of Teaching, Leiden University, Leiden, The Netherlands
³Community-based Medical Education, University of Manchester, Manchester, UK

Correspondence: Erik Driessen, Department of Educational Development and Research, Maastricht University, 6200 MD Maastricht, The Netherlands. Tel: 00 31 43 388 5774; Fax: 00 31 43 388 5779; E-mail: [email protected]

Received 14 August 2007; accepted for publication 20 August 2007

Medical Education 41(12):p 1224-1233, December 2007.

Context

The move towards competence-based medical education has created a need for instruments that support and assess competence development. Portfolios seem suitable but mixed reports of their success are emerging.

METHODS

To examine the effectiveness of portfolios, we searched PubMed and EMBASE using the keyword ‘portfol*’, PsychInfo and ERIC using the keywords ‘portfol*’ and ‘medical education’ and references of retrieved papers for empirical studies on portfolios in all phases of medical education. Thirty of 1939 retrieved papers met the inclusion criteria and were analysed. Data were collated against the research question, number of subjects, design, setting, findings and limitations, purpose and content, mentoring and assessment. We analysed impact using a modified version of Kirkpatrick's hierarchy.

RESULTS

Because differences across studies precluded statistical meta-analysis, the data were analysed by context, goals and procedure. Positive effects were strongest in undergraduate education. Important factors for success were: clearly communicated goals and procedures; integration with curriculum and assessment; flexible structure; support through mentoring, and measures to heighten feasibility and reduce required time. Moderately good inter-rater reliability was reported and global criteria and discussions among raters were beneficial. Formative and summative assessment could be combined. Without assessment, portfolios were vulnerable to competition from other summative assessment instruments.

Conclusions

For portfolios to be effective in supporting and assessing competence development, robust integration into the curriculum and tutor support are essential. Further studies should focus on the effectiveness and user-friendliness of portfolios, the merits of holistic assessment procedures, and the competences of an effective portfolio mentor.