Trends Med Educ Research.pdf
Documento PDF (81.3KB)

Exibir conteúdo Baixar

                    S P E C I A L

T H E M E

A R T I C L E

Trends in Medical Education Research
Glenn Regehr, PhD
ABSTRACT
The medical education community is reflecting increasingly on the role and nature of research in the field. Useful
sources of data to include in these reflections are a description of the topics in which we are investing our
energies, an analysis of the extent to which there is a sense
of progress on these topics, and an examination of the
mechanisms by which any progress has been achieved.
This article presents the results of a thematic review of the
medical education research literature in four key journals
since the turn of the 21st century. It describes four
examples of areas in which the community appears to be
investing its energies: curriculum and teaching issues,
skills and attitudes relevant to the structure of the profession, individual characteristics of medical students, and

I

the evaluation of students and residents. A discussion of
the recent publications in these domains highlights a
distinction between thematic categories of research, in
which many members of the community are working on
the same topic, and programmatic lines of research, in
which members of the community are working together
toward the shared goal of consensual understanding. The
author suggests that community-level, programmatic lines
of research are necessary to build knowledge and understanding of a domain and that, in the absence of such
communal effort, the value of research is limited to the
uncoordinated accrual of information.
Acad Med. 2004;79:939 –947.

n recent years, the medical education community has
been reflecting increasingly on the role of research in
the field. Among the issues associated with this reflection is a concern that the field is not developing
systematic or productive programs of research and, therefore,
does not seem to be advancing on “big questions.” One of the
reasons proposed by those who perceive this as a problem is
the absence of research studies that are motivated and
informed by useful theories. For example, Prideaux and
Bligh1 have suggested that one consequence of not locating
our work in a theoretical context has been the difficulty of
aggregating findings into consistent themes. Making a
slightly different argument, Colliver2 has suggested that the
theories we are using in the field are too weak to be productive. Whether there is a lack of “advancement” in our field
and whether the absence of “functional” theories is an
important contributor to this phenomenon requires further

reflection. One process for shedding additional light on this
issue would be to look at the areas where we as a community
have recently invested our research and development energies, to explore the extent to which these energies have led
to a sense of “advancement” in the field, and to examine the
mechanisms by which advancement has been achieved.
The first step in such a process is an examination of our
literature to provide insight into the domains where we are
investing our energies. This article, therefore, is an effort to
call our attention, as a community, to the question of what
we are doing. It is a preliminary effort to describe some of the
areas of thematic research in our literature as they have
manifested in the last several years, to explore how these
thematic areas have faired in advancing the field, and to
examine the extent to which theory has played a role.

Dr. Regehr is the Richard and Elizabeth Currie Chair in Health Professions
Education Research, associate director, Wilson Center for Research in Education, and professor, Department of Surgery, Faculty of Medicine, University of Toronto, Ontario, Canada.

Many research articles have been generated in the field of
medical education since the turn of the century. It would be
impossible to cite or categorize them all, even if it were
possible to find and read them all. Thus, this exercise focused
on only four of the journals that are central to the medical
education research enterprise: Academic Medicine, Advances

Correspondence should be addressed to Dr. Regehr, Wilson Center for
Research in Education, 200 Elizabeth Street, Eaton South 1-565, Toronto,
Ontario, Canada M5G2C4; e-mail: 具g.regehr@utoronto.ca典.

MEDICAL EDUCATION RESEARCH THEMES AND TRENDS

ACADEMIC MEDICINE, VOL. 79, NO. 10 / OCTOBER 2004

939

TRENDS IN MEDICAL EDUCATION RESEARCH, CONTINUED

in Health Sciences Education, Medical Education, and Teaching
and Learning in Medicine. These journals were selected for a
variety of reasons: they are explicitly dedicated to medical
(or health professional) education questions, but are generalist within this mandate (i.e., they focus on all aspects of the
medical education enterprise rather than specializing in an
area such as continuing education alone); they are research
focused; and the editors of each of these four journals have
been active and vocal in their pursuit of promoting and
shaping the face of medical education research.3–9 From my
review of the research articles in these journals since 2000,
many themes and trends emerged. The themes highlighted in
the following sections are some of the most populated,
suggesting that these are key domains in which the community has been investing its energies.

Applied Curriculum and Teaching Issues
Perhaps not surprising for an applied domain such as medical
education, a substantial number of research articles are dedicated to discussing and exploring applied issues of curriculum and teaching. The questions in this theme address
curriculum issues across a wide continuum, from large-scale
curriculum evaluation, through the description and evaluation of specific aspects of curricula, to implementations of
teaching strategies and techniques, and finally to descriptions
and evaluations of highly specific content-based courses or
workshops in the context of curricula.
At the level of large-scale curriculum evaluation, for
example, there are continuing questions about the relative
strengths and weaknesses of problem-based learning (PBL),
with continuing efforts to understand how it influences
learning and under what conditions it might be successful.
As just a few examples, Curet and Mennin10 addressed the
highly practical question of the influence of using long-term
versus short-term tutors in the PBL tutorial setting, and De
Grave et al.11 explored the impact of critical incidents in
PBL tutorial groups, how students interpreted these events,
and what they learned as a result. At a more theoretical level,
van den Hurk et al.12 developed and tested a causal model for
learning in a PBL curriculum as part of a research program
designed to increase our understanding of the critical factors
in making PBL successful. However, PBL is not the only
target of large-scale curriculum evaluation studies. Other
important questions include the integration of preclinical
and clinical training in medicine (with a frequent focus on
the integration of basic sciences13–15) and the shift from
bedside teaching to ambulatory or community teaching settings in clinical training.16,17 Although the exact target of
these curriculum-level studies varies, an important subset
of the studies has attempted not only to provide examples of

940

how a curriculum can be implemented, but also to inform our
understanding of how a curriculum can be shaped by the
interrelation among theoretical and practical factors. In
doing so, they have informed both theory and practice. In an
excellent illustration of theory informing practice and practice informing theory, Miflin et al.18 demonstrated and described how a facile interpretation and inappropriate implementation of the concept of self-directed learning can
undermine the development of an effective PBL curriculum.
Other studies focus not on the broad curriculum as a
whole, but on the integration and implementation of curriculum-level educational strategies or programs. One of the
largest and fastest-growing issues at this level of curriculum
design and evaluation is the integration of simulation into
the curriculum as a mechanism for teaching without direct
contact with patients. The types of simulation being addressed range widely, including standardized patients,19
bench models,20 and virtual reality simulators.21 However,
all these studies are focused on understanding the relative
strengths and weaknesses of simulation and its place in the
larger curriculum. A second, similarly active topic in the
domain of curriculum-level education strategies is the use of
technology as a vehicle for curriculum delivery. Broadly,
these technology-based instruction models involve the use of
synchronous visual mechanisms such as videoconferencing,22
asynchronous Web-mediated collaborative learning environments,23 or individual-access CD-ROM or Web-based databases of educational information.24 But, again, an important
subset of these studies has attempted to address this issue by
asking not merely whether technology “works” as an educational medium, but also how technology can be integrated
effectively with the larger goals and structures of the curriculum.
More specific but still theoretically driven programs of
research in this area address more localized content delivery
questions. Many of these content delivery questions are tied
to specific skills to be learned but often have broader implications for the theory of learning and the practice of education. In the tradition of cognitive psychology, researchers
have addressed issues such as the relative value of mixed
versus blocked practice25 or the relative value of using
complete clinical vignettes versus chief complaints26 when
teaching diagnostic skills to students. In a tradition more
grounded in the marriage of sociology and education, researchers have addressed issues such as the impact of integrating real (HIV) patients into preclinical learning groups27
and the use of team learning in the context of an evidencebased medicine course.28
More narrowly focused questions in this area provide
examples of highly specific educational implementations for
the delivery of highly specific content areas. Here the questions are not so much addressing a theoretical question of the
best way to deliver content; rather they ask whether local

ACADEMIC MEDICINE, VOL. 79, NO. 10 / OCTOBER 2004

TRENDS IN MEDICAL EDUCATION RESEARCH, CONTINUED

efforts at a particular content delivery mechanism were
successful in delivering a particular content. Studies of this
type are quite ubiquitous, and a few examples include the
delivery of content such as spiritual history-taking,29 domestic violence,30,31 geriatrics assessment,32 attitudes toward
substance-abusing patients,33 cardiology skills,34 lesbian and
gay health care,35 the thyroid examination,36 end-of-life
care,37 deafness and hearing impairments,38 the pelvic examination,39 and microscopic urinalysis.40 In most studies of
this type, the authors provide an example of how they teach
a particular content domain, offering an implied invitation
to readers to use a similar model. Thus, these studies are
relevant to the larger community of educators and researchers to the extent that the particular content domain is seen
as important and as either lacking or poorly taught in the
reader’s own curriculum.
Finally, related to this last category of questions is a set of
studies designed to determine the general level of knowledge
of students or residents in highly specific content domains.
Here, there is often no implementation of a content delivery
mechanism to be evaluated. Rather, these studies attempt
broad-scale evaluations of curricula in general for their success in teaching specific skills or knowledge, such as interns’
prescribing for common clinical conditions,41 graduating
nurses’ medication calculation skills,42 family medicine residents’ ability to interpret electrocardiograms,43 or medical
students’ skill sets in addressing domestic violence.44 Again,
the content domains being addressed in these large-scale
needs assessments are often quite specific, and they function
either implicitly or explicitly as a call to the community to
focus greater attention on the delivery of this particular
content. The relevance of the study’s findings and conclusions for the larger community of educators is largely dependent on the relative importance that individuals within the
community place on these content domains compared with
the vast number of other content domains that could be
examined in this way.

Skills and Attitudes Relevant to the Structure of the
Profession
A second large theme of research questions in medical
education addresses the underlying skills and attitudes that
form the substrate of professional competence. These “core
competencies” appear to be quite similar to, if not always
exactly synonymous with, those elaborated by the Accreditation Council for Graduate Medical Education (ACGME)
and by the Royal College of Physicians and Surgeons of
Canada (RCPSC) in its CanMeds roles. These competencies
have generally been seen as important for some time, but
more recently they have been the focus of increasing study

and have been accorded the concentration of the medical
education community’s resources.
One area where a substantial number of educators and
researchers have focused their collective efforts is on developing an understanding of “professionalism” in its broadest
terms. Many are asking questions such as, What do we think
professionalism is?45 What do our trainees think it is?46 – 48
How do we teach it or instill it?49 –51 How do we measure
it?52–54 As a group, the researchers and educators who have
chosen to work in this area appear to have formed a loose
consortium, a microcommunity within medical education,
who meet, share their ideas, pool resources, and work toward
a conceptualization of professionalism that is coherent and
broadly accepted as valuable and meaningful to the community at large.
Similar concerted community efforts are being directed at
several specific skills and attitudes that compose the professional substrate. Although they appear somewhat less developed as microcommunities, there are growing pools of researchers, educators, and administrators who are focusing
their time and effort on the understanding, teaching, and
evaluation of domains such as self-directed learning,55,56
self-assessment,57– 62 and interprofessional collaboration and
communication.63– 68

Students’ Characteristics
A somewhat less integrated but nonetheless identifiable area
of research is in the domain of students’ characteristics and,
frequently, the interactions of these characteristics with
curricula and performance. From the psychological and
learning perspective, the focus has tended to be on individual
differences in learning styles69 –72 and motivation.73 These
studies generally are fueled by the theory-based assumption
that there are stable characteristics (or aptitudes) in individuals, and that researchers are motivated by the search for
evidence of “aptitude by treatment interactions,” such as the
interaction between learning style and content delivery, that
are predicted by the theory. Although the evidence for these
interactions has been somewhat elusive, their demonstration
would offer both support for the theory and practical implications for education.
From a more sociological perspective, researchers have
addressed issues of systematic differences in medical school
experiences as a function of many individual characteristics
including age74,75 gender,74 –79 and race or ethnicity.80,81
These studies often highlight the systemic barriers to developing and maintaining diversity in the medical curriculum
broadly and/or within specific programs within medicine.
An issue obviously related to the general topic of students’
characteristics is the question of medical school admission

ACADEMIC MEDICINE, VOL. 79, NO. 10 / OCTOBER 2004

941

TRENDS IN MEDICAL EDUCATION RESEARCH, CONTINUED

policies.82,83 Again, the questions being asked in this domain
generally separate into two broad categories. The first and
larger of these addresses efforts to increase the reliability and
predictive validity of admissions policies (that is, selecting
students who will be successful).84 – 89 The second but equally
important category addresses the continuing effort to increase access to medical school for underrepresented populations.90 –92 Also related to these topics are the issues of
career choice and practice patterns, with studies addressing
the individual and social factors that influence decisions such
as trainees’ selection of a specialty93–96 or practice setting.97–99

Evaluation of Individuals
Of course, no discussion of research themes in medical
education could ignore the massive effort being exerted in
the field of measurement and testing. Student evaluation has
long been a staple of medical education, and, if anything,
attention to it is being accelerated by political forces such as
the ACGME’s and RCPSC’s shifts in policy from accreditation requirements based on education processes to requirements based on learners’ outcomes. In short, these organizations have made a clear statement that they will no longer be
satisfied with evidence that programs are teaching certain
content; they are now interested in evidence that the students are learning it. This mandate has not so much altered
the direction of measurement in medical education as it has
fueled changes that were already evolving.
Thus, increasingly, there is an emphasis on authentic,
performance-based assessments of clinical competencies,
with these assessments occurring both in the clinical setting
and in the simulated environment. Within the clinical
setting, continuing efforts are being made to redress past
concerns about the reliability and validity of end-of-rotation
clinical marks.100 Many researchers are attempting to understand the cognitive, social, and environmental influences
that affect the reliability and validity of longitudinal clinical
ratings.101,102 Many others are exploring mechanisms to
improve these clinical ratings, such as the use of formal
evaluation sessions103 and the use of multiple, independent
perspectives in the evaluation process.104,105 Still others are
attempting to address the problem by shifting the focus of “in
vivo” clinical evaluation away from longitudinally based
assessments to more localized, “in-the-moment” assessments
of specific performances using tools such as the mini– clinical
evaluation exercise.106 –108 In addition, there are efforts to
develop novel tools for assessing clinical competence,109,110
with portfolio assessments becoming increasingly popular.111
Perhaps the greatest amount of energy, however, is being
invested in the use of simulation for the purposes of evalu-

942

ation. As described earlier, these simulations use many media
including the computer interface112,113 and both hightech114 and low-tech115 bench models. But by far, the most
common simulation technology in evaluation is the standardized patient-based examination format widely known as the
objective structured clinical examination (OSCE). Literally
hundreds of articles have been published on the OSCE, and this
concerted effort has led to an extensive understanding of the
technology. We now have a body of literature that provides an
understanding of how to mark it,116 –120 many of the potential sources of error in scores,121–126 methods for setting
passing standards,127–129 potential innovations in its administration,130 –134 the impact of its use on the educational
milieu,135 as well as a sociological critique of its use.136
Finally, as in the context of educational innovation, there
are individual examples of innovation in evaluation for the
purposes of measuring specific aspects of clinical competence,
such as student reflection,137,138 bedside neurology,139 and
attitudes towards the homeless.140 Again, as in the context of
educational innovation, these tools are useful to the extent
that the community sees the skills as needing explicit assessment and to the extent that individual evaluators within the
community lack their own tools to make these evaluations
(as was clearly the case, for example, with the OSCE).

DISCUSSION
The scope of topics that are addressed by medical education
research is broad. The field’s breadth has the advantage of
providing many opportunities for research and discovery, but
it also risks generating a sense that the field lacks coherence
and communal effort toward the resolution of “big questions.” In addition, the work of researchers and educators in
medical education is broadly published not only across many
journals that would identify themselves as relevant to medical education but also across many clinical journals. This
breadth and scope of publication venues reflects positively on
the state of medical education scholarship. Clearly the messages of medical education are infiltrating many places, to
great benefit. However, again, this breadth does have implications for the community’s ability to locate and integrate
the research into coherent, programmatic efforts. These concerns can only be redressed through constant communal
reflection regarding our choices of topics to address and our
success in addressing them. This article is intended to be one
contribution to this process. The purpose of the exercise was
not to generate a comprehensive literature review of all
medical education research published in the last half decade,
nor to impose a set of themes on the literature that should
considered comprehensive, prescriptive, or definitive.
Rather, this article is intended to function as the early stages

ACADEMIC MEDICINE, VOL. 79, NO. 10 / OCTOBER 2004

TRENDS IN MEDICAL EDUCATION RESEARCH, CONTINUED

of an exercise in community-level reflective practice, to
provide some examples of the content areas where we are
choosing to invest our energies, and to establish the basis of
an explicit dialogue regarding the nature and relative success
of these choices.
Although such dialogues are important, they should not be
limited to debates or discussions of what areas we are studying, which areas are relatively more or less valuable, and who
fits into what categories. Rather, it is equally important to
address why the community might see such an exercise as
valuable and how we could take advantage of the results. In
this sense, taking the initial step of trying to develop this
article has been an important personal learning exercise, in
that it has highlighted for me an important distinction
regarding the pursuit of knowledge-building in a community
of scholars: the distinction between scholars who are working on
the same topic and scholars who are working together toward a
shared goal. It is, to use a common analogy, the difference
between parallel play and interactive play.
During the process of reflecting on the literature, several
themes jumped out as being very apparent. As examples, the
lines of research related to the content area of professionalism and those related to the development of the OSCE were
obvious, at least in part because of their coherence of purpose. For these themes, I suspect that participants would
recognize themselves as part of a community of researchers
with common goals and questions, and would see the “valueadded” in discussing their various ideas with each other. By
contrast, other themes were possible to develop, but seemed
to be substantially less satisfying upon development, more
theoretical than real, more abstract than functional. To
individuals who find their work placed in these categories, I
apologize, as I suspect they feel much like the conference
presenter who has discovered herself to be in the session
entitled “Potpourri.” Although it was possible to see studies
within these themes as “thematic,” they were not in any
practical sense “programmatic.” The problems being addressed by individual researchers were similar at some level,
but as a whole the individual studies did not seem to inform
each other. It is in this sense that these themes were marked
by a quality of institutionalized parallel play. Perhaps this
merely speaks to the weakness of the themes that I created as
an outsider to these domains. However, as described in the
introduction, I am not the first to make such observations.1
Returning to a discussion of the role of theory in determining a sense of coherence, it is not clear, based on this
review, that locating our work in a theoretical framework as
suggested by Prideaux and Bligh1 is either necessary or
sufficient for coherence to emerge in a particular domain. For
example, with a few notable exceptions,137 much of the
OSCE work appears more driven by practical innovation
than by theoretical exploration despite its apparent coher-

ence as a domain of research and development. By contrast,
there are some domains that have theoretical grounding and
intent but nonetheless generate a relatively weak sense (at
least to an outsider) of communal effort. The research on
experts’ cognitive knowledge structures, for example, seems
to have been marked by this quality of institutionalized
parallel play, with many of the community’s members arguing and engaging at conferences, but largely ignoring or
dismissing each other’s data as they develop their own
theories about the domain. Interestingly, despite its previous
prominence in the medical education literature, this field of
work was not a strong presence during the last five years in the
four journals reviewed. It appears that these researchers, to the
extent that they are still engaged in this work, have found other
communities of researchers with whom to interact.
It is possible therefore, that the problem being perceived
by key members of our community is not merely the absence
of effective, guiding theory. Although the absence of theory
in our writings and considerations may be a problem, this
absence may be epiphenomenal to, rather than the cause of,
a larger issue. Perhaps the real issue is the absence of a sense
of community effort to build understanding of the phenomena we care about, and the absence of a community where
data and ideas are not merely described, but listened to, and
not merely dismissed or ignored but addressed, incorporated,
and improved upon by other members of the community.
Community-level consensually mandated directions in research and development arise from a variety of sources:
political institutions, funding agencies, particularly successful
lines of research, and like-minded groups of researchers from
the community of practice who acknowledge each other’s
work and incorporate it into their thinking rather than
working in isolation. But for mandated directions in research
to become programmatically knowledge-building at the community level, all these sources must be present, each making
important and interacting contributions to the effort. As a
medical education community, we have the capacity to affect
each of these aspects. As administrators and lobbyists, we can
work jointly toward the accrual of political and economic
support for key areas of research and development. But as a
community of researchers and developers, we must also do
our part to work programmatically, both individually and as
a community. We must seek out and support individuals who
are pursuing promising ways of thinking and promising lines
of research. And we must work more collaboratively to build
mutual understanding. It is only through such communitybased efforts that we will create a vibrant and evolving field
in which we are not merely accruing information, but building knowledge and understanding about the enterprise of
medical education.

ACADEMIC MEDICINE, VOL. 79, NO. 10 / OCTOBER 2004

943

TRENDS IN MEDICAL EDUCATION RESEARCH, CONTINUED

REFERENCES
1. Prideaux D, Bligh J. Research in medical education: asking the right
questions. Med Educ. 2002;36:1114 –5.
2. Colliver JA. Educational theory and medical education practice: a cautionary note for medical school faculty. Acad Med. 2002;77:1217–20.
3. Whitcomb ME. Research in medical education: what do we know
about the link between what doctors are taught and what they do?
Acad Med. 2002;77:1067– 8.
4. Whitcomb ME. The journal continues to evolve. Acad Med. 2002;77:
3– 4.
5. Norman G. Holding on to the philosophy and keeping the faith. Med
Educ. 2001;35:820 –1.
6. Bligh J. Research in medical education at the start of the century. Med
Educ. 2002;36:1000 –1.
7. Bligh J. Evaluating the effects of research in medical education. Med
Educ. 2001;35:1094.
8. Colliver JA. The research enterprise in medical education. Teach
Learn Med. 2003;15:154 –5.
9. Colliver JA. Constructivism with a dose of pragmatism: a cure for what
ails educational research. Adv Health Sci Educ Theory Pract. 1999;4:
187–90.
10. Curet MJ, Mennin SP. The effect of longterm vs shortterm tutors on
the quality of the tutorial process and student performance. Adv
Health Sci Educ Theory Pract. 2003;8:117–26.
11. De Grave WS, Dolmans DH, Van Der Vleuten CP. Student perspectives on critical incidents in the tutorial group. Adv Health Sci Educ
Theory Pract. 2002;7:201–9.
12. Van den Hurk MM, Dolmans DH, Wolfhagen IH, Van der Vleuten
CP. Testing a causal model for learning in a problem-based curriculum.
Adv Health Sci Educ Theory Pract. 2001;6:141–9.
13. Custers EJ, Cate OT. Medical students’ attitudes towards and perception of the basic sciences: a comparison between students in the old
and the new curriculum at the University Medical Center Utrecht,
The Netherlands. Med Educ. 2002;36:1142–50.
14. Prince KJ, van Mameren H, Hylkema N, Drukker J, Scherpbier AJ,
van der Vleuten CP. Does problem-based learning lead to deficiencies
in basic science knowledge? An empirical case on anatomy. Med Educ.
2003;37:15–21.
15. Rudland JR, Rennie SC. The determination of the relevance of basic
sciences learning objectives to clinical practice using a questionnaire
survey. Med Educ. 2003;37:962–5.
16. Carney PA, Eliassen MS, Pipas CF, Genereaux SH, Nierenberg DW.
Ambulatory care education: how do academic medical centers, affiliated residency teaching sites, and community-based practices compare?
Acad Med. 2004;79:69 –77.
17. Bowen JL, Irby DM. Assessing quality and costs of education in the
ambulatory setting: a review of the literature. Acad Med. 2002;77:621– 80.
18. Miflin BM, Campbell CB, Price DA. A conceptual framework to guide
the development of self-directed, lifelong learning in problem-based
medical curricula. Med Educ. 2000;34:299 –306.
19. Davidson R, Duerson M, Rathe R, Pauly R, Watson RT. Using
standardized patients as teachers: a concurrent controlled trial. Acad
Med. 2001;76:840 –3.
20. Kneebone R. Simulation in surgical training: educational issues and
practical implications. Med Educ. 2003;37:267–77.
21. Bearman M. Is virtual the same as real? Medical students’ experiences
of a virtual patient. Acad Med. 2003;78:538 – 45.
22. Callas PW, Bertsch TF, Caputo MP, Flynn BS, Doheny-Farina S, Ricci

944

MA. Medical student evaluations of lectures attended in person or
from rural sites via interactive videoconferencing. Teach Learn Med.
2004;16:46 –50.
23. Stromso HI, Grottum P, Hofgaard Lycke K. Changes in student
approaches to learning with the introduction of computer-supported
problem-based learning. Med Educ. 2004;38:390 – 8.
24. Treadwell I, de Witt TW, Grobler S. The impact of a new educational
strategy on acquiring neonatology skills. Med Educ. 2002;36:441– 8.
25. Hatala RM, Brooks LR, Norman GR. Practice makes perfect: the
critical role of mixed practice in the acquisition of ECG interpretation
skills. Adv Health Sci Educ Theory Pract. 2003;8:17–26.
26. Nendaz MR, Raetzo MA, Junod AF, Vu NV. Teaching Diagnostic
Skills: Clinical Vignettes or Chief Complaints? Adv Health Sci Educ
Theory Pract. 2000;5:3–10.
27. Solomon P, Guenter D, Salvatori P. Integration of persons with HIV
in a problem-based tutorial: a qualitative study. Teach Learn Med.
2003;15:257– 61.
28. Hunt DP, Haidet P, Coverdale JH, Richards B. The effect of using
team learning in an evidence-based medicine course for medical
students. Teach Learn Med. 2003;15(2):131–9.
29. King DE, Blue A, Mallin R, Thiedke C. Implementation and assessment of a spiritual history taking curriculum in the first year of medical
school. Teach Learn Med. 2004;16:64 – 8.
30. Korenstein D, Thomas DC, Foldes C, Ross J, Halm E, McGinn T. An
evidence-based domestic violence education program for internal medicine residents. Teach Learn Med. 2003;15:262– 6.
31. Haist SA, Wilson JF, Pursley HG, et al. Domestic violence: increasing
knowledge and improving skills with a four-hour workshop using
standardized patients. Acad Med. 2003;78(10 suppl):S24 –S26.
32. Tandeter H, Peleg R, Menahem S, Biderman A, Fried VA. Teaching
geriatric assessment in home visits: the family physician/geriatrician
attachment. Teach Learn Med. 2003;15:123– 6.
33. Christison GW, Haviland MG. Requiring a one-week addiction treatment experience in a six-week psychiatry clerkship: effects on attitudes
toward substance-abusing patients. Teach Learn Med. 2003;15:93–7.
34. Issenberg SB, McGaghie WC, Gordon DL, et al. Effectiveness of a
cardiology review course for internal medicine residents using simulation
technology and deliberate practice. Teach Learn Med. 2002;14:223– 8.
35. McGarry KA, Clarke JG, Cyr MG, Landau C. Evaluating a lesbian and
gay health care curriculum. Teach Learn Med. 2002;14:244 – 8.
36. Houck WA, Soares-Welch CV, Montori VM, Li JT. Learning the
thyroid examination–a multimodality intervention for internal medicine residents. Teach Learn Med. 2002;14:24 – 8.
37. Lorenz KA, Steckart MJ, Rosenfeld KE. End-of-life education using the
dramatic arts: the Wit educational initiative. Acad Med. 2004;79:
481– 6.
38. Lock E. A workshop for medical students on deafness and hearing
impairments. Acad Med. 2003;78:1229 –34.
39. Herbers JEJr, Wessel L, El-Bayoumi J, Hassan SN, St Onge JE. Pelvic
examination training for interns: a randomized controlled trial. Acad
Med. 2003;78:1164 –9.
40. Canaris GJ, Flach SD, Tape TG, Stierwalt KM, Haggstrom DA,
Wigton RS. Can internal medicine residents master microscopic urinalysis? Results of an evaluation and teaching intervention Acad Med.
2003;78:525–9.
41. Pearson S, Smith AJ, Rolfe IE, Moulds RF, Shenfield GM. Intern
prescribing for common clinical conditions. Adv Health Sci Educ
Theory Pract. 2000;5:141–50.
42. Grandell-Niemi H, Hupli M, Leino-Kilpi H. Medication calculation
skills of graduating nursing students in Finland. Adv Health Sci Educ
Theory Pract. 2001;6:15–24.

ACADEMIC MEDICINE, VOL. 79, NO. 10 / OCTOBER 2004

TRENDS IN MEDICAL EDUCATION RESEARCH, CONTINUED

43. Boltri JM, Hash RB, Vogel RL. Are family practice residents able to
interpret electrocardiograms? Adv Health Sci Educ Theory Pract.
2003;8:149 –53.
44. Miller AW, Coonrod DV, Brady MJ, Moffitt MP, Bay RC. Medical
student training in domestic violence: a comparison of students entering
residency training in 1995 and 2001. Teach Learn Med. 2004;16:3– 6.
45. Cruess SR, Johnston S, Cruess RL. “Profession”: a working definition
for medical educators. Teach Learn Med. 2004;16:74 – 6.
46. Rizk DE, Elzubeir MA. Self-reported assessment by medical students and
interns of unprofessional practice. Teach Learn Med. 2004;16:39– 45.
47. Ginsburg S, Regehr G, Lingard L. To be and not to be: the paradox of
the emerging professional stance. Med Educ. 2003;37:350 –7.
48. Goldie J, Schwartz L, McConnachie A, Morrison J. Students’ attitudes
and potential behaviour with regard to whistle blowing as they pass
through a modern medical curriculum. Med Educ. 2003;37:368 –75.
49. Kenny NP, Mann KV, MacLeod H. Role modeling in physicians’
professional formation: reconsidering an essential but untapped educational strategy. Acad Med. 2003;78:1203–10.
50. Baernstein A, Fryer-Edwards K. Promoting reflection on professionalism: a comparison trial of educational interventions for medical students. Acad Med. 2003;78:742–7.
51. Hatem CJ. Teaching approaches that reflect and promote professionalism. Acad Med. 2003;78:709 –13.
52. Fontaine S, Wilkinson TJ. Monitoring medical students’ professional
attributes: development of an instrument and process. Adv Health Sci
Educ Theory Pract. 2003;8:127–37.
53. Arnold L. Assessing professional behavior: yesterday, today, and tomorrow. Acad Med. 2002;77:502–15.
54. Gordon J. Assessing students’ personal and professional development
using portfolios and interviews. Med Educ. 2003;37:335– 40.
55. Lloyd-Jones G, Hak T. Self-directed learning and student pragmatism.
Adv Health Sci Educ Theory Pract. 2004;9:61–73.
56. Harvey BJ, Rothman AI, Frecker RC. Effect of an undergraduate
medical curriculum on students’ self-directed learning. Acad Med.
2003;78:1259 – 65.
57. Tousignant M, DesMarchais JE. Accuracy of student self-assessment
ability compared to their own performance in a problem-based learning medical program: a correlation study. Adv Health Sci Educ Theory
Pract. 2002;7:19 –27.
58. Ward M, Gruppen L, Regehr G. Measuring self-assessment: current
state of the art. Adv Health Sci Educ Theory Pract. 2002;7:63– 80.
59. Biernat K, Simpson D, Duthie E Jr. , Bragg D, London R. Primary care
residents self assessment skills in dementia. Adv Health Sci Educ
Theory Pract. 2003;8:105–10.
60. Reiter HI, Eva KW, Hatala RM, Norman GR. Self and peer assessment
in tutorials: application of a relative-ranking model. Acad Med. 2002;
77:1134 –9.
61. Mattheos N, Nattestad A, Falk-Nilsson E, Attstrom R. The interactive
examination: assessing students’ self-assessment ability. Med Educ.
2004;38:378 – 89.
62. Fitzgerald JT, White CB, Gruppen LD. A longitudinal study of selfassessment accuracy. Med Educ. 2003;37:645–9.
63. Hawryluck LA, Espin SL, Garwood KC, Evans CA, Lingard LA.
Pulling together and pushing apart: tides of tension in the ICU team.
Acad Med. 2002;77(10 suppl):S73–S76.
64. Lingard L, Reznick R, Espin S, Regehr G, DeVito I. Team communications in the operating room: talk patterns, sites of tension, and
implications for novices. Acad Med. 2002;77:232–7.
65. Hawk C, Buckwalter K, Byrd L, Cigelman S, Dorfman L, Ferguson K.
Health professions students’ perceptions of interprofessional relationships. Acad Med. 2002;77:354 –7.
66. Farmer EA, Beard JD, Dauphinee WD, LaDuca T, Mann KV.

Assessing the performance of doctors in teams and systems. Med Educ.
2002;36:942– 8.
67. Reeves S, Freeth D, McCrorie P, Perry D. “It teaches you what to
expect in future . . .”: interprofessional learning on a training ward for
medical, nursing, occupational therapy and physiotherapy students.
Med Educ. 2002;36:337– 44.
68. Gardner SF, Chamberlin GD, Heestand DE, Stowe CD.
Interdisciplinary didactic instruction at academic health centers in the
United States: attitudes and barriers. Adv Health Sci Educ Theory
Pract. 2002;7:179 –90.
69. Lindblom-Ylanne S, Lonka K. Students’ perceptions of assessment
practices in a traditional medical curriculum. Adv Health Sci Educ
Theory Pract. 2001;6:121– 40.
70. Engleberg NC, Schwenk T, Gruppen LD. Learning styles and perceptions of the value of various learning modalities before and after a
2nd-year course in microbiology and infectious diseases. Teach Learn
Med. 2001;13:253–7.
71. Mattick K, Dennis I, Bligh J. Approaches to learning and studying in
medical students: validation of a revised inventory and its relation to
student characteristics and performance. Med Educ. 2004;38:535– 43.
72. Martin IG, Stark P, Jolly B. Benefiting from clinical experience: the
influence of learning style and clinical experience on performance in
an undergraduate objective structured clinical examination. Med
Educ. 2000;34:530 – 4.
73. Perrot LJ, Deloney LA, Hastings JK, Savell S, Savidge M. Measuring
student motivation in health professions’ colleges. Adv Health Sci
Educ Theory Pract. 2001;6:193–203.
74. Haist SA, Wilson JF, Elam CL, Blue AV, Fosson SE. The effect of
gender and age on medical school performance: an important interaction. Adv Health Sci Educ Theory Pract. 2000;5:197–205.
75. Kick S, Adams L, O’Brien-Gonzales A. Unique issues of older medical
students. Teach Learn Med. 2000;12:150 –5.
76. Bienstock JL, Martin S, Tzou W, Fox HE. Medical students’ gender is
a predictor of success in the obstetrics and gynecology basic clerkship.
Teach Learn Med. 2002;14:240 –3.
77. Levy BT, Merchant ML. Differences in clinical experiences based on
gender of third-year medical students in a required family medicine
preceptorship. Acad Med. 2002;77:1241– 6.
78. O’Hara BS, Maple SA, Bogdewic SP, Saywell RM Jr, Zollinger TW,
Smith CP. Gender and preceptors’ feedback to students. Acad Med.
2000;75:1030.
79. de Saintonge DM, Dunn DM. Gender and achievement in clinical
medical students: a path analysis. Med Educ. 2001;35:1024 –33.
80. Gartland JJ, Hojat M, Christian EB, Callahan CA, Nasca TJ. African
American and white physicians: a comparison of satisfaction with
medical education, professional careers, and research activities. Teach
Learn Med. 2003;15:106 –12.
81. Tekian A, Han Y, Hruska L, Krainik AJ. Do underrepresented minority medical students differ from non-minority students in problemsolving ability? Teach Learn Med. 2001;13:86 –91.
82. Elam CL, Stratton TD, Scott KL, Wilson JF, Lieber A. Review,
deliberation, and voting: a study of selection decisions in a medical
school admission committee. Teach Learn Med. 2002;14:98 –103.
83. Searle J, McHarg J. Selection for medical school: just pick the right
students and the rest is easy! Med Educ. 2003;37:458 – 63.
84. Turnbull D, Buckley P, Robinson JS, Mather G, Leahy C, Marley J.
Increasing the evidence base for selection for undergraduate medicine:
four case studies investigating process and interim outcomes. Med
Educ. 2003;37:1115–20.
85. Stratton TD, Elam CL, McGrath MG. A liberal arts education as
preparation for medical school: how is it valued? How do graduates
perform? Acad Med. 2003;78(10 suppl):S59 –S61.

ACADEMIC MEDICINE, VOL. 79, NO. 10 / OCTOBER 2004

945

TRENDS IN MEDICAL EDUCATION RESEARCH, CONTINUED

86. Albanese MA, Snow MH, Skochelak SE, Huggett KN, Farrell PM.
Assessing personal qualities in medical school admissions. Acad Med.
2003;78:313–21.
87. Gilbert GE, Basco WT Jr, Blue AV, O’Sullivan PS. Predictive validity
of the Medical College Admissions Test Writing Sample for the
United States Medical Licensing Examination Steps 1 and 2. Adv
Health Sci Educ Theory Pract. 2002;7:191–200.
88. Kulatunga-Moruzi C, Norman GR. Validity of admissions measures in
predicting performance outcomes: the contribution of cognitive and
non-cognitive dimensions. Teach Learn Med. 2002;14:34 – 42.
89. Eva KW, Rosenfeld J, Reiter HI, Norman GR. An admissions OSCE:
the multiple mini-interview. Med Educ. 2004;38:314 –26.
90. Thomson WA, Ferry PG, King JE, Martinez-Wedig C, Michael LH.
Increasing access to medical education for students from medically
underserved communities: one program’s success. Acad Med. 2003;78:
454 –9.
91. Blakely AW, Broussard LG. Blueprint for establishing an effective
postbaccalaureate medical school pre-entry program for educationally
disadvantaged students. Acad Med. 2003;78:437– 47.
92. Strayhorn G. A pre-admission program for underrepresented minority
and disadvantaged students: application, acceptance, graduation rates
and timeliness of graduating from medical school. Acad Med. 2000;
75:355– 61.
93. Ciechanowski PS, Russo JE, Katon WJ, Walker EA. Attachment
theory in health care: the influence of relationship style on medical
students’ specialty choice. Med Educ. 2004;38:262–70.
94. Owen JA, Hayden GF, Connors AF Jr. Can medical school admission
committee members predict which applicants will choose primary care
careers? Acad Med. 2002;77:344 –9.
95. Ward AM, Kamien M, Lopez DG. Medical career choice and practice
location: early factors predicting course completion, career choice and
practice location. Med Educ. 2004;38:239 – 48.
96. Zun LS, Downey L. Is a third year clerkship in emergency medicine
correlated with a career choice in emergency medicine? Teach Learn
Med. 2004;16:14 –7.
97. Wilkinson D, Laven G, Pratt N, Beilby J. Impact of undergraduate and
postgraduate rural training, and medical school entry criteria on rural
practice among Australian general practitioners: national study of
2414 doctors. Med Educ. 2003;37:809 –14.
98. Woloschuk W, Tarrant M. Does a rural educational experience influence students’ likelihood of rural practice? Impact of student background and gender. Med Educ. 2002;36:241–7.
99. Brooks RG, Walsh M, Mardon RE, Lewis M, Clawson A. The roles of
nature and nurture in the recruitment and retention of primary care
physicians in rural areas: a review of the literature. Acad Med.
2002;77:790 – 8.
100. Speer AJ, Solomon DJ, Fincher RM. Grade inflation in internal
medicine clerkships: results of a national survey. Teach Learn Med.
2000;12:112– 6.
101. Williams RG, Klamen DA, McGaghie WC. Cognitive, social and
environmental sources of bias in clinical performance ratings. Teach
Learn Med. 2003;15:270 –92.
102. Kreiter CD, Ferguson KJ. The empirical validity of straight-line responses on a clinical evaluation form. Acad Med. 2002;77:414 – 8.
103. Battistone MJ, Milne C, Sande MA, Pangaro LN, Hemmer PA,
Shomaker TS. The feasibility and acceptability of implementing formal evaluation sessions and using descriptive vocabulary to assess
student performance on a clinical clerkship. Teach Learn Med. 2002;
14:5–10.
104. Norcini JJ. Peer assessment of competence. Med Educ. 2003;37:539– 43.

946

105. Joshi R, Ling FW, Jaeger J. Assessment of a 360-degree instrument to
evaluate residents’ competency in interpersonal and communication
skills. Acad Med. 2004;79:458 – 63.
106. Durning SJ, Cation LJ, Markert RJ, Pangaro LN. Assessing the reliability and validity of the mini-clinical evaluation exercise for internal
medicine residency training. Acad Med. 2002;77:900 – 4.
107. Holmboe ES, Huot S, Chung J, Norcini J, Hawkins RE. Construct
validity of the miniclinical evaluation exercise (miniCEX). Acad Med.
2003;78:826 –30.
108. Kogan JR, Bellini LM, Shea JA. Feasibility, reliability, and validity of
the mini-clinical evaluation exercise (mCEX) in a medicine core
clerkship. Acad Med. 2003;78(10 suppl):S33–S35.
109. Bulik RJ, Frye AW, Callaway MR, Romero CM, Walters DJ. Clinical
performance assessment and interactive video teleconferencing: an
iterative exploration. Teach Learn Med. 2002;14:124 –32.
110. Olson L, Schieve AD, Ruit KG, Vari RC. Measuring inter-rater
reliability of the sequenced performance inventory and reflective
assessment of learning (SPIRAL). Acad Med. 2003;78:844 –50.
111. Wilkinson TJ, Challis M, Hobma SO, Newble DI, Parboosingh JT,
Sibbald RG, Wakeford R. The use of portfolios for assessment of the
competence and performance of doctors in practice. Med Educ. 2002;
36:918 –24.
112. Lieberman SA, Frye AW, Litwins SD, Rasmusson KA, Boulet JR.
Introduction of patient video clips into computer-based testing: effects
on item statistics and reliability estimates. Acad Med. 2003;78(10
suppl):S48 –S51.
113. Guagnano MT, Merlitti D, Manigrasso MR, Pace-Palitti V, Sensi S.
New medical licensing examination using computer-based case simulations and standardized patients. Acad Med. 2002;77:87–90.
114. Tsai TC, Harasym PH, Nijssen-Jordan C, Jennett P, Powell G. The
quality of a simulation examination using a high-fidelity child manikin. Med Educ. 2003;37(1 suppl):S72–S78.
115. Friedlich M, Wood T, Regehr G, Hurst C, Shamji F. Structured
assessment of minor surgical skills (SAMSS) for clinical clerks. Acad
Med. 2002;77(10 suppl):S39 –S41.
116. Margolis MJ, Clauser BE, Swanson DB, Boulet JR. Analysis of the
relationship between score components on a standardized patient
clinical skills examination. Acad Med. 2003;78(10 suppl):S68 –S71.
117. Wilkinson TJ, Frampton CM, Thompson-Fawcett M, Egan T.
Objectivity in objective structured clinical examinations: checklists
are no substitute for examiner commitment. Acad Med. 2003;78:
219 –23.
118. McIlroy JH, Hodges B, McNaughton N, Regehr G. The effect of
candidates’ perceptions of the evaluation method on reliability of
checklist and global rating scores in an objective structured clinical
examination. Acad Med. 2002;77:725– 8.
119. Solomon DJ, Szauter K, Rosebraugh CJ, Callaway MR. Global ratings
of student performance in a standardized patient examination: is the
whole more than the sum of the parts? Adv Health Sci Educ Theory
Pract. 2000;5:131– 40.
120. De Champlain AF, Margolis MJ, Macmillan MK, Klass DJ. Predicting
mastery level on a large-scale standardized patient test: a comparison of
case and instrument score-based models using discriminant function
analysis. Adv Health Sci Educ Theory Pract. 2001;6:151– 8.
121. Colliver JA, Swartz MH, Robbs RS. The effect of examinee and
patient ethnicity in clinical-skills assessment with standardized patients. Adv Health Sci Educ Theory Pract. 2001;6:5–13.
122. Humphris GM, Kaney S. Examiner fatigue in communication skills
objective structured clinical examinations. Med Educ. 2001;35:444 –9.
123. Wilkinson TJ, Fontaine S. Patients’ global ratings of student competence. Unreliable contamination or gold standard?. Med Educ. 2002;
36:1117–21.

ACADEMIC MEDICINE, VOL. 79, NO. 10 / OCTOBER 2004

TRENDS IN MEDICAL EDUCATION RESEARCH, CONTINUED

124. Chambers KA, Boulet JR, Furman GE. Are interpersonal skills ratings
influenced by gender in a clinical skills assessment using standardized
patients? Adv Health Sci Educ Theory Pract. 2001;6:231– 41.
125. McKinley DW, Boulet JR. Detecting score drift in a high-stakes
performance-based assessment. Adv Health Sci Educ Theory Pract.
2004;9:29 –38.
126. Boulet JR, McKinley DW, Whelan GP, Hambleton RK. The effect of
task exposure on repeat candidate scores in a high-stakes standardized
patient assessment. Teach Learn Med. 2003;15:227–32.
127. Kaufman DM, Mann KV, Muijtjens AM, van der Vleuten CP. A
comparison of standard-setting procedures for an OSCE in undergraduate medical education. Acad Med. 2000;75:267–71.
128. Kramer A, Muijtjens A, Jansen K, Dusman H, Tan L, van der Vleuten
C. Comparison of a rational and an empirical standard setting procedure for an OSCE. Objective structured clinical examinations. Med
Educ. 2003;37:132-9 关published erratum appears in Med Educ. 2003;
37:574兴.
129. Wilkinson TJ, Newble DI, Frampton CM. Standard setting in an
objective structured clinical examination: use of global ratings of
borderline performance to determine the passing score. Med Educ.
2001;35:1043–9.
130. Mavis BE, Ogle KS, Lovell KL, Madden LM. Medical students as
standardized patients to assess interviewing skills for pain evaluation.
Med Educ. 2002;36:135– 40.
131. Sibbald D, Regehr G. Impact on the psychometric properties of a
pharmacy OSCE: using 1st-year students as standardized patients.
Teach Learn Med. 2003;15:180 –5.

132. Humphris GM, Kaney S. The objective structured video exam for
assessment of communication skills. Med Educ. 2000;34:939 – 45.
133. Petrusa ER, Hales JW, Wake L, Harward DH, Hoban D, Willis S.
Prediction accuracy and financial savings of four screening tests for a
sequential test of clinical performance. Teach Learn Med. 2000;12:4 –
13.
134. Fliegel JE, Frohna JG, Mangrulkar RS. A computer-based OSCE
station to measure competence in evidence-based medicine skills in
medical students. Acad Med. 2002;77:1157– 8.
135. Duerson MC, Romrell LJ, Stevens CB. Impacting faculty teaching and
student performance: nine years’ experience with the Objective Structured Clinical Examination. Teach Learn Med. 2000;12:176 – 82.
136. Hodges B. OSCE! Variations on a theme by Harden. Med Educ.
2003;37:1134 – 40.
137. Boenink AD, Oderwald AK, De Jonge P, Van Tilburg W, Smal JA.
Assessing student reflection in medical practice. The development of
an observer-rated instrument: reliability, validity and initial experiences. Med Educ. 2004;38:368 –77.
138. Charlin B, Roy L, Brailovsky C, Goulet F, van der Vleuten C. The
Script Concordance test: a tool to assess the reflective clinician. Teach
Learn Med. 2000;12:189 –95.
139. Millos RT, Gordon DL, Issenberg SB, et al. Development of a reliable
multimedia, computer-based measure of clinical skills in bedside neurology. Acad Med. 2003;78(10 suppl):S52–S54.
140. Lester HE, Pattison HM. Development and validation of the attitudes
towards the homeless questionnaire. Med Educ. 2000;34:266 – 8.

ACADEMIC MEDICINE, VOL. 79, NO. 10 / OCTOBER 2004

947