Select any filter and click on Apply to see results
Table of Contents
Scientific Thinking and Integrative Reasoning Skills (STIRS): Essential Outcomes for Medical Education and for Liberal Education
The Scientific Foundations for Future Physicians report recognized that evidence-based medicine is central to medical education and the practice of medicine. The report states: “It is essential not only to read the medical and scientific literature of one’s discipline, but to examine it critically to achieve lifelong learning. These activities require knowledge and skills in critical analysis, statistical inference, and experimental design” (Association of American Medical Colleges and Howard Hughes Medical Institute 2009).
There is strong evidence that most medical students and residents are not equipped with the tools to critically read the evidence and apply it to solve clinical or research problems. For instance, investigators from Yale surveyed Connecticut internal medicine residents state-wide and found, in an investigation published in JAMA, that only about 40 percent were able to correctly answer key study design and statistical questions (Committee to Evaluate the USMLE 2008)
If students are to gain competence in the ability to understand study designs and analyze findings, these skills need to be introduced and reinforced through practical application as part of the undergraduate curriculum as well as medical school and residency. The National Board of Medical Examiners has recognized this need and is piloting questions that utilize abstracts of articles and simulated journal advertisements to test understanding of study design and statistics as well as evidence-based problem solving (Windish, Huot, and Green 2007). Significantly, a basic approach to study design and analysis has been adopted by the MR5, the fifth comprehensive review of the MCAT, with the development of a framework for Scientific Inquiry and Reasoning Skills (SIRS) and the integration of this framework across the physical and biological sciences as well as the behavioral and social sciences components of the new MCAT exam.
The SIRS framework comprises the following skills:
- Knowledge of Scientific Concepts and Principles
Recognizing and using fundamental scientific concepts and principles, retrieving information, identifying the relationships between closely-related concepts, and using mathematical equations
- Scientific Reasoning and Evidence-Based Problem Solving
Using scientific principles, models, and theories to describe, explain, and make predictions about natural phenomena; working with theories and models to solve problems; and evaluating arguments about causes and consequences
- Reasoning about the Design and Execution of Research
Evaluating features of research studies to determine if conclusions of the research are warranted, and making predictions based on the features of a research design
- Data-Based and Statistical Reasoning
Using quantitative and qualitative data to describe and explain phenomena in the natural world; explain the relationships between variables; test hypotheses; solve problems; interpret, draw conclusions, and make comparisons; interpret and make predictions from qualitative or quantitative data presented in tables, figures, and graphs
(Association of American Medical Colleges 2011):
The MR5 identified and AAMC has endorsed a more detailed list of capacities that students should gain in order to demonstrate each of the SIRS skills as part of the new MCAT exam. See Figure 1.
Figure 1. Capacities that students should gain to demonstrate SIRS Skills
Skill 1—Knowledge of scientific concepts and principles: Students should be able to
Skill 2—Scientific reasoning and evidence-based problem solving: Students should be able to
Skill 3—Reasoning about design and execution of research: Students should be able to
Skill 4—Data-based and statistical reasoning: Students should be able to
The AAMC MCAT 2015 Preview emphasizes that SIRS is still in the development phase and cannot yet be formulated as a comprehensive list of performance expectations. Consequently, these lists are illustrative, not exhaustive. (Association of American Medical Colleges 2011).
Cross Linking SIRS with Content Exams
SIRS will not form the basis for a separate component of the MCAT—rather its four Skills will be cross linked with the Foundational Concepts and content categories from each of the three content area components of the new MCAT exam. The content areas are
- Biological and Biochemical Foundations of Living Systems
- Chemical and Physical Foundations of Biological Systems
- Psychological, Social and Biological Foundations of Behavior
Each of these three sections will be organized around three dimensions:
- Foundational Concepts—the “big ideas” in the sciences that provide the foundation for learning in medical school;
- Content Categories—the topics and subtopics that are needed to understand the foundational concepts; and
- Scientific Inquiry and Reasoning Skills—the inquiry and reasoning skills that are required to solve scientific problems.
For instance, a question about laboratory studies of radiation exposure might require a student to examine the chemical and physical impacts of radiation on living systems and draw conclusions based on laboratory as well as population data. Such questions would link SIRS Skill 2 (Scientific Reasoning and Evidence-based Problem Solving) and SIRS Skill 4 (Data-based and Statistical Reasoning), with Chemical and Physical Foundations Concept 4 (Complex living organisms transport materials, sense their environment, process signals, and respond to changes using processes understood in terms of physical principles.) and Concept 5 (The principles that govern chemical interactions and reactions form the basis for a broader understanding of the molecular dynamics of living systems.)
In another case, SIRS skills could be tested with a set of questions focused on the relationship between neural-tube defects such as spina bifida and folic acid. For instance, a passage might test principles of folic acid biochemistry using Biological and Biochemical Foundational Concept 1 (biomolecules have unique properties that determine how they contribute to the structure and function of cells and how they participate in the processes necessary to maintain life.) and principles of neurological development using Biological and Biochemical Foundational Concept 2 (highly organized assemblies of molecules, cells, and organs interact to carry out the functions of living organisms). It might also incorporate Psychological, Sociological, and Biological Foundations of Behavior Foundational Concept 9 (social and cultural differences influence well-being) and Psychological, Sociological, and Biological Foundations of Behavior Foundational Concept 10 (social stratification affects access to resources and well-being) to ask students to examine the advantages and disadvantages of options for individual folic acid supplementation of women of childbearing age as well as supplementation of the food supply. SIRS Skill 3 (Reasoning About the Design and Execution of Research), as well as SIRS Skill 4 (Data-Based and Statistical Reasoning), can be cross-linked to issues of neural-tube defects and folic acid by asking students to evaluate the randomized controlled trials as well as observational studies used to demonstrate the effectiveness and safety of increased folic acid intake in reducing neural tube defects.
In examples like those above, SIRS should be viewed as the glue binding the diverse components of the MCAT exam and the competencies needed to prepare for medical school. These skills form a solid foundation on which to build a comprehensive sequenced curriculum in evidence-based medicine in medical school and beyond.
Scientific Thinking and Integrative Reasoning Skills (STIRS): A New Project at AAC&U
Integrating Scientific Inquiry and Research Skills into the MCAT and other preprofessional examinations is both a challenge and an opportunity because higher education remains without a comprehensive approach to teaching such basic skills, including research design and analysis, as part of an undergraduate education.
This issue of Peer Review marks the launch of a new project at AAC&U that incorporates these essential skills into a broader effort to encourage evidence-based thinking as a core outcome of general education. The Scientific Thinking and Integrative Reasoning Skills (STIRS) project aims to help reinvigorate the science and social science components of general education for all students, not just those preparing for the health professions. The STIRS project builds on existing AAC&U efforts, especially Liberal Education and America’s Promise (LEAP), to emphasize the importance of integrating evidence-based thinking into a wide range of disciplines including, but not limited to, the sciences and social sciences.
The STIRS project will expand the basic framework developed as part of the MR5 process to include analytical and logical reasoning and decision-making skills such as those used currently in the LSATs, as well as other standardized examinations, such as the GMAT and GREs. AAC&U will incorporate the STIRS outcomes into its ongoing efforts to help campuses create more intentional pathways between learning goals, curricular designs, and demonstrated student learning. In particular, AAC&U will use STIRS to highlight the integrative power of general education and to help students connect the learning that occurs in general education courses with their majors and with applications to real-world problems and projects.
STIRS will build on significant efforts at AAC&U and across the academy to define and advocate a set of Essential Learning Outcomes that inform curricular designs, pedagogical approaches, and resource allocation across undergraduate education. STIRS will focus important attention on efforts to ensure that all students can
- Use scientific reasoning to gather and evaluate evidence
- Understand how scientific and social science studies are designed and executed
- Understand the implications of design choices
- Use statistical reasoning to evaluate data and use data to communicate effectively
- Make decisions based on analysis of evidence, logic, and ethics.
The draft STIRS curricular framework includes the following skills:
- Scientific Reasoning and Evidence-Based Problem Solving
- Study Design, Execution, and Implications
- Data-Based and Statistical Thinking
- Analytical and Logical Reasoning and Evidence-Based Decision Making
The STIRS framework is further illuminated through the development of “Enduring Understandings” designed to identify the key components that students need to take away from the curriculum, incorporate into their majors, and utilize as part of their continuing education for many years to come. Excerpts from a draft version of “Enduring Understandings” for each of the components of the framework are being developed and are illustrated below in figure 2.
Figure 2. Enduring Understandings of the SIRS Project
The following are examples of the types of enduring understanding that are being developed for each of the components of the AAC&U STIRS project:
Scientific Reasoning and Evidence-Based Problem Solving
Scientific reasoning may be divided into reductionist science and integrative science or systems thinking. Social and behavioral sciences as well as the biological and physical sciences may be structured as either reductionist or integrative sciences with integrative sciences often building on reductionist science. Reductionist science aims to create study and control groups that are as similar as possible except for the factor under investigation. Reductionist science is grounded in disciplinary approaches often determined by the levels of organization of the material and social world. Subatomic and atomic physics, inorganic and organic chemistry, zoology and botany, and population sciences such as epidemiology all may take a reductionist approach focusing on one relationship or association at a time.
Study Design, Execution, and Implications
Study designs may be divided into experimental and observational. In experimental designs the investigator intervenes to change the conditions and compare the outcomes in the intervention or study group(s) compared to outcomes in the control group(s) without the intervention. In observational studies the investigator observes the occurrence of events without intervening. The investigator may begin by identifying an independent variable and observing the subsequent occurrence or lack of occurrence of the outcome measured by the dependent variable. Alternatively, the investigator may first identify the occurrence or lack of occurrence of the outcome measured by the dependent variable and look back in time to identify the occurrence or absence of occurrence of the factor measured by the independent variable. Observational studies are potentially capable of establishing the first two criteria of contributory cause or efficacy: (a) that there is an association between the independent and the dependent variable at the individual level and (b) that the “cause” precedes the “effect.” Experimental interventions are often required to definitively establish the third criterion, namely that altering the “cause” alters the “effect.”
Data-Based and Statistical Reasoning
Statistical analysis aims to draw conclusions about large groups or populations based on investigations of smaller groups called samples. Samples are designed to be representative of the larger population of interest. The type of statistical analysis that is conducted depends on the goals of the analysis. Potential goals of statistical analysis include estimation of the magnitude or strength of the relationship, statistical significance or inference, adjustment or taking into account potential confounding variables, and prediction or prognostication. The specific statistical methods used also depends on the type of data used to measure the independent and dependent variables, the distribution of the data, and the frequency of measurement of the dependent or outcome variable.
Analytical and Logical Reasoning and Evidence-Based Decision Making
Analytical reasoning requires understanding of the structure of relationships and drawing logical conclusions based on that structure. Logical reasoning includes the analysis and critical evaluation of statements or arguments and the drawing of well supported conclusions. Reasoning by analogy, determining how additional evidence affects an argument, applying principles or rules, and identifying flaws in arguments are all key skills in critical analysis.
Excerpted from the draft of enduring understandings, developed for components of the AAC&U STIRS project.
STIRS project participants will
- Develop and sustain a community of educators committed to teaching STIRS skills to all students as part of an integrative approach to general education.
- Encourage, review, and reward the development of curricular modules that assist in teaching and assessing components of the STIRS framework through peer review recognition and the designation of “STIRS Scholars”
- Utilize existing AAC&U VALUE Rubrics as an instrument to assess institutional achievement of the STIRS framework.
For more information, or to get involved in the STIRS project, contact Kevin Hovland at email@example.com.
The changes in the MCAT set the stage for broader reform in general education, as well as in science and social science majors, that takes an integrative approach to evidence-based thinking. These skills have long been valued as part of liberal education but have not yet been integrated as core skills for all undergraduates. The AAC&U STIRS project aims to take the next steps by developing a curriculum framework encouraging a wide range of faculty to submit curricular materials that may be used to implement the framework. Peer review of the curricular materials and recognition of excellence by designation of “STIRS Scholars” will move scientific thinking and integrative reasoning skills to the center of conversations about the essential learning outcomes associated with general education and liberal learning. Consequently, higher education will be better able to equip students with key skills they need to understand—and perhaps begin to solve—complex, real-world problems.
To receive a complete set of draft enduring understandings and provide to feedback, please contact Richard Riegelman at firstname.lastname@example.org
Association of American Medical Colleges. 2011.Preview Guide for MCAT2015. Washington, DC: Association of American Medical Colleges.
Association of American Medical Colleges and Howard Hughes Medical Institute. 2009. Scientific Foundations for Future Physicians. Washington, DC: Association of American Medical Colleges.
Committee to Evaluate the USMLE. 2008. USMLE, The Comprehensive Review. http://www.usmle.org/pdfs/cru/CEUP-Summary-Report-June2008.pdf (Accessed October 10, 2012).
Windish D. M , S. J. Huot, M. L. Green. 2007. “Medicine Residents’ Understanding of the Biostatistics and Results in the Medical Literature.” Journal of the American Medical Association 298 (9): 1010–1022.
Richard K. Riegelman is a professor and the founding dean of the School of Public Health and Health Services at The George Washington University; Kevin Hovland is the senior director for global learning and curricular change at AAC&U.