Summary
The paper presents a study that focuses mainly on systems thinking in science and engineering education. Authors reviewing the literature about systems thinking research in science education, engineering education, and technology education found different definitions and assessment frameworks within each discipline (e.g. biology education, earth system(s) education). Therefore, the creation of a common language – a shared terminology – for systems thinking to bridge this gap between and within the disciplines was the first purpose of their study. A second purpose of the study was the assessment of science and engineering – pre- and in-service –teachers’ systems thinking based on this common language.
Authors administered two assignments to teacher teams: first, modeling the same adapted scientific text, and second, modeling a synthesis of peer-reviewed articles in science and engineering education, with teams selecting a topic from a list and summarizing them. They assessed those models using a rubric for systems thinking that has been developed based on the literature review of this topic, through collaboration of three experts: a science education expert, an engineering education expert, and a systems engineering expert.
Systems thinking rubric was developed by synthesizing, adapting and dividing the terms that were found in the literature review into three systems thinking aspects, each with two systems thinking categories: (1) system function – divided into general properties and emergent properties; (2) system structure – divided into structural components and structural relationships; and (3) system behavior – divided into procedural relationships and change over time. Authors derived two, three, or four attributes of systems thinking from each category, making nine attributes in total. These attributes were meant to be relevant for both science and engineering education, i.e. for both descriptive and normative conceptual models of artificial systems or natural phenomena. They excluded the following attributes from the present study: system boundary and temporary objects and decision nodes.
Authors assessed the interrater reliability of the rubric by having three experts (an expert in conceptual modeling, a systems engineering expert, and an engineering education expert) score the same eight conceptual models created by teacher teams, and calculated Spearman correlations between raters’ scores of each systems thinking attribute using. They also assessed the internal consistency of the rubric by calculating Cronbach's α for all 34 conceptual models submitted by teacher teams. Finally, they conducted confirmatory factor analysis of the construct of systems thinking which their rubric was based on – the three system aspects of function, structure, and behavior, each one with its respective attributes of systems thinking.
Authors report high interrater reliability and validity of the rubric’s theoretical construct for the system aspects of function, structure and behavior.
Then, they applied the systems thinking rubric for assessing systems thinking of teacher teams as expressed in the conceptual models of science and engineering phenomena they created.
They also found differences in scores between the assignments in favor of the second assignment (modeling an article synthesis), for two attributes of systems thinking: ‘expected outcome/intended purpose’ and ‘main object and its sub-objects’. Authors explain the first attribute difference as stemming from the modelers’ domain expertise as science or engineering teachers, rather than as scientists or engineers, and the second attribute difference – from the larger amount of information available for modeling the articles synthesis assignment. The theoretical contribution of this study lies in the definition of the systems thinking construct as a first step in establishing a common language for the science education and engineering education communities. The study's methodological contribution lies in the rubric that was developed and validated, which can be used for assessing the systems thinking of teachers and potentially also of undergraduate students.
Relevance for Complex Systems Knowledge
The paper deals with systems thinking. Authors describe systems thinking as the dual ability to understand systems and analyze circumstances, questions, or problems from a systems perspective. They provide a working definition of system to use of the term systems thinking in the context of science, technology, and engineering education: (1) a system is an entity made up of interacting parts; (2) this entity provides a function for a specific intended purpose, or end (in engineering), or outcome (in science); (3) this purpose or outcome is achieved through the interaction of all (or the main) parts of the system; (4) the interaction between the system parts are maintained by cause and effect relationships; (5) systems feature multiple levels of system integration; (6) each level of system integration exhibits whole-system properties not belonging to parts or combination of parts at lower levels of the system; (7) in engineering, systems are artificial, while in science, natural phenomena can be described as systems; and (8) artificial systems include means-ends relationships, while natural phenomena do not.
