This project (2020-1-SE01-KA203-077872) has been funded with support from the European Commission. This web site reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

Assessing systems thinking: A tool to measure complex reasoning through ill-structured problems

Partners' Institution
Ionian University
Reference
Grohs, J.R., Kirk, G.R., Soledad, M.M., & Knight, D.B. (2018). Assessing systems thinking: A tool to measure complex reasoning through ill-structured problems. Thinking Skills and Creativity, 28, 110-130.
Thematic Area
Systems thinking-Theoretical framework and assessment
Summary
Authors argue for the need of a framework that draws across disciplines as a response to both the global problems that continue to grow in complexity and the blurred lines that consist between the physical and social sciences. They claim that the most challenging global problems contain combinations of intertwined complexities and are characterized by (a) the unique nature of each problem; (b) the interplay between attempts to address the problem and how it is framed; and (c) the ambiguity of causality, particularly as it relates to the temporal distance between an intervention and any direct effects. Such situations call for a flexible way of framing, reasoning, and acting within multiple dimensions, which is conceptualized as “systems thinking.”
Based on a literature review, authors developed the Dimensions of Systems Thinking Framework, which includes three dimensions —problem, perspective, time —along with measurable constructs associated with those dimensions. These three dimensions and their interactions provide a framework for understanding the elements of a systems thinking approach to problem-solving that is sensitive to the complex and ambiguous nature of global problems. The framework may be characterized as interdisciplinary, because it is strongly informed by problem-solving literature in engineering education, critical thinking literature in philosophy, as well as theory and scholarship related to leadership and community development, organizational studies, and public policy. It positions systems thinking as a metacognitive strategy for flexibly and iteratively considering problems and organizes the attempts to measure systems thinking ability using a scenario-based tool.
Authors also developed a purposefully designed, hypothetical problem scenario to challenge college student respondents to consider a set of information, approach problem definition, develop decision making and implementation processes, and create and evaluate potential situational solutions. The Dimensions of Systems Thinking Framework guided the scenario design and the delivery process
Moreover, authors describe the process and present the product of an iteratively developed scoring rubric that includes (1) a Criteria and Rating Guide (i.e., traditional scoring rubric), (2) a Mapping Document that directs the rater to specific responses when assessing quality within a particular construct, (3) a bank of Working Definitions to provide additional clarity to the key distinguishing traits that are featured in the Criteria and Rating Guide across different levels of response quality, and (4) example responses with rater scores and commented rationale in order to provide additional context.
Finally, authors present information to enable others to adopt the tool for research purposes or to assess student outcomes from university programs.
Relevance for Complex Systems Knowledge
The paper deals with interdisciplinarity, systems thinking, and complexity.
A framework for an interdisciplinary development of systems thinking competency is presented by this paper. Systems thinking is considered as a critical interdisciplinary skill that describes the cognitive flexibility needed to collaboratively work on problems facing society, which continue to grow in complexity. In taking this interdisciplinary approach, authors leverage the unique and common insights from various fields to create a broadly framed approach to systems thinking competency.
Point of Strength
The strengths of the publication are the presented framework for an interdisciplinary development of systems thinking competency and the shared scenario and scoring rubric for measuring systems thinking competency.
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License