DOI
https://doi.org/10.1002/sdr.198

Summary
This publication argues the need for the development of systems thinking. Although several courses in all education levels, ranging from one-day workshops with no mathematics to graduate level courses stressing formal modeling, teach people to think systemically, the question still remains: How do people learn to think systemically? What skills are required? Does a particular type of academic background improve one’s ability to think systemically? What systems concepts are most readily understood? Which concepts tend to be most difficult to grasp? As effective systems thinking also requires good scientific reasoning skills such as the ability to use a wide range of qualitative and quantitative data, and familiarity with domain-specific knowledge of the systems under study, the challenge faced by the educators is not only to develop ways to teach these skills, but also to measure the impact of such courses on students’ ability to think dynamically and systemically. Doing so requires instruments to assess students’ systems thinking abilities prior to and after exposure to the concepts.
This paper describes the first steps toward the development of an inventory of items that measure people’s performance on specific systems thinking concepts. Authors developed and tested items that consist of brief tasks designed to assess some of the most basic systems thinking concepts: stocks and flows, time delays, and negative feedback. Additional items under development will address other dimensions of systems thinking. Several tests were created by combination of suitable items to explore students’ baseline systems thinking abilities. Each test consisted of a few paragraphs posing a problem. Participants were asked to respond by drawing a graph of the expected behavior over time. The items were designed to be simple and can be answered without use of mathematics beyond high school (primarily simple arithmetic).
In this paper, authors describe the use of the inventory to assess understanding of basic systems concepts in subjects with little prior exposure to systems thinking. Initial findings indicate that participants from an elite business school with essentially no prior exposure to system dynamics concepts have a poor level of understanding of stock and flow relationships and time delays. Performance did not vary systematically with prior education, age, national origin, or other demographic variables.
Authors discuss the implications of these initial results and explore steps for future research, hoping the inventory will eventually provide a means for testing the effectiveness of training and decision aids used to improve systems thinking skills.

Relevance for Complex Systems Knowledge
The paper deals with systems thinking and complex systems. The systems thinking skills are related to people’s learning to better understand interdependency and change, and thereby their dealing more effectively with the forces that shape the consequences of their actions.
Authors argue that there are several lists of systems thinking skills with each of them stressing different concepts, from the ability to deduce behavior patterns and see circular cause–effect relations to the use of ‘‘synthesis’’ to reveal a system’s structure, to the view of systems thinking as a discipline of organizational learning for ‘‘seeing wholes’’. Most advocates of systems thinking agree that much of the art of systems thinking involves the ability to represent and assess dynamic complexity (e.g., behavior that arises from the interaction of a system’s agents over time), both textually and graphically. For example, the ability, known as graphical integration, is considered as basic to understanding the dynamics of complex systems. Feedback processes and time delays are pervasive in complex systems and often have a significant effect on their dynamics. Authors also mention dimensions of complexity such as the ability to recognize and interpret feedback relationships, the ability to recognize and analyze nonlinear relationships between cause and effect, and the ability to estimate and analyze the impact of time delays.

Point of Strength
The strength of the publication is the development of items to assess dimensions of complexity such as the ability to recognize and interpret feedback relationships, the ability to recognize and analyze nonlinear relationships between cause and effect, and the ability to estimate and analyze the impact of time delays.