Action and anticipation. We have been particularly interested in actions that involve physical tools and mechanisms. For example, we have examined how action facilitates people’s abilities to draw inferences about interacting gears, water moving in a glass, and blocks turning on a spool. We have emphasized tools, because tool use, particularly multi-part tool use, may provide a unique window on the flexible integration of action and understanding. To accommodate our findings and other’s, we have found it necessary to develop a novel but rudimentary model. In this model, knowledge plus the timing generated by bodily action, becomes a prime mover of imagery.  Our model helps explain how action facilitates inference, why people can perceptually anticipate the effects of tools, how non-spatial information can influence spatial reasoning, why some inferences are unavailable to reflection, why imagery for physically connected objects needs to be highly penetrable to beliefs, and how learning influences the relation between action and anticipation. Recent studies provide hypothetical-deductive support for the model by showing that it generates correct predictions about a navigation task whereas other models are silent.  For example, it correctly predicts how different rates and manners of movement to the same spatial target affect accuracy when people's eyes are closed.

Action and symbolic understanding. A foundational question for the learning sciences asks how people relate perceptual and symbolic (e.g., verbal) experiences during learning and problem solving. There are a variety of opinions on these matters that range from assigning priority to one form of knowledge over the other, or defining their relation so that symbols “stand for” perceptual experiences. We begin with the assumption that different forms of knowledge have particular cognitive benefits. For example, we have found that people use gestures to simulate physical interactions when they lack rules for predicting behaviors, but they use verbal rules when they are available.

One educational realm where this becomes particularly important is early education. Many prescriptions suggest the use of hands-on materials. For example, teachers often use manipulatives to teach basic fraction concepts. There are multiple perspectives on how manipulatives help students learn mathematics, though little evidence firmly supporting any one view. One idea is that exposure to multiple representations generates better understanding of underlying mathematical principles. Another hypothesis is that useful manipulatives have structures that mirror the semiotic systems they are meant to represent, such that each action on a manipulative corresponds to a semiotic action, one-to-one. Another view is that external resources primarily help problem solvers keep track of problem elements without wasting internal memory resources.

We suggest a use of manipulatives that is complementary but different from current alternatives. It begins with a characterization of quantitative development as an increase in the ability to organize the environment to help maintain multiple interpretations of the same referent (i.e., to interpret a single piece as a member of a group and a whole). So, rather than assuming that the quantitative meaning of a physical situation is manifest for children, it may be the process of adapting the situation through their physical actions that helps develop an interpretation that best prepares children to understand quantity. This implies that simply showing children how to use a manipulative to solve a mathematical problem does not guarantee they will develop appropriate interpretations, and in fact, it may block the adaptation-reinterpretation activity. Instead, it may be better to provide children a chance to grapple with structures and interpretations that can prepare them to learn subsequently.

The figure provides an example of the types of analyses we have been conducting to understand the relation between perceptual and symbolic activity. The figure plots how 10-year old children transferred fraction concepts to solve problems using new manipulatives. Children in the left figure were learning how to add fractions using tile pieces, while students in the right figure were learning how to add fractions using pie pieces. The figures plot how both groups of students performed when they had to transfer their knowledge to  solve problems with new types of manipulatives (e.g., beans, bars, etc.)

During the transfer with the new manipulatives, children completed problems at the levels they had learned to solve during a learning phase (e.g., adding fractions with a '1' in the numerator and a common denominator).. For each problem level, they used three transfer materials. The horizontal axis in figure represents the number of appropriate physical structures they made across the three transfer materials for a given problem level (from 0 to 3). The vertical axis represents the number of accurate interpretations (answers) for each of the three transfer materials. The 16 regions in the graph represent different ratios of physical to interpretive accuracy. To help read the figure, imagine it only showed a single point. The point would stand for a student’s performance across the three materials for a single problem level on a single day. It would represent the student’s success in physical restructuring relative to interpretative success. If the point were in a region on the diagonal, it would mean the student correctly partitioned and interpreted the materials at the same rate; the upper-right corner would mean 100% correct on both. If the point were below the diagonal, it would indicate the child made more good physical structures than interpretations. If the point were above the diagonal, the child made more good interpretations than physical structures. The figure uses arrows rather than points to indicate trajectories of change. The arrows represent performance changes between pairs of days. The tail of an arrow indicates the region a student was in for a given problem level on one day, and the head of the arrow points to the region the student attained on the next day for the same level of problem. Students typically moved to adjacent squares, with only 20% jumping across a square (omitted to simplify the figure). Circular arrows mean there was no change between days.

Though the sample is too small for a meaningful statistical analysis, the physical-interpretive space reveals some interesting patterns. Begin with the plot for the tile students. One thing to notice is that most of the students adapted the space better than they interpreted it before reaching full understanding; they are generally at or below the diagonal. The second thing to notice is that the upper-right region of 100% performance is a strong attractor state. The flow of arrows leads through the physical structure to the upper-right corner. Once students enter the region, they rarely leave on subsequent trials. Next, consider the plot for the pie students. In this case, there is more movement towards and through the upper-diagonal of the space. Student’s interpretations are driving the performance of the system. Also, the upper-right region is not a strong attractor state. It does not always draw the students from the remote regions; fewer students end in the region; and, even when they arrive, they are likely to leave. Compared to the tiles, learning with the pies does not yield a stable and directed physical-interpretive space when students transfer to new materials. Across the two graphs, physical structuring leads the way towards stable and full understanding, whereas correct interpretations do not. Evidently, a strong pathway to knowledge comes through the ability to make physical structures that are just ahead of one’s interpretations. People manipulate the perceptual environment until a structure emerges that they can interpret meaningfully.

Another way of looking at action and symbolic understanding is to determine whether people  spontaneously simulate or imagine actions to help understand symbolic materials. One line of this work has examined imagery during sentence comprehension. A special focus has been made on “fictive motion” sentences. Fictive motion refers to sentences that use action verbs that do not describe motion. For example, “The fence runs along the shore.” Evidence using response times in semantic sensibility judgments indicates that people imagine motion to read these sentences. This is a nice test of an embodiment hypothesis because it is not “necessary” to imagine motion, yet people do. For more information on this work, click the following url to download a Powerpoint presentation. (url coming soon)