Gesture in Spatial Cognition: Expressing,Communicating, and Thinking About Spatial Information

Do hand gestures play a role in spatial cognition? This paper reviews literature addressing the roles of gestures in (1) expressing spatial information, (2) communicating about spatial information, and (3) thinking about spatial information. Speakers tend to produce gestures when they produce linguistic units that contain spatial information, and they gesture more when talking about spatial topics than when talking about abstract or verbal ones. Thus, gestures are commonly used to express spatial information. Speakers use gestures more in situations when those gestures could contribute to communication, suggesting that they intend those gestures to communicate. Further, gestures influence addressees' comprehension of the speech they accompany, and addressees also detect information that is conveyed uniquely in gestures. Thus, gestures contribute to effective communication of spatial information. Gestures also play multiple roles in thinking about spatial information. There is evidence that gestures activate lexical and spatial representations, promote a focus on spatial information, and facilitate the packaging of spatial information in speech. Finally, some of the observed variation across tasks in gesture production is associated with task differences in demands on spatial cognitive processes, and individual differences in gesture production are associated with individual differences in spatial and verbal abilities. In sum, gestures appear to play multiple roles in spatial cognition. Central challenges for future research include: (1) better specification of the mental representations that give rise to gestures, (2) deeper understanding of the mechanisms by which gestures play a role in spatial thinking, and (3) greater knowledge of the sources of task and individual differences in gesture production.     

Effects of Communication Methods on Communication Patterns and Performance in a Remote Spatial Orientation Task

Abstract

An experiment was conducted to examine the impact of communication methods (text-only, audio-only, and audio-plus-video) on communication patterns and effectiveness in a 2-person remote spatial orientation task. The task required a pair of participants to figure out the cardinal direction of a target object by communicating spatial information and perspectives. Results showed that overall effectiveness in the audio-only condition was better than the text-only and audio-plus-video conditions, and communication patterns were more predictive of errors than individual differences in spatial abilities. Discourse analysis showed that participants in the audio-plus-video condition performed less mental transformation of spatial information when communicating, which led to more interpretation errors by the listener. Participants in the text-only conditions performed less confirmation and made more errors by misreading their own display. Results suggested that speakers in the audio-plus-video condition minimized effort by communicating spatial information based on their own perspective but speakers in the audio-only and text-only conditions would more likely communicate transformed spatial information. Analysis of gestures in the audio-plus-video condition confirmed that iconic gestures tended to co-occur with spatial transformation. Iconic gesture rates were negatively correlated with transformation errors, indicating that iconic gestures more likely co-occurred with successful communication of spatial transformation. Results show that when visual interactive feedback is available, speakers tend to adopt egocentric spatial perspectives to minimize effort in mental transformation and rely on the feedback to ensure that the hearer correctly interprets the information. When visual interactive feedback is not available, speakers will put more effort in transforming spatial information to help the hearer to understand the information. The current result demonstrated that allowing two persons to see and communicate with each other during a remote spatial reasoning task can lead to more errors because of the use of a suboptimal communication strategy.     

The Psychological Validity of Qualitative Spatial Reasoning in One Dimension

One of the central questions of spatial reasoning research is whether the underlying processes are inherently visual, spatial, or logical. We applied the dual task interference paradigm to spatial reasoning problems in one dimension, using Allen's interval calculus, in order to make progress towards resolving this argument. Our results indicate that spatial reasoning with interval relations is largely based on the construction and inspection of qualitative spatial representations, or mental models, while no evidence for logical proofs of derivations or the involvement of visual representations and processes was found.     

Interactivity in map learning: The effect of cognitive load

The hypothesis that active learning is beneficial relative to passive observation was assessed in the context of spatial knowledge derived from maps. Active and passive participants studied a map either while performing a simultaneous spatial tapping task (high cognitive load) or in the absence of this task (low cognitive load). Active participants controlled how the map was learned, with passive participants observing map learning without exercising control. Spatial recall was assessed in two tests, directional judgements and map drawing. Map drawing and directional judgments showed a similar pattern of results, with performance detrimentally affected by a high load for active participants, but not for passive participants. The results indicate that activity and cognitive load interact, suggesting that active learning can be detrimental to spatial learning in cognitively demanding tasks.     

Preferred and Alternative Mental Models in Spatial Reasoning

The mental model theory postulates that spatial reasoning relies on the construction, inspection, and the variation of mental models. Experiment 1 shows that in reasoning problems with multiple solutions, reasoners construct only a single model that is preferred over others. Experiment 2 shows that inferences conforming to these preferred mental models (PMM) are easier than inferences that are valid for alternatives. Experiments 3 and 4 support the idea that model variation consists of a model revision process. The process usually starts with the PMM and then constructs alternative models by local transformations. Models which are difficult to reach are more likely to be neglected than models which are only minor revisions of the PMM.     

A Neuro-Cognitive Theory of Deductive Relational Reasoning with Mental Models and Visual Images

Abstract

Many neuro-imaging studies have provided evidence that the parietal cortex plays a key role in reasoning based on mental models, which are supposed to be of abstract spatial nature. However, these studies have also shown concurrent activation in vision-related cortical areas which have often been interpreted as evidence for the role of visual mental imagery in reasoning. The aim of the paper is to resolve the inconsistencies in the previous literature on reasoning and imagery and to develop a neurally and cognitively plausible theory of human relational reasoning. The main assumption is that visual brain areas are only involved if the problem information is easy to visualize and when this information must be processed and maintained in visual working memory. A regular reasoning process, however, does not involve visual images but more abstract spatial representations—spatial mental models—held in parietal cortices. Only these spatial representations are crucial for the genuine reasoning processes.     

Expert Algorithmic and Imagistic Problem Solving Strategies in Advanced Chemistry

Abstract

Visualization and imagistic reasoning appear central to expert practice in science; however, expert use of these strategies on authentic tasks has not been examined in detail. This study documents how science experts use both algorithms and imagistic reasoning to solve problems. Using protocol analysis, we report expert chemists' preferential use of algorithms for solving spatial problems and imagistic reasoning for deducing spatial transformations. We observed experts employ algorithms to solve the majority of spatial tasks while reserving imagistic strategies to solve a class of tasks that required translating between representations. Strategy used varied widely among experts and tasks.     

When gestures show us the way: Co-thought gestures selectively facilitate navigation and spatial memory

How does gesturing during route learning relate to subsequent spatial performance? We examined the relationship between gestures produced spontaneously while studying route directions and spatial representations of the navigated environment. Participants studied route directions, then navigated those routes from memory in a virtual environment, and finally had their memory of the environment assessed. We found that, for navigators with low spatial perspective-taking performance on the Spatial Orientation Test, more gesturing from a survey perspective predicted more accurate memory following navigation. Thus, co-thought gestures accompanying route learning relate to performance selectively, depending on the gesturers’ spatial ability and the perspective of their gestures. Survey gestures may help some individuals visualize an overall route that they can retain in memory.     

Timing and sex effects on the “Spatial Orientation Test”: A World War II map reading test

The present study aimed to document the psychometric properties of a recently declassified map reading test used in World War II, known as the Spatial Orientation Test (SOT), which measures mostly the ability to match map and aerial images. We also examined the influence of timing conditions and sex of the participants as correlates of performance. A total of 101 participants (41 males, 60 females) completed the SOT, along with other spatial tests and spatial affect scales (tapping spatial anxiety and confidence). Results showed a significant male advantage on this rediscovered map reading test. Additionally, timing affected overall performance but not the magnitude of sex differences. Finally, significant correlations were found between the SOT and other spatial tests and scales. The processes underlying this test and its potential for future use are discussed.     

List of reviewers

Studies of map-learning have suggested that viewing maps from several orientations results in orientation independent encoding. The present investigation examined this claim. Three groups of participants were presented with a map from one orientation, or four orientations, or with the map constantly rotating. Participants' spatial knowledge was then assessed in a pointing task. Response latencies were fastest following presentations of the map from a single orientation. Contrary to earlier results, multiple views led to orientation dependent encoding when account was taken of participants'preferred map orientations. The findings are consistent with the notion that similar mental representations are acquired following single and multiple map views.     

Multiple Systems for Spatial Imagery: Transformations of Objects and Bodies

Problem-solving often requires imagining spatial changes. Object-based transformations allow imagining an object in a different orientation. Perspective transformations allow imagining changes in one's viewpoint. Three experiments tested the hypothesis that these two transformations are dissociable and specialized for different situations, by manipulating instructions and task parameters and measuring response times, errors, and introspective reports. Human experience with small objects such as telephones and clothes irons consists mostly of manipulation or observed manipulation, which is characterized by object-based transformations. Consistent with this experience, when participants made judgments about small manipulable objects, they showed a strong tendency to use object-based transformations. Experience with human bodies is more varied, including both object-like interactions and interactions in which one must estimate another's perspective. Accordingly, when making judgments about pictures of bodies, participants' selection of a spatial transformation depended on the type of judgment that needed to be made. When instructions violated these natural mappings, performance was impaired. These data argue for the view that multiple spatial transformation systems evolved to solve different spatial reasoning problems.     

Spatial Thinking Across the College Curriculum: A Report on a Specialist Meeting

This report presents findings from a specialist meeting of spatially-minded researchers and administrators from education and industry to consider prospects for introducing courses and curricula on spatial thinking in higher education. More than 40 participants explored the rationale for expanding student exposure to concepts, tools, and applications of spatial reasoning across a range of science, engineering, and humanities disciplines. The focus was on what we know and what we need to know to make the case for space, underscoring basic research on what is meant by spatial thinking and on variations in the spatial reasoning skills required in different domains of knowledge. The need for rigorous assessments of learning outcomes associated with different approaches to teaching spatial thinking was emphasized.     

The role of motor processes in mental rotation: selective shaping of cognitive processing via specific sensorimotor experience

The involvement of motor processes in mental rotation is experience-dependent: different levels of expertise in sensorimotor interactions lead to different strategies in mental rotation. In the present study, wrestlers, gymnasts, and nonathletes physically rotated objects that were either light (wooden) or heavy (lead) but otherwise having the same sizes and shapes. They then performed a mental rotation task using photographs of these objects in which the material and therefore the weight was visible. I hypothesized that wrestlers would rely more heavily on experience-based sensorimotor strategies in performing mental rotation because during their athletic practice they not only manipulate external “objects” (i.e., their opponent) but also have to plan future actions taking into account past experience of these “objects” (for example their weight). All participants reported that lead objects were harder to physically rotate than wooden ones. However, only wrestlers mentally rotated lead objects more slowly than wooden ones—as they would if they were physically rotating them—suggesting the involvement of motor processes. These findings show that the involvement of motor processes in mental rotation depends on specific rather than mere sensorimotor experience.     

Cell morphological, ontogenic, and genetic reactions to 0-g simulation and hyper-g

Organisms use gravity for spatial orientation, and differentiation into species during evolution follows geological processes which are caused by gravity. On the other hand, the task of most organismic functions which have or may have a relation to gravity is to compensate gravity. Furthermore, today it is very obvious that organisms do not disintegrate under the conditions of weightlessness, at least for the currently tested durations. These previous statements indicate a large field of still unknown regulation and adaptation mechanisms. Experiments to simulate weightlessness on the fast clinostat and with hyper-g show a highly developed ability of the genetic chain and of differentiating cells in being autonomous against mechanical stresses caused by outer accelerations. Nevertheless, different strong and slight changes of different tested end points were found. The question remains if the cells react actively or only passively.     

Qualitative Spatial Reasoning in Interpreting Text and Narrative

Abstract

Simple natural language texts and narratives often raise problems in commonsense spatial knowledge and reasoning of surprising logical complexity and geometric richness. In this article, I consider a dozen short texts—five taken from literature, the remainder contrived as illustrations—and discuss the spatial reasoning involved in understanding them. I conclude by summarizing their common features, and by tentatively drawing some morals for research in this area.     

Virtual to Real Transfer of Spatial Learning in a Complex Environment: The Role of Path Network and Additional Features

The effects of the amount and type of spatial information on the orientation accuracy and metric properties of mental representations of a large-scale built environment were evaluated in a virtual-real transfer study. Four groups of participants explored different virtual versions of a campus before being tested in the real environment. Learning with or without additional features and with or without a network of pathways led to different patterns of data. Although direction measures and straight-line and route distance estimates were least accurate in the absence of either type of information, the combination of both types of information did not produce more accurate knowledge than either did alone. In particular, the presence of additional features had a facilitating effect only on the direction estimates. The results highlight the respective importance of amount and type of spatial information in the acquisition and use of mental spatial representations.     

Analyzing Qualitative Spatio-Temporal Calculi using Algebraic Geometry

Abstract

Qualitative spatial reasoning is based on calculi which comprise relations and operation tables that encode operations like relation composition. Designing a calculus involves determining these tables and analyzing reasoning properties—a demanding task that is susceptible to errors if performed manually. This paper is concerned with automating computation of operation tables and analysis of qualitative calculi over real-valued domains like the plane ². We present an approach to specify qualitative relations using polynomial equations that allows methods from algebraic geometry to be applied. This paper shows how reasoning with qualitative relations can be posed algebraically and demonstrates algebraic reasoning using Gröbner base analysis. We evaluate this approach and describe our implementation, which is freely available as part of the spatial reasoning toolbox SparQ.     

Spatial Reorientation: Effects of Verbal and Spatial Shadowing

ABSTRACT

Spatially disoriented adults flexibly conjoin geometric information (macroscopic shape) and nongeometric information (e.g., the color of a wall) to re-establish their bearings. It has been proposed that non-geometric information is incorporated into a geometric frame of reference through the use of spatial language. Support for this assumption comes from dual-task studies in which participants failed to use non-geometric features when they shadowed a verbal message concurrent to the reorientation task. These studies were performed in small rectangular spaces. In contrast, our study was performed in a large square room. Experiment 1 showed that verbal shadowing did not disrupt non-geometric feature use in this environment. In Experiment 2, we found that a spatial shadowing task that required the encoding of frequently changing spatial directions impaired reorientation behavior. Our study shows that nongeometric information can be used for reorientation without recourse to linguistic processes, and suggests that the use of non-geometric features is dependent upon a spatial coding process.     

Initial moments of adaptation to microgravity of human orientation behavior, in parabolic flight conditions

The first ethological studies of astronauts' adaptation to microgravity dealt with the behavioral strategies observed during short-term space missions. No attempts had however been made to consider the initial moments of adaptation dynamics, when the subject is first submitted to conditions allowing body orientations in the full three dimensions of space. The present experimental approach was both longitudinal and transversal. It consisted of analysing, during a goal-directed orientation task in parabolic flight, the orientation behavior of 12 subjects with a past experience of 0, 30 or more than 300 parabolas. During each microgravity phase, the subjects were asked to orientate their bodies and touch, with the dominant hand, four coloured targets arranged inside the aircraft.

Results showed that for inexperienced subjects, the time between two target contacts was longer than experienced subjects. They often failed to reach all targets in the series during the first parabolas. They showed right-left confusion and a preference for the “up-down” vertical body orientation. Their performance, described by the efficiency of orientation in all three dimensions, improved over time and according to the level of experience. The results are discussed for the spontaneous, preliminary and integrative stages of adaptation, emphasizing new relationships between the body references and those of the surroundings. Such experiences lead the subject to develop a new mental representation of space.