Olmstead, Viswanathan, Aicher & Fowler (2009)

Fellow Leeds Met PhD student Liam Cross and I have collaborated this week to review a paper investigating a simulation theory approach to language comprehension.  This theory comes under the umbrella term of embodiment, but is distinct from RECS and will be revisited in future posts.

Liam conducts research on the role of synchrony in social behaviour and his blog can be found at: http://tosyncornottosync.wordpress.com/

HIBAR – Sentence comprehension affects the dynamics of bimanual co-ordination: Implications for embodied cognition

Simulation theory (Barsalou, 1999) is a subset of embodied approaches that attempts to reconcile embodiment within a representational framework.  This account seeks to ground high-level cognition in sensorimotor representations, in an attempt to overcome issues of how purely symbolic or abstract representations might be instantiated in the brain.  Within the domain of language, it predicts that mental simulation of the appropriate sensorimotor representation is integral to the process of comprehension.  This would predict interaction of activities which draw on the same representations, in either an interfering or facilitatory manner (e.g. Glenberg & Kaschak, 2002).  For example, motions congruent with an action specified in a statement should facilitate comprehension, while incongruent motions should interfere.


Olmstead et al. (2009) used a pre-established bilateral rhythmic co-ordination task (Kugler & Turvey, 1987) involving well-understood behavioural measures to investigate how language comprehension of performable sentences interferes with the behavioural organisation.

ᶲ = relative phase, V = potential

ᶲ = relative phase, V = potential

The task involved bimanual movements, where participants were required to swing two pendulums from both wrists.  This was tested when participants were swinging the pendulums in-phase (00) and anti-phase (1800).  Without interference, people show stable relative phase during these particular movements, but 00 is more stable.  1800 tends to see more variability in performance around its attractor location; this variability is the standard deviation of relative phase (SDRP).

While performing this task, participants were asked to read sentences on a screen and indicate verbally whether they were plausible or implausible sentences.  While plausibility was varied, the variable under investigation was that of performability vs. inanimacy of the sentences.  Performability sentences implied movements concerning the hands/fingers/arms.  Differences in the dynamic of the movement were then compared to a baseline, where participants engaged in a swinging only condition.

The researchers also had a detuned condition, in which the preferred frequencies of the limbs were manipulated by giving participants pendulums of different natural frequencies.  Detuning has the effect of shifting the attractors away from 00 and 1800, captured by “relative phase shift” (RPS), and also has the effect of increasing SDRP.

Under simulation theory, it would be predicted that movements would differ in the performable sentences condition compared to the inanimate sentences condition and the baseline (swinging only).  Since performable sentence comprehension and the movement task should be using overlapping neural resources with respect to the limb involved, simulation theory would expect a change in SDRP, but not a shift in RPS.  Sentence comprehension is presumed to cause intermittent interruption of the continuous neural oscillators active in the control of the movement task (Grossberg, Pribe, & Cohen, 1997).  In other words, performable sentence comprehension would act as a perturbation to the system and thus increase variability (SDRP).


Performability of sentences did not interact with detuning or required phase for either RPS or SDRP leading the researchers to aggregate the detuned and non-detuned conditions, comparing the single and dual-task conditions.

Further analysis revealed a main effect of comprehension task on RPS, but none on SDRP.  Judging performable sentences was found to affect relative phase (RP) of the co-ordination, where inanimate sentences saw no shift from baseline RP.  There was no effect of task on SDRP; in other words, the attractor shape did not change, it merely shifted significantly from baseline when judging performable sentences, but not when judging inanimate sentences.

The authors conclude that the results are inconsistent with predictions made by simulation theory as it currently stands, since an unexpected shift in attractor occurred and no increase in SDRP was observed.

While this is indeed an interesting effect, there are some questions and issues we would have raised, had we been reviewers of this paper:

Nature of Landscape Shift:

The authors plot task against difference scores between single/dual task conditions on grounds of clarity.  We argue that the figure is in fact misleading, especially coupled with its use of graph lines between conditions; these give the impression of RPS directionality i.e. that the significant shift in RP for performable sentences followed the n.s. rightward deviation in the inanimate condition.  We feel it odd that the authors did not explicitly mention in the results that the significant shift was actually a leftward shift from the baseline.

graphWithout access to the raw data for the detuned and non-detuned conditions at baseline and during sentence comprehension, it is also unclear whether this can be characterised as a leftward shift or one of facilitation towards the desired phase, as the raw RPS scores reported are aggregates of the two.  Because of this, the aggregated baseline RPS lies at -8.660, not 00, inanimate sentence comprehension lies at -9.07 and performable sentence comprehension at -4.78.  The leftward shift is briefly mentioned in the discussion, but could have benefitted from further explication in the results.  The authors report that right-handed participants appeared to be the culprit for the left-leading behaviour in the performable condition, but this is not substantiated by any formal analysis.

Temporal Resolution of Movement Analysis

The authors indicate that they chose to measure the global effects of the sentence types (averaging across all performable sentences vs. inanimate sentences), rather than adopt an event-based design, but give no explanation as to why this was opted for, whether for practicality’s sake or otherwise.  The current design does not allow analysis of potentially interesting data about the course of the movement change over time during comprehension.

An event-based design would allow observation of when movement changes occur and how these directly link on a moment-to-moment basis to presentation of the stimulus. Here, sentences are not considered as discrete events, rather some sort of continuous activity averaged over three sentence instances.  Temporal resolution could be further facilitated through auditory stimuli or eye-tracking for visual stimuli, as this means movement data can be accurately time stamped with relation to the information present in the environment at any one time.  It is curious that the lack temporal resolution was not mentioned, given the time spent in the discussion considering what sort of influence transient perturbations (comprehension of performable sentences) would have on the continuous control task under a simulation account.

Control of Facilitatory/Competition Effects

It would be helpful to investigate sentence comprehension that involves the same effector when comprehending both congruent or incongruent actions; there are key predictions in simulation theory about how this should affect neural resource-sharing.  Despite it being unclear why the shift took the form it did in the performable sentences condition, it would have been interesting to see whether actions predicted to facilitate or compete for resources under simulation theory would show qualitatively distinct shifts in behaviour.

The authors present this as a paradigm to be utilised in the study of embodied cognition.  While we applaud the use of a well-defined movement task with clear behavioural measures, we feel that the design could be smoothed out for future use.


Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and brain sciences, 22(04), 577-660.

Grossberg, S., Pribe, C., & Cohen, M. (1997). Neural control of interlimb oscillations: 1. Human bimanual coordination. Biological cybernetics, 77, 131–140.

Kugler, P. N., & Turvey, M. T. (1987). Information, natural law, and the self-assembly of rhythmic movement. Lawrence Erlbaum Associates, Inc.
Olmstead, A. J., Viswanathan, N., Aicher, K. A., & Fowler, C. A. (2009). Sentence comprehension affects the dynamics of bimanual coordination: Implications for embodied cognition. The Quarterly Journal of Experimental Psychology, 62(12), 2409-2417.

Introduction to mainstream Cognitive Science or: How we learned to stop worrying and love the representation

My first post is just going to set out a very brief of history of cognitive science.  While this will be woefully incomplete as a comprehensive historical account, the aim is to focus on some key movements that will hopefully shed some light on ideological shifts responsible for its current incarnation.

Psychology is generally accepted to have begun experimentally in the 1880s in the laboratory of Wilhelm Wundt, a man who focused his efforts on discovering the basic units of consciousness such as sensations through phenomenological report or “introspection”.

The early 1900s saw a shift in practice in the face of increasing criticism concerning the unreliability of introspective methods in a world where science aimed to champion the “objective”.  This led to the eventual development of behaviourism, an approach which switched the focus from fuzzy, undefined conceptions of mind and constructs (e.g. beliefs and feelings) to overt behaviour, which could be empirically tested and more easily experimentally controlled.  Those such as Watson and Skinner pioneered this science of behaviour, in which attempts were made to characterse human capacities in terms of stimulus-response associations and reinforcement schedules.

However, in the 1950s the American linguist Noam Chomsky famously reviewed Skinner’s account of language acquisition, critiquing the scope of operant conditioning to explain the complex and productive nature of language competence.  He argued that a fundamental part of the picture was missing, as mere exposure to natural language would be insufficient to account for the robust knowledge of language humans display.  Concerns were highlighted about the lack of information available in speech signals and also the lack of negative evidence (feedback on errors in production) to account for language competence.  This is known as the poverty of the stimulus argument and conclusions were drawn that internal mental structures must exist to account for enhancement of impoverished input.

This came during a period when the increasing prominence of information processing led to parallels being drawn between the way computers deal with information and how a mind might process sensory input.  The computer metaphor captivated the cognitive science community and had a profound impact on the direction Psychology would take up to the present day, in what is now termed the “cognitive revolution”.

The focus had turned full circle in under a century and it was squarely back on the empirically inaccessible mind and its internal causal contents, whose nature could only be inferred.  It was not solely language that was receiving this treatment.  The study of perception became a domain in which the basic building blocks of perception started at the level of sense data.  Since stimulation of a sense cell contains no information of the causal stimulus itself, the job of the brain was conceived to be to derive and infer the content of the world on the basis of this sparse input.  The role of the environment in any explanatory sense was theoretically dispensable; the conception was not that it was the world, but the world-as-represented, ­which influenced behaviour.

This approach has enjoyed a lengthy period of stability in the history of the science and countless models of human behaviour take the form of input/output models in which representations are computed over by something akin to a computer’s central processor to explain human behaviour.  Representations have done a lot of the heavy lifting for a wide range of cognitive tasks and highly complex calculations ascribed to them in order to account for simple and complex human behaviour.  It is rarely detailed how these calculations are performed, but these explanations are frequently believed to be on the horizon and thought unlikely to be a problem for the frighteningly complex human brain.

Mental representations nominally play the necessary explanatory role in most accounts of human behaviour and have to be presupposed, since they cannot be empircally accessed.  As a consequence, research is frequently dedicated, not to proving their existence, but to supposedly indirectly probing their character and modelling how they might be put to work.

Is this a problem or a scientific necessity?  For example, are mental representations providing a similar function to the hypothesised role of dark matter in astrophysics?  Can models simply not function without invoking mental representations?

One obvious benefit of invoking mental representations is their decouplability.  They appear to cater as a simple explanation of how we achieve competent behaviour when a stimulus is currently absent in the real world.  Things do not come in and out of existence for us just because they are not in contact with our senses at any one time, so the brain must surely contain some form of stand-in for the world.  This also seems to serve to explain highly competent behaviour that appears to require knowledge of future states of the world, such as knowing when and where to be to catch a fly ball.  How else could this be achieved, but through some hidden computation that allows prediction?

Language is perhaps the most obvious domain begging for a representational narrative.  With no lawful relationship between arbitrary sounds and their referents, a mental stand-in seems called for to explain how language and concepts can be used to control action.  How could we create sense-making but novel utterances, if there are not abstract structures in place to scaffold this?  Why else would children overgeneralise regular grammatical rules to irregular forms they had already mastered?

In essence, the resounding chorus in the cognitive science community was and continues to be: What else could it be?

A future post will look at an altenative approach to the current paradigm and why assumptions of mental representations are not as benign as they initially appear.