Last night I both learnt something new about language and cognition, and also developed a new trick for creativity!

In the dream in question I was in a meeting. I know, a sad topic for a dream, and perhaps even sadder it had started with me filling in forms!  The meeting was clearly one after I’d given a talk somewhere as a person across the table said she’d been wanting to ask me (obviously as a sort of challenge) if there was a relation between … and here I’ll expand later … something like evolutionary and ecological something.  Ever one to think on my feet I said something like “that’s an interesting question”, but it was also clear that the question arose partly because the terms sounded somewhat similar, so had some of the sense of a rhyming riddle “what’s the difference between a jeweller and a jailor”.  So I went on to mention random metaphors as a general creativity technique and then, so as to give practical advice, suggested choosing two words next to each other in a dictionary and then trying to link them.

Starting with the last of these, the two words in a dictionary method is one I have never suggested to anyone before, not even thought about. It was clearly prompted by the specific example where the words had an alliterative nature, and so was a sensible generalisation, and after I woke realised was worth suggesting in future as an exercise.  But it was entirely novel to me, I had effectively done the exactly sort of thinking / problem solving that I would have done in the real life situation, but while dreaming.

One of the reasons I find dreams fascinating is that in some ways they are so normal — we clearly have no or little sensory input, and certain parts of our brain shut down (e.g. motor control to stop us thrashing about too much in our sleep) — but other parts seem to function perfectly as normal.  I have written before about the cognitive nature of dreams (including maybe how to model dreaming) and what we may be able to learn about cognitive function because not everything is working, rather like running an engine when it is out of the car.

In this dream clearly the ‘conscious’ (I know an oxymoron) problem-solving part of the mind was operating just the same as when awake.  Which is an interesting fact about dreaming, but  I was already aware of it from previous dreams.

In this dream it was the language that was interesting, the original conundrum I was given.  The problem came as I woke up and tried to reconstruct exactly what my interlocutor had asked me.  The words clearly *meant* evolutionary and ecological, but in the dream had ’sounded’ even closer aurally, more like evolution and elocution (interesting to consider, images of God speaking forth creation).

So how had the two words sound more similar in my dream than in real speech?

For this we need the Jabberwocky circuit.

There is a certain neurological condition that arises, I think due to tumours or damage in particular areas of the grain, which disrupts particular functions of language.   The person speaks interminably; the words make sense and the grammar is flawless, but there is no overall sense.  Each small snippet of speech is fine, just there is no larger scale linkage.

When explaining this phenomenon to people I often evoke the Jabberwocky circuit.  Now I should note that this is not a word used by linguists, neurolinguists, or cognitive scientists, and is a gross simplification, but I think captures the essence of what is happening.  Basically there is a part of your mind (the conscious, thinking bit) that knows what to say and it asks another bit, the Jabberwocky circuit, to actually articulate the words.  The Jabberwocky circuit knows about the sound form of words and how to string them together grammatically, but basically does what it is told.  The thinking bit needs to know enough about what can be said, but doesn’t have time to deal with precisely how they are strung together and leaves that to Jabberwocky.

Even without brain damage we can see occasional slips in this process.  For example, if you are talking to someone (and even more if typing) and there is some other speech audible (maybe radio in the background), occasionally a word intrudes into your own speech that isn’t part of what you meant to say, but is linked to the background intruding sound.

Occasionally too, you find yourself stopping in mid sentence when the words don’t quite make sense, for example, when what would be reasonable grammar overlaps with a colloquialism, so that it no longer makes sense.  Or you may simply not be able to say a word that you ‘know’ is there and insert “thingy” or “what’s it called” where you should say “spanner”.

The relationship between the two is rather like a manager and someone doing the job: the manager knows pretty much what is possible and can give general directions, but the person doing the job knows the details.  Occasionally, the instructions get confused (when there is intruding background speech) or the manager thinks something is possible which turns out not to be.

Going back to the dream I thought I ‘heard’ the words, but examining more closely after I woke I realised that no word would actually fit.  I think what is happening is that during dreaming (and maybe during imagined dialogue while awake), the Jabberwocky circuit is not active, or not being attended to.  It is like I am hearing the intentions to speak of the other person, not articulated words.  The pre-Jabberwocky bit of the mind does know that there are two words, and knows what they *mean*.  It also knows that they sound rather similar at the beginning (“eco”, “evo”), but not exactly what they sound like throughout.

I have noticed a similar thing with the written word.  Often in dreams I am reading a book, sheet of paper or poster, and the words make sense, but if I try to look more closely at the precise written form of the text, I cannot focus, and indeed often wake at that point¹.  That is the dream is creating the interpretation of the text, but not the actual sensory form, although if asked I would normally say that I had ’seen’ the words on the page in the dream, it is more that I ’see’ that there are words.

Fiona does claim to be able to see actual letters in dreams, so maybe it is possible to recreate more precise sensory images, or maybe this is just the difference between simply writing and reading, and more conscious spelling-out or attending to words, as in the well known:

Paris in the
the spring

Anyway, I am awake now and the wiser.  I know a little more about dreaming, which cognitive functions are working and which are not;  I know a little more about the brain and language; and I know a new creativity technique.

Not bad for a night in bed.

What do you learn from your dreams?

1. The waking is interesting, I have often noticed that if the ‘logic’ of the dream becomes irreconcilable I wake.  This is a long story in itself, but I think similar to the way you get a ‘breakdown’ situation when things don’t work as expected and are forced to think about what you are doing.  It seems like the ‘kick’ that changes your mode of thinking often wakes you up! [back]

The central premise of Tomasello’s book is that:

(1) only cultural development can explain the remarkable development of the human race in the past 200 thousand years, as the changes we have seen are simply not explainable in terms of genetic evolution during that timescale

(2) the crucial genetic step that has fuelled this cultural explosion and the essential difference between humans and other animals is our ability to attribute intentionality to each other, to interpret others’ actions as being for some purpose.

Early on Tomasello describes this as:

“the ability of individual organisms to understand conspecifics as beings like themselves who have intentional and mental lives like their own” (p. 5, Tomasello’s emphasis)

While I agree with the broad argument, this specific statement is almost the opposite of the hypothesis, which I often talk about, concerning the origins of self-consciousness. In particular I suggest that the very concept of self may be an accident of sociality; we are aware of ourselves as intentional beings because we are aware of the intentionality of others.

By self-consciousness here I mean not feeling awkward in company, but the explicit awareness of oneself. This is not the same as consciousness, or simply being aware (one of the deepest mysteries), but more the declarative knowledge that one has intentions, actions, and thoughts.

My own argument runs like this.

In order to survive and prosper we need to be able to predict the actions of other creatures and our fellow humans. When chasing a rabbit it is useful to know that the rabbit will run away when it sees you approach, and that it will try to reach a nearby rabbit hole. Similarly, it is useful to know that one’s fellow hunters will attempt to cut off its escape route. These reactions during hunting could be purely instinctive, and probably are for many creatures such as pack animals, but with higher reasoning we can be more creative in terms of the strategies we use whether as hunter or prey; and this higher-order thinking is most effective when we can predict the actions of other creatures.

When the creature we are predicting is behaving largely instinctively, then our predictions can be similarly relatively simple. However, if the creature we wish to predict, a fellow human, is also able to employ these higher-order strategies, then we need to understand these in order to understand the other’s behaviour. In order to predict the behaviour of our fellow hunter, we need to take into account her understanding of the rabbit; and moreover, her understanding of ourself¹.

That is, to understand others one has to think about oneself as if from the outside — self consciousness!

Returning to Tomasello, his argument is about mutual understanding as a means to learn through creatively emulating others, whereas my argument above is more about the instrumental understanding of others. Being able to understand motivation helps both. The instrumental case, I’ve already described – by understanding what motivations drove your behaviour I can more accurately predict under what circumstances you will behave similarly.

Tomasello’s developmental case is similar. If I am able to imitate others then I have additional behaviours that I can employ, but I still have to learn pretty much for myself when they are appropriate. However, if I know why someone else is behaving in the way that they do then I can instantly know when those behaviours are appropriate for me. When the reasons for behaviour are readily visible in the environment, for example, a sound in the bushes and everyone running, then no model of mind is necessary to learn the association between stimulus and imitated behaviour, but where the behaviour is the result of inner thoughts and drives then we need correspondingly more complex responses.

Tomasello (p. 99) argues that this is also essential for language development as we have to understand the perspective of others as we interpret or frame utterances.

One of the key aspects he identifies is precisely that language requires us to see ourselves “from the outside”, which is entirely consonant with my own argument that the notion of self is an accident of social intercourse. The issue is about which comes first phylogenetically, self or other. Tomasello (p. 70) notes that social theorists “from Vico and Dilthey to Cooley and Mead” stress that our understanding of others rests on parallels to our understanding of ourselves; I would simply add that the reason we have access to knowledge about ourselves may be precisely in order to understand others.

When discussing how children acquire a sense of self, he notes that research has shown that infants do not conceptualise or explicitly talk about themselves before they do about others. So, while it is not true that ontogeny inevitably recapitulates phylogeny, this is certainly suggestive evidence that self is at least no more primitive than other.

While my own and Tomasello’s position both rely on the understanding of the motives and intentions of others, there is also that much deeper sharing of feeling and emotion when we empathise with others. It maybe that empathy is more primitive than the awareness of our own or others intentions as we do not need to explicitly know what someone else is feeling, nor be able to articulate one’s own, in order to simply feel with them.

It is easy to see why it is useful to understand others emotions – if someone bigger than me is feeling upset and angry it may be better to steer clear. But the roots of empathy are less clear and obviously rooted in social cohesion and bonding; it is a feeling not just with others, but intrinsically for them.

This getting alongside others is exactly what Christmas is about “the word became human” (John 1:14, New Living Tr.) and Immanuel means precisely “God with us”; the ineffable becoming an infant.

Another term in the original Tomasello quote is “conspecifics”. We have a special understanding other creatures of the same species as ourselves. This is clearly important for imitation and learning, there is no sense in imitating the behaviour of creatures very different from ourselves, such as birds, as we may be physically not able to do the same things (can’t fly!) and anyway may not share the same kinds of motivations (e.g. making a place to lay eggs).

This works also within species, we need to learn the things that we are able to and need to perform and so it is those closest to us in terms of aspirations and abilities who are the most obvious to imitate. Yet it maybe those who are more different and more experienced who have most to offer. Diligent students understand this and step beyond the obvious peer group, but also the best teachers are able to see the world from the point of view of their students.

I read with fascination as Tomasello described many experiments of his own and others that look at small infants acquiring language. However, I also noted that the focus of them all was on the way in which the infant had to make sense of the parent or other adults words and gestures. In fact, it is also the parents who try to make sense of the inarticulate sounds and embryonic gestures of their child.

look down		bend down		stoop down

People differ in the way they interact with small children: some stand fully up and look down, some bend over the child from the waist, and some squat down or sit on a low chair so that they are the child’s level. It is the latter I always know are going to be the ‘naturals’ with children².

To be a good teacher you sometimes need to become like a little child – Christmas.

1. This is effectively a second order model of mind, First order model of mind is when we understanding that others have beliefs, motivations etc.; that is that they have mind. Second order is when we reason about their understanding of our minds, third order when we think about how they think about us thinking about them! One of Piaget’s critical development steps is when a child moves away form ego centrality to be able to understand other people’s different knowledge and physical point of view – first order model of mind. In autism this does not develop normally with corresponding social and other developmental impact. While most of us manage first order and second order model of mind without difficulty, but third order is more difficult and fourth and higher orders get hard to deal with except more analytically. This was wonderfully demonstrated by the Kursaal Flyers‘ 1976 one hit wonder which as the opening line: “Little does she know that I know that she knows that I know she’s two timing me.” (music at lastfm, lyrics at justsomelyrics) – fourth order model of mind! There was a video at the time that acted out the scene described in the song lyrics. [back]
2. Of course, while people tend to interact naturally in one way or another, you can explicitly choose how to address a child. [back]

Mostly philosophy of the mind and materiality … the latter to help as we work on the DEPtH book on Physicality, TouchIT

• Shaun Gallagher, Dan Zahavi. The Phenomenological Mind: An Introduction to Philosophy of Mind and Cognitive Science, Routledge, 2007.
• John Lechte. Fifty Key Contemporary Thinkers: From Structuralism to Post-Humanism, 2nd Edition, Routledge, 2007.
• Jean-Paul Sartre.  Being and Nothingnes: An Essay on Phenomenological Ontology, 1943.  Routledge Classics, , 2nd Edition, 2003.
• Jay Friedenberg. Artificial Psychology, Routledge , 2008.
• Max Velmans.  Understanding Consciousness, Routledge, 2009.
• Peter Carruthers. The Nature of the Mind, Routledge, 2003.

In fact, with these and the previous  set I had far too many even for a month of evenings, and below you can see the books I actually brought.

As well as a selection from the academic books also some fiction/leisure reading, some old favourites and some new ones:

• How Green was My Valley, Richard Llewellyn – a Welshman has to read this :-/
• The Catcher in the Rye, J.D. Salinger – a classic I’ve never read
• More of the Good Life – the TV series was formative for me as a child, but 40 seemed so far away
• Lark Rise to Candleford, Flora Thompson – some years since I’ve read it last, and have been loving the TV series, but I don’t think it has stayed very close to the book!
• Nella Last’s War – this is the book that was the basis for the TV drama Housewife 49 and part of the Mass Observation that collected diaries from ordinary people across Britain during the Second World War.
• Ruth, Elizabeth Gaskill – another classic that I’ve not read yet!
• As I Walked Out One Midsummer Morning.  Laurie Lee’s account of travelling in Spain in the run up to the Civel War.  I read it in school for O’level.
• Swallowdale, Arthur Ransome – Couldn’t find Swallow’s an Amazons, I think one of the girls might have it on their shelves!
• The Shining Company, Rosemary Sutcliff – we have loads of her histroical novels for children.  I find that good children’s writing is so much better than most adult books, which often feel they need to be incomprehensible to be good.
• The Growing Summer, Noel Streatfield – lovely story, children visiting a quirky old lady in west coast of Ireland.
• Hovel in the Hills, Elizabeth West  – another book I’ve read many times, but not for many years.  True story about a couple who buy an old house on a Welsh hillside.

In addition, but missing from the picture, is one I borrowed from my daughter, Tamara Pierce’s  The Healing in the Vine, and one I’ve borrowed from Tiziana Catarci during my visit the Languages of Art.

So, two weeks in and how far have I got …

Well, been a little busy, two journal papers, a book chapter, an interfaces article, two 3 hour lectures to the masters students here, a seminar, reading thesis chapters and helping with two grant proposals … so not got very far through the bookshelf.

In fact, to be brutally honest, so far only finished the Tamora Pierce and nearly finished Gibson (just conclusions to go):

As you can see LOTS of notes on Gibson, I will write a very long blog sometime about this, but several others in line first!

But next week several train journeys, so may get through a few more books

One talk was by Stuart Hameroff on A New Marriage of Brain and Computer.  He is the guy that works with Penrose on the possibility that quantum effects in microtubules may be the source of consciousness.  I notice that he used calculations for computational capacity based on traditional neuron-based models that are very similar to my own calculations some years ago in “the brain and the web” when I worked out that the memory and computational capacity of a single human brain is very similar to those of the entire web. Hameroff then went on to say that there are an order of magnitude more microtubules (sub-cellular structures, with many per neuron), so the traditional calculations do not hold!

Microtubules are fascinating things, they are like little mechano sets inside each cell.  It is these microtubules that during cell division stretch out straight the chromosomes, which are normally tangled up the nucleus.  Even stranger those fluid  movements of amoeba gradually pushing out pseudopodia, are actually made by mechanical structures composed of microtubules, only looking so organic because of the cell membrane – rather like a robot covered in latex.

The main reason for going to the text talks was one by Steve Souders “Life’s Too Short – Write Fast Code” that has lots of tips for on speeding up web pages including allowing Javascript files to download in parallel.  I was particularly impressed by the quantification of costs of delays on web pages down to 100ms!

This is great.  Partly because of my long interest in time and delays in HCI. Partly because I want my own web scripts to be faster and I’ve already downloaded the Yahoo! YSlow plugin for FireFox that helps diagnose causes of slow pages.  And partly  because I get so frustrated waiting for things to happen, both on the web and on the desktop … and why oh why does it take a good minute to get a WiFi connection ….  and why doesn’t YouTube introduce better controls for skimming videos.

… and finally, because I’d already spent too much time skimming the tech talks, I looked at one last talk: David Levy, “No Time To Think” … how we are all so rushed that we have no time to really think about problems, not to mention life¹.  At least that’s what I think it said, because I skimmed it rather fast.

1. see also my own discussion of Slow Time [back]

“Why did the dinosaur cross the road?”
…

It reminded me yet again of the incredible richness of apparently trivial day-to-day thought.  Not the stuff of Wittgenstein or Einstein, but the ordinary things we think as we make our breakfast or chat to a friend.

There is a whole field of study looking at computational humour, including its use in user interfaces¹, and also on the psychology of humour dating back certainly as far as Freud, often focusing on the way humour involves breaking the rules of internal  ‘censors’ (logical, social or sexual) but in a way that is somehow safe.

Of course, breaking things is often the best way to understand them, Graeme Ritchie wrote²:

“If we could develop a full and detailed theory of how humour works, it is highly likely that this would yield interesting insights into human behaviour and thinking.”

In this case the joke starts to work, even before you hear the answer, because of the associations with its obvious antecessor³ as well as a whole genre of question/answer jokes: “how did the elephant get up the tree?”⁴, “how did the elephant get down from the tree?”⁵.  We recall past humour (and so neurochemically are set in a humourous mood), we know it is a joke (so socially prepared to laugh), and we know it will be silly in a perverse way (so cognitively prepared).

The actual response was, however, far more complex and rich than is typical for such jokes.  In fact so complex I felt an almost a palpable delay before recognising its funniness; the incongruity of the logic is close to the edge of what we can recognise without the aid of formal ‘reasoned’ arguments.  And perhaps more interesting, the ‘logic’ of the joke (and most jokes) and the way that logic ‘fails’, is not recognised in calm reflection, but in an instant, revealing complexity below the level of immediate conscious thought.

Indeed in listening to any language, not just jokes, we are constantly involved in incredibly rich, multi-layered and typically modal thinking⁶. Modal thinking is at the heart of simple planning and decision making “if I have another cake I will have a stomach ache”, and when I have studied and modelled regret⁷ the interaction of complex “what if” thinking with emotion is central … just as in much humour.  In this case we have to do an extraordinary piece of counterfactual thought even to hear the question, positing a state of the world where a dinosaur could be right there, crossing the road before our eyes.  Instead of asking the question “how on earth could a dinosaur be alive today?”, we are instead asked to ponder the relatively trivial question as to why it is doing, what would be in the situation, a perfectly ordinary act.  We are drawn into a set of incongruous assumptions before we even hear the punch line … just like the way an experienced orator will draw you along to the point where you forget how you got there and accept conclusions that would be otherwise unthinkable.

In fact, in this case the punch line draws some if its strength from forcing us to rethink even this counterfactual assumption of the dinosaur now and reframe it into a road then … and once it has done so, simply stating the obvious.

But the most marvellous and complex part of the joke is its reliance on perverse causality at two levels:

temporal – things in the past being in some sense explained by things in the future⁸.

reflexive – the explanation being based on the need to fill roles in another joke⁹.

… and all of this multi-level, modal and counterfactual cognitive richness in 30 seconds chatting over the garden gate.

So, why did the dinosaur cross the road?

“Because there weren’t any chickens yet.”

1. Anton Nijholt in Twente has studied this extensively and I was on the PC for a workshop he organised on “Humor modeling in the interface” some years ago, but in the end I wasn’t able to attend [back]
2. Graeme Ritchie (2001) “Current Directions in Computer Humor”, Artificial Intelligence Review. 16(2): pages 119-135 [back]
3. … and in case you haven’t ever heard it: “why did the chicken cross the road?” – “because it wanted to get to the other side” [back]
4. “Sit on an acorn and wait for it to grow” [back]
5. “Stand on a leaf and wait until autumn” [back]
6. Modal logic is any form of reasoning that includes thinking about other possible worlds, including the way the world is at different times, beliefs about the world, or things that might be or might have been.  For further discussion of the modal complexity of speech and writing, see my Interfaces article about “writing as third order experience“ [back]
7. See “the adaptive significance of regret” in my essays and working papers [back]
8. The absence of chickens in prehistoric times is sensible logic, but the dinosaur’s action is ‘because ‘ they aren’t there – not just violating causality, but based on the absence.  However, writing about history, we might happily say that Roman cavalry was limited because they hadn’t invented the stirrup. Why isn’t that a ridiculous sentence? [back]
9. In this case the dinosaur is in some way taking the role of the absent chicken … and crossing the Jurassic road ‘because’ of the need to fill the role in the joke.  Our world of the joke has to invade the dinosaur’s word within the joke.  So complex as modal thinking … yet so everyday. [back]

In this case the issue is the distributed nature of cognition within the brain and the inadequacy of central executive models. In support of this, Clark (p.39) cites Mitchel Resnick at length and I’ll reproduce the quote:

“people tend to look for the cause, the reason, the driving force, the deciding factor. When people observe patterns and structures in the world (for example, the flocking patterns of birds or foraging patterns of ants), they often assume centralized causes where none exist. And when people try to create patterns or structure in the world (for example, new organizations or new machines), they often impose centralized control where none is needed.” (Resnick 1994, p.124)¹

The take home message is that we tend to think in terms of centralised causes, but the world is not like that. Therefore:

(i) the way we normally think is wrong

(ii) in particular we should expect non-centralised understanding of cognition

However, if our normal ways of thinking are so bad, why is it that we have survived as a species so long? The very fact that we have this tendency to think and design in terms of centralised causes, even when it is a poor model of the world, suggests some advantage to this way of thinking.

Of course, this may simply be an accident of our neural architecture … in which case it would be important for (ii), or may be adapted for a hunter gatherer life, but not 21st Century living – but again would be interesting for (ii). However, the fact that we are still here means it is certainly not too unsuccessful.

Whatever the reason, the fact that we think in these terms is itself an empirical data point for understanding human cognition. We have brains that tend to seek centralised solutions – what are the neural and cognitive mechanisms that drive this and what are the environmental reasons that make it work.²

There are two factors at work here, one is about the way we see the world and the other about the way we plan and act on it.

At the level of perception, one of the Gestalt laws is that things that move together belong together. Even if bushes hide most of a predator from view, the several disconnected tiny moving fragments still form one large animal you need to avoid.

Yes this is a gross simplification of reality. Look at a rock – it is an ‘it’, a single thing – but in fact it is not, it is simply the decentralised activities of millions of millions of millions of millions of atoms interacting, largely locally, with one another. It is not so far unlike Resnick’s flock of birds. However, their general coherence of motion and substance makes it sensible to regard it as one thing. As a scientist understanding the decentralised emergent phenomenon is interesting, but as a gardener wanting to move the rock it is an intellectual luxury and the (incorrect) centralised view of a the rock makes sense.

In real world problems, sometimes decentralised solutions work, other times they don’t.

I was once driving in Rome on a Saturday night (happily with a native Roman to guide me). It was after 11pm so they turned off all the traffic lights (as Italians ignore them anyway) and we came to a massive crossroads with completely full three lane roads in all directions. The space was filled with a criss-cross of apparently grid-locked cars and I thought we would be stuck there until a policeman came, but my navigator told me to simply drive. Every time the slightest gap opened, be it only a few inches, I would edge forward. Eventually, but after a relatively short time, we found ourselves at the other side. Thinking afterwards I realised that always some car was able to get out and I fact the ‘greedy’, decentralised algorithm worked perfectly.

Driving is different in north-west Scotland where there are long stretches of narrow single-track roads with passing places. When you spot a vehicle coming you watch out for a passing place and whichever of you gets to the one first waits there. If you notice too late and meet, then the person closest to a passing place may need to reverse. This is another local, slightly more polite, but semi-greedy algorithm, with each person making independent choices and trying to proceed, but taking into account immediately close road users. However, when single track roads get too full, this can fail. In situations, like the passing goods-train puzzles, where lines of vehicles have to pass with only single passing place, then often long lines of vehicles have to backup, go forward again, reverse again, in apparently disorganised ways – and ways in which each single driver cannot understand from local conditions alone. People have to get out their cars and start to coordinate their efforts.

Note that the decentralised strategies work remarkably well, and when they do require less effort than coordination. However, the reason that we do more than that, and think in ways that have an (at least behavioural) appearance of centralised control is because for certain problems this is needed – not least in complex social and technological situations.

With a HCI hat on, when we come to designing for people, we get the best solutions not when we ignore one aspect or another, but when we recognise the relative strengths of the two and how they can work together.

I recall when I was a child (yes over 30 years ago), seeing a television report about Benetton’s new CAD systems. The problem was cutting out rolls of cloth to make pieces for clothing. Traditionally an experienced cutter would arrange the pieces for a single garment as tightly as possible (to avoid waste), whilst ensuring proper orientations. These were then cut using a special form of guillotine. The new system of cutting from the roll allowed them to take the pieces for several garments and organise them over a long run of cloth for cutting. Doing several garments at once offered savings in terms of less wastage, but was a more challenging arrangement tasks … hence computer aid. The computer would take pieces initially arranged on (virtual) fabric and ‘jiggle’ them until they fitted closer with les waste. However, an experienced cutter would oversee this process and make large scale changes, “what if we tried this large piece over here?” The computer’s activity would have been serial, but could have been parallelised as it involved effectively lots of small local decisions. However, the human made strategic decisions, that themselves made use of the human’s internal associative pattern recognition, but from the point of view of the large system were effectively more centralised. Here a combination of centralised and decentralised thinking/computation together addressed a problem neither could solve on their own.

Finally, it is interesting to reflect on the ability demonstrated in both Clark and Resnick’s writing. They look in at our modes of thinking, see that they are often over-simplistic in terms of assuming central control when there is none, and then consider how to address this. This highly reasoned and reflective process does not arise naturally from decentralised thought that would simply go on using the same old ways of thinking, but is the product of exactly the more ‘rational’ linear, centralised thinking that they seek to expose as outmoded.

1. Mitchel Resnik (1994). Turtles Termites and Traffic Jams: Explorations in Massively Parallel Microworlds. MIT Press. [back]
2. This is similar to the argument in my previous post on the power of sequential thinking, where I pondered the complexity of establishing sequence within an underlying parallel and distributed neural superstructure … but also discussed the advantages it brings. Sequentiality and central control are of course closely linked. [back]

This reminded me of results of brain scans (sadly, I can’t recall the source), which showed that the areas in the brain where you store concepts like ‘chair’ are different form those where you store the sound of the word … and also I’m sure the spelling of it also.

This makes sense of the ‘tip of the tongue’ phenomenon, you know that there is a word for something, but can’t find the exact word. Even more remarkable is that of you know words in different languages you can know this separately for each language.

So, musing on this, there seem to be very good reasons why, even within our own mind, we hold multiple representations for the ‘same’ thing, such as chair, which are connected, but loosely coupled.

In an artificial ‘brain’ like COG, the computational units are physically separate. In our brains things are much less discrete, but we know do have well defined locality for certain functions (e.g. Broca’s area for speech). Also, while there is debate about the extent to which we know what we are doing … or perhaps more important know what we are about to do … still it is clear that at least low-level functions operate semi-autonomously, but for instant reaction (pain withdrawal) and also for controlled actions (play a guitar chord).

In particular, there are particular brain lesions that mean that the patient cannot choose what to say, yet still vaguely grammatical but entirely meaningless utterances are constantly made … rather like James Joyce In between are cases where ‘nearly right’ words come out, perhaps table, or cushion instead of chair. So for speech it seems a ‘higher level’ part of the brain decides what we want to say and makes gentle suggestion for what this should be, but a different part does the final stringing together of words and it is this part that ‘knows’ the rules of grammar, the way words connect into each other and what the words sound like (although the grammar and aural elements may themselves be generated separately). This is rather like the intention to walk and the fine movement of muscles needed to move each leg.

Obviously the representations needed for saying ‘chair’ in a sentence are about the way it fits into grammatical structure, agreement with verbs, the sound of the word, and eventually (maybe at another level again) the way the lips need to be formed and air expelled. In contrast for choosing what to say, it is the semantics of chairs, the fact that you sit on them, they have legs, etc. that are important. The ‘planning to speak’ bit needs to know there is a suitable word, what that word means and whether it will fit with other words, but does not need to know the details of how to say it. Similarly the ‘planning to move’ bit needs to know roughly that legs can move in the desired way, but not the details of movement. Planning needs a model of action (speech or movement) and the model needs to be close enough to reality for it to work most of the time, but without all the details.

So the representations at a higher level need to share or independently represent enough of the lower level functions to be able to make appropriate suggestions for lower level action, but each will also represent different things. In addition, there needs to be some linkage between the two representations. Suppose you form the intention to say something like ‘The chair has four legs’. (Note ‘something like’ because as you form the intention to say the thing the exact words will probably not be there.) In order for this intention to lead to the words ‘The chair has four legs’, something has to link the planning ‘chair’, with the saying ‘chair’.

This is not unlike human communication; we need both shared vocabulary and a level of shared meaning: so that when I say to a garage mechanic ‘the clutch is not working’ it is the same thing we are referring to. However, we can also each have additional meanings, annotations etc.: the mechanic will know how the clutch works better than I do.

However, whereas human verbal communication has to be pushed through a discrete medium of signs, it seems more likely that there is a level of direct (but diffuse) connection between the ‘concepts’ used in our brains at different levels of activity.

1. Andy Clark. Being There. MIT Press. 1997. ISBN 0-262-53156-9. book@MIT [back]

“that’s impossible” he said.

“why” I asked, “I’m not saying they are similar, just that there is the same computational potential”

“Computers are sequential” he said, “brains are associative”.

Further attempts to reason, likening it to other forms of simulation or emulation, simply met with the same flat response, a complete unwillingness to entertain the concept.

Partly this is to do with the feeling that this somehow diminishes us as people, what for me was a form of play with numbers, for him was perhaps an assault on his integrity as a human. I guess as a Christian I’m used to the idea that the importance of a person is not that we are clever or anything else, but that we are loved and chosen. So, I guess, for me this is less of an insult to my idea of being who I am.

This aside it is interesting that the reason given was about the mode of computation: “computers are sequential” vs. the massively parallel associativity of the human brain.

Of course if the computational substrate is all the PCs connected to the Intenet then this is hardly purely sequential and in fact one of the reasons that you could not ‘run’ a brain simulation on the Internet is that communication is too slow. Distributed computation over 100s of millions of PCs on the internet could not synchronise in the way that long-range synapses do within our brains.

Amongst other things it is suggested that our sense of consciousness is connected with the single track of synchronised activity enabled by the tight interconnections and rapid feedback loops within our brains¹. In contrast, individual computers connected to the onternet compute far faster than they can communicate, there could be not single thread of attention switching at the rate that our minds can.

If the internet were to think it would be schizophrenic.

Sequence is also imprtant in other ways. As the man said, our brians are associative. When considering spreading activation mechanisms for intelligent internet interfaces, one of the problems is that associative stuff gets ‘mixed up’. If London has a high level of activation, why is that? In a designed computational framework it is possible to consider mutiple ‘flavours’ of activations spreading through a network of concepts, but our brains do not do this, so how do they mange to separate things.

Now to some extent they don’t – we get an overall feel for things, not seeing the world as little pieces. However, it is also important to be able to more or less accurately ascribe feelings and associations to things. Consider one of those FBI training ranges were bank terrorists and hostages pop out from behind windows or doors. Your aim is to shoot the terrorists and save the hostages. But, if you see a robber holding a hostage how do you manage to separate the ‘bad and kill’ feelings and properly ascribe them only to the terrorist and not the hostage.

The answer may well be due exactly to the switching of attention. Even with both terrorist and hostage are next to each other, as mental attention shifts momentarily to one and then the other, the mental associations also shift. Rodney Cotterill in Enchanted Looms describes two levels of attention switich². One near conscious and taking around 500ms and one connected with more low-level visual attention (sometimes called a visual searchlight) at 20-50ms. It is probably the slower timescales that allow fuller webs of association to build and decay, but maybe there are other intermediate timescales of attention switching as well.

If this is right then the rapid sequential shifts of attention could be essential for maintaining the individual identity of percepts and concepts.

If we look at concepts on their own, another story of sequence unfolds.

There is a bit of a joke among neuroscientists about grandmother cells. This is the idea that there is a single neuron that in someway encodes or represents your grandmother³

Looking at this purely from a computing science perspective, even if there were not neurological reasons for looking for more distrubuted representations, there are computational ones. If concepts were stored in small local assemblies of neurons (not single ones to allow some redundancy and robsutness) and even a reasonably large part of our brains were dedicated to concept memory, then there just seems too few ‘concept-slots’.

If we used 100 neurons per concept and 10% of the brain for concept memory, we would only have space for around 10 million concepts. A quick scan through the dictionary suggests I have a reconition vocabuary of arounf 35,000 words, so that means I’d have less than 300 other concepts per dictinary word one. Taking into account memories of various kinds, it justs seems a little small. If we take into account the interconnections then we have plenty of potential long-term storage capacity (1/2 petabyte or so), but not if we try to use indiviudal groups of neorons to represent things. Gradmother cells are simpy an inefficient use of neurons!

Now there is also plenty of neurological evidence for more distributed storage. Walter Freeman describes how he and his team lovingly chopped the tops off rabbits’ skulls, embeded electrodes into their olfactory bulbs and then gently nursed them back to health⁴. The rabbits were then presented with different smells and each smell produced a distinctive pattern of neuron firings, but these patterns exteded across the bulb, not localised to a few neurons.

If neurons had ‘continuous’ levels of activation it would be possible to represent things like “1/2 think it is a dog 1/2 think it is a fox”, simply as an overlay of the activation of each. However, if this were the case, and one could have in mind any blend of concepts, then an assembly of N neurons would still only be able to encode up to N concepts as the concepts patterns would form a set of basis vectors for the N-dimensional vector space of possible activation levels (a bit of standard linear algebra).

In fact, neurons tend to behave non-linearly and in many areas there are patterns of inhibition as well as mutual excitement and disinhibition, leading to winner-takes-all effects. If this is true of the places where we represent concepts for short term memory, conscious attention, etc., then this means instead of representations that ‘add up’, we have each pattern potentially completely different, similar to the way binary numers are encoded in computer memory: 1010 is not a combination of 1000 and 0010 but completely different.

In principle this kind of representation allows 2^N (two to the power of N) rather than N different concepts using the same N neurons … In reality, almost certainly representations are less ‘precise’ allowing some levels of similarity in representations etc., so the real story will be more complex, but the basic principle holds that combinations of thresholding and winner-takes-all allow more distinct concepts than would be possible if combinations of concepts can occur more freely.

However, notice again that higher capacity to deal with more concepts is potentially bought at the cost of being able to think of less things ‘at once’ – and the side effect is that we have to serialise.

Returning back to the “computers are sequential, brains are associative” argument, whilst not denying the incredible parallel associativity of human memory, actually there seems as much to wonder about in the mechanisms that the brain ‘uses’ for sequentiality and the gains it gets because of this.

1. see Gerald Edelman, Wider then the Sky, Yale University Press, 2004, ISBN 0-300-10229-1 [back]
2. Rodney Cotterill, Enchanted Looms: Conscious Networks in Brains and Computers, Cambridge University Press, 1998, ISBN 0-521-62435-5. See p. 244 for 500ms switching and pp. 261 and 265 for 20-50ms spotlight/searchlight of attention [back]
3. Although the grandmother cell this is generally derided as oversimplisitic, there is evidence that there is more neuron specialisation then previously thought [[see Mind Hacks: evidence for 'Grandmother Cells']]. Also it is easier to encode relationships if there are single patches than configuratiin sof neurons, so perhaps we have both mechanisms at work. [back]
4. Walter J. Freeman, How Brains Make Up Their Minds, Phoenix, 1999, ISBN 0-75381-068-9. See p. 95 onwards for rabbit olfactory bulb experiments. [back]

The book is largely about one message – that a scientific study of consiousness can only take into account third party accessible knowledge about first part experience. In other words I can only base a scientific study on what I can tell of other people’s consciousness from their actions, words and any available brain scanning etc.

Dennett has a meticulous rhetoric, but I found two broad facets of his argument weak, one more about rheteric and one substance.

First somewhat disingenuously he does not say that a scientific study of consciousness would yield a complete account of consciouness, but effectively the implication is there. That is he does not say that consciouness is no more than its phenomenial effects … but implies it.

Second, being a philosopher he focuses on incontrovertible evidence, whereas as scientists and humans often reasonable evidence is sufficient.

The first point is obvious and yet easily underestimated. A ’scientific’ study of literature could formulate many known literary rules (aliteration, rhyme, etc.) and may even find new ones, and indeed poets in particular are happy to perform such analyses. However, we do not expect such rules to be a complete account of literture.

The second point is more substantive, but does interact with the first.
Dennett takes issue with philosophers who posit some form of non-sentient zombie (often called ‘Mary’) who/which nonetheless behaves otherwise exactly like a human including things that might appear to be conscious. They then say “but of course Mary is not conscious”. Dennett objects to the ‘of course’, which is really a statement about prior beliefs/assumptions (although Dennett, of course, frequently does the same with his beliefs!).

Dennett posits a Robo-Mary which is entirely mechanical/electronic and yet emulates perfectly the brain circuitry of a person and so can work out how the person would react and then reacts similarly. From the outside and by all her (emulated) subjective reactions she appears to be conscious. She would pass any ‘Turing Test’ for consciousness and yet many, perhaps most, would say she is not. The implication (from the first weakness) is that we are no more conscius than she (it?).

Actually I don’t object to the idea that such a creature may indeed be conscious, but I’d need more evidence than I would for a human, not because Robo-Mary is a machine, but becasue she is designed to appear conscious.

Robo-Mary is in fact a Robo-Mata-Hari, a spy, a robot in human clothing.

A good enough actor may convince you he is feeling happy, sad, or in love, and you may not be able to tell the differece between the act and the real thing, but that does not mean happiness, saddness and love are no more than their appearance.

As a philosopher, you cannot have incontrovertible evidence that a person’s emotions are real, not just a facade. However, as a human it would be unreasonable to therefore dismiss all expressions of emotion.

Some (well many) years ago, I worked with people at York who creating one of the first ADA compilers. There was a validation suite of programs that had to compile and run correctly for the compiler to get an official stamp from the ADA standards agency. I used to wonder about writing a program that recognised each of the tests cases and simply spat out the right code for each one. Any other program given to the program would simply print an error message and stop. The program would pass the test suite and could get the stamp as being a validated compiler, and yet would be completely useless. It would be a cheat ADA compiler.

Imagine if I sold such a cheat compiler. Any judge would regard it as fraud – whilst it passed the test, it is clearly not an ADA compiler. The test is there to validate things that are designed to be ADA compilers, not things designed to pass the test. So, the cheat ADA compiler is not adequately validated by the test, just becase it is designed to pass it.

Robo-Mary is designed to pass the consciousness test … indeed any consciousness test. We perhaps could never incontrovertibly tell whether Robo-Mary was conscious or simply acting conscious. However, when faced with another human being, an ordinary Mary, who is not designed specifically to appear conscious, it is reasonable to assume that she experiences similar things to me when she describes her experience in similar terms. I can never incontrovertibly tell that Mary is conscious, but it is reasonable to believe so. And it is equally reasonable to base a scientific study on such defeasible observations.

Turning back to Robo-Mary; convincing machine cosciousness would not come from machines designed to appear conscious, but more ‘by accident’. Perhaps one day my intelligent automated vacuum cleaner will say to me “Alan, have you ever watched those dust motes in the sunlight”.