Last night I both learnt something new about language and cognition, and also developed a new trick for creativity!

In the dream in question I was in a meeting. I know, a sad topic for a dream, and perhaps even sadder it had started with me filling in forms!  The meeting was clearly one after I’d given a talk somewhere as a person across the table said she’d been wanting to ask me (obviously as a sort of challenge) if there was a relation between … and here I’ll expand later … something like evolutionary and ecological something.  Ever one to think on my feet I said something like “that’s an interesting question”, but it was also clear that the question arose partly because the terms sounded somewhat similar, so had some of the sense of a rhyming riddle “what’s the difference between a jeweller and a jailor”.  So I went on to mention random metaphors as a general creativity technique and then, so as to give practical advice, suggested choosing two words next to each other in a dictionary and then trying to link them.

Starting with the last of these, the two words in a dictionary method is one I have never suggested to anyone before, not even thought about. It was clearly prompted by the specific example where the words had an alliterative nature, and so was a sensible generalisation, and after I woke realised was worth suggesting in future as an exercise.  But it was entirely novel to me, I had effectively done the exactly sort of thinking / problem solving that I would have done in the real life situation, but while dreaming.

One of the reasons I find dreams fascinating is that in some ways they are so normal — we clearly have no or little sensory input, and certain parts of our brain shut down (e.g. motor control to stop us thrashing about too much in our sleep) — but other parts seem to function perfectly as normal.  I have written before about the cognitive nature of dreams (including maybe how to model dreaming) and what we may be able to learn about cognitive function because not everything is working, rather like running an engine when it is out of the car.

In this dream clearly the ‘conscious’ (I know an oxymoron) problem-solving part of the mind was operating just the same as when awake.  Which is an interesting fact about dreaming, but  I was already aware of it from previous dreams.

In this dream it was the language that was interesting, the original conundrum I was given.  The problem came as I woke up and tried to reconstruct exactly what my interlocutor had asked me.  The words clearly *meant* evolutionary and ecological, but in the dream had ‘sounded’ even closer aurally, more like evolution and elocution (interesting to consider, images of God speaking forth creation).

So how had the two words sound more similar in my dream than in real speech?

For this we need the Jabberwocky circuit.

There is a certain neurological condition that arises, I think due to tumours or damage in particular areas of the grain, which disrupts particular functions of language.   The person speaks interminably; the words make sense and the grammar is flawless, but there is no overall sense.  Each small snippet of speech is fine, just there is no larger scale linkage.

When explaining this phenomenon to people I often evoke the Jabberwocky circuit.  Now I should note that this is not a word used by linguists, neurolinguists, or cognitive scientists, and is a gross simplification, but I think captures the essence of what is happening.  Basically there is a part of your mind (the conscious, thinking bit) that knows what to say and it asks another bit, the Jabberwocky circuit, to actually articulate the words.  The Jabberwocky circuit knows about the sound form of words and how to string them together grammatically, but basically does what it is told.  The thinking bit needs to know enough about what can be said, but doesn’t have time to deal with precisely how they are strung together and leaves that to Jabberwocky.

Even without brain damage we can see occasional slips in this process.  For example, if you are talking to someone (and even more if typing) and there is some other speech audible (maybe radio in the background), occasionally a word intrudes into your own speech that isn’t part of what you meant to say, but is linked to the background intruding sound.

Occasionally too, you find yourself stopping in mid sentence when the words don’t quite make sense, for example, when what would be reasonable grammar overlaps with a colloquialism, so that it no longer makes sense.  Or you may simply not be able to say a word that you ‘know’ is there and insert “thingy” or “what’s it called” where you should say “spanner”.

The relationship between the two is rather like a manager and someone doing the job: the manager knows pretty much what is possible and can give general directions, but the person doing the job knows the details.  Occasionally, the instructions get confused (when there is intruding background speech) or the manager thinks something is possible which turns out not to be.

Going back to the dream I thought I ‘heard’ the words, but examining more closely after I woke I realised that no word would actually fit.  I think what is happening is that during dreaming (and maybe during imagined dialogue while awake), the Jabberwocky circuit is not active, or not being attended to.  It is like I am hearing the intentions to speak of the other person, not articulated words.  The pre-Jabberwocky bit of the mind does know that there are two words, and knows what they *mean*.  It also knows that they sound rather similar at the beginning (“eco”, “evo”), but not exactly what they sound like throughout.

I have noticed a similar thing with the written word.  Often in dreams I am reading a book, sheet of paper or poster, and the words make sense, but if I try to look more closely at the precise written form of the text, I cannot focus, and indeed often wake at that point¹.  That is the dream is creating the interpretation of the text, but not the actual sensory form, although if asked I would normally say that I had ‘seen’ the words on the page in the dream, it is more that I ‘see’ that there are words.

Fiona does claim to be able to see actual letters in dreams, so maybe it is possible to recreate more precise sensory images, or maybe this is just the difference between simply writing and reading, and more conscious spelling-out or attending to words, as in the well known:

Paris in the
the spring

Anyway, I am awake now and the wiser.  I know a little more about dreaming, which cognitive functions are working and which are not;  I know a little more about the brain and language; and I know a new creativity technique.

Not bad for a night in bed.

What do you learn from your dreams?

1. The waking is interesting, I have often noticed that if the ‘logic’ of the dream becomes irreconcilable I wake.  This is a long story in itself, but I think similar to the way you get a ‘breakdown’ situation when things don’t work as expected and are forced to think about what you are doing.  It seems like the ‘kick’ that changes your mode of thinking often wakes you up! [back]

Earlier today I was using Google to look up the author of a book I was reading and one of the top links was a listing on ratemyprofessors.com.  For interest I clicked through and saw:

“He sucks.. hes mean and way to demanding if u wanan work your ass off for a C+ take his class¹”

Hmm I wonder what this student’s course assignment looked like?

and a little web-usability tag story

In case you are wondering, yes I did try to look to see if I was listed (although I am sure all my British students have perfect grammar :-/ ).  However, unfortunately (or maybe fortunately), I could not tell.  In order to search UK universities you need to use a pull-down menu.  But², of course, being an up-to-date and cool site, ratemyprofessors.com uses a funky Javascript+DOM menu not a plain HTML-form one .  Notice the little gap between the button for the pull-down and the menu itself.  As you try to move your mouse over the menu it disappears!  So for a Firefox user like me it is a US-only site.

1. In case you think I’m a complete pedant, personally, I am happy with both the slang ‘sucks’ and ‘ass’ (instead of ‘arse’!), and the compressed speech ‘u’. These could be well-considered choices in language. The mistyped ‘wanna’ is also just a slip. It is the slightly more proper “hes mean and way to demanding” that seems to show  general lack of understanding.  Happily, the other comments, were not as bad as this one, but I did find the student who wanted a “descent grade” amusing [back]
2. Yes, that was a sentence starting with a conjunction.  And, yes, you may have heard this is bad grammar, but only when used carelessly; see “Grammar- Starting a Sentence with Or, And or But” @ Suite101 or “Beginning a Sentence with And or But” for an apposite quote. [back]

Reading language

“The parrot, which was continually watched by the big black cat, who sometimes sat on the mat until displaced by the bigger brown Doberman and sometimes on the garden wall in the hope of catching the sparrow, flew through the window.” (sentence 1)

Why is this difficult to read?  It is grammatically well-formed and has commas in all the right places.  However, the large subordinate clause (if I have got my grammatical terms right) means that by the time you have go to “flew through the window” you have ‘forgotten’ the parrot who it was all about.

In contrast, a computer parser has no such trouble. Take the fragment:

if ( y > 10 ) {
    x1 = y+1;
    x2 = y+2;
    . . .   // 97 more lines of code
    x100 = y+100
} else { . . .

The compiler does not ‘forget’ about the ‘if’, because it is part of a relatively simple syntactic structure:

if ( condition ) { statement_block } else { statement_block }

In the English sentence about the parrot, like the program fragment, even the ‘depth’ of the parsing stack is not deep, yet still there is confusion.
Years ago when I was a student in Cambridge a friend who was studying Politics came to me with a passage in Nozick’s “Anarchy State and Utopia“.  I read the paragraph in question several times.  Each time it made sense, until I came to the last few words, which seemed to be a total non sequitur.  Only after many readings did I realise that the paragraph was a single sentence (all 10 lines of it!), and that the sentence was of precisely the same form as the parrot one, except with nested sub-clauses.   The words that appeared to come from nowhere were in fact the equivalent of “flew through the window” … indeed imagine if the description of the Doberman had extended to another seven lines!

Clearly our ability to hold on to relatively simple hierarchical parsing structures is quite limited, and is definitely smaller than the 7+/- 2 for working memory¹.  However, there is also another mechanism at work.

Consider the following alternative sentences:

“The policeman, which was continually watched by the big black cat, who sometimes sat on the mat until displaced by the bigger brown Doberman and sometimes on the garden wall in the hope of catching the sparrow, flew through the window.” (sentence 2)

“The parrot, which was continually watched by the big black cat, who sometimes sat on the mat until displaced by the bigger brown Doberman and sometimes on the garden wall in the hope of catching a snail, flew through the window.” (sentence 3)

It is hard to read these ‘fresh’ as if you hadn’t read the initial sentences, but notice that the first is far harder than the second.  This is because we have an additional sequential associative mechanism at work as well as syntactic parsing.

Consider instead the much simpler  ‘sentence’:

“bone dog eat”

We have little problem making sense of this even though the structure is not grammatical.  The verb ‘eat’ is ‘looking for’ something that eats to act as its subject, and something that can be eaten to be its object.  As there is only one eatable and one eater in the ‘sentence’, the parsing (or perhaps connecting) happens purely by association.

If there are several ‘matches’ for the words then the word combinations become ambiguous:

“policeman demonstrator hit”

Some languages, such as English, largely deal with this ambiguity by use of syntactic rules (subject verb object), others, such as Latin, by tagging the words with their role … and I don’t know my Latin enough to give a proper example, but I think something like, in pigeon Latin: “policemanus demonstratori hit” vs “policemani demonstratorus hit”!  I believe Latin has quite strong syntactic rules as well, but there are certainly languages where order is less significant, you can think of them as Lego block languages; you throw the words into a bag, shake them up, and they stick together where the words fit together.

When we are reading a sentence both forms of parsing are happening at once.  There is a level of grammatical parsing, but also when a word, like ‘parrot’ in sentence 1, is encountered and does not get ‘bound’ it is ‘waiting’ for a verb to connect to.  Later in the sentence,  when we encounter ‘flew’ it wants something that can fly to be its subject, if ‘parrot’ is still sufficiently in mind then all is well and we get “The parrot flew through the window”.  However, the last thing we read was ‘sparrow’ a thing that can fly.  The sequential associative parsing says “sparrow flew” and even if  the syntactic parsing ‘wins’, the conflict still makes the sentence hard to grasp.

Sentence 2 was chosen to make the binding of the grammatical subject ‘policeman’ unusual to bind to ‘flew’ and so more confusing, whereas sentence 3 is less confusing as there is no obvious flying candidate except the parrot.

Poets make good use of these multiple mechanisms as it allows them to bend grammar and yet have (sometimes) comprehensible sentences.  For my students it is not usually poetry I am after in their PhD theses, but understanding the role of sequential parsing can help us to construct more comprehensible English.

Reading action

It was only quite recently I realised there was a very close parallel to work we had been doing in the TIM project on tasks² – that is action rather than language.

There are various algorithms used in intelligent user interface research to help predict the next action based on the previous one.  Mostly these make use of some sort of Markov model or similar sliding window where the next command is the one that has occurred most often after the preceding N commands.  However, there have been various proposals for performing more hierarchical pattern formation effectively building task structures as found in hierarchical task analysis.

A fundamental block to both approaches is that human activity is not always focused on one task at a time, but we do a little of one, then a little of another, just as I might break off from writing to make a cup of tea.

In the TIM project we have been using a personal ontology for various purposes including helping to propose values for actions such as web forms.  If there is a ‘name’ field, then the ontology includes names of friends so these can be proposed as potential options to the user.

Because of the ontology, we are also able to detect that the telephone number used in one web form is the telephone number of the person in the mail that has just been received.  This semantic binding through the ontology means that we can effectively see that individual actions are or are not related depending on whether they have semantic relationships through the ontology, that is we can thread together the actions that belong a single task sequence and pull them out of the chronological sequence that involves a mix of task and activities.

There are many differences between this and the sequential associative parsing, not least that the basic actions we are considering are typically parameterised (the completion of a web form) while words are singular, but there are also connections.  Previously, the relationship between task analysis and traces of actions has been compared with that between grammar and sentences, basically treating task analysis as a way to ‘parse’ activity³. This parallels the conventional parsing approach.  However, what I only belatedly realised was the use of semantic relationship between actions we are adopting in the TIM project also parallels the use of semantic connections between words in our task inference.

Reading the world

The above is about the way a computer can make sense of human action.   However, as humans we need to make sense of the things that happen around us in the world not least other people.

When we sense the world we are confronted with myriad stimuli, as I write the sound of an ambulance passing outside, the insistent hiss of the kettle on the stove, all clamour against the words I type.  Against this visual, aural and tactile cacophony, we have to unravel the threads of meaning.  If my leg aches I do not ascribe it to the cup of tea I am drinking, or the ambulance that recently passed, but instead to the run round the Circo Massimo earlier this morning.  However, equally I do not ascribe it to running on the beach a weak ago.  Temporal and semantic proximity both play their part in our reading of the signs of day-to-day life.  It is not surprising that these same tools are at play as we listen or read language.

We also perceive patterns of actions in the world, most significantly in the actions of other creatures.  In a keynote at Tamodia in Pisa last year, I discussed the production of human action and the rich intertwining of sequential planned or proceduralised activity with more stimulus-driven reactions.  Both could be explicit or tacit.  The computer aid is trying to ‘read’ these patterns of intentional (and reactive) activity.

It is of course likely that our ability to produce and understand language builds on our ability to act within and to comprehend the world.  However, the key difference is that language is about intentional communication.  As well as expecting parts to link together in fragments, we also expect the whole to make sense.  In a detective novel each incident may seem disparate, but we are sure that in the end they will come together in the dénouement.  This is a large-scale application of Grice’s cooperation principle.  For us interpreting the world and each other’s actions, or for the computer interpreting human action, no such overarching message is expected with the exception of life as a whole: God, or death, depending on your beliefs, as the ultimate dénouement.

1. George A. Miller. The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information. The Psychological Review, 1956, vol. 63, pp. 81-97 [back]
2. The best account of this task inference at present is probably my keynote Tasks = data + action + context: automated task assistance through data-oriented analysis at Engineering Interactive Systems 2008 last September in Pisa. [back]
3. Personally, I first used this ‘task analysis as grammar’ approach in teaching task analysis and it appeared in the task model slides for the third edition if the HCI book (but not in the book itself, maybe fourth edition).  However, it has been elaborated in work with Stavros Asimakopoulos and Robert Fyldes “Grammatically interpreted task analysis for supply chain forecasting“. [back]

I suspect the word arose because ‘thus’ often precedes a verb and so has acquired an adverb ‘-ly’ ending … a form of lexical over-generalisation. … and if you have not already stopped reading because you can’t understand why anyone would be even interested in this level of language … as I did the web search found a lovely discussion site called INTERACTION (and no, this one is not the synonymous BCS HCI Group) where one person mildly mocked another for using ‘thusly’ in the 21st Century … obviously not a Lancashire lad.

This has also made me reflect on my own frequent use of ‘thus’ to mean ‘therefore’ as opposed to ‘in this way’ … argh am I really a grammar nerd :-/