I like a nice cup of tea in the morning,
Just to start the day you see;
And at half past eleven,
Well my idea of heaven,
Is a nice cup of tea.

I like a nice cup off tea with my dinner,
And a nice cup of tea with my tea,
And about this time of night,
What goes down a treat, you’re right,
It’s a nice cup of tea.

As well as the deep truth underlying the words, I suddenly became aware of the beginning of the second stanza: “a nice cup of tea with my dinner, and a nice cup of tea with my tea“.

I’d guess the last part of this may be confusing to a non-UK audience, or it may conjure up images of period-drama afternoon tea with cucumber sandwiches and parasols over a game of croquet.

Now the meaning of ‘dinner’ has been a matter of discussion in my household for years.

When I was a child ‘dinner’ was the light meal in the middle of the day, whereas ‘tea’ was the main meal at around 6 o’clock.

In contrast, Fiona takes a more pragmatic approach: ‘dinner’ is the main meal whether taken midday or in the evening.

My impression is that, when I was a child, this was part of a general class distinction. Posh (middle class) people ate lunch at midday, dinner in the evening, watched BBC and drank coffee. The working class ate dinner at midday, ate tea in the evening, watched ITV (the channel with adverts), and drank tea.

Weirdly in school one had ‘school dinners’ or ‘free dinners’ if on benefits, but had ‘packed lunches’.

We have sometimes discussed whether the tea/dinner distinction was more a Welsh-ism. But the advertising jingle clearly shows it was widespread².

Now-a-days I tend to use the words rather interchangeably, and certainly happy to use ‘lunch’. Is this because I have become part of the professional classes or a general shift of language?

What do you call meals? Is it the same as when you were little? Is it still a class distinction?

1. According to responses in AnswerBank, this was from an original 1937 song for Brook Bond ‘D’ brand … and in fact the word ‘tea’ was replaced by ‘D’ … but I obviously missed this and remember it as ‘tea’!  The original lyrics have slightly different final lines, “And when it’s time for bed, There’s a lot to be said, For a nice cup of tea“, or maybe I simply misremembered the advert. [back]
2. even in 1937 [back]

I recall my own interest in the origins of language began young, as a seven year old over breakfast one day, asking whether ‘night, was a contraction of ‘no light’.  While this was an etymological red herring, it is very much the kind of change that Deutscher documents in detail showing the way a word accretes beginnings and ending through juxtaposition of simpler words followed by erosion of hard to pronounce sounds.

One of my favourites examples was the French “aujourd’hui”.  The word ‘hui, was Old French for ‘today’, but was originally Latin “hoc die”, “(on) this day”. Because ‘hui’ is not very emphatic it became “au jour d’hui”, “on the day of this day” , which contracted to the current ‘aujourd’hui’. Except now to add emphasis some French speakers are starting to say “au jour aujourd’hui”, “on the day on the day of this day”!  This reminds me of Longsleddale in the Lake District (inspiration for Postman Pat‘s Greendale),  a contraction of “long sled dale”, which literally means “long valley valley” from Old English “slaed” meaning “valley” … although I once even saw something suggesting that ‘long’ itself in the name was also “valley” in a different language!

Deutscher gives many more prosaic examples where words meaning ‘I’, ‘you’, ‘she’ get accreted to verbs to create the verb endings found in languages such as French, and how prepositions (themselves metaphorically derived from words like ‘back’) were merged with nouns to create the complex case endings of Latin.

However, the most complex edifice, which Deutscher returns to repeatedly, is that of the Semitic languages with a template system of vowels around three-consonant roots, where the vowel templates change the meaning of the root.  To illustrate he uses the (fictional!) root ‘sng’ meaning ‘to snog’ and discusses how first simple templates such as ‘snug’ (“I snogged”) and then more complex constructions such as ‘hitsunnag’ (“he was made to snog himself”) all arose from simple processes of combination, shortening and generalisation.

“The Unfolding of Language” begins with the 19th century observation that all languages seem to be in a process of degeneration where more complex  forms such as the Latin case system or early English verb endings are progressively simplified and reduced. The linguists of the day saw all languages in a state of continuous decay from an early linguistic Golden Age. Indeed one linguist, August Schleicher, suggested that there was a process where language develops until it is complex enough to get things done, and only then recorded history starts, after which the effort spent on language is instead spent in making history.

As with geology, or biological evolution, the modern linguist rejects this staged view of the past, looking towards the Law of Uniformitarianism, things are as they have always been, so one can work out what must have happened in the pre-recorded past by what is happening now.  However, whilst generally finding this convincing, throughout the book I had a niggling feeling that there is a difference.  By definition, those languages for which we have written records are those of large developed civilisations, who moreover are based on writing. Furthermore I am aware that for biological evolution small isolated groups (e.g. on islands or cut off in valleys) are particularly important for introducing novelty into larger populations, and I assume the same would be true of languages, but somewhat stultified by mass communication.

Deutscher does deal with this briefly, but right at the very end in a short epilogue.  I feel there is a whole additional story about the interaction between culture and the grammatical development of language.  I recall in school a teacher explained how in Latin the feminine words tended to belong to the early period linked to agriculture and the land, masculine words for later interests in war and conquest, and neuter for the still later phase of civic and political development. There were many exceptions, but even this modicum of order helped me to make sense of what otherwise seemed an arbitrary distinction.

The epilogue also mentions that the sole exception to the ‘decline’ in linguistic complexity is Arabic with its complex template system, still preserved today.

While reading the chapters about the three letter roots, I was struck by the fact that both Hebrew an Arabic are written as consonants only with vowels interpolated by diacritical marks or simply remembered convention (although Deutscher does not mention this himself). I had always assumed that this was like English where t’s pssble t rd txt wth n vwls t ll. However, the vowels are far more critical for Semitic languages where the vowel-less words could make the difference between “he did it” and “it will be done to him”.  Did this difference in writing stem from the root+template system, or vice versa, or maybe they simply mutually reinforced each other?

The other factor regarding Arabic’s remarkable complexity must surely be the Quran. Whereas the Bible was read for a over a millennium in Latin, a non-spoken language, and later translated focused on the meaning; in contrast there is a great emphasis on the precise form of the Quran together with continuous lengthy recitation.  As the King James Bible has been argued to have been a significant influence on modern English since the 17th century, it seems likely the Quran has been a factor in preserving Arabic for the last 1500 years.

Early in “The Unfolding of Language” Deutscher dismisses attempts to look at the even earlier prehistoric roots of language as there is no direct evidence. I assume that this would include Mithin’s “The Singing Neanderthals“, which I posted about recently. There is of course a lot of truth in this criticism; certainly Mithin’s account included a lot of guesswork, albeit founded on paleontological evidence.  However, Deutscher’s own arguments include extrapolating to recent prehistory. These extrapolations are based on early written languages and subsequent recorded developments, but also include guesswork between the hard evidence, as does the whole family-tree of languages.  Deutscher was originally a Cambridge mathematician, like me, so, perhaps unsurprisingly, I found his style of argument convincing. However, given the foundations on Uniformitarianism, which, as noted above, is at best partial when moving from history to pre-history, there seems more of  a continuum rather than sharp distinction between the levels of interpretation and extrapolation in this book and Mithin’s.

Deutscher’s account seeks to fill in the gap between the deep prehistoric origins of protolanguage (what Deutscher’s calls ‘me Tarzan’ language) and its subsequent development in the era of media-society (starting 5000BC with extensive Sumerian writing). Rather than seeing these separately, I feel there is a rich account building across various authors, which will, in time, yield a more complete view of our current language and its past.

“The Singing Neanderthal”, as its title suggests, is about the role of music in the evolutionary development of the modern human. We all seem to be born with an element of music in our heart, and Mithin seeks to understand why this is so, and how music is related to, and part of the development of, language. Mithin argues that elements of music developed in various later hominids as a form of primitive communication², but separated from language in homo sapiens when music became specialised to the communication of emotion and language to more precise actions and concepts.

The book ‘explains’ various known musical facts, including the universality of music across cultures and the fact that most of us do not have perfect pitch … even though young babies do (p77). The hard facts of how things were for humans or related species tens or hundreds of thousands of years ago are sparse, so there is inevitably an element of speculation in Mithin’s theories, but he shows how many, otherwise disparate pieces of evidence from palaeontology, psychology and musicology make sense given the centrality of music.

Whether or not you accept Mithin’s thesis, the first part of the book provides a wide ranging review of current knowledge about the human psychology of music. Coincidentally, while reading the book, there was an article in the Independent reporting on evidence for the importance of music therapy in dealing with depression and aiding the rehabilitation of stroke victims³, reinforcing messages from Mithin’s review.

The topic of “The Singing Neanderthal” is particularly close to my own heart as my first personal forays into evolutionary psychology (long before I knew the term, or discovered Cosmides and Tooby’s work), was in attempting to make sense of human limits to delays and rhythm.

Those who have been to my lectures on time since the mid 1990s will recall being asked to first clap in time and then swing their legs ever faster … sometimes until they fall over! The reason for this is to demonstrate the fact that we cannot keep beats much slower than one per second⁴, and then explain this in terms of our need for a mental ‘beat keeper’ for walking and running. The leg shaking is to show how our legs, as a simple pendulum, have a natural frequency of around 1Hz, hence determining our slowest walk and hence need for rhythm.

Mithin likewise points to walking and running as crucial in the development of rhythm, in particular the additional demands of bipedal motion (p150). Rhythm, he argues, is not just about music, but also a shared skill needed for turn-taking in conversation (p17), and for emotional bonding.

In just the last few weeks, at the HCI conference in Newcastle, I learnt that entrainment, when we keep time with others, is a rare skill amongst animals, almost uniquely human. Mithin also notes this (p206), with exceptions, in particular one species of frog, where the males gather in groups to sing/croak in synchrony. One suggested reason for this is that the louder sound can attract females from a larger distance. This cooperative behaviour of course acts against each frog’s own interest to ‘get the girl’ so they also seek to out-perform each other when a female frog arrives. Mithin imagines that similar pressures may have sparked early hominid music making. As well as the fact that synchrony makes the frogs louder and so easy to hear, I wonder whether the discerning female frogs also realise that if they go to a frog choir they get to chose amongst them, whereas if they follow a single frog croak they get stuck with the frog they find; a form of frog speed dating?

Mithin also suggests that the human ability to synchronise rhythm is about ‘boundary loss’ seeing oneself less as an individual and more as part of a group, important for early humans about to engage in risky collaborative hunting expeditions. He cites evidence of this from the psychology of music, anthropology, and it is part of many people’s personal experience, for example, in a football crowd, or Last Night at the Proms.

This reminds me of the experiments where a rubber hand is touched in time with touching a person’s real hand; after a while the subject starts to feel as if the rubber hand is his or her own hand. Effectively our brain assumes that this thing that correlates with feeling must be part of oneself⁵. Maybe a similar thing happens in choral singing, I voluntarily make a sound and simultaneously everyone makes the sound, so it is as if the whole choir is an extension of my own body?

Part of the neurological evidence for the importance of group music making concerns the production of oxytocin. In experiments on female prairie voles that have had oxytocin production inhibited, they engage in sex as freely as normal voles, but fail to pair bond (p217). The implication is that oxytocin’s role in bonding applies equally to social groups. While this explains a mechanism by which collaborative rhythmic activities create ‘boundary loss’, it doesn’t explain why oxytocin is created through rhythmic activity in the first place. I wonder if this is perhaps to do with bipedalism and the need for synchronised movement during face-to-face copulation, which would explain why humans can do synchronised rhythms whereas apes cannot. That is, rhythmic movement and oxytocin production become associated for sexual reasons and then this generalises to the social domain. Think again of that chanting football crowd?

I should note that Mithin also discusses at length the use of music in bonding with infants, as anyone who has sung to a baby knows, so this offers an alternative route to rhythm & bonding … but not one that is particular to humans, so I will stick with my hypothesis

Sexual selection is a strong theme in the book, the kind of runaway selection that leads to the peacock tail. Changing lifestyles of early humans, in particular longer periods looking after immature young, led to a greater degree of female control in the selection of partners. As human size came close to the physical limits of the environment (p185), Mithin suggests that other qualities had to be used by females to choose their mate, notably male singing and dance – prehistoric Saturday Night Fever.

As one evidence for female mate choice, Mithin points to the overly symmetric nature of hand axes and imagines hopeful males demonstrating their dexterity by knapping ever more perfect axes in front of admiring females (p188). However, this brings to mind Calvin’s “Ascent of Mind“, which argues that these symmetric, ovoid axes were used like a discus, thrown into the midst of a herd of prey to bring one down. The two theories for axe shape are not incompatible. Calvin suggests that the complex physical coordination required by axe throwing would have driven general brain development. In fact these forms of coordination, are not so far from those needed for musical movement, and indeed expert flint knapping, so maybe it was this skills that were demonstrated by the shaping of axes beyond that immediately necessary for purpose.

Mithin’s description of the musical nature of mother-child interactions also brought to mind Broomhall’s “Eternal Child“. Broomhall ‘s central thesis is that humans are effectively in a sort of arrested development with many features, not least our near nakedness, characteristic of infants. Although it was not one of the points Broomhall makes, his arguments made sense to me in terms of the mental flexibility that characterises childhood, and the way this is necessary for advanced human innovation; I am always encouraging students to think in a more childlike way. If Broomhall’s theories were correct, then this would help explain how some of the music making more characteristic of mother-infant interactions become generalised to adult social interactions.

I do notice an element of mutual debunking amongst those writing about richer cognitive aspects of early human and hominid development. I guess a common trait in disciplines when evidence is thin, and theories have to fill a lot of blanks. So maybe Mithin, Calvin and Broomhall would not welcome me bringing their respective contributions together! However, as in other areas where data is necessarily scant (such as sub-atomic physics), one does feel a developing level of methodological rigour, and the fact that these quite different theoretical approaches have points of connection, does suggest that a deeper understanding of early human cognition, while not yet definitive, is developing.

In summary, and as part of this wider unfolding story, “The Singing Neanderthal” is an engaging and entertaining book to read whether you are interested in the psychological and social impact of music itself, or the development of the human mind.

… and I have another of Mithin’s books in the birthday pile, so looking forward to that too!

1. See particularly my essay on the role of imagination in bringing together our different forms of ‘specialised intelligence’. “The Prehistory of the Mind” highlighted the importance of this ‘cognitive fluidity’, linking social, natural and technological thought, but lays this largely in the realm of language. I would suggest that imagination also has this role, creating a sort of ‘virtual world’ on which different specialised cognitive modules can act (see “imagination and rationality“). [back]
2. He calls this musical communication system Hmmmm in its early form – Holistic, Multiple-Modal, Manipulative and Musical, p138 – and later Hmmmmm – Holistic, Multiple-Modal, Manipulative, Musical and Mimetic, p221. [back]
3. “NHS urged to pay for music therapy to cure depression“, Nina Lakhani, The Independent, Monday, 1 August 2011 [back]
4. Professional conductors say 40 beats per minute is the slowest reliable beat without counting between beats. [back]
5. See also my previous essay on “driving as a cyborg experience“. [back]

However, I have been wonderfully forced by circumstances back to the page.

I have already written about one of these “The Shadow of the Wind” while I was travelling, itself a book about books, and while travelling I also read Kathryn Harrison’s “A Thousand Orange Trees” (an eye opening but unrelentingly depressing vision of women’s life during the Spanish inquisition), Frank McCourt’s “Angela’s Ashes” (a sometimes depressing, but also glorious account of a hard Irish childhood), Jodi Picoult’s “Change of Heart” (on the death penalty and religion) and Elizabeth Gaskell’s “Ruth” (a wonderful book about the small mindedness and great generosity of the human spirit, especially remarkable when seen against its time).

… and then I had some new books for Christmas …

At first Keith Gray’s “Ostrich Boys” seems like a classic boy’s book with three friends ‘kidnapping’ the ashes of their friend Ross in order to give him a proper send off at Ross in Scotland.   Like all journey stories, a mixture of going on against the odds and self discovery, not in the league of Cynthia Voigt’s “Homecoming“, but more likely to be read by boys wanting a good adventure and being stretched in the process.

Mal Peet’s “Tamar” is clearly for ‘young adults’, a claustrophobic tale of war time resistance in 1945 cut through with a ‘modern day’ tale.  This parallel tale is a hard genre, and, like Joan Lingard “Natasha’s Will“, I felt Peet managed the 1945 tale better than the current day one¹. Although Peet is writing for an older audience, I was reminded of the way Nina Bawden manages to get me to identify, however unwillingly at times, with the flawed characters in her children’s novels.

Susan Hill’s “Howards End is on the Landing” is, like “The Shadow of the Wind“, a book about books, but whereas Zafon’s Novel is set against a fictional library, Susan Hill tells us about her own bookshelves, which seem to coat and fill, like windblown snow, every wall and nook in her house.  She decides to spend a year reading only the books she already has on her shelves, a decision that coincides with a resolution to minimise use of the internet:

Too much internet usage fragments the brain and dissipates concentration so that after a while, one’s ability to spend long, focused hours immersed in a single subject becomes blunted.  Information comes pre-digested in small pieces, one grazes on endless ready-meals and snacks of the mind, and the result is mental malnutrition. (p.2)

This reminds me a little of Andrew Keen’s “The Cult of the Amateur” except Susan Hill says it more succinctly and elegantly.

Hill’s reading is both eclectic and catholic, encompassing Ian Flemming along with Trollope and Chaucer.  She takes Enid Blyton and J.K. Rowling seriously for their contribution to literacy (although unlike me does not re-read the former), and is happy to say that she never feels comfortable with Austin.  As she describes the titles she finds, sometimes lost between unlikely bedfellows, I am inspired to read them all and also to look to my own shelves:

A book which is left on a shelf for a decade is a dead thing, but it is also a chrysalis, packed with the potential to burst into new life. (p.2)

Susan Hill’s knowledge is amazing and the book is filled with anecdotes about authors she has met, known and corresponded with, giving hints of the inside story of many 20th century writers.  Sometimes I am surprised at her choices or rather not what she chooses, but more what she rejects.  In her list of books she has not read she includes:

Buddenbrooks. Thomas Mann

I want to read this . I mean to read this. I really do.

but also:

Romola. George Elliot

I do understand how I can have not read it.  (p.70)

Why?  Does she mean she can understand why it has not crossed her path, or that she does not want to read it? It is a book I have re-read several times, although always wishing the ending could be different.  I know she does not like “A Tale of Two Cities” feeling that Dickens is at his best when dealing with (for him) the contemporary; maybe she fears the same is true of Elliot?

However, Hill never assumes that her tastes are her readers’ tastes, she does not select the ‘good’ books, but the books she wants to read.

Sadly she does not supply a list of all the books she read during the year, but at the end she gives a ‘top 40′ list, with some I know well such as “The Mayor of Casterbridge“, some I know of but have never read, such as “A Passage to India“, and some I have never heard of such as “Flaubert’s Parrot“.  An instant Amazon Wish List!

Top of her top 40 are the Book of Common Prayer and the Bible (King James Version) — in the text (p173/174) deliberately in that order, but for some reason in the list at the end the Bible comes first, did she rethink after writing, or is it an editorial decision to make the list look neater?

This reminds me that I need to retrieve my battered school bible from underneath the pew where I left it after the Christmas morning service.  Also, the last of my Christmas reading (helped enormously by the enforced internet blackout due to the power cut), a book of Fiona’s “Whose Bible Is It?” by a biblical scholar and champion of inter-faith relations Joroslav Pelikan.  It is a book about a book, or rather a book about books, as Pelikan reminds us that the word “Bible” strictly means “little books”. I know some of the history of the forming of the modern Bible, but Pelikan’s encyclopaedic and detailed knowledge shines through.  I had not realised that it was the Greek Septuagint rather than the Hebrew Tanakh (Old Testament) that is quoted by the New Testament writers, making odd the decision of the early Protestants to excise the ‘Apocrypha’ (the writings in the Septuagint but not in the Hebrew).

It is a short and very readable book, but at times I found myself wanting to know a little more on some points.  When discussing the dates of the Gospels Pelikan notes that Mark is usually dated at 70CE, but doesn’t explain why.  Previously I’ve seen the same date quoted with the reason being that the Gospel appears to predict the fall of Jerusalem which happened in 70AD and therefore must have been written after.  This argument seems to presuppose that prediction is impossible and by analogy would inevitably lead to future historians excising or re-dating  several of Vince Cable’s statements prior to the 2008 financial crash. Maybe there is a better reason, or maybe, like other academic disciplines, biblical scholarship is a servant of its assumptions.

And now … no more power cuts and the internet is flawless, but trying not loose momentum with Sheila Stewart’s “Pilgrims of the Mist“, Mike Parker’s “Map Addict” and George Basalla’s “The Evolution of Technology” all on the go … maybe another post in a couple of weeks.

1. Try also Susan Cooper’s “Victory” for a story that blends past and current narrative with equal conviction. [back]

In computer science, and in particular programming languages, a whole vocabulary has arisen to talk about names: scope, binding, referential transparency. As in philosophy, it is typically the association between a name and its ‘meaning’ that is of interest. Names and words, whether in programming languages or day-to-day language, are, what philosophers call, ‘intentional‘: they refer to something else. In computer science the ‘something else’ is typically some data or code or a placeholder/variable containing data or code, and the key question of semantics or ‘meaning’ is about how to identify which variable, function or piece of data a name refers to in a particular context at a particular time.

The emphasis in computing has tended to be about:

(a) Making sure names have unambiguous meaning when looking locally inside code. Concerns such as referential transparency, avoiding dynamic binding and the deprecation of global variables are about this.

(b) Putting boundaries on where names can be seen/understood, both as a means to ensure (a) and also as part of encapsulation of semantics in object-based languages and abstract data types.

However, there has always been a tension between clarity of intention (in both the normal and philosophical sense) and abstraction/reuse. If names are totally unambiguous then it becomes impossible to say general things. Without a level of controlled ambiguity in language a legal statement such as “if a driver exceeds the speed limit they will be fined” would need to be stated separately for every citizen. Similarly in computing when we write:

function f(x) { return (x+1)*(x-1); }

The meaning of x is different when we use it in ‘f(2)’ or ‘f(3)’ and must be so to allow ‘f’ to be used generically. Crucially there is no internal ambiguity, the two ‘x’s refer to the same thing in a particular invocation of ‘f’, but the precise meaning of ‘x’ for each invocation is achieved by external binding (the argument list ‘(2)’).

Come the web and URLs and URIs.

Fiona@lovefibre was recently making a test copy of a website built using WordPress. In a pure html website, this is easy (so long as you have used relative or site-relative links within the site), you just copy the files and put them in the new location and they work Occasionally a more dynamic site does need to know its global name (URL), for example if you want to send a link in an email, but this can usually be achieved using configuration file. For example, there is a development version of Snip!t at cardiff.snip!t.org (rather then www.snipit.org), and there is just one configuration file that needs to be changed between this test site and the live one.

Similarly in a pristine WordPress install there is just such a configuration file and one or two database entries. However, as soon as it has been used to create a site, the database content becomes filled with URLs. Some are in clear locations, but many are embedded within HTML fields or serialised plugin options. Copying and moving the database requires a series of SQL updates with string replacements matching the old site name and replacing it with the new — both tedious and needing extreme care not to corrupt the database in the process.

Is this just a case of WordPress being poorly engineered?

In fact I feel more a problem endemic in the web and driven largely by the URL.

Recently I was experimenting with Firefox extensions. Being a good 21st century programmer I simply found an existing extension that was roughly similar to what I was after and started to alter it. First of course I changed its name and then found I needed to make changes through pretty much every file in the extension as the knowledge of the extension name seemed to permeate to the lowest level of the code. To be fair XUL has mechanisms to achieve a level of encapsulation introducing local URIs through the ‘chrome:’ naming scheme and having been through the process once. I maybe understand a bit better how to design extensions to make them less reliant on the external name, and also which names need to be changed and which are more like the ‘x’ in the ‘f(x)’ example. However, despite this, the experience was so different to the levels of encapsulation I have learnt to take for granted in traditional programming.

Much of the trouble resides with the URL. Going back to the two issues of naming, the URL focuses strongly on (a) making the name unambiguous by having a single universal namespace;  URLs are a bit like saying “let’s not just refer to ‘Alan’, but ‘the person with UK National Insurance Number XXXX’ so we know precisely who we are talking about”. Of course this focus on uniqueness of naming has a consequential impact on generality and abstraction. There are many visitors on Tiree over the summer and maybe one day I meet one at the shop and then a few days later pass the same person out walking; I don’t need to know the persons NI number or URL in order to say it was the same person.

Back to Snip!t, over the summer I spent some time working on the XML-based extension mechanism. As soon as these became even slightly complex I found URLs sneaking in, just like the WordPress database The use of namespaces in the XML file can reduce this by at least limiting full URLs to the XML header, but, still, embedded in every XML file are un-abstracted references … and my pride in keeping the test site and live site near identical was severely dented¹.

In the years when the web was coming into being the Hypertext community had been reflecting on more than 30 years of practical experience, embodied particularly in the Dexter Model². The Dexter model and some systems, such as Wendy Hall’s Microcosm³, incorporated external linkage; that is, the body of content had marked hot spots, but the association of these hot spots to other resources was in a separate external layer.

Sadly HTML opted for internal links in anchor and image tags in order to make html files self-contained, a pattern replicated across web technologies such as XML and RDF. At a practical level this is (i) why it is hard to have a single anchor link to multiple things, as was common in early Hypertext systems such as Intermedia, and (ii), as Fiona found, a real pain for maintenance!

1. I actually resolved this by a nasty ‘hack’ of having internal functions alias the full site name when encountered and treating them as if they refer to the test site — very cludgy! [back]
2. Halasz, F. and Schwartz, M. 1994. The Dexter hypertext reference model. Commun. ACM 37, 2 (Feb. 1994), 30-39. DOI= http://doi.acm.org/10.1145/175235.175237 [back]
3. Hall, W., Davis, H., and Hutchings, G. 1996 Rethinking Hypermedia: the Microcosm Approach. Kluwer Academic Publishers. [back]

Last night I both learnt something new about language and cognition, and also developed a new trick for creativity!

In the dream in question I was in a meeting. I know, a sad topic for a dream, and perhaps even sadder it had started with me filling in forms!  The meeting was clearly one after I’d given a talk somewhere as a person across the table said she’d been wanting to ask me (obviously as a sort of challenge) if there was a relation between … and here I’ll expand later … something like evolutionary and ecological something.  Ever one to think on my feet I said something like “that’s an interesting question”, but it was also clear that the question arose partly because the terms sounded somewhat similar, so had some of the sense of a rhyming riddle “what’s the difference between a jeweller and a jailor”.  So I went on to mention random metaphors as a general creativity technique and then, so as to give practical advice, suggested choosing two words next to each other in a dictionary and then trying to link them.

Starting with the last of these, the two words in a dictionary method is one I have never suggested to anyone before, not even thought about. It was clearly prompted by the specific example where the words had an alliterative nature, and so was a sensible generalisation, and after I woke realised was worth suggesting in future as an exercise.  But it was entirely novel to me, I had effectively done the exactly sort of thinking / problem solving that I would have done in the real life situation, but while dreaming.

One of the reasons I find dreams fascinating is that in some ways they are so normal — we clearly have no or little sensory input, and certain parts of our brain shut down (e.g. motor control to stop us thrashing about too much in our sleep) — but other parts seem to function perfectly as normal.  I have written before about the cognitive nature of dreams (including maybe how to model dreaming) and what we may be able to learn about cognitive function because not everything is working, rather like running an engine when it is out of the car.

In this dream clearly the ‘conscious’ (I know an oxymoron) problem-solving part of the mind was operating just the same as when awake.  Which is an interesting fact about dreaming, but  I was already aware of it from previous dreams.

In this dream it was the language that was interesting, the original conundrum I was given.  The problem came as I woke up and tried to reconstruct exactly what my interlocutor had asked me.  The words clearly *meant* evolutionary and ecological, but in the dream had ‘sounded’ even closer aurally, more like evolution and elocution (interesting to consider, images of God speaking forth creation).

So how had the two words sound more similar in my dream than in real speech?

For this we need the Jabberwocky circuit.

There is a certain neurological condition that arises, I think due to tumours or damage in particular areas of the grain, which disrupts particular functions of language.   The person speaks interminably; the words make sense and the grammar is flawless, but there is no overall sense.  Each small snippet of speech is fine, just there is no larger scale linkage.

When explaining this phenomenon to people I often evoke the Jabberwocky circuit.  Now I should note that this is not a word used by linguists, neurolinguists, or cognitive scientists, and is a gross simplification, but I think captures the essence of what is happening.  Basically there is a part of your mind (the conscious, thinking bit) that knows what to say and it asks another bit, the Jabberwocky circuit, to actually articulate the words.  The Jabberwocky circuit knows about the sound form of words and how to string them together grammatically, but basically does what it is told.  The thinking bit needs to know enough about what can be said, but doesn’t have time to deal with precisely how they are strung together and leaves that to Jabberwocky.

Even without brain damage we can see occasional slips in this process.  For example, if you are talking to someone (and even more if typing) and there is some other speech audible (maybe radio in the background), occasionally a word intrudes into your own speech that isn’t part of what you meant to say, but is linked to the background intruding sound.

Occasionally too, you find yourself stopping in mid sentence when the words don’t quite make sense, for example, when what would be reasonable grammar overlaps with a colloquialism, so that it no longer makes sense.  Or you may simply not be able to say a word that you ‘know’ is there and insert “thingy” or “what’s it called” where you should say “spanner”.

The relationship between the two is rather like a manager and someone doing the job: the manager knows pretty much what is possible and can give general directions, but the person doing the job knows the details.  Occasionally, the instructions get confused (when there is intruding background speech) or the manager thinks something is possible which turns out not to be.

Going back to the dream I thought I ‘heard’ the words, but examining more closely after I woke I realised that no word would actually fit.  I think what is happening is that during dreaming (and maybe during imagined dialogue while awake), the Jabberwocky circuit is not active, or not being attended to.  It is like I am hearing the intentions to speak of the other person, not articulated words.  The pre-Jabberwocky bit of the mind does know that there are two words, and knows what they *mean*.  It also knows that they sound rather similar at the beginning (“eco”, “evo”), but not exactly what they sound like throughout.

I have noticed a similar thing with the written word.  Often in dreams I am reading a book, sheet of paper or poster, and the words make sense, but if I try to look more closely at the precise written form of the text, I cannot focus, and indeed often wake at that point¹.  That is the dream is creating the interpretation of the text, but not the actual sensory form, although if asked I would normally say that I had ‘seen’ the words on the page in the dream, it is more that I ‘see’ that there are words.

Fiona does claim to be able to see actual letters in dreams, so maybe it is possible to recreate more precise sensory images, or maybe this is just the difference between simply writing and reading, and more conscious spelling-out or attending to words, as in the well known:

Paris in the
the spring

Anyway, I am awake now and the wiser.  I know a little more about dreaming, which cognitive functions are working and which are not;  I know a little more about the brain and language; and I know a new creativity technique.

Not bad for a night in bed.

What do you learn from your dreams?

1. The waking is interesting, I have often noticed that if the ‘logic’ of the dream becomes irreconcilable I wake.  This is a long story in itself, but I think similar to the way you get a ‘breakdown’ situation when things don’t work as expected and are forced to think about what you are doing.  It seems like the ‘kick’ that changes your mode of thinking often wakes you up! [back]

The central premise of Tomasello’s book is that:

(1) only cultural development can explain the remarkable development of the human race in the past 200 thousand years, as the changes we have seen are simply not explainable in terms of genetic evolution during that timescale

(2) the crucial genetic step that has fuelled this cultural explosion and the essential difference between humans and other animals is our ability to attribute intentionality to each other, to interpret others’ actions as being for some purpose.

Early on Tomasello describes this as:

“the ability of individual organisms to understand conspecifics as beings like themselves who have intentional and mental lives like their own” (p. 5, Tomasello’s emphasis)

While I agree with the broad argument, this specific statement is almost the opposite of the hypothesis, which I often talk about, concerning the origins of self-consciousness. In particular I suggest that the very concept of self may be an accident of sociality; we are aware of ourselves as intentional beings because we are aware of the intentionality of others.

By self-consciousness here I mean not feeling awkward in company, but the explicit awareness of oneself. This is not the same as consciousness, or simply being aware (one of the deepest mysteries), but more the declarative knowledge that one has intentions, actions, and thoughts.

My own argument runs like this.

In order to survive and prosper we need to be able to predict the actions of other creatures and our fellow humans. When chasing a rabbit it is useful to know that the rabbit will run away when it sees you approach, and that it will try to reach a nearby rabbit hole. Similarly, it is useful to know that one’s fellow hunters will attempt to cut off its escape route. These reactions during hunting could be purely instinctive, and probably are for many creatures such as pack animals, but with higher reasoning we can be more creative in terms of the strategies we use whether as hunter or prey; and this higher-order thinking is most effective when we can predict the actions of other creatures.

When the creature we are predicting is behaving largely instinctively, then our predictions can be similarly relatively simple. However, if the creature we wish to predict, a fellow human, is also able to employ these higher-order strategies, then we need to understand these in order to understand the other’s behaviour. In order to predict the behaviour of our fellow hunter, we need to take into account her understanding of the rabbit; and moreover, her understanding of ourself¹.

That is, to understand others one has to think about oneself as if from the outside — self consciousness!

Returning to Tomasello, his argument is about mutual understanding as a means to learn through creatively emulating others, whereas my argument above is more about the instrumental understanding of others. Being able to understand motivation helps both. The instrumental case, I’ve already described – by understanding what motivations drove your behaviour I can more accurately predict under what circumstances you will behave similarly.

Tomasello’s developmental case is similar. If I am able to imitate others then I have additional behaviours that I can employ, but I still have to learn pretty much for myself when they are appropriate. However, if I know why someone else is behaving in the way that they do then I can instantly know when those behaviours are appropriate for me. When the reasons for behaviour are readily visible in the environment, for example, a sound in the bushes and everyone running, then no model of mind is necessary to learn the association between stimulus and imitated behaviour, but where the behaviour is the result of inner thoughts and drives then we need correspondingly more complex responses.

Tomasello (p. 99) argues that this is also essential for language development as we have to understand the perspective of others as we interpret or frame utterances.

One of the key aspects he identifies is precisely that language requires us to see ourselves “from the outside”, which is entirely consonant with my own argument that the notion of self is an accident of social intercourse. The issue is about which comes first phylogenetically, self or other. Tomasello (p. 70) notes that social theorists “from Vico and Dilthey to Cooley and Mead” stress that our understanding of others rests on parallels to our understanding of ourselves; I would simply add that the reason we have access to knowledge about ourselves may be precisely in order to understand others.

When discussing how children acquire a sense of self, he notes that research has shown that infants do not conceptualise or explicitly talk about themselves before they do about others. So, while it is not true that ontogeny inevitably recapitulates phylogeny, this is certainly suggestive evidence that self is at least no more primitive than other.

While my own and Tomasello’s position both rely on the understanding of the motives and intentions of others, there is also that much deeper sharing of feeling and emotion when we empathise with others. It maybe that empathy is more primitive than the awareness of our own or others intentions as we do not need to explicitly know what someone else is feeling, nor be able to articulate one’s own, in order to simply feel with them.

It is easy to see why it is useful to understand others emotions – if someone bigger than me is feeling upset and angry it may be better to steer clear. But the roots of empathy are less clear and obviously rooted in social cohesion and bonding; it is a feeling not just with others, but intrinsically for them.

This getting alongside others is exactly what Christmas is about “the word became human” (John 1:14, New Living Tr.) and Immanuel means precisely “God with us”; the ineffable becoming an infant.

Another term in the original Tomasello quote is “conspecifics”. We have a special understanding other creatures of the same species as ourselves. This is clearly important for imitation and learning, there is no sense in imitating the behaviour of creatures very different from ourselves, such as birds, as we may be physically not able to do the same things (can’t fly!) and anyway may not share the same kinds of motivations (e.g. making a place to lay eggs).

This works also within species, we need to learn the things that we are able to and need to perform and so it is those closest to us in terms of aspirations and abilities who are the most obvious to imitate. Yet it maybe those who are more different and more experienced who have most to offer. Diligent students understand this and step beyond the obvious peer group, but also the best teachers are able to see the world from the point of view of their students.

I read with fascination as Tomasello described many experiments of his own and others that look at small infants acquiring language. However, I also noted that the focus of them all was on the way in which the infant had to make sense of the parent or other adults words and gestures. In fact, it is also the parents who try to make sense of the inarticulate sounds and embryonic gestures of their child.

look down		bend down		stoop down

People differ in the way they interact with small children: some stand fully up and look down, some bend over the child from the waist, and some squat down or sit on a low chair so that they are the child’s level. It is the latter I always know are going to be the ‘naturals’ with children².

To be a good teacher you sometimes need to become like a little child – Christmas.

1. This is effectively a second order model of mind, First order model of mind is when we understanding that others have beliefs, motivations etc.; that is that they have mind. Second order is when we reason about their understanding of our minds, third order when we think about how they think about us thinking about them! One of Piaget’s critical development steps is when a child moves away form ego centrality to be able to understand other people’s different knowledge and physical point of view – first order model of mind. In autism this does not develop normally with corresponding social and other developmental impact. While most of us manage first order and second order model of mind without difficulty, but third order is more difficult and fourth and higher orders get hard to deal with except more analytically. This was wonderfully demonstrated by the Kursaal Flyers‘ 1976 one hit wonder which as the opening line: “Little does she know that I know that she knows that I know she’s two timing me.” (music at lastfm, lyrics at justsomelyrics) – fourth order model of mind! There was a video at the time that acted out the scene described in the song lyrics. [back]
2. Of course, while people tend to interact naturally in one way or another, you can explicitly choose how to address a child. [back]

Earlier today I was using Google to look up the author of a book I was reading and one of the top links was a listing on ratemyprofessors.com.  For interest I clicked through and saw:

“He sucks.. hes mean and way to demanding if u wanan work your ass off for a C+ take his class¹”

Hmm I wonder what this student’s course assignment looked like?

and a little web-usability tag story

In case you are wondering, yes I did try to look to see if I was listed (although I am sure all my British students have perfect grammar :-/ ).  However, unfortunately (or maybe fortunately), I could not tell.  In order to search UK universities you need to use a pull-down menu.  But², of course, being an up-to-date and cool site, ratemyprofessors.com uses a funky Javascript+DOM menu not a plain HTML-form one .  Notice the little gap between the button for the pull-down and the menu itself.  As you try to move your mouse over the menu it disappears!  So for a Firefox user like me it is a US-only site.

1. In case you think I’m a complete pedant, personally, I am happy with both the slang ‘sucks’ and ‘ass’ (instead of ‘arse’!), and the compressed speech ‘u’. These could be well-considered choices in language. The mistyped ‘wanna’ is also just a slip. It is the slightly more proper “hes mean and way to demanding” that seems to show  general lack of understanding.  Happily, the other comments, were not as bad as this one, but I did find the student who wanted a “descent grade” amusing [back]
2. Yes, that was a sentence starting with a conjunction.  And, yes, you may have heard this is bad grammar, but only when used carelessly; see “Grammar- Starting a Sentence with Or, And or But” @ Suite101 or “Beginning a Sentence with And or But” for an apposite quote. [back]

In my recent post about “language and action: sequential associative parsing“, I compared the role of semantics in the parsing of language with the similar role semantics plays in linking disparate events in our interpretation of the world and most significantly the actions of others. The two differ however in that language is deliberative, intentionally communicative, and hence has a structure, a rule-iness resulting from conventions; it is chosen to make it easier for the recipient to interpret. In contrast, the events of the world have structure inherent in their physical nature, but do not structure themselves in order that we may interpret them, their rule-iness is inherent not intentional. However, the actions of other people and animals often fall between the two.

In this post I will focus in on individual actions of creatures in the world and the way that observing others tells us about their current activities and even their intended actions, and thus how these observations becomes a resource for planning our own actions. However, our own actions are also the subject of observation and hence available to others. We may deliberately hide or obfuscate our intentions and actions if we do not wish others to ‘read’ what we are doing; however, we may also exaggerate them, making them more obvious when we are collaborating. That is, we shape our actions in the light of their potential observation by others so that they become an explicit communication to them.

This exaggeration is evident in computer environments and the physical world, and may even be the roots of iconic gesture and hence language itself.

The CSCW Framework

In the early 1990s I began to use variants of the following diagram as a way to understand the different kinds of systems found in groupware and CSCW research. It has proved useful in many papers and notably in the groupware chapter of the HCI textbook.

communication through the artefact
(from “Computer Supported Cooperative Work – A Framework“).

At the top, the two circles labelled ‘P’ represent participants in a collaborative interaction. Although there are two they are taken to represent any number in a group¹. The ‘A’ represents the ‘artefacts of work’, the things people are creating or working with in order to achieve their collaborative purpose. The simple diagram helps to distinguish direct communication from the control of and feedback from shared artefacts, and also to discuss the role of diexis (and even barcodes!) in collaborative work.

Most important for the current argument is that the actions of one participant on shared artefacts can often be observed by the other participants. That is, in addition to feedback of one’s own actions, one obtains feedthrough about the actions of others. This creates a path of communication through the artefact in addition to those of direct communication.

The framework was formulated to discuss computer-mediated interactions, so the core examples of shared artefacts were shared documents, diaries and files; now-a-days it would be Google docs, but at the time it was various collaborative authoring and editing tools such as DistEdit² and Quilt³.

However, the framework diagram was typically illustrated through the same kinds of interactions on physical world objects, and most often moving furniture.

“Compare two persons moving an object, say a piano, by hand with them moving it with the aid of two (small) forklift trucks. … In the manual case, much of the time you hear little (except grunts). As they manoeuvre it round corners, one of them (probably the one in front) may tip it and the other, feeling the movement, moves in concert. …”

extract from “Computer Supported Cooperative Work – A Framework“.

Our skills in understanding actions on virtual artefacts are inherited from our real world interactions.

So, let’s move from the computer systems of the 21st century and take ourselves to a place far away from all technology, or maybe back to a simpler time before computers.

Conflict and camouflage

Somewhere on an African plain a lioness watches a zebra. The zebra has more meat than the many bucks that roam freely and would make a better meal for her cubs, but it is fast and has sufficient stamina to outrun her in a long chase. If the lioness makes a dash from too far away, the zebra will escape and the meal is lost. So the lioness slowly moves through the grass, hoping to get close enough to go for the kill. Meanwhile the zebra has noticed the lioness, but the grass is good and the lioness may not be hungry or may have had a recent kill. It is better to stay around and eat the good grass than to run unnecessarily. So the zebra watches the lioness, but tries not to show that it is doing so. The lioness is also watching for signs that the zebra has taken fright. She is trying to get as close to possible, but if the zebra starts to run she will chase it; the slightest movement that looks like flight is her trigger to move.

Each animal is watching the other; reading the movements of action and looking for subtle signs of the intention to act. But they are in conflict, their goals diverge in the extreme, so they obfuscate and hide both action and intention. Some of this is played out in the moment, the lioness holding her body low amongst the grass. Some has been set by millennia of adaptations: the golden coat of the lioness is almost invisible amongst the sun-scorched grass and red-oxide earth; the stripes of the zebra confuse its form against the bush.

Collaboration and communication

Now let’s shift our attention to the Siberian snows. A pack of wolves are stalking a herd of deer and the deer take fright. The younger, weaker and slower deer are evident, and as the chase begins the pack leader spots a likely prey and runs for it. One of its pack mates sees the wolf’s movements, and that the deer, while weak, is making a break; so it heads off the deer turning it to bay and the rest of the pack moves in.

Again the animals read each other’s movement and the subtle signs of action and intention. However, here the pack needs to act in concert, they are collaborating in the kill and so it is to their advantage if the visible (and aural) signs of their actions and intentions are clear to one another. Indeed before the chase begins the pack leader may have identified the likely target and the rest of the pack seen this through the direction of gaze. A class of hunting dogs are called ‘pointers’ because they literally point their noses towards prey; the modern hunter with green wellies and shotgun is reading their signs just as the pack mates of their ancestors did thousands of years before.

In some ways the reading of the actions of the other wolves is just like the lioness reading the actions and intentions of the zebra, or vice versa. However, because the lioness and zebra are in conflict, they each seek (whether intentionally, or instinctively) to obfuscate the signs of their actions. In contrast, the wolf pack are cooperating and so while they may need to obfuscate their intentions from the deer, it is to their advantage to make these clear to one another.

Like the lioness and the zebra, this may include long-term changes in their appearance, but certainly changes in instinctive behaviours. The ‘pointing’ of pointer dogs is tapping into instincts. However, the form of this is particular. It is natural for the act of seeking out and staking prey that the dog (or its wolf ancestor) looks at the selected target. The other animals are attuned to these natural signs. However, at some point the ‘looking at’ becomes exaggerated beyond what is essential for the action itself; it is just like the original action, but ‘more so’. So without any change in the perceptions of others a form of message is given, the sign becomes a signal⁴.

Of course, the process is symmetric, over time animals adapt to not only notice the ordinary signs of action, but to especially notice the exaggerated movements signalling intentions. Sometimes the exaggerated signal becomes divorced from the original signs and takes on a life of its own⁵. This is especially evident in mating rituals, which are not moderated by the need to obfuscate actions from prey.

Cooperation in conflict

Some conflict situations, particularly predator-prey, are zero sum; there is a winner and a loser and no middle ground. However, in other situations, between animals competing for food, or between males competing for leadership of herd or pack. There is a zero sum competition for the food or position, but it is in both competitors’ interests to avoid actually fighting, which will hurt both.

This young bull elephant below clearly means business: the extended ears say, “I’m getting upset, I don’t want you near”. However, it is not in the interest of the elephant to charge a car, nor in the interests of the occupants to stay around. We drove away!

In this case the elephant and the car occupants are cooperating to prevent a conflict about territory becoming a physical confrontation. In this case the elephant is using the same signal of raised ears as he would use to another elephant or animal. It is partly a deception, making the elephant look bigger than it is (it wants to win in the conflict!), but also a signal “I don’t want to fight but I will”, which is understood by other elephants … and car drivers.

Towards gesture

Of course as deliberative creatures, we humans not only instinctively act, but also do so deliberatively, with forethought and consideration.

Some of this we learn without being aware that we do so. The baby may cry because it is hurting and the mother comforts it, but later may cry to obtain attention. Watch a small child who has fallen over and bursts into tears. The tears will often abruptly stop if no one notices. It is not that there is no hurt, nor that the tears are purely a show, but they are exaggerated to become a signal to the parent, a message, not just a sign of the pain.

As adults our behaviours are full of these unconscious signals: smiles and frowns, looking at our wrist (even if we do not have a watch) if it feels time to leave, or slightly exaggerated shuffling of papers when a meeting has gone on too long.

The borders between this and more thought-out deliberate action is blurred. Certainly politicians and seniors managers both naturally give out these signals, but also may learn explicitly how to read and how to create these impressions, from shoulder pads to the power stare.

A few years ago I was putting a note on my door as I was leaving my office for a few minutes, but knew a visitor was due. I suddenly noticed myself adjusting the note to make it less vertical. This was not a conscious action, but clearly deliberate. On reflection I realised that the door was covered in various permanent posters and notices. There was clearly a danger that the visitor might knock at the door and never notice the temporary note amongst this background. However, a temporary note will typically be put up in a hurry, and will not be carefully aligned; so its disposition⁶, its skew-whiff angle, will act as a sign to the visitor that it is temporary and so worth reading. Without explicitly realising it, I was exaggerating this sign of temporariness. Now that I know (explicitly) the ‘trick’ I deliberately do this; but note the unconscious complexity of every-day thought that was achieving this without my awareness.

This is not a one-off phenomena. I have also found myself leaving my office light on when I leave the office as a signal that I am ‘in’. Again emulating what could be an accidental sign of presence … albeit not very environmentally sound.

One of the Equator experiments took place in the Mack Room, a gallery in the Lighthouse arts centre in Glasgow⁷. The experiment linked three participants; one was in the Mack room itself with a PDA tracked using ultrasonic sensors, the second was using desktop VR navigating a model of the room, and the third was using plain web pages. They were all able to speak to one another through microphones, and each could see a representation of themselves and the others in a map view.

Even when still, the person in the room would sway slightly (and maybe the ultrasonic sensors had some noise), so there was a slight movement of the avatar, but the avatar of the VR participant was rock steady as soon as hands left the cursor keys (used for moving). After a while the VR participant was seen to periodically ‘wiggle’, deliberately using small cursor movements to draw attention to speech or location by emulating the natural wiggle of a standing person.

Onomatopoeic action

These effects are similar to what I used to call onomatopoeic action. When you are driving down the road and want to turn right at the next junction you may gradually move the car to the middle of the road. In a wide road this is the correct road position. However, if the road is narrow, then you need more room to turn and the appropriate road position to start the manoeuvre is on the opposite side of the road. However, even in such situations, one often initially moves to the centre, and only at the last minute sweeping to the left then making the right turn. Why?

Turning right into a narrow road:
(1) normal road position, (2) initially move towards centre of road, then
(3) move to the left ready for (4) swing into the narrow entrance.

The initial movement to the centre (labelled (2) in the diagram) is not necessary for the action of turning, but is a signal to other road users that you intend to turn right. Obviously you can use your indicators as well, but we often trust our reading of other driver’s road position more than explicit signals and the person behind seeing someone pull over to the left might mistake it for stopping and erroneously and dangerously decide to overtake just you make your turn

Things go wrong: (1) normal road position with car following,
(2) move to left ready for manoeuvre, following car assumes you are stopping so
(3) begins to overtake at which point you complete your manoeuvre turning right
and … (4) boom!

Words, like ‘boom’ or ‘tinkle’, that imitate in their sound what they denote in meaning are called onomatopoeic. The action of moving into the middle of the road conveys in its form (rightness) the intended future action (turning right), so can be thought of as an onomatopoeic action. The skew-whiff placement of notes on the door, leaving the light on, looking at a non-existent watch all have this onomatopoeic element.

To language: from exaggeration to representation

Note that the development of the ‘move to the centre to signal a right turn’ action will almost certainly have started out as exaggeration. On a wide road it is not necessary to move to right or left, but simply to turn from the normal road position. However, the movement to the centre begins the action ahead of time, exaggerating it. It is the start of the actual action, but also emphasises it to others.

By the time we have the right turn in the narrow road, the action of moving to the centre of the road has become divorced from the actual action of turning right. It is representative of or maybe even symbolic of the action of turning right, but is not actually a part of the action (which really needs a position to the left).

This is similar to the runaway mating displays of birds and other animals, but whereas these are solely instinctive, our own actions of this kind often mix reactive and deliberative elements.

This is reminiscent of the idea of accountability in ethnomethodology: as we act in a social situation we know that our actions and activities are available to others and must therefore be meaningful to others within the particular cultural and social settings. Similarly Goffman regarded day-to-day activity as a performance; just as an actor performs on-stage, we perform moment to moment for those around us and often for ourselves. However, while the actor emulates fictive actions, when we perform in an exaggerated way we effectively emulate what we are doing anyway!

The important shift occurs at the moment when we no longer emulate what we are doing, but emulate in order to convey what we may be about to do. We then move from observable actions to representative gestures and surely the beginnings of symbolic language.

1. Note the choice of two participants in the CSCW framework diagram seemed unproblematic, until some years later I realised it had limited the scope of analysis of awareness. A third participant is needed in order to talk about the way one person is aware of the communications or collaborative actions of others. The diagram had proved incredibly powerful in suggesting questions and issues, but by its nature had precluded others; good lesson about any diagram, notation or system of thought! [back]
2. Knister, M.J. and Prakash, A., (1990), DistEdit: a distributed toolkit for supporting multiple group editors, in CSCW’90 Proceedings of the Conference on Computer-Supported Cooperative Work, ACM SIGCHI & SIGOIS, 343-355 [back]
3. Leland, M.D.P., Fish, R.S. and Robert E, K., (1988), Collaborative document production using Quilt, in Proceedings Of CSCW’88, 206-215 [back]
4. I am aware that I am using the words ‘sign’ and ‘signal’ in nearly the opposite way to those used in some branches of semiotics. However, the sense here is the normal sense in the English language, where one talks about the crocus being a sign of spring, or smoke is a sign of fire, but smoke deliberately covered and uncovered to create a characteristic pattern is a signal. In ordinary language a signal suggest intention, something emitted deliberately to be noticed, the possession of the sender.  In contrast, signs are noticed, but not necessarily intended; indeed the thing that has a sign may not even be animate; the meaning of a sign belongs to the recipient. In fact, doing a quick check it seems that in Pierce’s semiotics, ‘sign’ is very close to the meaning here. [back]
5. I’m not an expert on Pierce, but I think he would refer to the point at which the sign becomes detached from its ‘natural’ connection to the action as the change from sinsign to legisign. [back]
6. We have used the term ‘disposition’ is used in previous work, such as “Trigger Analysis – understanding broken tasks” and  “Artefacts as designed, Artefacts as used: resources for uncovering activity dynamics“.  Often the location of an artefact is important (papers placed on a chair for reminders, or in a particular position on a desk).  However, often the way in which it is placed in that location is also significant – the disposition (straight or at an angle, front-side up or down). [back]
7. See the Equator pages about the Mack Room project also the Mackintosh Interpretation Centre web site, and the HCI book case study on the Mack Room. [back]

Reading language

“The parrot, which was continually watched by the big black cat, who sometimes sat on the mat until displaced by the bigger brown Doberman and sometimes on the garden wall in the hope of catching the sparrow, flew through the window.” (sentence 1)

Why is this difficult to read?  It is grammatically well-formed and has commas in all the right places.  However, the large subordinate clause (if I have got my grammatical terms right) means that by the time you have go to “flew through the window” you have ‘forgotten’ the parrot who it was all about.

In contrast, a computer parser has no such trouble. Take the fragment:

if ( y > 10 ) {
    x1 = y+1;
    x2 = y+2;
    . . .   // 97 more lines of code
    x100 = y+100
} else { . . .

The compiler does not ‘forget’ about the ‘if’, because it is part of a relatively simple syntactic structure:

if ( condition ) { statement_block } else { statement_block }

In the English sentence about the parrot, like the program fragment, even the ‘depth’ of the parsing stack is not deep, yet still there is confusion.
Years ago when I was a student in Cambridge a friend who was studying Politics came to me with a passage in Nozick’s “Anarchy State and Utopia“.  I read the paragraph in question several times.  Each time it made sense, until I came to the last few words, which seemed to be a total non sequitur.  Only after many readings did I realise that the paragraph was a single sentence (all 10 lines of it!), and that the sentence was of precisely the same form as the parrot one, except with nested sub-clauses.   The words that appeared to come from nowhere were in fact the equivalent of “flew through the window” … indeed imagine if the description of the Doberman had extended to another seven lines!

Clearly our ability to hold on to relatively simple hierarchical parsing structures is quite limited, and is definitely smaller than the 7+/- 2 for working memory¹.  However, there is also another mechanism at work.

Consider the following alternative sentences:

“The policeman, which was continually watched by the big black cat, who sometimes sat on the mat until displaced by the bigger brown Doberman and sometimes on the garden wall in the hope of catching the sparrow, flew through the window.” (sentence 2)

“The parrot, which was continually watched by the big black cat, who sometimes sat on the mat until displaced by the bigger brown Doberman and sometimes on the garden wall in the hope of catching a snail, flew through the window.” (sentence 3)

It is hard to read these ‘fresh’ as if you hadn’t read the initial sentences, but notice that the first is far harder than the second.  This is because we have an additional sequential associative mechanism at work as well as syntactic parsing.

Consider instead the much simpler  ‘sentence’:

“bone dog eat”

We have little problem making sense of this even though the structure is not grammatical.  The verb ‘eat’ is ‘looking for’ something that eats to act as its subject, and something that can be eaten to be its object.  As there is only one eatable and one eater in the ‘sentence’, the parsing (or perhaps connecting) happens purely by association.

If there are several ‘matches’ for the words then the word combinations become ambiguous:

“policeman demonstrator hit”

Some languages, such as English, largely deal with this ambiguity by use of syntactic rules (subject verb object), others, such as Latin, by tagging the words with their role … and I don’t know my Latin enough to give a proper example, but I think something like, in pigeon Latin: “policemanus demonstratori hit” vs “policemani demonstratorus hit”!  I believe Latin has quite strong syntactic rules as well, but there are certainly languages where order is less significant, you can think of them as Lego block languages; you throw the words into a bag, shake them up, and they stick together where the words fit together.

When we are reading a sentence both forms of parsing are happening at once.  There is a level of grammatical parsing, but also when a word, like ‘parrot’ in sentence 1, is encountered and does not get ‘bound’ it is ‘waiting’ for a verb to connect to.  Later in the sentence,  when we encounter ‘flew’ it wants something that can fly to be its subject, if ‘parrot’ is still sufficiently in mind then all is well and we get “The parrot flew through the window”.  However, the last thing we read was ‘sparrow’ a thing that can fly.  The sequential associative parsing says “sparrow flew” and even if  the syntactic parsing ‘wins’, the conflict still makes the sentence hard to grasp.

Sentence 2 was chosen to make the binding of the grammatical subject ‘policeman’ unusual to bind to ‘flew’ and so more confusing, whereas sentence 3 is less confusing as there is no obvious flying candidate except the parrot.

Poets make good use of these multiple mechanisms as it allows them to bend grammar and yet have (sometimes) comprehensible sentences.  For my students it is not usually poetry I am after in their PhD theses, but understanding the role of sequential parsing can help us to construct more comprehensible English.

Reading action

It was only quite recently I realised there was a very close parallel to work we had been doing in the TIM project on tasks² – that is action rather than language.

There are various algorithms used in intelligent user interface research to help predict the next action based on the previous one.  Mostly these make use of some sort of Markov model or similar sliding window where the next command is the one that has occurred most often after the preceding N commands.  However, there have been various proposals for performing more hierarchical pattern formation effectively building task structures as found in hierarchical task analysis.

A fundamental block to both approaches is that human activity is not always focused on one task at a time, but we do a little of one, then a little of another, just as I might break off from writing to make a cup of tea.

In the TIM project we have been using a personal ontology for various purposes including helping to propose values for actions such as web forms.  If there is a ‘name’ field, then the ontology includes names of friends so these can be proposed as potential options to the user.

Because of the ontology, we are also able to detect that the telephone number used in one web form is the telephone number of the person in the mail that has just been received.  This semantic binding through the ontology means that we can effectively see that individual actions are or are not related depending on whether they have semantic relationships through the ontology, that is we can thread together the actions that belong a single task sequence and pull them out of the chronological sequence that involves a mix of task and activities.

There are many differences between this and the sequential associative parsing, not least that the basic actions we are considering are typically parameterised (the completion of a web form) while words are singular, but there are also connections.  Previously, the relationship between task analysis and traces of actions has been compared with that between grammar and sentences, basically treating task analysis as a way to ‘parse’ activity³. This parallels the conventional parsing approach.  However, what I only belatedly realised was the use of semantic relationship between actions we are adopting in the TIM project also parallels the use of semantic connections between words in our task inference.

Reading the world

The above is about the way a computer can make sense of human action.   However, as humans we need to make sense of the things that happen around us in the world not least other people.

When we sense the world we are confronted with myriad stimuli, as I write the sound of an ambulance passing outside, the insistent hiss of the kettle on the stove, all clamour against the words I type.  Against this visual, aural and tactile cacophony, we have to unravel the threads of meaning.  If my leg aches I do not ascribe it to the cup of tea I am drinking, or the ambulance that recently passed, but instead to the run round the Circo Massimo earlier this morning.  However, equally I do not ascribe it to running on the beach a weak ago.  Temporal and semantic proximity both play their part in our reading of the signs of day-to-day life.  It is not surprising that these same tools are at play as we listen or read language.

We also perceive patterns of actions in the world, most significantly in the actions of other creatures.  In a keynote at Tamodia in Pisa last year, I discussed the production of human action and the rich intertwining of sequential planned or proceduralised activity with more stimulus-driven reactions.  Both could be explicit or tacit.  The computer aid is trying to ‘read’ these patterns of intentional (and reactive) activity.

It is of course likely that our ability to produce and understand language builds on our ability to act within and to comprehend the world.  However, the key difference is that language is about intentional communication.  As well as expecting parts to link together in fragments, we also expect the whole to make sense.  In a detective novel each incident may seem disparate, but we are sure that in the end they will come together in the dénouement.  This is a large-scale application of Grice’s cooperation principle.  For us interpreting the world and each other’s actions, or for the computer interpreting human action, no such overarching message is expected with the exception of life as a whole: God, or death, depending on your beliefs, as the ultimate dénouement.

1. George A. Miller. The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information. The Psychological Review, 1956, vol. 63, pp. 81-97 [back]
2. The best account of this task inference at present is probably my keynote Tasks = data + action + context: automated task assistance through data-oriented analysis at Engineering Interactive Systems 2008 last September in Pisa. [back]
3. Personally, I first used this ‘task analysis as grammar’ approach in teaching task analysis and it appeared in the task model slides for the third edition if the HCI book (but not in the book itself, maybe fourth edition).  However, it has been elaborated in work with Stavros Asimakopoulos and Robert Fyldes “Grammatically interpreted task analysis for supply chain forecasting“. [back]