Is this culture or genes, nature or nurture?

As with most things, I’d guess the answer is a mix.  But, if of culture, what? There is a tradition of Biblical numerology, but hardly widespread enough to make the substantial effects. Is it to do with the discipline of learning Hebrew, maybe just discipline, or perhaps is it that mathematics is one of the fields where there has been less prejudice in academic appointments³.

I have just read a paper, “Disembodying Cognition” by Anjan Chatterjee, that may shed a little light on this.  The paper is an excellent overview of current neuroscience research  on embodiment and also its limits (hence ‘disembodying’).  One positive embodiment result related to representations of actions, such as someone kicking a ball, which are often depicted with the agent on the left and the acted upon object on the right.  However, when these experiments are repeated for Arab participants, the direction effects are reversed (p.102).  Chaterjee surmises that this is due to the right-to-left reading direction in Arabic.

In mathematics an equation is strictly symmetrical, simply stating that two thinsg are equal.  However, we typically see equations such as:

y = 3x + 7

where the declarative reading may well be:

y is the same as “3x + 7″

but the more procedural ‘arithmatic’ reading is:

take x, multiple by three, add seven and this gives y

In programming languages this is of course the normal semantics … and can give rise to confusion in statements such as:

x = x + 1

This is both confusing if read as an equation (why some programming languages have := read as “becomes equal to”), but also conflicts with the left-to-right reading of English and European languages.

COBOL which was designed for business use, used English-like syntax, which did read left to right:

ADD Tiree-Total TO Coll-Total GIVING Overall-Total.

Returning to Jewish mathematicians, does the right-to-left reading of Hebrew help in early understanding of algebra?  But if so then surely there should be many more contemporary Arab mathematicians also.  This is clearly not the full story, but maybe it is one contributory factor.

And, at the risk of confusing all of us brought up with the ‘conventional’ way of writing equations, would it be easier for English-speaking children if they were introduced to the mathematically equivalent, but linguistically more comprehensible:

3x + 7 = y

1. Although they did have to ‘forget’ while they were there otherwise they would have starved on the all-pork cuisine [back]
2. Source jews.org “Jews in Mathematics“. [back]
3. The Russians did not send a team to the IMO in 1978.  There were three explanations of this (i) because it was in Romania, (ii) because the Romanians had invited a Chinese team and (iii), because the Russian national mathematical Olympiad had also produced an all Jewish team and the major Moscow university that always admitted the team did not want that many Jewish students.  Whether the last explanation is true or not, it certainly is consonant with the levels of explicit discrimination in the USSR at the time. [back]

I said I would send some links, so this is both a short report on the visit and also a few links to appropriation and adoption and a big long list of links to physicality!

Evening talk: Designing for adoption and designing for appropriation

I started¹ with my three use words: useful, usable and used … it doesn’t matter if your design is the most perfect ever, if it is never actually used it is useless! The talk centred around the dynamics of use:

adoption – the path between no use and use

appropriation – from plain use to the artefact becoming deeply embedded in the user’s life

In the questions I was asked how my useful-usable-used, differed from useful-usable-desirable, which is used in some ‘design’ texts. I would say that desirable is just one of the things that makes something used, in addition there are things like organisational acceptability, availability, … not to mention cost! However, the question, and subsequent talk after dinner, reminded me that I have very little knowledge of the ‘design’ literature².

The appropriation part of the talk was largely based on my HCI2007 short paper “designing for appropriation“. See also blogs about the topic by Palojono and Dan Perkel.

The adoption part was based on a draft section on designing for adoption (for next edition of HCI book) and draws heavily on my experiences in the dot.com days on 1998-2000. During that time I wrote a number of eBusiness bulletins covering various aspects of internet product development and marketing; partiicularly relevent for this talk are bulletins about lattice of value, marketplace ecology, network effects, and the websharer vision.

Some of the analysis is based on Grudin’s critical mass arguments for CSCW systems, but it was in Andy Cockburn‘s thesis that I first saw this being used as a positive design tool.

During the talk I passed round some of the little Firefly units that we have developed at Lancaster that are still on display in Lancaster city centre (see blog entry).

PhD workshop on physicality

[larger image]

I really enjoyed the afternoon with the Prometei students. I asked them to bring things with them natural and man-made³, which we used as props in the discussion during the afternoon.

Everyone had two minutes to write down some differences between different kinds of things on the natural/artificial and physical/digital spectrum:

(i) natural – such as rock, wood, etc., (but not animal)

(ii) man made but simple – such as hammers or spoons

(iii) mechanical – some hand-powered such as a hand-drill, some electrically powered such as a small food mixer

(iv) digital products – such as phones, PDAs, etc.

We then spent the rest of the three hour session simply discussing what was written down – the time seemed to fly by. The flipchart notes try to reduce each element down to a single word or short phrase, but can not do justice to the rich discussion around each item.

One thing I found fascinating was that, while there was overlap between the issues raised by different people, there was also a huge difference in the kinds of issues that different people considered. The very first points on the flip chart were all about prior expectations and focused on the experiential aspects of physical and digital devices. This contrasted with my own first thoughts and that of some others in the group, which tended to be about physical properties. Interestingly several of the ‘techie’ artefacts that people brought had their own personal stories, such as Habakuk’s binocular that belonged to his grandfather. … but I don’t recall any such stories about the purely digital artefacts. Interestingly the binocular was I guess at least 40 years old but still functioned well, a 4 year old digital device is completely out of date!

Several people mentioned the relative fragility of digital devices. This is partly because of the materials used to make them (often plastic), but also because with a digital or mechanical object it is the relative configuration of parts that matter as much as the parts themselves. Hit a stone hard enough and you get two stones, but take a computer to bits and you get … just bits.

My daughter once gave me some stones for my birthday. At first they looked just like a collection of pretty grey pebbles with streaks of white through them. Then I recognised an ‘A’ on one of them, and then that they were, in fact, all the letters of my name. The stones themselves were natural, but when suitable selected and put in the right relation to one another they were a word.

In Doc Hollywood when the local garage rebuilds his precious car they give him a part “there’s always one bit left over” they say … of course the joke is that with a car if you leave one bit out it usually doesn’t run. However, with a tree if you chop one branch off it is still a tree, or take one grain from the Sahara and it is still dessert. With complexity comes fragility.

With many of the aspects that came up there were often contradictions. Although digital devices were regarded as fragile, digital information was the opposite. It is not bound by particular materials, but can be captured and replicated in the configurations of different materials at different places and times (from text on a page to electrons in circuits, or magnetised particles on a spinning disc surface).

We covered many topics (look a the enlarged flipchart image to see) some more physical (e.g. physical has size, digital virtually no size), some about interaction (e.g. affordance related ability to immediately ascertain how to manipulate a physical object vs. instruction manual for complex devices), and some communicative (continued sense of the creator, physical sign vs digital symbol).

As the evening talk was to be abut affordance, it was interesting that several people picked up on the ability to use physical items including less complex man-made ones in new ways, whereas digital devices tended to be used as designed. However, this was contrasted with the fact that in other ways a digital device (in principle) are able to be reconfigured and repurposed.

Later on, Habakuk told me about a list of aspects of physical/digital differences they they made for a TEI conference session, that they label PIBA (Physicality is Better At) vs DIBA (Digitality is Better At) [see list]. Also he and others are organising a SIG at CHI on “Designing for Intuitive Use“.

Physicality some of my own writing

I said that I would distribute a list of things I’ve done related to physicality. So including it here. Just a list I’m afraid … maybe something more annotated at another time …

• www.physicality.org – for info on DEPtH project, proceedings of Physicality 2006 & Physicality 2007 worhshops, and upcoming workshops
• First Steps in Physicality – preface to Physicality 2006 has breakdown of different physicality issues
• various keynotes and talks including: Physicality, rationality and imagination, and driving as a cyborg experience.
• my blog entries on physicality: especially physicality and middle ages tech support and matterealities and the physical embodiment of code.
• work with Masitah and others on natural interaction and fluidity: Aladdin’s lamp: understanding new from old, Knowledge of Today for the Design of Tomorrow, Visceral Interaction, Natural Inverse: Physicality, Interaction and Meaning,
• more formal approaches: Modelling Devices for Natural Interaction, Formalising Performative Interaction, and loads on status-event analysis.
• about paper documents and related: Finding Decisions Through Artefacts, Artefact-centred analysis – transect and archaeological approaches, Accidents of Infornation.
• about the physical nature of space: Welsh Mathematician walks in Cyberspace (the cartography of cyberspace), Exploiting space and location as a design framework for interactive mobile systems, Managing multiple spaces (also as book chapter), The auditability of public space – approaching security through social visibility, Paths and Patches – patterns of geognosy and gnosis.

1. Well to be strictly honest I started talking about the graffiti in the toilets … which did relate to appropriation: the toilet walls can be appropriated for graffiti precisely because they do not serve a function. I mentioned the Viking graffiti at Maes Howe in Orkney, which is remarkable similar in content to its modern equivalent. [back]
2. ‘design’ in quotes as the word is used in many ways and by this I sort of mean mainly product design. [back]
3. and we even had a short discussion about gendered language where the men told me that it wasn’t an issue in German and the women said the opposite! [back]

The people there included someone who studied people coding about DNA, someone interested in text, anthropologosts, artists and an ex-AI man. We talked about embodied computation¹, the human body as part of computation, the physical nature of code, the role of the social and physical environment in computation … and briefly over lunch I even strayed onto the modeling of regret … but actually a little off topic.

physicality – Played a little with sticks and stones while talking about properties of physical objects: locality of effect, simplicity of state, proportionality and continuity of effect².

physical interaction – Also talked about the DEPtH project and previous work with Masitah on natural interaction. Based on the piccie I may have acted out driving when talking about natural inverse actions

ubiquity of computation – I asked the question I often do “How many computers do you have in your house” … one person admitted to over 10 … and she meant real computers³. However, as soon as you count the computer in the TV and HiFi, the washing machine and microwave, central heating and sewing machine the count gets bigger and bigger. Then there is the number you carry with you: mobile phone, camera, USB memory stick, car keys (security codes), chips on credit cards.

However at the Firefly demo later in the morning they got to see what may be the greatest concentration of computers in the UK … and all on a Christmas Tree. Behind each tiny light (over 1000 of them) is a tiny computer, each as powerful as the first PC I owned allowing them to act together as a single three dimensional display.

embodiment of computation – Real computation always happens in the physical world: electrons zipping across circuit boards and transistors routing signals in silicon. For computation to happen the code (the instruction of what needs to happen) and the data (what it needs to happen with and to) need to be physically together.

The Turing Machine, Alan Turing’s thought experiment, is a lovely example of this. Traditionally the tape in the Turing machine is thought of as being dragged across a read-write head on the little machine itself.

However … if you were really to build one … the tape would get harder and harder to move as you used longer and longer tapes. In fact it makes much more sense to think of the little machine as moving over the tape … the Turing machine is really a touring machine (ouch!). Whichever way it goes, the machine that knows what to do and the tape that it must do it to are brought physically together⁴.

This is also of crucial importance in real computers and one of the major limits on fast computers is the length of the copper tracks on circuit boards – the data must come to the processor, and the longer the track the longer it takes … 10 cm of PCB is a long distance for an electron in a hurry.
brain as a computer - We talked about the way each age reinvents humanity in terms of its own technology: Pygmalion in stone, clockwork figures, pneumatic theories of the nervous system, steam robots, electricity in Shelley’s Frankenstein and now seeing all life through the lens of computation.

This withstanding … I did sort of mention the weird fact (or is it a factoid) that the human brain has similar memory capacity to the web⁵ … this is always a good point to start discussion

While on the topic I did just sort of mention the socio-organisational Church-Turing hyphothesis … but that is another story

more … I recall counting the number of pairs of people and the number of seat orderings to see quadratic (n squared) and exponential effects, the importance of interpretation, why computers are more than and less than numbers, the Java Virtual Machine, and more, more, more, … it was very full hour

1. I just found notes I’d made for web page in embodied computation 5 years ago … so have put the notes online [back]
2. see preface to Physicality 2006 proceedings [back]
3. I just found an online survey on How many computers in your house [back]
4. Yep I know that Universal Turing machine has the code on the tape, but there the ‘instructions’ to be executed are basically temporarily encoded into the UTM’s state while it zips off to the data part of the tape. [back]
5. A. Dix (2005). the brain and the web – a quick backup in case of accidents. Interfaces, 65, pp. 6-7. Winter 2005.
   http://www.hcibook.com/alan/papers/brain-and-web-2005/ [back]

In this case the issue is the distributed nature of cognition within the brain and the inadequacy of central executive models. In support of this, Clark (p.39) cites Mitchel Resnick at length and I’ll reproduce the quote:

“people tend to look for the cause, the reason, the driving force, the deciding factor. When people observe patterns and structures in the world (for example, the flocking patterns of birds or foraging patterns of ants), they often assume centralized causes where none exist. And when people try to create patterns or structure in the world (for example, new organizations or new machines), they often impose centralized control where none is needed.” (Resnick 1994, p.124)¹

The take home message is that we tend to think in terms of centralised causes, but the world is not like that. Therefore:

(i) the way we normally think is wrong

(ii) in particular we should expect non-centralised understanding of cognition

However, if our normal ways of thinking are so bad, why is it that we have survived as a species so long? The very fact that we have this tendency to think and design in terms of centralised causes, even when it is a poor model of the world, suggests some advantage to this way of thinking.

Of course, this may simply be an accident of our neural architecture … in which case it would be important for (ii), or may be adapted for a hunter gatherer life, but not 21st Century living – but again would be interesting for (ii). However, the fact that we are still here means it is certainly not too unsuccessful.

Whatever the reason, the fact that we think in these terms is itself an empirical data point for understanding human cognition. We have brains that tend to seek centralised solutions – what are the neural and cognitive mechanisms that drive this and what are the environmental reasons that make it work.²

There are two factors at work here, one is about the way we see the world and the other about the way we plan and act on it.

At the level of perception, one of the Gestalt laws is that things that move together belong together. Even if bushes hide most of a predator from view, the several disconnected tiny moving fragments still form one large animal you need to avoid.

Yes this is a gross simplification of reality. Look at a rock – it is an ‘it’, a single thing – but in fact it is not, it is simply the decentralised activities of millions of millions of millions of millions of atoms interacting, largely locally, with one another. It is not so far unlike Resnick’s flock of birds. However, their general coherence of motion and substance makes it sensible to regard it as one thing. As a scientist understanding the decentralised emergent phenomenon is interesting, but as a gardener wanting to move the rock it is an intellectual luxury and the (incorrect) centralised view of a the rock makes sense.

In real world problems, sometimes decentralised solutions work, other times they don’t.

I was once driving in Rome on a Saturday night (happily with a native Roman to guide me). It was after 11pm so they turned off all the traffic lights (as Italians ignore them anyway) and we came to a massive crossroads with completely full three lane roads in all directions. The space was filled with a criss-cross of apparently grid-locked cars and I thought we would be stuck there until a policeman came, but my navigator told me to simply drive. Every time the slightest gap opened, be it only a few inches, I would edge forward. Eventually, but after a relatively short time, we found ourselves at the other side. Thinking afterwards I realised that always some car was able to get out and I fact the ‘greedy’, decentralised algorithm worked perfectly.

Driving is different in north-west Scotland where there are long stretches of narrow single-track roads with passing places. When you spot a vehicle coming you watch out for a passing place and whichever of you gets to the one first waits there. If you notice too late and meet, then the person closest to a passing place may need to reverse. This is another local, slightly more polite, but semi-greedy algorithm, with each person making independent choices and trying to proceed, but taking into account immediately close road users. However, when single track roads get too full, this can fail. In situations, like the passing goods-train puzzles, where lines of vehicles have to pass with only single passing place, then often long lines of vehicles have to backup, go forward again, reverse again, in apparently disorganised ways – and ways in which each single driver cannot understand from local conditions alone. People have to get out their cars and start to coordinate their efforts.

Note that the decentralised strategies work remarkably well, and when they do require less effort than coordination. However, the reason that we do more than that, and think in ways that have an (at least behavioural) appearance of centralised control is because for certain problems this is needed – not least in complex social and technological situations.

With a HCI hat on, when we come to designing for people, we get the best solutions not when we ignore one aspect or another, but when we recognise the relative strengths of the two and how they can work together.

I recall when I was a child (yes over 30 years ago), seeing a television report about Benetton’s new CAD systems. The problem was cutting out rolls of cloth to make pieces for clothing. Traditionally an experienced cutter would arrange the pieces for a single garment as tightly as possible (to avoid waste), whilst ensuring proper orientations. These were then cut using a special form of guillotine. The new system of cutting from the roll allowed them to take the pieces for several garments and organise them over a long run of cloth for cutting. Doing several garments at once offered savings in terms of less wastage, but was a more challenging arrangement tasks … hence computer aid. The computer would take pieces initially arranged on (virtual) fabric and ‘jiggle’ them until they fitted closer with les waste. However, an experienced cutter would oversee this process and make large scale changes, “what if we tried this large piece over here?” The computer’s activity would have been serial, but could have been parallelised as it involved effectively lots of small local decisions. However, the human made strategic decisions, that themselves made use of the human’s internal associative pattern recognition, but from the point of view of the large system were effectively more centralised. Here a combination of centralised and decentralised thinking/computation together addressed a problem neither could solve on their own.

Finally, it is interesting to reflect on the ability demonstrated in both Clark and Resnick’s writing. They look in at our modes of thinking, see that they are often over-simplistic in terms of assuming central control when there is none, and then consider how to address this. This highly reasoned and reflective process does not arise naturally from decentralised thought that would simply go on using the same old ways of thinking, but is the product of exactly the more ‘rational’ linear, centralised thinking that they seek to expose as outmoded.

1. Mitchel Resnik (1994). Turtles Termites and Traffic Jams: Explorations in Massively Parallel Microworlds. MIT Press. [back]
2. This is similar to the argument in my previous post on the power of sequential thinking, where I pondered the complexity of establishing sequence within an underlying parallel and distributed neural superstructure … but also discussed the advantages it brings. Sequentiality and central control are of course closely linked. [back]

This reminded me of results of brain scans (sadly, I can’t recall the source), which showed that the areas in the brain where you store concepts like ‘chair’ are different from those where you store the sound of the word – and also I’m sure the spelling of it also.

This makes sense of the “tip of the tongue” phenomenon, you know that there is a word for something, but can’t find the exact word. Even more remarkable is that of you know words in different languages you can know this separately for each language.

So, musing on this, there seem to be very good reasons why, even within our own mind, we hold multiple representations for the “same” thing, such as chair, which are connected, but loosely coupled.

In an artificial “brain” like COG, the computational units are physically separate. In our brains things are much less discrete, but we know do have well defined locality for certain functions (e.g. Broca’s area for speech). Also, while there is debate about the extent to which we know what we are doing (or perhaps more important know what we are about to do) still it is clear that at least low-level functions operate semi-autonomously, but for instant reaction (pain withdrawal) and also for controlled actions (play a guitar chord).

In particular, there are particular brain lesions that mean that the patient cannot choose what to say, yet still vaguely grammatical but entirely meaningless utterances are constantly made – rather like James Joyce In between are cases where “nearly right” words come out, perhaps table, or cushion instead of chair. So for speech it seems a “higher level” part of the brain decides what we want to say and makes gentle suggestion for what this should be, but a different part does the final stringing together of words and it is this part that ‘knows’ the rules of grammar, the way words connect into each other and what the words sound like (although the grammar and aural elements may themselves be generated separately). This is rather like the intention to walk and the fine movement of muscles needed to move each leg.

Obviously the representations needed for saying ‘chair’ in a sentence are about the way it fits into grammatical structure, agreement with verbs, the sound of the word, and eventually (maybe at another level again) the way the lips need to be formed and air expelled. In contrast for choosing what to say, it is the semantics of chairs, the fact that you sit on them, they have legs, etc. that are important. The “planning to speak” bit needs to know there is a suitable word, what that word means and whether it will fit with other words, but does not need to know the details of how to say it. Similarly the “planning to move” bit needs to know roughly that legs can move in the desired way, but not the details of movement. Planning needs a model of action (speech or movement) and the model needs to be close enough to reality for it to work most of the time, but without all the details.

So the representations at a higher level need to share or independently represent enough of the lower level functions to be able to make appropriate suggestions for lower level action, but each will also represent different things. In addition, there needs to be some linkage between the two representations. Suppose you form the intention to say something like “The chair has four legs”. (Note “something like” because as you form the intention to say the thing the exact words will probably not be there.) In order for this intention to lead to the words “The chair has four legs”, something has to link the planning ‘chair’, with the saying ‘chair’.

This is not unlike human communication; we need both shared vocabulary and a level of shared meaning: so that when I say to a garage mechanic “the clutch is not working” it is the same thing we are referring to. However, we can also each have additional meanings, annotations etc.: the mechanic will know how the clutch works better than I do.

However, whereas human verbal communication has to be pushed through a discrete medium of signs, it seems more likely that there is a level of direct (but diffuse) connection between the ‘concepts’ used in our brains at different levels of activity.

1. Andy Clark. Being There. MIT Press. 1997. ISBN 0-262-53156-9. book@MIT [back]