Abstract
This preface to the Proceedings of Physicality 2006 describes some of the work at Lancaster and in the Equator project that were the initial inspiration for this workshop. We wish to understand physicality both because it is interesting in its own right and also because the understanding can help us design novel digital and hybrid digital–physical artefacts. Our own existing work is used to propose some initial properties and issues of physicality including rules of 'natural' interaction, issues of 'it-ness' and continuity in time and space, the physicality and instrumentation of the human body and issues of embodiment and spatiality.
Keywords: Physicality, spatiality, user-interface design, tangible computing, physiological sensing.
LOCAL ORIGINS
This workshop stemmed from a growing recognition that issues of the nature of physicality were emerging in several aspects of our research work here at Lancaster and also as part of the inter-disciplinary, multi-site Equator project.
This theme, in different aspects, is very clear in the work of the organisers. Eva Hornecker has worked alongside and studied people in several institutions who are deeply involved in the design of tangible interaction, where the importance of physicality is explicit, even if its precise nature is seldom articulated [9]. Devina Ramduny and I have studied the importance of physical artefacts in work environments (evident in virtually all ethnographic accounts), and more particularly the computational role of those artefacts in socio-technical systems and how studying artefacts and their disposition in the office ecology exposes tacit and often hidden work processes [10]. In Masitah Ghazali's work, perhaps even more explicitly, the precise nature of physicality has been critical in making sense of the way in which the fine details of physical design in day-to-day consumer products enables fluid interaction [6].
In addition, Nicolas Villar et al's work on Pin and Play emphasises the role of spatial arrangement and this is beginning to be teased out in more theoretical accounts of the role of spatiality [13]. Jennifer Sheridan's studies of technological interventions in artistic performance, with myself and colleagues at Nottingham, has again surfaced critical aspects of 'normal' physicality and how these have been systematically broken or bent over many years, for example the way in which limelight creates visual asymmetry between audience and player [1].
The papers in the workshop touch on some of the themes emerging above, but also other aspects such as the physicality of the body and the importance of physicality within the design process.
DEEPER UNDERSTANDING
The workshop brought together a multi-disciplinary group of researchers and designers, whose work, in various ways, relates to physicality. However, the primary goal of the workshop was to start to dig more deeply into the nature of physicality itself. What is it about physical things that we take for granted but perhaps makes them easy to grasp mentally and hence becomes problematic in digital environments? What aspects of physicality by being broken or bent in digital environments are thus surfaced, problematised and become things we can consider and reflect upon?
Put more operationally: what can understanding physicality do to help digital design and what about digital design can help our understanding of physicality?
The first of these questions is clearly moot for many involved in this workshop. The second is perhaps more philosophical, and not one we would expect to answer in two days! Indeed, over the years, when discussing with philosophers the issue of "it-ness" (what it is about a thing that makes it an 'it' before 'it' is a something), I have found it interesting that they repeatedly refer back to the singular spatial and temporal nature of objects, qualities that are not necessarily essential to computational artefacts.
So, to seed discussion, here are a few properties of physicality to be challenged, enriched, rejected or added to.
THREE RULES TO BE BROKEN
Some years ago, in teaching a software engineering course, I tried to characterise what it was that made software engineering more difficult than many areas of more physical engineering. I listed three properties of 'ordinary' physical things – that is inanimate, non-mechanical, 'natural' things like rocks.
directness of effort – You push a little, things move a little, you push a lot, things move a lot.
locality of effort – Things happen here and now. If you push something it moves at the moment you push it, not earlier or later. If you push in space, not touching an object, it does not move!
visibility of state – Whilst the more static appearance of an object may be complex, its dynamic state is defined largely by immediately apparent properties such as location, orientation, and (albeit harder to apprehend) speed and direction of movement and rotation.
These properties are systematically violated by computational artefacts. Take a mobile phone:
directness of effort – violated You press a single digit wrong, and end up dialling Brisbane rather than Blackburn.
locality of effort – violated Looking at spatial directness – the whole purpose of the phone is to talk to someone far away! Temporally as well: voice mail, alarms, waiting for a connection – all are non-local.
visibility of state – violated Hidden within the phone you have a large address book, old text messages and personal settings. Moreover the phone appears to be the repository of distant information and is influenced by invisible, almost magical, electromagnetic fields as you waft the phone near a window seeking signal.
The mention of magic in the last of these is not inconsequential; the violation of these principles often leads to magical explanations, either explicitly or implicitly, in people's understanding of phenomena. Imagine you put a glass down near the edge of a table, then as you turn your back the glass topples and crashes to the floor; it is hard to shake off that momentary chill down the spine even when you realise it has simply overbalanced.
Understanding these properties helps us realise why the design of software (or for that matter complex chemical plants, telecoms networks, and clockwork mice) is difficult. And from understanding comes better design. Many years ago, Harold Thimbleby included proportionality of effort among his GUEPS (generative usability design principles) [12], and in Masitah Ghazali's work the 'three properties', and others derived from and related to them, have been central.
Pinning down these properties is itself difficult; however even more problematically Masitah and others have been trying to experimentally 'test' the effects of some of them in interfaces – but we find it is quite hard to violate single properties without breaking many. Interestingly, we found similar difficulties 'testing' GUEPS back in the 1980s!
IT-NESS AND CONTINUITY OF IDENTITY
As noted, I have often found that philosophers refer back to the singularity of location in space at a given time in their explanations of the most fundamental aspects of thing-ness. Whilst the previous three properties were explicitly about inanimate things (kick a dog now and it may bite you in 5 seconds time), the more fundamental continuity in time and space is true also of animate things. In magical worlds this may not be the same. Hermione is able to study so many subjects at Hogwarts because she travels back and forth in time and is able to be in two places simultaneously [11]. The Frog Prince is continuous in time and space, but fundamentally changes what it/he is as frog becomes prince.
Digital objects (whilst usually not enabling time travel) also violate this fundamental spatio-temporal continuity. When I copy the file containing this paper I suddenly have two copies of 'the same' document. And whilst these are arguably different due to their different file names, the same could not be said for information stored redundantly on a RAID disk. Variables continually change their values. of course, but this could be argued to be like me changing the smile on my face; however, in the Smalltalk programming environment you can say that an object, like the Frog Prince, 'becomes' another of a completely different kind and type.
Even more strangely, the UNIX operating system allows you to write to parts of a file at arbitrary character locations. You can create a new file and then write a single 'X' at character location 1,000,000,000,000,000, but nothing else. If you look at the directory listing, the file appears to be a peta-byte in extent – far bigger than your hard disk drive. In fact the intermediate data is not explicitly stored anywhere as it is 'known' to be zero (the initialised value) and is only 'called into existence' if you subsequently attempt to read the file. (This is a sure way to frighten your system administrator!)
THE BODY
From earlier more 'inside-to-outside' Cartesian conceptions of cognition, there has developed, over a number of years, an increasing acceptance of the importance of physical embodiment for cognition and even selfhood. This is explicit in frameworks such as distributed cognition [8], where the role of physical artefacts and multiple actors is seen as essential for 'cognition' to occur – thinking as transactional with and within the world, as opposed to disembodied then acted out – imposed upon the world. This has also been central to the conceptions of the computational role of artefacts that Devina, Julie Wilkinson and I developed in the "socio-organisational Church–Turing hypothesis" [4].
Looking more fundamentally at 'the body' itself, Damasio locates consciousness effectively in our brain's self-image of our physical body [3]. Personally I have tended to look slightly more broadly to self-consciousness emerging as an 'accident' of third-order model of mind. In order to understand you I 'construct' a model of your intentions. Because I do this, I may either directly apply the same process to myself (as an 'other'), or more indirectly construct myself as an intentional being in order to understand myself in your eyes ... the first party 'I' developing from the third party 'me'. Others place consciousness in a neurological short-cut between mouth and ear, a silent narrative about oneself to oneself [2].
All of these are deeply embodied views where internal self-hood is intimately derived from or tied to external body-hood.
At a physiological level our bodies have an odd role of being both our own and yet also to some extent 'other': 'my' stomach does not always digest at 'my' will. One of the roles that physiological computing has played is in exposing aspects of users' bodily and mental functioning and making these explicit and apparent to 'their' mind. Alternatively, we may keep these 'secret' from the user, creating systems that respond to unconscious (but not necessarily unfelt) signs and signals. For example, Kiel Gilleade and others have worked on physiological sensing to influence computer game play – the body becomes sucked into the digital environment [7].
Strangely, the physical world does not always match our 'natural' understandings of physicality. The early twentieth century saw a series of discoveries, in particular quantum mechanics and relativity, that fundamentally challenged our understandings of the world and led to paradigm shifts of science, and to some extent broader public conceptions.
Perhaps even more strangely, our bodies also exhibit this relativity as nerve signals take a short but appreciable time to travel from eye to brain to hand ... not massively different to the fastest Internet packets travelling from end to end of the earth. In computing terms our body is a distributed system; the 'now' we experience is spread over approximately a second and the apparent continuity of movement as we trace our finger across a frosty window pane is, Zeno-like, composed of a series of more discrete commands from central control and observations from field units at a leisurely 5–10 Hz pace.
EMBODIED COMPUTATION AND SPATIALITY
It is a truism (but also a potential fallacy) that computation and information are always physical: electrons speeding along copper tracks and through silicon junctions, magnetic regions polarising, even ink on paper.
Just as the embodiedness of the human body is critical to understanding cognition, physical embodiment reminds us of crucial features of computation; for example, that you can only perform finite computation in finite time and space and that memory 'space' consumes physical space (a peta-byte is currently the size of a collection of large filing cabinets!). The simple Turing Machine, whilst being a conceptual computation engine, apparently moves a tape through itself, or possibly, as a real 'touring' machine, drives along the tape, finding memory externally represented outside of the core (and finite) engine itself.
However, this truism of embodiment is also misleading, as there is a difference between computation and computer, between word and page. Whilst the representatum [note 1] is physical the idea of the information is not. As highlighted by the word and page, this is not a new thing but has been a feature of literary works, and, before that, ideas and classes from times when silicon was still simply sand. Indeed the ontological status of ideas has been a difficult and contested topic within philosophy from at least Plato onwards.
So far I have only mentioned space briefly in passing, but clearly spatiality is a crucial aspect of physicality. In my previous writing about the development of ideas, expressed most recently in a keynote and chapter for the 2004 Space and Spatiality workshop at Napier, the relationship between spatial thinking and words and that of ideas and concepts has been central [5]. In particular, in the understanding of issues such as transarticulation – the way words form meanings – there are strong parallels between our partially imposed and partially determined namings and understandings of the physical landscape and the parallel namings and understandings of the intellectual landscape.
The physicality of representata, whether paper, silicon or neuron, does not determine but does shape the information, computation, and ideas that flow over and through them: linear narrative, planar graphs and patterns of thinking influenced by our physical existence.
ACKNOWLEDGEMENTS
This paper is drawing on the work of my fellow workshop co-organisers: Masitah Ghazali, Eva Hornecker and Devina Ramduny-Ellis as well as other colleagues at Lancaster and in Equator.
In addition, the Equator project (www.equator.ac.uk) sponsored this workshop and has supported my own work and much of the work referenced here.
NOTES
- Here I am using 'representatum' very narrowly to mean the material on or with which the information is represented (e.g. paper, ink). Some semioticians use the word effectively as a synonym for 'sign', which encompasses both the material and the form of representation on it (e.g. a black ink 'x' on paper).
REFERENCES
- Benford, S., Crabtree, A., Reeves, S., Sheridan, J., Dix, A., Flintham, M., and Drozd, A. (2006). Designing for the opportunities and risks of staging digital experiences in public settings. In Proceedings of CHI '06. ACM Press, pp. 427– 436.
- Bownds, M.D. (1999). Biology of Mind, Fitzgerald Science Press. (Page 94&95, discussion of Daniel Dennett's ideas)
- Damasio, A. (1994). Descartes' Error: Emotion, Reason, and the Human Brain, Avon Books.
- Dix, A., Wilkinson, J. and Ramduny, D. (1998). Redefining Organisational Memory - artefacts, and the distribution and coordination of work. in Workshop on Understanding Work and Designing Artefacts, York, 21st September 1998. http://www.hiraeth.com/alan/papers/artefacts98/
- Dix, A. (2004). Paths and Patches - patterns of geognosy and gnosis. Keynote at Second Workshop on Spaces, Spatiality and Technology Napier University, Edinburgh, 13-14 December 2004. http://www.hcibook.com/alan/papers/space2-2004/
- Ghazali, M. and Dix, A. (2005). Knowledge of Today for the Design of Tomorrow. in Proceedings of the 2nd International Design and Engagibility Conference (IDEC) 2005, Edinburgh, 6th September 2005
- Gilleade K., Dix A. and Allanson J, (2005). Affective Videogames and Modes of Affective Gaming: Assist Me, Challenge Me, Emote Me. in Online Proceedings of DIGRA 2005, June 2005. http://www.gamesconference.org/digra2005/viewabstract.php?id=256
- Hollan, J., Hutchins, E. and Kirsh, D. (2002). Distributed Cognition: toward a new foundation for human–computer interaction research. in Human–Computer Interaction in the New Millenium, J. Carroll (ed.), Addison Wesley, pp. 74–94
- Hornecker, E. and Buur, J. (2006). Getting a grip on tangible interaction: a framework on physical space and social interaction. In Proceedings of CHI '06. ACM Press, pp. 437–446
- Ramduny-Ellis, D., Dix, A., Rayson, P., Onditi, V., Sommerville, I. and Ransom, J. (2005). Artefacts as designed, Artefacts as used: resources for uncovering activity dynamics. Cognition, Technology and Work, Vol. 7, No. 2, pp: 76–87
- Rowling, J.K. (1999). Harry Potter and the Prisoner of Azkaban. Scholastic.
- Thimbleby, H. (1984): Generative User-Engineering Principles for User Interface Design. In: Proceedings of INTERACT 84, 1st IFIP International Conference on Human-Computer Interaction. pp.661–666.
- Villar, N., Lindsay, A. T., and Gellersen, H. (2004). Pin & Play & Perform: a rearrangeable interface for musical composition and performance. In Proceedings of the 2005 Conference on New Interfaces For Musical Expression (NIME). Singapore, pp. 188–191.
|