Alan Dix > 2009

Physigrams: Modelling Devices for Natural Interaction

Alan Dix1, Masitah Ghazali2, Steve Gill3, Joanna Hare3 and Devina Ramduny-Ellis1
1 Computing Department, Infolab21 Lancaster University, Lancaster, UK
2 Information Systems Department, Universiti Putra Malaysia, Malaysia
3 Cardiff School of Art & Design, UWIC, Cardiff, Wales, UK
< Alan on the Web > < Masitah on the Web > < Steve on the Web > <Jo on the Web > < Devina on the Web >

published Formal Aspects of Computing Volume 21, Number 6 / December, 2009
full text available through SpringerLink online first doi:10.1007/s00165-008-0099-y
download draft paper (PDF, 728K)

Abstract

This paper explores the formal specification of the physical behaviour of devices "unplugged" from their digital effects. By doing this we seek to better understand the nature of physical interaction and the way this can be exploited to improve the design of hybrid devices with both physical and digital features. We use modified state transition networks of the physical behaviour, which we call physiograms, and link these to parallel diagrams of the digital state. These are used to describe a number of features of physical interaction exposed by previous work and relevant properties expressed using a formal semantics of the diagrams. As well as being an analytic tool, the physigrams have been used in a case study where product designers used and adapted them as part of the design process.

Keywords: physicality, interaction modelling, affordance, natural interaction, physical devices, product design, physigrams
Full reference:
A. Dix, M. Ghazali, S. Gill, J. Hare and D. Ramduny-Ellis (2009). Physigrams: Modelling Devices for Natural Interaction Formal Aspects of Computing , Springer, 21(6):613-641, doi:10.1007/s00165-008-0099-y
http://www.hcibook.com/alan/papers/
FAC-physical-2009/
more:
download draft paper (PDF, 728K)
full text at Springer DOI: 10.1007/s00165-008-0099-y
physicality.org - investigating issues of physicality in human interaction
Aladdin's lamp: understanding new from old ... about other kinds on natural interaction
Visceral Interaction paper at HCI'2005
Modelling Devices for Natural Interaction earlier conference paper
 

References

  1. Anson, E.: The device model of interaction, SIGGRAPH Comput. Graph., 16:3, 107–114 (1992).
  2. Avrahami, D. and Hudson, S.: Forming interactivity: a tool for rapid prototyping of physical interactive products, Proc. of the 4th Conf. on Designing interactive Systems: Processes, Practices, Methods, and Techniques (DIS '02). ACM, pp. 141–146, 2002. http://doi.acm.org/10.1145/778712.778735
  3. Benford, S., H. Schnadelbach, B. Koleva, B. Gaver, A. Schmidt, A. Boucher, A. Steed, R. Anastasi, C.Greenhalgh, T. Rodden and H. Gellersen, “Sensible, Sensable and Desirable: A Framework for Designing Physical Interfaces,” Technical Report Equator-03-003, Equator, (2003) http://www.equator.ac.uk/.
  4. Booker, S. and Vertelney, L.: Designing the Whole-Product User Interface, In: Laural, B. (ed) The Art of Computer Interface Design. Addison Wesley, pp. 57–63, 1990.
  5. Buxton, W.: There's More to Interaction than Meets the Eye: Some Issues in Manual Input, In: Norman, D. and Draper, S. (eds) User Centered System Design: New Perspectives on Human–Computer Interaction, Lawrence Erlbaum Associates, Hillsdale, New Jersey, pp. 319–337, 1986.
  6. Buxton, W., A Three-State Model of Graphical Input, in: Human–Computer Interaction – INTERACT
    ’90. Amsterdam: Elsevier Science Publishers B.V, (North-Holland), (1990), 449–456.
  7. Card, S., Mackinlay, J. and Robertson, G.: The design space of input devices, Proc. of CHI'90, ACM Press, pp. 117–124, 1990.
  8. Card, S., Mackinlay, J. and Robertson, G.: A morphological analysis of the design space of input devices, ACM Trans. Inf. Syst., 9:2, 99–122 (1991).
  9. Carr, D.: Specification of interface interaction objects, Proc. of CHI '94, ACM, pp. 372–378, 1994.
  10. Coutrix, C. and Nigay, L.: Mixed Reality: A Model of Mixed Interaction, Proc. of AVI'06, ACM Press, pp. 43–50, 2006.
  11. Coutrix, C. and Nigay, L.: Balancing physical and digital properties in mixed objects, Proc. of AVI'08, the Working Conf. on Advanced Visual interfaces, ACM Press, pp. 305–308, 2008.
  12. Curzon, P., Ruksenas, R. and Blandford, A.: An approach to formal verification of human-computer interaction, Formal Aspects of Computing, 19:4, 513–550 (2007).
  13. Degani, A.: Taming HAL: Designing Interfaces Beyond 2001, Palgrave Macmillan, 2004.
  14. Dix, A. and Runciman, C.: Abstract models of interactive systems, People and Computers: Designing the Interface, Cambridge University Press, pp. 13–22, 1985.
  15. Dix, A.: Formal Methods for Interactive Systems, Academic Press, 1991. http://www.hiraeth.com/books/formal/
  16. Dix, A.: Status and events: static and dynamic properties of interactive systems, Proc. of the Eurographics Seminar: Formal Methods in Computer Graphics, 1991. http://www.hcibook.com/alan/papers/euro91/
  17. Dix, A. and G. Abowd, Modelling status and event behaviour of interactive systems. Software Engineering Journal, 11:6 (1996), 334–346 http://www.hcibook.com/alan/papers/SEJ96-s+e/.
  18. Dix, A. and Abowd, G.: Delays and temporal incoherence due to the mediated status–status mappings, SIGCHI Bulletin, 28:2, 47–49 (1996).
  19. Dix, A., Getting Physical. keynote at: OZCHI 2003, Brisbane, Australia, 26-28 Nov 2003 http://www.hcibook.com/alan/talks/ozchi2003-keynote/.
  20. Dix, A., J. Finlay, G. Abowd and R. Beale, “Human-Computer Interaction. Third Edition,” Prentice Hall, (2004) http://www.hcibook.com/e3/.
  21. Dix, A.: Designing for Appropriation, Procedings of BCS HCI 2007, People and Computers XXI, Volume 2, BCS eWiC, 2007. http://www.bcs.org/server.php?show=ConWebDoc.13347
  22. Dix, A., Ghazali, M. and Ramduny-Ellis, D.: Modelling Devices for Natural Interaction, Proc. of Second Intnl. Workshop on Formal Methods for Interactive Systems, FMIS2007, ENTCS, Elsevier, 2007.
  23. Dix, A., Leite, J. and Friday, A.: XSED – XML-based Description of Status–Event Components and Systems, Proc. of Engineering Interactive Systems 2007 (EIS 2007), LNCS 4940, Springer, 2008.
  24. Dubois, E., Silva, P. and Gray, P.: Notational Support for the Design of Augmented Reality Systems, Proc. of the 9th International Workshop on interactive Systems. Design, Specification, and Verification, DSVIS2002, LNCS 2545, Springer, pp. 74–88, 2002.
  25. Dubois, E. and Gray, P.: A Design-Oriented Information-Flow Refinement of the ASUR Interaction Model, Engineering Interactive Systems (incorporating EHCI, HCSE, DSV-IS), LNCS 4940, Springer, 2008.
  26. Dubois, E., Gray, P. and Ramsay. A.: A Model-Based Approach to Describing and Reasoning about the Physicality of Interaction, Proc. of Physicality 2007, UWIC Press, Cardiff, Wales, pp. 77-82, 2007.
  27. Eslambolchilar, P.: Making Sense of Interaction Using a Model-Based Approach, PhD thesis, Hamilton Institute, National University of Ireland, NUIM, Ireland, 2006.
  28. Evans, M. and Gill, S.: Rapid Development of Information Appliances, Proc. of International Design Conf. – Design 2006, (Croatia, 15–18 May 2006)
  29. Gaver, W., Technology affordances. in: Proceedings of CHI ’91. ACM Press, (1991),79–84.
  30. Ghazali, M. and A. Dix, Aladdin’s lamp: understanding new from old. In 1st UK-UbiNet Workshop, 25-26th September 2003, Imperial College London (2003) http://www.hcibook.com/alan/papers/ubinet-2003/.
  31. Ghazali, M. and A. Dix, Visceral Interaction. in: Proceedings of the 10th British HCI conference, Vol 2, September 5-9, Edinburgh. (2005), 68–72 http://www.hcibook.com/alan/papers/visceral-2005/.
  32. Gibson, J., “The Ecological Approach to Visual Perception,” Houghton Mifflin Company, USA, 1986.
  33. Gill, S., Loudon, G., Hewett, B. and Barham, G.: How to Design and Prototype an Information Appliance in 24 Hours – Integrating Product \& Interface Design Processes, in: Proc. of the 6th International Conf. on Computer Aided Industrial Design and Concept Design, University of Delft, The Netherlands, 2005.
  34. Green, T. and Petri, M.: Usability analysis of visual programming environments: A 'cognitive dimensions' framework, Journal of Visual Languages and Computing, 7, 131–174 (1996).
  35. Greenberg, S. and Fitchett, C.: Phidgets: easy development of physical interfaces through physical widgets, Proc. of the 14th Annual ACM Symposium on User interface Software and Technology (UIST '01), pp. 209–218, 2001. http://doi.acm.org/10.1145/502348.502388
  36. Harel, D.: Statecharts: A visual formalism for complex systems, Science of Computer Programming, 8:3, 231–274 (1987).
  37. Harrison, M., Kray, C. and Campos, J.: Exploring an option space to engineer a ubiquitous computing system, 2nd International Workshop on Formal Methods for Interactive Systems (FMIS 2007), Electronic Notes in Theorectical Computer Science 208, Elsevier, pp. 41–55, 2007/2008.
  38. Hartman, B., Klemmer, S., Bernstein, M. and Mehta, N.: d.tools: Visually Prototyping Physical UIs through Statecharts, Extended Abstracts of UIST 2005, ACM, 2005.
  39. Hartson, H., Cognitive, physical, sensory, and functional affordances in interaction design. Behaviour & Information Technology 22:5 (2003), 315–338.
  40. Ishii, H.: Tangible User Interfaces, Chapter 24 in: Sears, A. and Jacko, J. (eds) The Human-Computer Interaction Handbook Fundamentals, Evolving Technologies, and Emerging Applications (second edition). Laurence Earlbaum, London, pp. 469–487, 2008.
  41. Jacob, J.: Deligiannidis, L., and Morrison, S.: A software model and specification language for non-WIMP user interfaces, ACM Trans. Computer–Human Interaction, 6:1, 1–46 (1999).
  42. Johnson, C.: The Evaluation Of User Interface Design Notations, Proc. of Design, Specification and Verification of Interactive Systems '96, Springer, pp. 188–206, 1996. http://www.dcs.gla.ac.uk/\tilda johnson/papers/chris\_jarle/
  43. Landay, J. and Myers, B.: Interactive sketching for the early stages of user interface design, Proc. of CHI'95, ACM Press/Addison-Wesley, pp. 43–50, 1995. http://doi.acm.org/10.1145/223904.223910
  44. Loer, K. and Harrison, M.: Towards Usable and Relevant Model Checking Techniques for the Analysis of Dependable Interactive Systems, Proc. 17th International Conf. on Automated Software Engineering, IEEE Computer Society, pp. 223–226, 2002.
  45. Massink, M., Duke, D. and Smith, S.: Towards hybrid interface specification for virtual environments, DSV-IS 1999 Design, Specification and Verification of Interactive Systems, Springer, pp. 30–51, 1999.
  46. Milner, N.: A Review of Human Performance and Preferences with Different Input Devices to Computer Systems, Proc. of HCI’88, People and Computers IV, Cambridge University Press, pp. 341–362, 1988.
  47. Moher, T., Dirda, V., Bastide, R. and Palanque, P.: Monolingual, Articulated modelling of Users, Devices and Interfaces, 3rd EUROGRAPHICS workshop on "design, specification and verification of Interactive systems, Springer, pp. 312–329, 1996.
  48. Monk, A.: Mode Errors: A User-Centered Analysis and Some Preventative Measures Using Keying-Contingent Sound, International Journal of Man–Machine Studies, 24:4, 313–327 (1986).
  49. Nielsen, J., and R. Mack, “Usability Inspection Methods,” John Wiley & Sons, New York, 1994.
  50. Nigay, L. and Coutaz, J.: Building User Interfaces: Organizing Software Agents, ESPRIT `91 Conf., pp. 707–719, 1991.
  51. Nigay, L. and Coutaz, J.: A generic platform for addressing the multimodal challenge, Proc. of CHI'95, ACM, pp. 98–105, 1995.
  52. Norman, D., Affordance, conventions, and design. Interactions 6:3 (1999), 38–43
  53. Palanque, P. and Paterno, F. (eds): Formal Methods in Human–Computer Interaction, Springer, 1997.
  54. Parnas, D.: On the use of transition diagrams in the design of a user interface for an interactive computer system, Proc. of the 1969 24th National Conf., ACM, pp. 379–385, 1969.
  55. Payne, S. and Green, T.: Task–action grammars: a model of mental representation of task languages, Human–Computer Interaction, 2:2, 93–133 (1986).
  56. Pfaff, G., and P. Hagen, (Eds.). “Seeheim Workshop on User Interface Management Systems,” Springer- Verlag, Berlin, 1985.
  57. Phidgets Inc., 2008. http://www.phidgets.com/
  58. Pierce, J., Stearns, B. and Pausch, R.: Voodoo Dolls: Seamless Interaction at Multiple Scales in Virtual Environments, Proc. of the 1999 Symposium on Interactive 3D Graphics, pp. 141-145, 1999.
  59. Ramduny-Ellis, D., Dix, A., Rayson, P., Onditi, V., Sommerville, I. and Ransom, J.: Artefacts as designed, Artefacts as used: resources for uncovering activity dynamics, Cognition, Technology and Work, 7:2, 76–87 (2005).
  60. Reisner, P.: Formal grammar and human factors design of an interactive graphics system, IEEE Transactions on Software Engineering, SE-7:2, 229–240 (1981).
  61. Shneiderman, B., Direct manipulation: a step beyond programming languages. IEEE Computer 16:8 (1983), 57–69.
  62. Shneiderman, B.: Response Time and Display Rate in Human Performance with Computers, ACM Compuer. Surveys, 16:3, 265–285 (1984).
  63. Smith, S.: Exploring the specification of haptic interaction, Interactive Systems: Design, Specification and Verification (DSVIS 2006), LNCS 4323, Springer, pp. 171-184, 2007.
  64. Sufrin, B.: Formal specification of a display editor, Science of Computer Programming, 1, 157–202 (1982).
  65. Thimbleby, H. and Harrison, M.: Formal Methods in Human-Computer Interaction, Cambridge University Press, 1990.
  66. Thimbleby, H.: "Press On: Principles of Interaction Programmin"', MIT Press (2007).
  67. Thimbleby, H.: Using the Fitts Law with State Transition Systems to Find Optimal Task Timings. in: Pre-Proc. of Second Intnl. Workshop on Formal Methods for Interactive Systems, FMIS2007, 2007. http://www.dcs.qmul.ac.uk/research/imc/hum/fmis2007/preproceedings/FMIS2007preproceedings.pdf
  68. UIMS: A Metamodel for the Runtime Architecture of an Interactive System: the UIMS Tool Developers Workshop, SIGCHI Bulletin, 24:1, 32–37 (1992).
  69. Ullmer, B., Ishii, H. and Jacob, R.: Token+constraint systems for tangible interaction with digital information, ACM Trans. Computer–Human Interaction, 12:1, 81–118 (2005).
  70. Wensveen, S., J. Dja jadiningrat and C. Overbeeke, Interaction Frogger: A Design Framework to Couple Action and Function. in: Proceedings of the DIS’04, ACM Press, (2004) 177–18
  71. Willans, J. and Harrison, M.: Verifying the behaviour of virtual world objects, Proc. of DSV-IS'2000, P. Palanque and F. Paternó (eds). Springer, pp. 65–77, 2001.
  72. Wüthrich, C.: An analysis and model of 3D interaction methods and devices for virtual reality, Proc. of DSV-IS'99, Springer, pp. 18–29, 1999.
  73. Young, R., Green, T. and Simon, T.: Programmable user models for predictive evaluation of interface design, Proc. of CHI ’89: Human Factors in Computing Systems, ACM Press, 1989.


FIGURE 1.


FIGURE 2.

   
FIGURE 3.


FIGURE 4.

x
FIGURE 5.


FIGURE 6 (i).


FIGURE 6 (ii).

  
FIGURE 7 (i).


FIGURE 7 (ii).


FIGURE 8.

   
FIGURE 9.


FIGURE 10.

  
FIGURE 11.

  
FIGURE 12.

  
FIGURE 13.

     
FIGURE 14.


FIGURE 15.

  
FIGURE 16.

http://www.hcibook.com/alan/papers/FAC-physical-2009/

Alan Dix 12/11/2008