“30% of the UK population have no internet access at home”

I couldn’t find the exact source of this, however, another  guardian article “UK internet audience rises by 1.9 million over last year” dated Wednesday 30 June 2010 has a similar figure.  This says that Internet use  has grown to 38.8 million. The National Statistics office say the overall UK population is 61,792,000 with 1/5 under 16, so call that 2 in 16 under 10 or around 8 million. That gives an overall population of a little under 54 million over 10 years old, that is still only 70% actually using the web at all.

My guess is that some of the people with internet at home do not use it, and some of the ones without home connections use it using other means (mobile, use at school, cyber cafe’s), but by both measures we are hardly a society where the web is as ubiquitous as one might have imagined.

I mentioned ‘Project Phoenix’ in my last previous post, and this was it – getting vfridge up and running again.

Ten years ago I was part of a dot.com company aQtive¹ with Russell Beale, Andy Wood and others.  Just before it folded in the aftermath of the dot.com crash, aQtive spawned a small spin-off vfridge.com.  The virtual fridge was a social networking web site before the term existed, and while vfridge the company went the way of most dot.coms, for some time after I kept the vfridge web site running on Fiona’s servers until it gradually ‘decayed’ partly due to Javascript/DOM changes and partly due to Java’s interactions with mysql becoming unstable (note very, very old Java code!).  But it is now back online

The core idea of vfridge is placing small notes, photos and ‘magnets’ in a shareable web area that can be moved around and arranged like you might with notes held by magnets to a fridge door.

Underlying vfridge was what we called the websharer vision, which looked towards a web of user-generated content.  Now this is passé, but at the time  was directly counter to accepted wisdom and looking back seem prescient – remember this was written in 1999:

Although everyone isn’t a web developer, it is likely that soon everyone will become an Internet communicator — email, PC-voice-comms, bulletin boards, etc. For some this will be via a PC, for others using a web-phone, set-top box or Internet-enabled games console.
…

The web/Internet is not just a medium for publishing, but a potential shared place.

Everyone may be a web sharer — not a publisher of formal public ‘content’, but personal or semi-private sharing of informal ‘bits and pieces’ with family, friends, local community and virtual communities such as fan clubs.

This is not just a future for the cognoscenti, but for anyone who chats in the pub or wants to show granny in Scunthorpe the baby’s first photos.

Just over a year ago I thought it would be good to write a retrospective about vfridge in the light of the social networking revolution.  We did a poster “Designing a virtual fridge” about vfridge years ago at a Computers and Fun workshop, but have never written at length abut its design and development.  In particular it would be good to analyse the reasons, technical, social and commercial, why it did not ‘take off’ the time.  However, it is hard to do write about it without good screen shots, and could I find any? (Although now I have)  So I thought it would be good to revive it and now you can try it out again. I started with a few days effort last year at Christmas and Easter time (leisure activity), but now over the last week have at last used the fact that I have half my time unpaid and so free for my own activities … and it is done

The original vfridge was implemented using Java Servlets, but I have rebuilt it in PHP.  While the original development took over a year (starting down in Coornwall while on holiday watching the solar eclipse), this re-build took about 10 days effort, although of course with no design decisions needed.  The reason it took so much development back then is one of the things I want to consider when I write the retrospective.

As far as possible the actual behaviour and design is exactly as it was back in 2000 … and yes it does feel clunky, with lots of refreshing (remember no AJAX or web2.0 in those days) and of course loads of frames!  In fact there is a little cleverness that allowed some client-end processing pre-AJAX².    Also the new implementation uses the same templates as the original one, although the expansion engine had to be rewritten in PHP.  In fact this template engine was one of our most re-used bits of Java code, although now of course many alternatives.  Maybe I will return to a discussion of that in another post.

I have even resurrected the old mobile interface.  Yes there were WAP phones even in 2000, albeit with tiny green and black screens.  I still recall the excitement I felt the first time I entered a note on the phone and saw it appear on a web page   However, this was one place I had to extensively edit the page templates as nothing seems to process WML anymore, so the WML had to be converted to plain-text-ish HTML, as close as possible to those old phones!  Looks rather odd on the iPhone :-/

So, if you were one of those who had an account back in 2000 (Panos Markopoulos used it to share his baby photos ), then everything is still there just as you left it!

If not, then you can register now and play.

1. The old aQtive website is still viewable at aqtive.org, but don’t try to install onCue, it was developed in the days of Windows NT. [back]
2. One trick used the fact that you can get Javascript to pre-load images.  When the front-end Javascript code wanted to send information back to the server it preloaded an image URL that was really just to activate a back-end script.  The frames  used a change-propagation system, so that only those frames that were dependent on particular user actions were refreshed.  All of this is preserved in the current system, peek at the Javascript on the pages.    Maybe I’ll write about the details of these another time. [back]

Devices are clearly everywhere, but the connections between them seem infrequent and poor.

Why is ubiquitous technology still so … well un-ubiquitous?

Why bother

One reason for lack of connectivity, is that for each production and purchase decision there is little need for it.  It is possible to wire up your whole home with sensors as Andy Stanford-Clark‘s twittering house demonstrates.  But for most people why put sensors in your home unless they are for something?  Why bother to buy things unless they give you value now? Home security, central hearing control, warning you when your fuel is running low — these are reasons for sensors, but in general we tend to buy things that do things for us, not things that might possibly plug together to do something interesting. This is maybe even more true of the new generation of digital natives, brought up with single function toys, and even Lego in  pre-determined models rather than generic kits.

Of course, we do buy things with sensors, the electric kettle has a temperature sensor, the TV knows what channel is tuned — but these are single purpose.  Even SCART connections for TV/DVD/satellite boxes are pretty limited unless the devices expect to be chained.  For the manufacturer, why bother to make stand-alone devices connectible; it will cost more in terms of a wireless chip, not to mention complex interfaces to manage privacy.  Even software rarely takes this step to make itself easy to interconnect — despite the mash-culture of Web2.0.  The barcode reader app on the iPhone is a rare exception; here the cost was relatively low and purely in  software (up front cost, but no cost per ‘device’), and yet even this is rare to see.  Once there are hardware costs even generic devices such as phones only include sensors for which they can see an immediate use: for ecxample, despite the presence of GPS and compass in the iPhone, the new iPad, with sightly different expected uses, has neither.  For physical devices, connectivity may come when individual devices require external interfaces, maybe to manage complex installations or monitoring via mini-webservers for less cost than dedicated displays, just as with most home routers.  Then the hardware cost of connectivity has a purpose for the dedicated device, and the additional cost of genericity is software only.

What’s there

It is interesting to look back at Mark Weiser‘s original vision of ubiquitous computing.  It is less connected than Adam’s ‘everywhere’, but equally rooted in research prototypes and potential, and yet has certainly come to pass as computation is certainly ‘ubiquitous’ in many ways.  Like others, I have always been struck by the disconnect between the opening words of Weiser’s The Computer for the 21st Century “The most profound technologies are those that disappear” and the rest of the paper, which focused on displays the visible face of computation.  But, with the exception of miniature MP3 players,  it is precisely these visible displays that are part of our lives.

Weiser classified displays into tabs, pads and boards, with associated scales (inch, foot, yard), and indeed the size of displays is crucial to their uses¹.  However, it is perhaps better to look at Wesier’s definitons of where and how these are used: tabs are ‘on things’ microwaves, some TV remotes, and old mobile phones (the kind that were primarily for telephoning people!), pads are held and carried and while decidely smaller than an foot, the iPhone and similar smartphones are clearly in this category, and finally boards are in the environment, stuck on walls (digital photo frames) and furniture (Microsoft surface).  All can be seen in many homes and offices, although the latter have perhaps had least impact so far,  laregly because of costs of production (foreseen by Wesier), but are clearly coming on stream.

The crucial thing is that all of these displays do something for us.  They are ubiquitous because they are not invisible.

How to use them

Adam’s book describes the profoundly different user experience of ‘everywhere’ interactions (pp:27-39), rather than being task-driven by the user, in situations such as such as automatic room lights “the system precedes the user“, “Whether or not you walk into the room in pursuance of a particular aim or goal, the system’s reaction to your arrival is probably tangential to that goal.“.  This what I have previosly called ‘incidental interaction‘, interactions not driven by the user’s immediate intentions, but by the system’s actions for the user.

Somewhat strangely the discussion in ‘Everywhere’ then turns to Bellotti et al’s “Making sense of sensing systems: five questions for designers and researchers“, which summarises some of the key issues of pervasive systems dating back to the 1984 Phone Slave project², such as knowing how to address a system that has no keyboard and knowing whether it is attending.  These are really important issues, but are about intentional use, getting the system, albeit embedded in the environment, to do something that the user has decided. At a more device level,  Abricht Schimdt‘s concept of implicit interaction³ is also in this area, making the interaction to achieve some purpose as natural and invisible as possible using sensors such as accelerometers. This is now common in devices such as the iPhone, but is still essentially about intentional interaction — again getting the device to do something.

In contrast, we seem to still have little understanding of the user experience of incidental interactions, where the system spontaneously does things for or to us, and they are still rare in the home and office, a few specific devices such as automatic lights, doors and central heating controllers, with little sign of widespread connectivity.  The main exceptions seem to be at a very large scale, Amazon book recommendations or traffic management systems.

The latter is perhaps an interesting comment on ubiquity as, for the ordinary user, the display is simply a red light.

1. see recent papers with Corina Sas “Mobile Personal Devices meet Situated Public Displays: Synergies and Opportunities” and with Lucia Terrenghi and Aaron Quigley “A taxonomy for and analysis of multi-person-display ecosystems“, which extend this size classification to larger and smaller displays. [back]
2. See “Phone Slave: A Graphical Telecommunications Interface” and YouTube video, although it was the longer envusonment video, that raised the user interaction design questions. [back]
3. Schmidt, A. Implicit Human Computer Interaction Through Context. Personal Technologies, 4(2&3), Springer-Verlag, 191–199, 2000 [back]

The USP for Dabble DB amongst other online data sharing apps, is that it appears to really be a complete database solution online … and its USB amongst conventional databses is the way they seem to have really thought about real use.  This focus on real use by ordinary users includes dynamically altering the structure of the data as you gradually understand it more.  The model they have is that you start with plain table data from a spreadsheet or other document and gradually add structure as opposed to the “first analyse and then enter” model of traditional DBs.

As I read Enrico’s blog I remembered that he had mailed me about the ‘magic/replace‘ feature ages ago.  This lets you tidy up  data during import (but apparently not data already imported … wonder why?), using a ‘by example’ approach and is a really nice example of all that ‘programming by example‘ and related work that was so hot 15 years ago eventually finding its way into real products.

The downside to Dabble DB is that editing is via forms only … it is often so much easier to enter data in a spreadsheet view, the API is quite limited, and while they have a ‘Dabble DB Commons‘ for public data (rather like Swivel), there is no directory or other way to see what people have put up

I was particularly hoping the API was better as it would have been nice to link it into my web version of Query-by-Browsing. or even integrate with the Query-through-Drilldown approach for constructing complex table joins that Damon Oram implemented more recently.

In general, while the DB and (many) UI features are strong it is not really looking outwards to creating shared linked data (in the broadest sense of the term, not just pure SemWeb world linked data), … so still room there for the absolute killer shared data app!

I came across the Google Zeitgeist 2008 page that analyses the popular and the rising search terms of 2008.  The rising ones reveal things in the media “sarah palin” way in there above “obama” in the global stats.  … if Google searches were votes!  However, the ‘most popular’ searches reveal longer term habits.  For the UK the 10 most popular searches are:

1. facebook
2. bbc
3. youtube
4. ebay
5. games
6. news
7. hotmail
8. bebo
9. yahoo
10. jobs

Some of these terms ‘games’, ‘news’, and ‘jobs’ (no Steve, not you) are generic categories … and suggests that people approach these from the search box, not a portal.  However, of these top 10, seven of them are simply domain names of popular sites.  Instead of typing this into the address bar (which certainly on Firefox autocompletes if I type any I’ve visited before), many users just Google it (and I’m sure the same is true for LiveSearch and others).

I was told some years ago that AOL browsers swapped the relative sizes (and locations I think) of the built-in search box and address bar on the assumption that their users rarelt tyoed in URLs (although I knew of AOL users who accidentally typed URLs into the search box).  Also recalling the company that used to sell net keywords that were used by Netscape (and possibly others) if you entered terms rather than a URL into the adders bar.

… of course if I try that now … FireFox  redirects me through Google “I feel lucky” … of course

Incidentally I came to this as I was trailing back the source of the, now shown to be incorrect, Sunday Times news story that said two Google seaches used the same electricity as boiling an electric kettle.  This got challenged in a TechCrunch blog, refuted by Google, and was effectively (but not explcitly) retracted in subsequent Times online item.  The source turns out to be a junior Harvard physicist, Alex Wissner-Gross, whose own source was a blog by Rolf Kersten, one of the Sun Green Team (Sun the computer manufacturer not Sun the newspaper!), so actually not an unreasonable basis.

In fact Rolf Kersten’s estimate, which was prepared for a talk in 2007, seemed to be based on sensible calculations, although he has recently posted a blog saying the figure was out by a factor of 35 … yes it actually takes 70 Google searches to boil that kettle.  Looking deeper the cause of the discrepancy appears to be the figure he used for the number of Google searches per day.  He took 2005 data about the size of the Google server farm and used a figure of 40 million searches per day.  Although Google did not publish their full workings in their response, it is clearly this figure of 40 million hits that was way too low for 2005 as a Feb 2001 Google press release quoted 60 million searches per day in 2000.  Actually with a moment’s reflection it is clear that 40 million hits per day (500 per second) would hardly have justified a major server farm and the figure is clearly in the billions.  However, it is surprisingly difficult to find the true figure and if you Google “google searches per day” you simply find lots of people asking the same question.  In fact, it was through looking for further Google press releases to find a more up-to-date figure that got me to the Zeitgeist page!

A Eamonn Fitzgerald’s Rainy Day blog nicely lays out the timeline of this story and sees it as a triumph of the power of media consumers to challenge the authority of the press due to what Jay Rosen refers to as  ‘audience atomization‘.   Fitzgerald also sees the paradox that the story itself was sourced from the somewhat broken sources on the internet; in the past the press would have perhaps used more authoritative sources … and as I noted couple of years ago at a Memories for Life panel at the British Library, the move from BBC to YouTube could be read as mass democratisation … or simply signal the end of history.

There is another lesson though, one that I picked up in a blog “keeping track of history” not long after the Memories for Life meeting, just how hard it is to find pretty straightforward information on the web.  At that point I was after Tony Blair’s statement about the execution of Saddam Husssein, in this case trying to find out the number of Google search hits.  Neither are secret, propriety or obscure, but both difficult to track down.

… but we still trust that single hit of a search button

My own keynote at PPIG2008 was “as we may code: the art (and craft) of computer programming in the 21st century” and was an exploration of the changes in coding from 1968 when Knuth published the first of his books on “the art of computer programming“.  On the web site for the talk I’ve made a relatively unstructured list of some of the distinctions I’ve noticed between 20th and 21st Century coding (C20 vs. C21); and in my slides I have started to add some more structure.  In general we have a move from more mathematical, analytic, problem solving approach, to something more akin to a search task, finding the right bits to fit together with a greater need for information management and social skills. Both this characterisation and the list are, of course, a gross simplification, but seem to capture some of the change of spirit.  These changes suggest different cognitive issues to be explored and maybe different personality types involved – as one of the attendees, David Greathead, pointed out, rather like the judging vs. perceiving personality distinction in Myers-Briggs¹.

One interesting comment on this was from Marian Petre, who has studied many professional programmers.  Her impression, and echoed by others, was that the heavy-hitters were the more experienced programmers who had adapted to newer styles of programming, whereas  the younger programmers found it harder to adapt the other way when they hit difficult problems.  Another attendee suggested that perhaps I was focused more on application coding and that system coding and system programmers were still operating in the C20 mode.

The social nature of modern coding came out in several papers about agile methods and pair programming.  As well as being an important phenomena in its own right, pair programming gives a level of think-aloud  ‘for free’, so maybe this will also cast light on individual coding.

Margaret-Anne Storey gave a fascinating keynote about the use of comments and annotations in code and again this picks up the social nature of code as she was studying open-source coding where comments are often for other people in the community, maybe explaining actions, or suggesting improvements.  She reviewed a lot of material in the area and I was especially interested in one result that showed that novice programmers with small pieces of code found method comments more useful than class comments.  Given my own frequent complaint that code is inadequately documented at the class or higher level, this appeared to disagree with my own impressions.  However, in discussion it seemed that this was probably accounted for by differences in context: novice vs. expert programmers, small vs large code, internal comments vs. external documentation.  One of the big problems I find is that the way different classes work together to produce effects is particularly poorly documented.  Margaret-Anne described one system her group had worked on² that allowed you to write a tour of your code opening windows, highlighting sections, etc.

I sadly missed some of the presentations as I had to go to other meetings (the danger of a conference at your home site!), but I did get to some and  was particularly fascinated by the more theoretical/philosophical session including one paper addressing the psychological origins of the notions of objects and another focused on (the dangers of) abstraction.

The latter, presented by Luke Church, critiqued  Jeanette Wing‘s 2006 CACM paper on Computational Thinking.  This is evidently a ‘big thing’ with loads of funding and hype … but one that I had entirely missed :-/ Basically the idea is to translate the ways that one thinks about computation to problems other than computers – nerds rule OK. The tenet’s of computational thinking seem to overlap a lot with management thinking and also reminded me of the way my own HCI community and also parts of the Design (with capital D) community in different ways are trying to say they we/they are the universal discipline  … well if we don’t say it about our own discipline who will …the physicists have been getting away with it for years

Luke (and his co-authors) argument is that abstraction can be dangerous (although of course it is also powerful).  It would be interesting perhaps rather than Wing’s paper to look at this argument alongside  Jeff Kramer’s 2007 CACM article “Is abstraction the key to computing?“, which I recall liking because it says computer scientists ought to know more mathematics

I also sadly missed some of Adrian Mackenzie‘s closing keynote … although this time not due to competing meetings but because I had been up since 4:30am reading a PhD thesis and after lunch on a Friday had begin to flag!  However, this was no reflection an Adrian’s talk and the bits I heard were fascinating looking at the way bio-tech is using the language of software engineering.  This sparked a debate relating back to the overuse of abstraction, especially in the case of the genome where interactions between parts are strong and so the software component analogy weak.  It also reminded me of yet another relatively recent paper³ on the way computation can be seen in many phenomena and should not be construed solely as a science of computers.

As well as the academic content it was great to be with the PPIG crowd they are a small but very welcoming and accepting community – I don’t recall anything but constructive and friendly debate … and next year they have PPIG09 in Limerick – PPIG and Guiness what could be better!

1. David has done some really interesting work on the relationship between personality types and different kinds of programming tasks.  I’ve seen him present before about debugging and unfortunately had to miss his talk at PPIG on comprehension.  Given his work has has shown clearly that there are strong correlations between certain personality attributes and coding, it would be good to see more qualitative work investigating the nature of the differences.   I’d like to know whether strategies change between personality types: for example, between systematic debugging and more insight-based scan and see it bug finding. [back]
2. but I can’t find on their website [back]
3. Perhaps 2006/2007 in either CACM or Computer Journal, if anyone knows the one I mean please remind me! [back]

Within the web it is easy to link things together. If I want to refer to my home page I just add a link like this. However, on the desktop things are not so simple and I end up copying chunks of mail messages into the notes field in iCal rather than simply being able to link to the mail message where I arranged the meeting.

Links from the desktop to the web are easy … just use the URL … many desktop applications including mail clients and word processors will allow you to embed clickable links. Indeed it is often easier to link to a web page than to another object on the desktop! However, things get more difficult if you want to link the other way round, from a web page to a local file or resource. In my browser’s favourites I have several links to local files, but you cannot easily do the same if your bookmarks are in a web service like del.icio.us or even my own Snip!t. It is hard to seamlessly weave your desktop into the global web.

A couple of events brought this issue to a head for me.

First at the CHI workshop on PIM entitled the Disappearing Desktop, I asked if anyone knew of work in the area and I heard from Leo Sauermann that they had made some progress on this as part of the Gnowsis project. Their proposal for a Desktop URI Scheme (edited by Leo) is targeted principally at the first of the scenarios above, being able to link between things within the desktop.

The second event was at the AVI workshop on designing multi-touch interaction techniques for coupled public and private displays. During discussions abut touch-based interactions such as the Microsoft Surface or Apple iPhone, we considered scenarios where peole got together for a meeting (as we were) in a hotel bar (where we split for small group discussion) and had screens on table tops and walls, laptops, tablets, phones … and wanted to seamlessly move material between devices. Clearly an essential requirement for which is some way to identify resources across ad hoc collections of devices.

Finally I was in Athens working with George Lepouras, Akrivi Katifori and others. George had developed a Thunderbird extension to allow Snip!t to snip from mail messages … but while we could snip the text there was no way for the Snip!t page to link back to the mail message. We need full round trip URIs that link desktop and web with no distinction – URIs that can be embedded in a web page and (assuming you have the right permissions and are in an appropriate place) can be clicked and the appropriate mail message, calendar entry or whatever is opened.

Based on this and discussions we had, I drafted a discussion document on globally accessible local URIs. Any feedback very welcome.

Over the summer we hope to put together a demonstrator / reference implementation – if anyone is interested let me know.

Has the usability world gone mad or is it just that our conception of HCI has become too narrow?

Now it may be that by ‘CSCW’ they were thinking ‘Groupware’ and I can sympathise with this to some extent – explicitly collaborative software is after all just one application area … and if we ignore SMS and photosharing, email, blogs and forums, industrial and transport control rooms, track changes and comments in MS Word, the collaborative features in Google office products, and I guess pretty much all of Web2.0 … well I guess there is stall plenty of software (?) that doesn’t have explicit collaboration.

But it sounded as if it was not just this, but CSCW as a whole that was rejected … and I guess any socio-technical considerations with it.

How does such a myopic conception of HCI arise?

I would guess it has two roots:

• fragmentation within HCI as a discipline
• narrowness of usability / interaction design vision

fragmentation

The first of these can be seen as a side effect of maturity. While there are still general CHI conferences such as CHI and British HCI, there are numerous more specific areas such as mobileHCI and in particular CSCW and ECSCW. These sub-areas have distinct communities and so it is not surprising to see a level of boundary setting in individuals’ interests.¹

What is more problematic is to see this community fragmentation reflected in education. In some cases there may be specific modules about socio-technical design, organisational issues or CSCW itself, where broader contextual design issues arise … but in many undergraduate and masters courses there is just one module and one shot at informing a generation of students that people (in the plural) matter in design. We may not have time to teach everything we would wish (and indeed in my own courses I only occasionally mention collaborative issues); however we should at least be giving our students a proper view of the scope of HCI. If they were going to enter a company alongside a team sociologists, organisational psychologists and other specialists maybe this would be OK, but it is likely they will be the lone voice for the user – if the user model they have is sociopathic what hope for rounded design!

narrowness of vision

The second issue of increasing narrowness within the scope of ‘usability’ is at least as worrying. In the 1980s HCI took a decade to escape the idea that you either (a) get in a usability person to evaluate the system your programmers have already put together, or at best (b) ‘put an interface on this system’. It seems we are coming full circle, just doing (b) to ourselves. How many of the problems I noted in my recent post “I just wanted to print a file” would be picked up by a standard usability evaluation? The problem with a too heavy evaluation focus demands a post of its own, but suffice to say it is odd to see so much focus on evaluation compared with design itself and even less on the knowledge needed for design.

Lists of guidelines and usability heuristics are an easy hit, but risk limiting us to:

• a superficial veneer of usability that misses the deeper functional user issues and broader social context
• a narrow range of pre-studied application domains

The latter is perhaps most obvious when we look at the resistance in some quarters to dealing with affective issues and user experience and the problems in dealing wth new domains such as Web2.0.²

At Talis they spent over a year trying to find a HCI specialist / interaction designer to work on design of their semantic web / web2.0 products. When they interviewed they found people who could go through a web site and fix usability bugs, but not with the deeper understanding to address new domains.

… it sounds as this is exactly how we are educating our next generation of students.

If your conception of usability is too narrow, it is time to branch out.

P.S. the post title is taken from J.B.Phillips’ classic book “Your God is Too Small” (Wyvern Books, 1952) – it is not only HCI that paints its conceptions too narrowly!

1. Indeed, a young researcher who attended the recent ECSCW in Limerick found that the concerns of that community also seemed very insular. [back]
2. See the HCI 2.0 panel and also palexa’s paper on YouTube evaluation. [back]

Freebase is interesting as its underlying representation is graph/relationship based like RDF, but its Metaweb Query Language (MQL) uses JSON which is a more programming-like whole and parts representation with arrays and slots. Facebook’s new Data Store API also has objects and associations, but does not use RDF or other obvious web technologies.

So the question is – if the closest things to Semantic Web apps on the internet don’t use SemWeb techology like RDF, SPARQL etc. … are these SemWeb techologies fit for purpose or indeed useful at all?

I think the answer is that (i) partly they are not fit for purpose – caught in a backwater by their history, but (ii) that is like all things and they are what we have got, and (iii) we can use some of the tools of computing to make them work …

Well, we can start off with XML. XML (from its roots in SGML) … and the ML gives it away … is a mark-up language for showing structure in text. However, we all (and I do it!) use it as a data representation notation. Its roots give it away – confusion of type and part naming,² very heavy-weight notation for dealing with small data items such as numbers, difficulties with binary data or even full character code sets for text³. There were so many IDLs that were so much better …. if only … and interestingly JSON (as used in MQL and many REST services) is in the tradition of programming language formats and, not surprisingly, more succinct for data representation.

On to RDF … again initially for meta data and annotation of other, more complex, data such as XML objects. However, (for good reasons) increasingly we are using it as the data itself. So we end up using something designed for meta data for data ??? … and furthermore the XML is now being used as a second-order notation!

The underlying semantic model of RDF is triples … and yes in the theoretical CS world this was recognised many years ago as a model that could, in principle, be used to represent anything. Now strangely enough this had virtually no effect on people writing real code. This is a bit like the fact that in mathematics Peano axioms of arithmetic represent any number as combinations of 0 and ‘+1′, but … surprisingly …. in supermarkets I have never been asked for 0+1+1+1+1+1+1+1+1+1 pounds and 0+1+1+1+1+1 pence.⁴

Triples are good for semantics, but programming languages use a variety of other constructs and again interesting to see Metaweb API using JSON as a ‘view’ on the underlying graph it supports.

Finally poor old SPARQL inherits from SQL which, like COBOL before it, was designed as an attempt to be an end-user query language … but in so doing was never well-fitted for programmatic manipulation. … And it is also interesting that the semantics of SPARQL seems to need tuples not just triples … :-/

However, to avoid leaving everyone gloomy at the weekend … of course we always live with the past, and there are good lessons to learn for the future …

1) the Sematic Web is a bit like Pascal programmers who have just discovered that everything could  be done in binary and have leapt upon machine code for its conceptual simplicity. But having seen the purity of the life of the aesthete, we probably need to come back to something more like Pascal

2) the graph/relation based model is very powerful, but other structures such as hierarchies, sequences, mappings will in different situations be either (a) more natural or (b) more efficient

3) we therefore need abstractions/layers/APIs/protocols that make triple stores more like other things … such as programing language structures in JSON … or maybe make triple stores look like relational databases

4) probably also abstractions that make non-triple structures more like the Semantic Web. These might be legacy structures (e.g. wrappers round relational databases) or specially designed ones (like the way Smalltalk made it look like even the number 2 was an object, but with efficient internal representations)

1. listen to Talis’ pod cast interview with Jamie Taylor Freebase’s ‘Minister of Information’ (sic). [back]
2. why attempts to do CSS-style print definitions in SGML always came a cropper, and what SOAP at enormous complexity tried to solve [back]
3. I recently had text that contained tabs … sorry not in the accepted characters for XML data! [back]
4. And similarly, despite its disciples, LISP is not the dominant language despite its data structure simplicity [back]