Alan’s blog

[another techie post - a problem I had and can see that other people have had too]

It is common in various web frameworks to pass pretty much everything through a central script using Apache .htaccess file and mod_rewrite.  For example enabling permalinks in a Wordpress blog generates an .htaccess file like this:

RewriteEngine On
RewriteBase /blog/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /blog/index.php [L]

I use similar patterns for various sites such as vfridge (see recent post “Phoenix rises“) and Snip!t.  For Snip!t however I was using not a local .htaccess file, but an AliasMatch in httpd.conf, which meant I needed to ask Fiona every time I needed to do a change (as I can never remember the root passwords!).  It seemed easier (even if slightly less efficient) to move this to a local .htaccess file:

RewriteEngine On
RewriteBase /
RewriteRule ^(.*)$ code/top.php/$1 [L]

The intention is to map “/an/example?args” into “/code/top.php/an/example?args”.

Unfortunately this resulted in a “500 internal server error” page and in the Apache error log messages saying there were too many internal redirects.  This seems to be a common problem reported in forums (see here, here and here).  The reason for this is that .htaccess files are encountered very late in Apache’s processing and so anything rewritten by the rules gets thrown back into Apache’s processing almost as if they were a fresh request.  While the “[L]“(last)  flags says “don’t execute any more rules”, this means “no more rules on this pass”, but when Apache gets back to the .htaccess in the fresh round the rule gets encountered again and again leading to an infinite loop “/code/top/php/code/top.php/…/code/top.php/an/example?args”.

Happily, mod_rewrite thought of this and there is an additional “[NS]” (nosubreq) flag that says “only use this rule on the first pass”.  The mod_rewrite documentation for RewriteRule in Apache 1.3, 2.0 and 2.3 says:

Use the following rule for your decision: whenever you prefix some URLs with CGI-scripts to force them to be processed by the CGI-script, the chance is high that you will run into problems (or even overhead) on sub-requests. In these cases, use this flag.

I duly added the flag:

RewriteRule ^(.*)$ code/top.php/$1 [L,NS]

This should work, but doesn’t.  I’m not sure why except that the Apache 2.2 documentation for NS|nosubreq reads:

NS|nosubreq

Use of the [NS] flag prevents the rule from being used on subrequests. For example, a page which is included using an SSI (Server Side Include) is a subrequest, and you may want to avoid rewrites happening on those subrequests.

Images, javascript files, or css files, loaded as part of an HTML page, are not subrequests – the browser requests them as separate HTTP requests.

This is identical to the documentation for 1.3, 2.0 and 2.3 except that quote about “URLs with CGI-scripts” is singularly missing.  I can’t find anything that says so, but my guess is that there was some bug (feature?) introduced 2.2 that is being fixed in 2.3.

Wordpress is immune from the infinite loop as the directive “RewriteCond %{REQUEST_FILENAME} !-f” says “if the file exists use that without rewriting”.  As “index.php” is a file, the rule does not rewrite a second time.  However, the layout of my files meant that I sometimes have an actual file in the pseudo location (e.g. /an/example really exists).  I could have reorganised the complete directory structure … but then I would have been still fixing all the broken links now!

Instead I simply added an explicit “please don’t rewrite my top.php script” condition:

RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_URI}  !^/code/top.php/.*
RewriteRule ^(.*)$ code/top.php/$1 [L,NS]

I suspect that this will be unnecessary when Apache upgrades to 2.3, but for now … it works

I was struggling to get PHPUnit to run under PHP 5.2.9. I’ve only used PHPUnit a little, so may have simply got something wrong, but I kept getting the error:

Catchable fatal error: Object of class AbcTest could not be converted to string in {dir}/PHPUnit/Framework/TestFailure.php on line 98

The error happens in the PHPUnit_Framework_TestFailure::toString method, which tries to implicitly convert a test case to a string.

The class AbcTest is my test case, which it is trying to display following a test failure.  PHPUnit test cases all extend PHPUnit_Framework_TestCase and while this has a toString method it does not have the ‘magic method‘ __toString required by PHP 5.2 onwards.

To fix the problem I simply added the following method to the class PHPUnit_Framework_TestCase in PHPUnit/Framework/TestCase.php .

public function __toString()
 {
 return $this->toString();
 }

I am using PHPUnit 3.4.9, but peeking at 3.5.0beta it looks the same.  I’m guessing the PHPUnit_Framework_TestFailure::toString method is not used much so has got missed since the change to PHP 5.2.x.

PHPUnit is now in GITHub so I really ought to work out how to submit corrections to that … but another day I think.

Over a cup of tea in bed I was pondering the future of business data processing and also general programming. Many problems of power-computing like web programming or complex algorithmics, and also end-user programming seem to stem from assumptions embedded in the heart of what we consider a programming language, many of which effectively date from the days of punch cards.

Often the most innovative programming/scripting environments, Smalltalk, Hypercard, Mathematica, humble spreadsheets, even (for those with very long memories) Filetab, have broken these assumptions, as have whole classes of ‘non-standard’ declarative languages.  More recently Yahoo! Pipes and Scratch have re-introduced more graphical and lego-block style programming to end-users (albeit in the case of Pipes slightly techie ones).

Yahoo! Pipes (from Wikipedia article)	Scratch programming using blocks

What would programming be like if it were more incremental, more focused on live data, less focused on the language and more on the development environment?

Two things have particularly brought this to mind.

First was the bootcamp team I organised at the Winter School on Interactive Technologies in Bangalore¹.  At the bootcamp we were considering “content development through the keyhole”, inspired by a working group at the Mobile Design Dialog conference last April in Cambridge.  The core issue was how one could enable near-end-use development in emerging markets where the dominant, or only, available computation is the mobile phone.  The bootcamp designs focused on more media content development, but one the things we briefly discussed was full code development on a mobile screen (not so impossible, after all home computers used to be 40×25 chars!), and where literate programming might offer some solutions, not for its original aim of producing code readable by others², but instead to allow very succinct code that is readable by the author.

if ( << input invalid >> )
    << error handling code >>
else
    << update data >>

(example of simple literate programming)

The second is that I was doing a series of spreadsheets to produce some Fitts’ Law related modelling.  I could have written the code in Java and run it to produce outputs, but the spreadsheets were more immediate, allowed me to get the answers I needed when I needed them, and didn’t separate the code from the outputs (there were few inputs just a number of variable parameters).  However, complex spreadsheets get unmanageable quickly, notably because the only way to abstract is to drop into the level of complex spreadsheet formulae (not the most readable code!) or VB scripting.  But when I have made spreadsheets that embody calculations, why can’t I ‘abstract’ them rather than writing fresh code?

I have entitled this blog ‘part 1′ as there is more to discuss  than I can manage in one entry!  However, I will return, and focus on each of the above in turn, but in particular questioning some of those assumptions embodied in current programming languages:

(a) code comes before data

(b) you need all the code in place before you can run it

(c) abstraction is about black boxes

(d) the programming language and environment are separate

In my PPIG keynote last September I noted how programming as an activity has changed, become more dynamic, more incremental, but probably also less disciplined.  Through discussions with friends, I am also aware of some of the architectural and efficiency problems of web programming due to the opacity of code, and long standing worries about the dominace of limited models of objects³

So what would programming be like if it supported these practices, but in ways that used the power of the computer itself to help address some of the problems that arise when these practices address issues of substantial complexity?

And can we allow end-users to more easily move from move seemlessly from filling in a spreadsheet, to more complex scripting?

1. The winter school was part of the UK-India Network on Interactive Technologies for the End-User.  See also my blog “From Anzere in the Alps to the Taj Bangelore in two weeks“ [back]
2. such as Knuth’s “TeX: the program” book consisting of the full source code for TeX presented using Knuth’s original literate programming system WEB. [back]
3. I have often referred to object-oriented programming as ‘western individualism embodied in code’. [back]

Last week I was giving a keynote at the annual workshop PPIG2008 of the Psychology of Programming Interest Group.   Before I went I was politely pronouncing this pee-pee-eye-gee … however, when I got there I found the accepted pronunciation was pee-pig … hence the logo!

My own keynote at PPIG2008 was “as we may code: the art (and craft) of computer programming in the 21st century” and was an exploration of the changes in coding from 1968 when Knuth published the first of his books on “the art of computer programming“.  On the web site for the talk I’ve made a relatively unstructured list of some of the distinctions I’ve noticed between 20th and 21st Century coding (C20 vs. C21); and in my slides I have started to add some more structure.  In general we have a move from more mathematical, analytic, problem solving approach, to something more akin to a search task, finding the right bits to fit together with a greater need for information management and social skills. Both this characterisation and the list are, of course, a gross simplification, but seem to capture some of the change of spirit.  These changes suggest different cognitive issues to be explored and maybe different personality types involved – as one of the attendees, David Greathead, pointed out, rather like the judging vs. perceiving personality distinction in Myers-Briggs¹.

One interesting comment on this was from Marian Petre, who has studied many professional programmers.  Her impression, and echoed by others, was that the heavy-hitters were the more experienced programmers who had adapted to newer styles of programming, whereas  the younger programmers found it harder to adapt the other way when they hit difficult problems.  Another attendee suggested that perhaps I was focused more on application coding and that system coding and system programmers were still operating in the C20 mode.

The social nature of modern coding came out in several papers about agile methods and pair programming.  As well as being an important phenomena in its own right, pair programming gives a level of think-aloud  ‘for free’, so maybe this will also cast light on individual coding.

Margaret-Anne Storey gave a fascinating keynote about the use of comments and annotations in code and again this picks up the social nature of code as she was studying open-source coding where comments are often for other people in the community, maybe explaining actions, or suggesting improvements.  She reviewed a lot of material in the area and I was especially interested in one result that showed that novice programmers with small pieces of code found method comments more useful than class comments.  Given my own frequent complaint that code is inadequately documented at the class or higher level, this appeared to disagree with my own impressions.  However, in discussion it seemed that this was probably accounted for by differences in context: novice vs. expert programmers, small vs large code, internal comments vs. external documentation.  One of the big problems I find is that the way different classes work together to produce effects is particularly poorly documented.  Margaret-Anne described one system her group had worked on² that allowed you to write a tour of your code opening windows, highlighting sections, etc.

I sadly missed some of the presentations as I had to go to other meetings (the danger of a conference at your home site!), but I did get to some and  was particularly fascinated by the more theoretical/philosophical session including one paper addressing the psychological origins of the notions of objects and another focused on (the dangers of) abstraction.

The latter, presented by Luke Church, critiqued  Jeanette Wing’s 2006 CACM paper on Computational Thinking.  This is evidently a ‘big thing’ with loads of funding and hype … but one that I had entirely missed :-/ Basically the idea is to translate the ways that one thinks about computation to problems other than computers – nerds rule OK. The tenet’s of computational thinking seem to overlap a lot with management thinking and also reminded me of the way my own HCI community and also parts of the Design (with capital D) community in different ways are trying to say they we/they are the universal discipline  … well if we don’t say it about our own discipline who will …the physicists have been getting away with it for years

Luke (and his co-authors) argument is that abstraction can be dangerous (although of course it is also powerful).  It would be interesting perhaps rather than Wing’s paper to look at this argument alongside  Jeff Kramer’s 2007 CACM article “Is abstraction the key to computing?“, which I recall liking because it says computer scientists ought to know more mathematics

I also sadly missed some of Adrian Mackenzie’s closing keynote … although this time not due to competing meetings but because I had been up since 4:30am reading a PhD thesis and after lunch on a Friday had begin to flag!  However, this was no reflection an Adrian’s talk and the bits I heard were fascinating looking at the way bio-tech is using the language of software engineering.  This sparked a debate relating back to the overuse of abstraction, especially in the case of the genome where interactions between parts are strong and so the software component analogy weak.  It also reminded me of yet another relatively recent paper³ on the way computation can be seen in many phenomena and should not be construed solely as a science of computers.

As well as the academic content it was great to be with the PPIG crowd they are a small but very welcoming and accepting community – I don’t recall anything but constructive and friendly debate … and next year they have PPIG09 in Limerick – PPIG and Guiness what could be better!

1. David has done some really interesting work on the relationship between personality types and different kinds of programming tasks.  I’ve seen him present before about debugging and unfortunately had to miss his talk at PPIG on comprehension.  Given his work has has shown clearly that there are strong correlations between certain personality attributes and coding, it would be good to see more qualitative work investigating the nature of the differences.   I’d like to know whether strategies change between personality types: for example, between systematic debugging and more insight-based scan and see it bug finding. [back]
2. but I can’t find on their website [back]
3. Perhaps 2006/2007 in either CACM or Computer Journal, if anyone knows the one I mean please remind me! [back]

Just read Adam Greenfield’s blog entry “Reality bites“. He describes how a design he produced for a friend’s new restaurant became a solid metal sign within days. Despite knowing about recent rapid fabrication techniques, actually seeing these processes in action for his own design was still shocking.

I too am still amazed at the relative ease that ideas can be turned into reality. In a presentation “As we may print” at the 2003 Interaction Design for Children, Michael Eisenberg described how he and his co-workers at University Colorado were using laser cutters to enable children to design their own 3D designs in card or even thin plywood. More recently at the National Centre for Product Design and Development Research in Cardiff, I saw 3D metal printers. I was aware of 3D printers working in various gels and foams, but did not realise it was possible to create parts in titanium and steel, simply printed from 3D CAD designs. Chasing one of Adam’s links I found instructions to make your own 3D printer on the MIT site … however, this constructs your designs in pasta paste not metal!

One of the arguments we are making about our FireFly technology is that it will change lighting from being a matter of engineering and electronics, to a digital medium where the focus moves form hardware to software. While FireFly allows more flexible 2D and 3D arrangements than other technologies we are aware of, it is certainly not alone in making this transformation in lighting. Last week I was talking to Art Lights London and they are planning some large installations using Barco’s LED lighting arrays. Soon anything that you can point on your computer screen you will also be able to paint in light from your own Christmas tree to London Bridge.

Although it sometimes seems that technology is simply fuelling war and environmental catastrophe, it is a joy to still glimpse these occasional moments of magic.

As it was new Years Day and it was too wet to go shift earth in the garden I thought I’d play a bit with Professor Alan’s puzzle square. I’ve had a ‘make your own’ version for years, but you had to chop an image into bits give them special names, etc. Now it works much more easily with any image (try it yourself). This are a couple I made with my own photos:

The key is that it is I am now using the CSS clip property which allows you to show selected parts of an image (or in fact any HTML element). This was made a little more complicated due to the fact that the W3C pages for clip give running examples for every other kind of visual effect … but not clip! Googling was a nightmare as it turns up page after page in forums saying “I can’t get clip to work”!

Happily I found seifi.org (a blog that looks like a really great web resource) and a post on Creating Thumbnails Using the CSS Clip Property. This was full of meticulously laid out examples … Mojo Seifi, you are a star!

(more…)

I just found a wonderful service spell with flickr that makes images like:

It is built from a pool of letters on flickr which have been collected independently (there are also pools for digits and punctuation), a lovely example of Web2.0 in action!

I’ve added it to Snip!t, so now if you snip a single word ’spell with flickr’ gets suggested as an action.

I had thought I had got used to the ineptitude of Microsoft programmers, that nothing would surprise me any more. However, yet again their inventiveness in producing obscure yet damaging effects has once again amazed me.

(more…)

tax collectors get a bad press, but we have just got to the end of filling in our annual tax returns online and is an amazingly trouble free process … I can still remember when it was paper forms and 2 days before the final deadline we’d discover we needed extra green pages for this thing or other. Now-a-days you just fill in the boxes on the web form, if you haven’t got the figures it just remembers it all for later, then at the end you press the button and it works out everything. And even better, they said they owed me £80

If only every online service was as good. A short while ago we tried to open a savings account with the Northern Rock and gave up as the applet-based system they use is not compatable with Apple Macs … other banks seem to be able to use SSL for security and browser-independent HTML, so why not them! Suffice to say we went elsewhere.

Even worse was an experience early last year. I’d given a seminar at another university and submitted an expense claim. The university sent me payment advice as an email, but it displayed oddly when I viewed it and got a high spamassassin rating. A bit of digging and I found that the high spam rating was due to the fact that there was not a closing body tag in the HTML. I was going to mail the university IT support and then saw that the company who supplied the software, Albany Software, was named in the email and decided to mail them directly to avoid embarassing them to their client.

So I went to their web site … but it didn’t display properly in Firefox, I tried Safari … even worse! Eventually I got their ’support’ contact email by using view source and mailed them, mentioning both the broken HTML in the email and the broken website.

The reply from their ’support’ email:

“Try using Microsoft Internet Explorer. Though Firefox is vastely superior, most websites/applications are only compatible with Internet Explorer.”

Who said the days of the old sys admins had gone!

The happy end to the last story is that I just revisited their site, they have at last got it working cross browser … well I guess better late than never.

Anyway thumbs up for Her Majesty’s Revenue & Customs, even if others need to catch up a little.

I’ve just installed the footnotes plugin in my Wordpress¹ . It seems to work a dream.

I’ve used the following CSS:

ol.footnotes{
font-size:.9em;
color:#666666;
margin-top:0px;
}

ol.footnotes li{
list-style-type:decimal;
}

1. makes footnotes like this [back]