I’ve been reading over old ubicomp papers in preparation for a new project at NICTA. So it was that I found myself reading “Charting Past, Present, and Future Research in Ubiquitous Computing“, by Gregory Abowd and Elizabeth Mynatt (whom, incidentally, should surely be listed among those ubiquitous computing researchers who inspire me – particularly Abowd, whose work I’ve followed since my Honours year in 2000, and whose books were often referenced in the HCI course I took a couple of years before that). One of the most important passages in that paper, to my mind, was tucked away in section 6.1.1, Finding a Human Need (the emphasis is mine):
It is important in doing ubicomp research that a researcher build a compelling story, from the end-user’s perspective, on how any system or infrastructure to be built will be used. The technology must serve a real or perceived human need, because, as Weiser [1993] noted, the whole purpose of ubicomp is to provide applications that serve the humans. The purpose of the compelling story is not simply to provide a demonstration vehicle for research results. It is to provide the basis for evaluating the impact of a system on the everyday life of its intended population. The best situation is to build the compelling story around activities that you are exposed to on a continuous basis. In this way, you can create a living laboratory for your work that continually motivates you to “support the story” and provides constant feedback that leads to better understanding of the use.
Designers of a system are not perfect, and mistakes will be made. Since it is already a difficult challenge to build robust ubicomp systems, you should not pay the price of building a sophisticated infrastructure only to find that it falls far short of addressing the goals set forth in the compelling story. You must do some sort of feasibility study of cutting-edge applications before sinking substantial effort into engineering a robust system that can be scrutinized with deeper evaluation. However, these feasibility evaluations must still be driven from an informed, user-centric perspective—the goal is to determine how a system is being used, what kinds of activities users are engaging in with the system, and whether the overall reactions are positive or negative. Answers to these questions will both inform future design as well as future evaluation plans. It is important to understand how a new system is used by its intended population before performing more quantitative studies on its impact.
It strikes me that too few ubicomp research groups heed this, seemingly obvious, advice, including our own. Though we might occasionally attempt to build a story, it is not often compelling, and I’ve read far too many papers that suffer from the same problem (caveat: I specifically exclude Karen‘s work from this blunt introspective analysis because her work is typically very well motivated, and compelling; and she read the paper I’ve just quoted early on in her Ph.D. and took note of it). I don’t think it’s a coincidence that the most successful ubiquitous computing researchers have taken this advice to heart. I want to make sure that the new project at NICTA does do these things properly.
