Tuesday, November 14, 2017

The Basic Anatomy of Interaction

What is interaction and how can we describe it? In our recent book "Things That Keep Us Busy--the elements of interaction" we take on this challenge and we develop, what we call, an anatomy of interaction.  We also develop a detailed account of when it is reasonable to say that interaction actually takes place. We do this by employing the notion of the "window of interaction" (more on that later).

Below I am briefly presenting some of the work on our anatomy of interaction (from Chapter 4 in the book, as a teaser :-)

The basic elements of the anatomy are artifact and user. Interaction takes places between a human and an artifact/system, as described in the figure below (4.3).

Some of the terms used in the figure need to be explained since they mean very specific things. First of all, an artifact has certain 'states':

internal states, or i-states for short, are the functionally important interior states of the artifact or system.
external states, or e-states for short, are the operationally or functionally relevant, user-observable states of the interface, the exterior of the artifact or system.

And then

world states, or w-states for short, are states in the world outside the artifact or system causally connected with its functioning.

To fully describe the anatomy of interaction some more terms are needed (as defined in the glossary in the book):

Action (with respect to an artifact or system): an action that a human interactant can do in its fullness, here defined to include also the intention with the action; only used for human interactants

Cue the user’s impression of a move of an artifact or system

Move (with respect to artifact or system) something the artifact or system can do, the counterpart of a human action; only applicable to nonhuman interactants

Operation (with respect to artifact or system) an artifact’s or system’s impression of an action by a human interactant; something the artifact or system is designed to take as input from a human interactant; only applicable to nonhuman interactants

So, how does it work. Here is an excerpt from the book, page 65.

"Let us first look at the artifact or system end of the interaction. States can change. They can change as a result of an operation triggered by a user action. For digital artifacts and systems i-states as well as e-states are usually affected by an operation. They can also change as a result of the functioning of the artifact or system itself, what we will call a move. For digital artifacts and systems the changes caused by a move will concern first of all i-states, but frequently also e-states, and sometimes w-states.

An operation can be seen as an artifact’s perception of a human action, a projection of an action. Operations can be seen as partially effective implementations of actions. A move can be seen as the artifact counterpart of a human action. To avoid confusion, we choose to call it “move” rather than “action.” Operations and moves are thus artifact centered: they change i-states always, e-states sometimes, and in some cases also w-states (see figure 4.3). .........

Turning now to the human end of the interaction, we have already pointed out that user actions appear to the artifact or system as operations. Similarly, the moves of an artifact or system appear as cues to the user. A cue is the user’s perception of an artifact move: it is what the user perceives or experiences of a move, the impression of a move. Actions and cues are user- centered concepts. Cues come via e-state changes or w-state changes. When using a word processor the cues mainly stem from the changing images and symbols on the display, but in the case of a robot vacuum cleaner, the important cues will come rather from watching its physical movements, hearing the sounds it makes, and seeing dust and dirt disappear from the floor (all a matter of moves that change w-states). .....

To summarize: User actions appear to the artifact as operations and are reciprocated by artifact moves that appear as cues to the user. Operations are projected actions. Cues are projected moves."

Well, that is a lot. If you find this interesting, read Chapter 4 in the book! Have fun.

Friday, November 03, 2017

Some great books on rationality

In my last post, I talked about my interest in the relationship between designing and rationality. Here are some of my major inspirational sources for this project.

New book project "Nature of design rationality"

I have since my early days of being a Ph.D. student been intrigued by the question of what it means to be rational and to act rationally. This interest manifested itself in my Ph.D. dissertation that translated to English had the title "The Hidden Rationality of Design Work".

Reading about rationality has since then been a lifelong side project, almost like a hobby. I have not done so much writing on the topic but I have read. Recently I have started a book project around designing and rationality (maybe with a title similar to my dissertation, however with different content).

The main idea of this project is that the designing, as a major human approach for change, still struggles with a "hidden rationality". Even though today the praise of designing is stronger than ever before, it is far from clear what is the distinguishing features of the approach compared to other approaches. What is the rationality underlying designing that makes it into a unique approach and makes it possible to achieve outcomes that seems difficult when using other approaches?

There are a lot of superficial ideas about designing presented today as a design approach. In many cases, designing is not seen as anything more than some steps or phases and the use of some simple techniques. It is obvious that we would not define science in the same sense. So what if we treated designing as an approach that has to be understood and explained at the same depth as we do with science. This is what I think is needed and where my interest in rationality can help, I hope. I understand that this is ambitious and maybe overwhelmingly difficult but it is very exciting and maybe I will be able to develop the book project to at least relate to some of these big issues.

[For a long time I have been inspired by the book "The Nature of Rationality" by Robert Nozick. It is a wonderful book that develops a fundamental understanding of rationality and also opens up for a form of rationality that seems to resonate with design.]

Monday, October 30, 2017

Book note: The Grace of Great Things - Creativity and Innovation by Robert Grudin

Looking through your bookshelves is exciting. I have recently experimented with looking at my books and almost randomly picked one to read in. Today I picked Robert Grudin's book "The Grace of Great Things - Creativity and innovation" from 1990.

I remember when I read the book the first time, probably in the mid-90s, I was intrigued and excited to read about creativity in a way that made much more sense to me than most other books on the topic. The typical books on creativity use examples of famous creative people and innovations. They tell stories and, in the best case, try to abstract some useful aspects that regular people could potentially use. In most cases, I find those book uninteresting and not very useful (even though they usually have good stories). Grudin's book is different. It is actually more philosophical but at the same time much more practical and useful.

Grudin sees creativity as something that we, if we do the right things, "deserve". This may sound strange but it makes sense. Creativity cannot be controlled. It can not be "purposed or designed" as Grudin writes. Instead, he writes "But even if we cannot specify or command inspiration, we can, I think, practice deserving it." (p 11). And the 'deserving' is not a talent or inborn virtue, it is something that can be cultivated and made into habits.

Anyone, who really want to understand creativity and innovation should read this book. It was great to find it again on my bookshelf and to be reminded of its ideas.

Wednesday, September 27, 2017

The things that keep us busy (first two pages)

Here is the first couple of pages from our book "Things that keep us busy - the elements of interaction" by Janlert and Stolterman.

1. The things that keep us busy

Despite strong misgivings, private eye Eddie Valiant eventually ventures into the city of Toontown (in Who framed Roger Rabbit, 1988). It is a truly nerve-racking experience: everything is throbbing with life, nervously responsive to his every move, incessantly calling for his attention—not just the usual toon animals, but plants, cars, buildings, everyday things like the elevator button—even the bullets in his toon revolver are alive.  Everything is on speed as it were, Tourettic, incessantly making faces, quipping, jesting, collectively whipping up the environment into a bedlam of interactions. Toontown, the viewer soon realizes, is a madhouse where you would not maintain your sanity for long.

Is this our future?

Even though there are early examples of amazing constructions and machines with interactive abilities, everyday interaction with technical devices is to common people a fairly recent phenomenon brought about by the modern revolution in information technology. An avalanche of interactive devices, artifacts and systems has followed in its path. With this change come new questions and challenges.

It is hard to deny that our artifacts and environments are becoming more and more complex, more and more “alive,” and as a consequence more and more demanding. We have to interact more. Interactivity seems to be everywhere. Why is this happening? There are of course many answers to this question, among them some short and simple. Because it can be done: One obvious cause is the extraordinary and powerful development of digital technology that makes it possible to complexify and infuse everything in our environment with computational and interactive capabilities. Because we want it: It brings on many benefits that we would not want to be without. Interactivity changes our everyday environments in ways that previous generations would have seen as science fiction or magic. We are today able to interact in advanced ways with a range of diverse artifacts and systems, from the smallest device to our homes and with our environments. Interactivity promises that we can be in control of our lives and that we can shape it in any way we desire.

These days, everybody seems to be talking confidently and comfortably about interaction—you interact with web services, with apps, appliances, vehicles, and any form of technical equipment, but also with people, and even entire environments. To be interactive is generally considered good—a positive feature or property associated with being modern, efficient, fast, flexible, reasonable, dynamic, adaptable, controllable—perhaps even smart, curious, caring, involved, engaged, informed, and democratic. Still, there seems to be no very precise idea of what interaction is and what being interactive means, beyond a vague notion that it is some kind of interplay, usually optimistically understood as good-natured cooperation. This vagueness would not be very surprising if it were just the idea of the general public, but even among researchers and experts in Human–Computer Interaction (HCI)—a field of research and development with “interaction” in its very heading—a deep, crisp, shared understanding is wanting. We will in this chapter stay close to the inclusive everyday understanding of ‘interaction.’ In the following chapters we will approach a narrower and more precise definition of ‘interaction’ focused on interaction with digital artifacts and systems, while still keeping in touch with and drawing inspiration from the broader and vaguer everyday sense.

The closely related notion of interface, which has a more technical flavor in everyday parlance has similar problems of depth, preciseness and shared understanding. Yet, it has over the years attracted more attention than interaction from researchers and developers. From a design point of view this is understandable. Interaction, whatever exactly it is thought to be, is something fluid, a dynamic relation played out in time, in use time, not design time—whereas the interface appears as a stable property of the artifact or system, which is there also when no user interaction is going on, hence directly accessible to the designer at design time.

For these reasons, the interface appears more designable than the interaction. Even if a designer is really focusing on designing the interaction, it is hard to see how this design can be effectively implemented except indirectly via the design of the interface. To some extent, it is of course possible to influence the user through education, training, or seeding behavioral patterns, for example, but this path to shaping interactions is not as direct nor usually as potent as the concrete design of an interface and we will not investigate it further in this book. Our examination of interactivity will rather take as its point of departure the interface itself and how the way we think about it has radically changed over time, from being a physical surface with knobs and dials, to clickable symbols, to gestures, and finally to its disappearance.

Today we can interact with some artifacts and environments without there being any visible surface presenting controls or displays of any kind. It is obvious that even if there is no interface, we still interact. We open doors just by walking towards them, we turn on the light by clapping our hands, we get the weather forecast spoken to us by just asking for it, the red light turns green triggered by our car, etc. Of course, as soon as we move towards interaction without any visible surface the questions of what an interface is and what interaction is become more complex.

This development combined with an ability and desire for interactivity fosters a common feeling that the level of interactivity will just keep rising, inexorably. We feel that there is more interactivity, more interaction between humans and things going on, than ever before in history, and it just keeps increasing. There seems to be no retreat or escape from interactivity. Some well-informed critics worry that the proliferation of interactions and interactive things has already gone too far. Their concerns raise many questions. Does interactivity in fact increase? How can we know? What does it really mean to claim that it does? And if indeed it is increasing, what does it mean? Should something be done?

To be able to answer any of these questions requires a more careful and penetrating examination of the concepts of “interactivity” and “interaction” than has been common in research on human-computer interaction. We believe that the answer to questions about rising interactivity and how it affects us humans is not just a matter of belief and conviction about the overall nature of technology and its influence, or how we experience it on a personal and social level. We think it requires a careful investigation into the aspects of artifacts and systems that causes interactivity with a purpose to develop some common understanding that in turn can inform our opinions and positions. This is also the purpose and ambition of this book.

But before we enter into such examinations, let’s first take a closer look at some of the concerns that recently have been recognized in relation to the proliferation of interactivity, concerns that taken together paints a picture with a lot of unknowns. Unknowns that have inspired our examinations.

Wednesday, September 20, 2017

When design philosophy becomes reality

One of the things I "preach" in my class on Design Theory is that everyone who designs act based on some kind of design philosophy. It may be explicit or implicit, but it is there. A design philosophy influences how you think about design, its role, its purpose, how to do it, etc. I push my students to do four things.

First, to examine and reveal their own (existing) design philosophy, to make it as explicit as possible, in an honest way (usually they do not think they have one).

Secondly, to critically examine their own design philosophy, what it means, its consequences for practices, its strengths, and weaknesses, etc.

And then thirdly, to reflect on if their existing philosophy is what they want. What are they missing, what do they want to emphasize, and what do they see as their future strength as a designer.

And finally, to reflect on how they can change and develop their design philosophy in a desired direction.

I think this article about how Logitch has changed their design philosophy is a great example of the importance of knowing your design philosophy. This is how design philosophy becomes reality.

The Meaning of Interactivity—Some Proposals for Definitions and Measures

Is it possible to define interaction and interactivity? And is it possible to measure it in some way? My colleague Lars-Erik Janlert and I have developed some concepts and definitions that we believe can help us answer these questions. In our article (that you can download here)

Lars-Erik Janlert & Erik Stolterman (2017) The Meaning of Interactivity—Some Proposals for Definitions and Measures, Human–Computer Interaction, 32:3, 103-138, DOI: 10.1080/07370024.2016.1226139

we present our work. [Even though this article is recently published, some of the materials in the article has been reworked and further developed in our new book. "Things that keep us busy -- the elements of interaction" (MIT Press, 2017). ]

What I like about this work is that we take the question "what is interaction" seriously and in detail try to define it, or at least frame it, in a way that makes sense and also makes it usable. I know that the way we do it seems strange to some (we have already heard that), but even in those cases, it seems as if our attempt opens up for new questions and invites to a conversation. And this is really what I think our field needs, we need some serious efforts and attempts to carefully frame and define what interaction is since it is our core object of study.

Tuesday, August 29, 2017

Book note: "Making Design Theory" by Johan Redström

It is great to see books being published by people you respect as scholars and thinkers. I am especially happy to see my colleague and friend John Redström's new book "Making Design Theory". Johan is one of the most thoughtful scholars in the world today when it comes to how to understand the relationship between design practice, design research, and knowledge production. Johan is one of the few who can, in a scholarly and successful way, grapple with fundamental questions around design as an approach of making things and of making theory.

One of the most important features of this book is that it presents a foundation of concepts and definitions that are philosophically sound and practically useful.  I am convinced that his thoughts around design research: what it is, how to think about it, but also how to actually do it, will soon be regarded as a fundamental corner stone in the field of design research and research about design.

This is a book I strongly encourage every PhD student who is involved in any form of design research to carefully read. It will provide them with an understanding that is solidly grounded and practically useful. It will help them to defend the way they do (design) research and it will lead to new kinds of theory development that will seriously improve the field.

Tuesday, August 22, 2017

Things That Keep Us Busy -- elements of interaction

Ok, now it is only a week or so until our new book is available (at least according to Amazon). Here is the title and short overview of the book.

Things That Keep Us Busy
The Elements of Interaction

By Lars-Erik Janlert and Erik Stolterman


We are surrounded by interactive devices, artifacts, and systems. The general assumption is that interactivity is good—that it is a positive feature associated with being modern, efficient, fast, flexible, and in control. Yet there is no very precise idea of what interaction is and what interactivity means. In this book, Lars-Erik Janlert and Erik Stolterman investigate the elements of interaction and how they can be defined and measured. They focus on interaction with digital artifacts and systems but draw inspiration from the broader, everyday sense of the word.

Viewing the topic from a design perspective, Janlert and Stolterman take as their starting point the for manipulation by designers, considering such topics as complexity, clutter, control, and the emergence of an expressive-impressive style of interaction. They argue that only when we understand the basic concepts and terms of interactivity and interaction will we be able to discuss seriously its possible futures.
interface, which is designed to implement the interaction. They explore how the interface has changed over time, from a surface with knobs and dials to clickable symbols to gestures to the absence of anything visible. Janlert and Stolterman examine properties and qualities of designed artifacts and systems, primarily those that are open

Tuesday, August 01, 2017

The limits of critique

My colleague and friend Harold Nelson sent me a link to a very interesting article. It is a review of Rita Felski's new book "The limits of critique". I have not read the book but just by reading the article I get a good sense of the major argument Feltski makes. And it really resonates with my own experience and thinking, of course, not so much when it comes to literature critique, but critique in general. Interesting!