Human-Computer Interaction Institute

Texture Displays: A Passive Approach to Tactile Presentation

Chris Harrison et al. — Fri, 17 Sep 2010 11:59:51 PDT

In this paper, we consider a passive approach to tactile presentation based on changing the surface textures of objects that might naturally be handled by a user. This may allow devices and other objects to convey small amounts of information in very unobtrusive ways and with little attention demand. This paper considers several possible uses for this style of display and explores implementation issues. We conclude with results from our user study, which indicate that users can detect upwards of four textural states accurately with even simple materials.

Where to Locate Wearable Displays? Reaction Time Performance of Visual Alerts from Tip to Toe

Chris Harrison et al. — Fri, 17 Sep 2010 11:59:50 PDT

Advances in electronics have brought the promise of wearable computers to near reality. Such systems can offer a highly personal and mobile information and communication infrastructure. Previous research has investigated where wearable computers can be located on the human body - critical for successful development and acceptance. However, for a location to be truly useful, it needs to not only be accessible for interaction, socially acceptable, comfortable and sufficiently stable for electronics, but also effective at conveying information. In this paper, we describe the results from a study that evaluated reaction time performance to visual stimuli at seven different body locations. Results indicate that there are numerous and statistically significant differences in the reaction time performance characteristics of these locations. We believe our findings can be used to inform the design and placement of future wearable computing applications and systems.

Stacks on the Surface: Resolving Physical Order Using Fiducial Markers With Structured Transparency

Tom Bartindale et al. — Fri, 17 Sep 2010 11:59:48 PDT

We present a method for identifying the order of stacked items on interactive surfaces. This is achieved using conventional, passive fiducial markers, which in addition to reflective regions, also incorporate structured areas of transparency. This allows particular orderings to appear as unique marker patterns. We discuss how such markers are encoded and fabricated, and include relevant mathematics. To motivate our approach, we comment on various scenarios where stacking could be especially useful. We conclude with details from our proof-of-concept implementation, built on Microsoft Surface.

Lean and Zoom: Proximity-Aware User Interface and Content Magnification

Chris Harrison et al. — Fri, 17 Sep 2010 11:59:47 PDT

The size and resolution of computer displays has increased dramatically, allowing more information than ever to be rendered on-screen. However, items can now be so small or screens so cluttered that users need to lean forward to properly examine them. This behavior may be detrimental to a user's posture and eyesight. Our Lean and Zoom system detects a user's proximity to the display using a camera and magnifies the on-screen content proportionally. This alleviates dramatic leaning and makes items more readable. Results from a user study indicate people find the technique natural and intuitive. Most participants found on-screen content easier to read, and believed the technique would improve both their performance and comfort.

Lightweight Material Detection for Placement-Aware Mobile Computing

Chris Harrison et al. — Fri, 17 Sep 2010 11:59:46 PDT

Numerous methods have been proposed that allow mobile devices to determine where they are located (e.g., home or office) and in some cases, predict what activity the user is currently engaged in (e.g., walking, sitting, or driving). While useful, this sensing currently only tells part of a much richer story. To allow devices to act most appropriately to the situation they are in, it would also be very helpful to know about their placement - for example whether they are sitting on a desk, hidden in a drawer, placed in a pocket, or held in one's hand - as different device behaviors may be called for in each of these situations. In this paper, we describe a simple, small, and inexpensive multispectral optical sensor for identifying materials in proximity to a device. This information can be used in concert with e.g., location information, to estimate, for example, that the device is "sitting on the desk at home", or "in the pocket at work". This paper discusses several potential uses of this technology, as well as results from a two-part study, which indicates that this technique can detect placement at 94.4% accuracy with real-world placement sets.

Pseudo-3D Video Conferencing with a Generic Webcam

Chris Harrison et al. — Fri, 17 Sep 2010 11:59:45 PDT

When conversing with someone via video conference, you are provided with a virtual window into their space. However, this currently remains both flat and fixed, limiting its immersiveness. Previous research efforts have explored the use of 3D in telecommunication, and show that the additional realism can enrich the video conference experience. However, existing systems require complex sensor and cameras setups that make them infeasible for widespread adoption. We present a method for producing a pseudo-3D experience using only a single generic webcam at each end. This means nearly any computer currently able to video conference can use our technique, making it readily adoptable. Although using comparatively simple techniques, the 3D result is convincing.

Scratch Input: Creating Large, Inexpensive, Unpowered and Mobile Finger Input Surfaces

Chris Harrison et al. — Fri, 17 Sep 2010 11:59:43 PDT

We present Scratch Input, an acoustic-based input technique that relies on the unique sound produced when a fingernail is dragged over the surface of a textured material, such as wood, fabric, or wall paint. We employ a simple sensor that can be easily coupled with existing surfaces, such as walls and tables, turning them into large, unpowered and ad hoc finger input surfaces. Our sensor is sufficiently small that it could be incorporated into a mobile device, allowing any suitable surface on which it rests to be appropriated as a gestural input surface. Several example applications were developed to demonstrate possible interactions. We conclude with a study that shows users can perform six Scratch Input gestures at about 90% accuracy with less than five minutes of training and on wide variety of surfaces.

iEPG: An Ego-Centric Electronic Program Guide and Recommendation Interface

Chris Harrison et al. — Fri, 17 Sep 2010 11:59:42 PDT

Conventional program guides present television shows in a list view, with metadata displayed in a separate window. However, this linear presentation style prevents users from fully exploring and utilizing the diverse, descriptive, and highly connected data associated with television programming. Additionally, despite the fact that program guides are the primary selection interface for television shows, few include integrated recommendation data to help users decide what to watch. iEPG presents a novel interface concept for navigating the multidimensional information space associated with television programming, as well as an effective visualization for displaying complex ratings data. Results from a user study indicate people appreciate the ability to search for content in non-linear ways and are receptive to recommendation systems and unconventional EPG visualizations.

CollaboraTV: Making Television Viewing Social Again

Mukesh Nathan et al. — Fri, 17 Sep 2010 11:59:41 PDT

With the advent of video-on-demand services and digital video recorders, the way in which we consume media is undergoing a fundamental change. People today are less likely to watch shows at the same time, let alone the same place. As a result, television viewing, which was once a social activity, has been reduced to a passive and isolated experience. To study this issue, we developed a system called CollaboraTV and demonstrated its ability to support the communal viewing experience through a month-long field study. Our study shows that users understand and appreciate the utility of asynchronous interaction, are enthusiastic about CollaboraTV's engaging social communication primitives and value implicit show recommendations from friends. Our results both provide a compelling demonstration of a social television system and raise new challenges for social television communication modalities.

Rethinking the Progress Bar

Chris Harrison et al. — Fri, 17 Sep 2010 11:59:40 PDT

Progress bars are prevalent in modern user interfaces. Typically, a linear function is employed such that the progress of the bar is directly proportional to how much work has been completed. However, numerous factors cause progress bars to proceed at non-linear rates. Additionally, humans perceive time in a non-linear way. This paper explores the impact of various progress bar behaviors on user perception of process duration. The results are used to suggest several design considerations that can make progress bars appear faster and ultimately improve users' computing experience

Providing Dynamically Changeable Physical Buttons on a Visual Display

Chris Harrison et al. — Fri, 17 Sep 2010 08:31:53 PDT

Physical buttons have the unique ability to provide low-attention and vision-free interactions through their intuitive tactile clues. Unfortunately, the physicality of these interfaces makes them static, limiting the number and types of user interfaces they can support. On the other hand, touch screen technologies provide the ultimate interface flexibility, but offer no inherent tactile qualities. In this paper, we describe a technique that seeks to occupy the space between these two extremes - offering some of the flexibility of touch screens, while retaining the beneficial tactile properties of physical interfaces.

The outcome of our investigations is a visual display that contains deformable areas, able to produce physical buttons and other interface elements. These tactile features can be dynamically brought into and out of the interface, and otherwise manipulated under program control. The surfaces we describe provide the full dynamics of a visual display (through rear projection) as well as allowing for multitouch input (though an infrared lighting and camera setup behind the display). To illustrate the tactile capabilities of the surfaces, we describe a number of variations we uncovered in our exploration and prototyping. These go beyond simple on/off actuation and can be combined to provide a range of different possible tactile expressions. A preliminary user study indicates that our dynamic buttons perform much like physical buttons in tactile search tasks.

Predicting Students’ Performance with SimStudent: Learning Cognitive Skills from Observation

Noboru Matsuda et al. — Wed, 21 Oct 2009 13:30:40 PDT

SimStudent is a machine-learning agent that learns cognitive skills by demonstration. SimStudent was originally built as a building block for Cognitive Tutor Authoring Tools to help an author build a cognitive model without significant programming. In this paper, we evaluate a second use of SimStudent, viz., student modeling for Intelligent Tutoring Systems. The basic idea is to have SimStudent observe human students solving problems. It then creates a cognitive model that can replicate the students’ performance. If the model is accurate, it would predict the human students’ performance on novel problems. An evaluation study showed that when trained on 15 problems, SimStudent accurately predicted the human students’ correct behavior on the novel problems more than 80% of the time. However, the current implementation of SimStudent does not accurately predict when the human students make errors.

Evaluating a Simulated Student using Real Students Data for Training and Testing

Noboru Matsuda et al. — Wed, 21 Oct 2009 13:30:39 PDT

SimStudent is a machine-learning agent that learns cognitive skills by demonstration. It was originally developed as a building block of the Cognitive Tutor Authoring Tools (CTAT), so that the authors do not have to build a cognitive model by hand, but instead simply demonstrate solutions for SimStudent to automatically generate a cognitive model. The SimStudent technology could then be used to model human students’ performance as well. To evaluate the applicability of SimStudent as a tool for modeling real students, we applied SimStudent to a genuine learning log gathered from classroom experiments with the Algebra I Cognitive Tutor. Such data can be seen as the human students’ “demonstrations” of how to solve problems. The results from an empirical study show that SimStudent can indeed model human students’ performance. After training on 20 problems solved by a group of human students, a cognitive model generated by SimStudent explained 82% of the problem-solving steps performed correctly by another group of human students.

Applying Machine Learning to Cognitive Modeling for Cognitive Tutors

Noboru Matsuda et al. — Wed, 21 Oct 2009 13:30:38 PDT

The aim of this study is to build an intelligent authoring environment for Cognitive Tutors in which the author need not manually write a cognitive model. Writing a cognitive model usually requires days of programming and testing even for a well-trained cognitive scientist. To achieve our goal, we have built a machine learning agent – called a Simulated Student – that automatically generates a cognitive model from sample solutions demonstrated by the human domain expert (i.e., the author). This paper studies the effectiveness and generality of the Simulated Student. The major findings include (1) that the order of training problems does not affect a quality of the cognitive model at the end of the training session, (2) that ambiguities in the interpretation of demonstrations might hinder machine learning, and (3) that more detailed demonstration can both avoid difficulties with ambiguity and prevent search complexity from growing to impractical levels.

What Characterizes a Better Demonstration for Cognitive Modeling by Demonstration?

Noboru Matsuda et al. — Wed, 21 Oct 2009 13:30:38 PDT

A simulated student is a machine learning agent that learns a set of cognitive skills by observing solutions demonstrated by human experts. The learned cognitive skills are converted into a cognitive model for a Cognitive Tutor that is a computerized tutor that teaches human students the cognitive skills. In this paper, we analyze the characteristics of the effective demonstrations that lead to quicker and more accurate learning. Results from empirical studies show that expressive demonstrations (as opposed to abbreviated demonstrations that involve implicit mental operations) are better for both speed and accuracy of learning. We also found that providing multiple demonstrations of the same cognitive skill with differing surface features accelerates learning. These findings imply that the ordering of training sequence as well as the level of detail in demonstration determines the efficiency with which a simulated student generates a cognitive model.

Building Cognitive Tutors with Programming by Demonstration

Noboru Matsuda et al. — Wed, 21 Oct 2009 13:30:37 PDT

The aim of this study is to incorporate the technique of programming by demonstration (PBD) into an authoring tool for Cognitive Tutors. The primary motivation of using PBD is to facilitate the authoring of Cognitive Tutors by educators, rather than AI programmers. That is, instead of asking authors to build a cognitive model representing a task to be taught, a machine-learning agent – called the Simulated Student – observes the author performing the target task and induces production rules that replicate the author’s performance. FOIL is used to learn conditions appearing in the production rules. An evaluation in an example domain of algebra equation solving shows that observing 10 problems solved in 44 steps induced 9 correct and 1 wrong production rules. Two of the correctly induced rules were overly general hence produced redundant solutions.

Applying Programming by Demonstration in an Intelligent Authoring Tool for Cognitive Tutors

Noboru Matsuda et al. — Wed, 21 Oct 2009 13:30:36 PDT

We are building an intelligent authoring tool for Cognitive Tutors, a highly successful form of computer-based tutoring. The primary target users (the authors) are educators who are not familiar with cognitive task analysis and AI programming, which are essential tasks in building Cognitive Tutors. Instead of asking authors to write a cognitive model by hand, a Simulated Student embedded in the authoring tool lets an author demonstrate how to perform the tasks in the subject domain, for instance, solving an algebra equation. The Simulated Student observes an author’s demonstration and induces a set of production rules that replicate the demonstrated performances. Correct production rules, as well as production rules that are incorrect but similar to those a human student might produce, can be directly embedded in the Cognitive Tutor. We give a preliminary evaluation of an implemented Simulated Students based on inductive logic programming and path-finding.

Position Paper on Design in HCI Education

John Zimmerman — Tue, 16 Jun 2009 07:08:56 PDT

In this position paper I address issues with the integration of design, an intuitive and craft based discipline, into the scientific based disciplines of computer science and behavioural science that traditionally make up HCI education. These issues include (i) clearly defining and communicating the purpose for design in HCI education, (ii) measuring the value of interdisciplinary classes, and (iii) the role and value of qualitative evaluation for students who come from a quantitative background. While no solutions for these issues are presented, I do indicate some directions for advancement.

A Multi-Agent TV Recorder

Kaushal Kurapati et al. — Tue, 16 Jun 2009 06:57:56 PDT

Personal Television is here via the advent of a new class of devices called personal video recorders (PVRs). These recorders change the user task from (a) selecting a specific channel to watch from the 100+ available channels to (b) finding something “good” to record from the 10,000+ shows broadcast each week. Recommender systems, such as the one described in this paper, will help track users’ preferences and aid users in choosing shows to record. In this paper we advance a multi-agent TV recommender system that encapsulates three user information streams--implicit view history, explicit preferences, and feedback information on specific shows--into adaptive agents and generates program recommendations for a TV viewer. We have tested the system in various agent combinations with real users drawn from a wide variety of living conditions. The combination of implicit and explicit agents seems to work best in our framework.

Interface Design of Video Scout: A Selection, Recording, and Segmentation System for TVs

John Zimmerman et al. — Tue, 16 Jun 2009 06:50:26 PDT

Video Scout is a prototype retrieval application that allows Personal Video Recorders to actually watch the TV programs they record. By analyzing the visual, audio, and transcript data, Scout can segment and index TV programs, finding and recording specific video clips that match requests in users’ profiles. For example: if users request information on Philips, Scout will watch news programs and capture any stories it finds on Philips. The Scout interface offers a familiar TV environment where users can interact with whole TV programs and video clips organized by topic. Scout also provides users with tools for managing their profiles. This paper captures the Video Scout interface design process, from concept sketches to user testing to final prototype design.