Projects

LuminAI is an interactive art installation that explores the improvisation of proto-narrative movement between humans and virtual AI agents using full body, expressive, movement-based interaction. Interactors can co-create movement with an autonomous virtual agent that learns movement, response, and improvisation directly from interacting with human teachers. It analyses their movement using Viewpoints movement theory.

Lumos is a visual data analysis tool that captures and shows the interaction history with data to increase awareness of such analytic behaviors. Using in-situ (at the place of interaction) and ex-situ (in an external view) visualization techniques, Lumos provides real-time feedback to users for them to reflect on their activities. For example, Lumos highlights datapoints that have been previously examined in the same visualization (in-situ) and also overlays them on the underlying data distribution (i.e., baseline distribution) in a separate visualization (ex-situ).

Machine Teaching is a collection of approaches explicitly aimed at solving the difficulties that lie in enabling domain experts to effectively teach machine learning systems. AI Systems, including agents like Jill Watson, face significant challenges in adapting to new domains.

Gestures for interfaces should be short, pleasing, intuitive, and easily recognized by a computer. However, it is a challenge for interface designers to create gestures easily distinguishable from users' normal movements. Our tool MAGIC Summoning addresses this problem. Given a specific platform and task, we gather a large database of unlabeled sensor data captured in the environments in which the system will be used (an "Everyday Gesture Library" or EGL). MAGIC can output synthetic examples of the gesture to train a chosen classifier.

Magic Window supports immersive augmented video experiences allowing viewers to change perspective, as if they are looking through a real window. A rich set of collaborative interactions with live and pre-recorded media content as well as connected devices are possible through gesture-based controls.

The project was inspired by the notion of a temporary, magical social space. It resulted in a tangible interface in the form of an umbrella that reacts to sound, movement, and tactile input. The result is a playful experience with the umbrella that can be shared as participants hide from the rain (thus creating an impromptu social space) and explore their umbrella's "magic" functionality together

MagTrack is a wearable alternative controller designed for power wheelchair users with limited upper-body mobility to regain better autonomy of their physical and digital life.

Interviews with 61 makers, along with observations in several maker communities, provide empirical insight on the nuances between different types of communities and how these differences are influenced by the space and place of the makerspaces. Our exploration led to the identification of five prototypical maker communities; closed and regulated, open and messy, hybrid, online large-scale, and online small-scale.

We describe our design considerations and iterative prototyping process from testing with youth and educators, and studying seven teachers in a week-long workshop. Our findings demonstrate how teachers perceived MCVT Cards for facilitating new computational practices, specifically for learning through exploratory making and connecting to examples of everyday computing.

We draw upon insights from design-based research in developing the TADA (Tangible Data) system, in order to understand how data literacy, data mapping, and visualization might be aided through the translation of data into dynamic learner-created sculptures. Through the PaperMech medium, we are also able to see how storytelling and creation might be used to further understand the information in the data. We describe workshop design and discuss how the TADA system could be a useful tool in the future of 6-8th grade data science pedagogy.

Our project was inspired by the Chinese traditional fairy tale Magic Brush. The interactor has a unique brush with a magic power that could turn anything he draws to life. They will use the magic brush to escape from a room by creating/interacting with the artifacts he draws. We want to design the agencies of creating interactive user-generated content in VR.

Mapping iThemba draws on ethnographic research that Professor Anne Pollock began in 2010 at iThemba Pharmaceuticals (pronounced ee-TEM-ba), a small start-up pharmaceutical company in the outskirts of Johannesburg that was founded in 2009 with the mission of drug discovery for TB, HIV, and malaria. The synthetic chemistry research that scientists do at iThemba is no different than what might be done in a well-equipped lab anywhere in the world. Yet, place matters. The interactive map is an opportunity to explore how.

As part of the exhibit, Mapping Place: Africa Beyond Paper, which contrasts western concepts of mapping (i.e. Cartesian plots of locations) with other traditional practices, Synlab students created an interactive tabletop installation that lets participants tell their own stories by creating a digital Lukasa, a mnemonic device used by the Luba people of central Africa to record genealogy and history. The exhibition was at the Robert C. Williams Paper Museum from February 27 to June 6, 2014.

Examining COVID-19 Crisis Visualizations and Information Behaviors

An ever-increasing number of smart technologies are being developed all over the world to coach people on healthier, as well as more responsible behaviors, providing them timely and ubiquitously with personalized information and support. Marlin is one such wearable swim coach, specifically for distance swimmers, which constantly monitors a swimmer's performance and provides them necessary real-time feedback through sonification while swimming. It is a tool for coaches to plan a detailed training program, set new targets and sync them to the swimmers' device.