(T)racing Eyes and Hearts: An Installation to Explore the Physiology of Empathy

Eyes darting, or maintaining a steady gaze straight ahead. Heartbeat racing, or maintaining a slow, even rhythm. If we encounter these phenomena in another, how do we respond – not just affectively, but physiologically?  Eye movements and heartbeats are among the most intuitively meaningful physiological characteristics that humans observe in one another.  Without necessarily consciously realizing it, we often respond empathetically.  This project brings together humanities scholars and physiology scholars to create an art installation that uses representation, tracking, and visualization to investigate and reflect upon the physiology of empathy.  The installation renders video of eye movements and audio of heartrate of a virtual person, and tracks the eye movements and heartrate of an observing user. We anticipate a mirroring, empathetic physiological response from the user, in which their heartrate also speeds and slows in conjunction with the virtual person.  Immediately after the experience, the user will be provided a visual and auditory representation of the data, in order to see and reflect on this empathetic engagement, and also provided with a link to a copy of the video by email if they so choose.  The playback could be either in real time, or in a time that is set to either the virtual person or the user’s heartrate as a metronome, to allow a distinctively human-centered exploration of the data. 

A Multimodal Human Computer Interface Combining Head Movement, Speech and Tongue Motion for the People with Severe Disabilities

Assistive technologies (ATs) play a crucial role in the lives of individuals with severe disabilities by enabling them to have greater autonomy in performing daily tasks. The Tongue Drive System (TDS) developed at the Georgia Tech Bionics Lab is such an AT, empowering people with severe Spinal Cord Injury (SCI) to be more independent. Earlier versions of the TDS have offered tongue motion and speech as means of driving mouse activity and keyboard input. In this project, we introduce a new multi-modal Tongue Drive System (mTDS), which incorporates head tracking to deliver proportional control of a mouse cursor. The mTDS integrates this new capability while preserving tongue motion and speech from previous versions and offers a richer means of driving computing interfaces, than previously available to individuals with severe disabilities.

A Social Q and A System for Young Adults with Autism

Users go to social network sites or online forums to get advice from members of their networks. Individuals with autism adopt and use such computer-mediated communication technology differently from typical users. They require advice about everyday situations ranging from very simple operations to complex social activities. We propose to develop a Q&A system with a robust network of people whom the user is not likely to know but who nonetheless may be willing to provide advice on everyday situations.

A Wireless Wearable Neckwear System for Health Monitoring

Wearable systems play an important role in continuous health monitoring and can contribute to early detection of abnormal health-related events and facilitate the advancement of personalized healthcare. The neck is a unique sensing location because it provides access to a set of health-related data that other wearable devices simply cannot obtain. Activities including breathing, chewing, clearing the throat, coughing, swallowing, speech and even heartbeat can be recorded from around the neck. Two applications of particular interest for this project include medication adherence monitoring and  food intake monitoring.

Medication non-compliance, especially for patients with chronic illnesses, is a global issue that has been associated with increased healthcare cost, rehospitalization, complications and disease progression. To address this problem, it is essential to have a portable, wearable health platform that can remind patients of their medication regimen, track medication ingestion, and monitor a patient’s overall health status. The proposed system in the form of a necklace will automatically track medication ingestion using the well established radio frequency (RF) technology in very high frequency (13.56MHz) band. For power management purposes, the system will be ‘asleep’ by default except during a swallowing event when there is a possibility of medication ingestion. For this reason automatic swallowing detection is essential; the ability to differentiate swallowing sounds from other tracheal sounds initiated by speaking, coughing, clearing the throat etc. In previous work, we developed a real-time swallowing detection algorithm based on acoustic signals and patterns that combines computationally-inexpensive features to achieve comparable performance with previously proposed offline methods using acoustic and non-acoustic data. With data from four healthy subjects that includes common tracheal events such as speech, chewing, coughing, clearing the throat, and swallowing of different liquids, our results show an overall recall performance of 79.9% and precision of 67.6%, which are slightly better or close to the offline results.

In our following work, we expanded our scope and explored tracheal activity recognition using a combination of promising acoustic features from related work and apply simplistic classifiers including K-NN and Naive Bayes. For wearable systems in which low power consumption is of primary concern, we have shown that with a sub-optimal sampling rate of 16 kHz, we achieved average classification results in the range of 86.6% to 87.4% using 1-NN, 3-NN, 5-NN and Naive Bayes. All classifiers obtained the highest recognition rate in the range of 97.2% to 99.4% for speech classification. This is promising to mitigate privacy concerns associated with wearable systems interfering with the user’s conversations. 

Accessible Bluetooth Cane

The Accessible Bluetooth Cane project allows visually impaired users to control their iPhone while using the white cane, without having to stop and take out the phone. This is achieved by embedding Bluetooth remote controls with tactile buttons inside the cane handle.

ActEarly: Redesign and Evaluation of an Android Mobile Application for Tracking Developmental Milestones

About 1 in 7 U.S. children will be affected with a developmental disability including Autism and Attention Deficit Hyperactivity Disorder (ADHD). Early diagnosis of developmental delays ensures proper intervention and overall improved qualities of life. In this research, we investigate how parents of young children use an Android mobile application, ActEarly, to log their children’s developmental progress through milestones tracking techniques. The app leverages information from the Centers for Disease Control and Prevention’s “Learn the Signs, Act Early” campaign to provide parents with information needed to identify signs of developmental disabilities, while empowering them to share their questions, doubts, and concerns with pediatricians. The goal this project is to evaluate the interactive ActEarly app and uncover users’ method keeping needs, as we design a solution to help parents become more proactive about milestone tracking and create a useful health care tool for the public sector.

Active Pathways

Active Pathways aims to support learning and discovery in systems biology by allowing users to construct and manipulate bio-chemical reaction network simulations using active tangibles on an interactive tabletop display surface. Researchers in systems biology currently run simulation programs that model different experimental parameters such as concentrations inside cells and reaction speeds. Parameters are adjusted algorithmically or by entering numbers into equations. The simulation results are then plotted as graphs in order to discover hidden patterns in the network. Using tangible and tabletop interaction techniques, we provide a direct hands-on way for researchers to construct and manipulate models in order to gain a better understanding of the systems they are studying.

Addressing Well-being and Social Support with an Online Platform

The goal of the research is to identify the ways in which social media could play a role in assisting Georgia Tech students find mental health support.  Mental health disorders are extremely prevalent on college campuses, and anxiety and depression in particular have been shown to have a negative effect on academic success.  Despite the fact that mental health professionals and program are available to students with mental health conditions, many are not seeking help from their campus resources.  Social support is a key component in preventing mental health issues from becoming serious problems, and it has been shown to be a top factor in preventing suicide attempts.  By examining the mental health status of current Georgia Tech students as well as their social media usage and behavior, the proposed project aims to discover how a social platform could be used to provide social support to Tech students facing mental health issues.  

Advanced Auditory Menus

Many electronic devices, from desktop computers to mobile phones to DVD players, can be thought of as a menu of functions. These functions can be accessible to a blind user if the menus are spoken aloud. However, this is extremely inefficient, so we have been enhancing auditory menus with sophisticated text-to-speech, spearcons, spindex, and other audio extensions. These can also be applied in many different languages and research is ongoing to look at more language applications, including tonal types.

Air Gesture-Based Systems for Accessible Stem Education

This project is exploring ways of using air gesture technologies, audio and haptic to facilitate exploration of  STEM concepts by blind and low vision learners. Efforts will establish the efficacy of this approach, as well as best practices for creating air gesture interfaces that support exploration of a virtual reality space such as a simulated atom, wind tunnel or electrical system- all without the use of vision.

Air Gestures in the Vehicle

Modern sensor technology is beginning to allow for cost-effective deployment of air gesture interfaces in the vehicle. Unlike the current standard of direct touch, air gesture interfaces do not require that the driver takes their eyes off the road, especially when coupled with properly applied auditory or tactile feedback.

While emerging systems like Apple Carplay and Android Auto support limited speech commands, the majority of tasks still require visually targeted touch interaction, which poses a safety hazard to drivers.

Research in the Sonification Lab centers on developing guidelines for automotive interface designers on how to create air gesture interfaces which provide minimal cognitive, motor and visual demand to drivers. We combine user-centered HCI design with comprehensive engineering psychology evaluation using eye tracking, physiological measures, performance measures and subjective measures to take a data-driven approach to air gesture systems in the vehicle.

Algorithmic and Historical Detection Patterns of Music Subcultures

The algorithmic detection of subcultural or niche taste trends is of growing importance in targeted advertising. This demonstration presents research using online music analysis tools from Spotify, Musicbrainz, and Rovi coupled with aggregated music listening behavior from Facebook users to detect individual tastes and emerging taste trends amongst social groups.

This research is presented alongside historical signifiers of music taste such as fashion, music collections, and subcultural knowledge.

The goal of the research is to display the growing importance of software based taste detection algorithms in determining niche markets for online content providers, and some of the new methodologies available in to such systems.

Algorithmically Bypassing Censorship on Sina Weibo with Nondeterministic Homophone Substitutions

Like traditional media, social media in China is subject to censorship. However, in limited cases, activists have employed homophones of censored keywords to avoid detection by keyword matching algorithms. In this paper, we show that it is possible to scale this idea up in ways that make it difficult to defend against. Specifically, we present a non-deterministic algorithm for generating homophones that create large numbers of false positives for censors, making it difficult to locate banned conversations. In two experiments, we show that 1) homophone-transformed weibos posted to Sina Weibo remain on-site three times longer than their previously censored counterparts, and 2) native Chinese speakers can recover the origi- nal intent behind the homophone-transformed messages, with 99% of our posts understood by the majority of our participants. Finally, we find that coping with homophone transformations is likely to cost the Sina Weibo censorship apparatus an additional 15 hours of human labor per day, per censored keyword. To conclude, we reflect briefly on the opportunities presented by this algorithm to build interactive, client-side tools that promote free speech. 

AlmaBase - Understanding you Alumni Network

Networking and peer inspiration from alumni of your program/school is important when making decisions about the next steps in your career. However, schools lose touch with alumni once they graduate and find it difficult to keep a track of where they are. Networking platforms such as Linkedin are helpful but do not provide a big picture of your alumni network. AlmaBase is a Linkedin extension, that shows a visualization of career trajectories of alumni from your program, for you to find the "right" alumni to network with and get inspired.

Amazing Me: Milestone tracking via an interactive e-book


Early development of children is a critical issue for young parents. However, symptoms of abnormality may occur in a subtle manner, and parents often fail to recognize them or seek for help at an early stage. This is often because they lack certain knowledge or professional guidance. CDC distributed brochures to promote knowledge of children early development. However, this form of publication contains large volume of information and is hard to popularize. Furthermore, even though parents’ role in solving this problem is significant, they could not do it alone. Enhancing the collaboration among different roles(parents, childcare givers and professionals) will make the most impact in this process. Therefore, this project would utilize a tablet-based interactive storybook to intervene with milestone tracking, and help improve chronic/health care management.

Why is it important?

◦Identify children at risk for developmental disabilities

◦Communicate concerns to primary care provider

◦Get necessary services to improve outcomes for the child in question

◦To understand developmental trajectories of children from different demographics

Ambient Alerting

No matter what age we are, we have likely forgotten to turn off the stove or oven, iron, heater or even water. Forgetfulness can lead to serious events that may result in costly damage to the home or even injury or death. Older adults are more prone to such forgetfulness. When an older adult forgets to turn off a hazardous appliance, it is often attributed to losing mental capacity and may lead to loss of self-confidence, embarrassment, and judgment from others. Many families turn to monitoring when they discover such hazards, but this can result in their loved one feeling a loss of independence. We feel there is an opportunity before monitoring to use technology to provide gentle reminders or cues that empower the resident to determine for themselves when such appliances should be turn off.  

To this end, we have performed in-home contextual interviews, designed prototypes of possible solutions, and performed Aware Home interviews and prototype evaluations with older adult participants to understand their needs for notifications and preferences for alert (audible and visual). As we envision it, an ambient alert system should consist of several ambient and/or wearable reminder products that would integrate with existing connected home systems and provide those gentle reminders both at and away from the primary hazard.

Ambient Visualization using Interactive Projection Mapping

Information visualization can augment human cognition in many ways, and has proved useful in professional application areas such as scientific visualization and business management. But what are the potentials of information visualization in everyday life? Using ambient visualization techniques, the opportunity to co-exist with an embodiment of data in the same physical space, and analyze such a metaphor in relation to the space around us could potentially lead to a greater learning environment. For such environments, how could information exist between
aesthetics and utility to support its cause? The project concerns an interactive weather installation that leverages interactive projection mapping to highlight an aesthetic quality to weather data, and signify its relation to space, movement and time. Working with digital projectors, a coding environment such as OpenCV and Processing, and projection-mapping tools, the project aims to create an interactive projection-mapped experience that provides a platform to analyze weather information in meaningful, aesthetic and engaging ways.

Analyzing blocking mechanisms on social media

In this project, we analyze blocking mechanisms on social media. We perform a comparative analysis of different technically and socially curated block-lists on Twitter. We also conduct interviews with users who are on such block-lists as well as those who subscribe to them. Our analysis reveals nuances of online harassment and the tactics used by harassers. We discuss the limitations of state of the art moderation used by social media platforms like Facebook, Twitter, etc. We examine how the harassment victims appropriate the online tools and resources available to them to cope with online abuse. We also suggest design implications for improved blocking mechanisms.

Analyzing the Affordances of a Location-Based App to Provide Safety

There are not many computing systems available that will help keep individuals safe when meeting up with strangers offline. Therefore, our team has developed an application that will help keep these individuals safe. In order to improve our current system, we will conduct a two-part research experiment: an interactive activity and interviews. This data, along with the data from the activity, will help to address whether or not using a computer system helps people feel safer when traveling alone. The study will be conducted on the campus of the Georgia Institute of Technology.


Animated Day of the Dead Puppets

Graduate studnets in the Prototyping Interactive Applications class taught by Gregory Abowd worked independently and with high school studnets from Latin American Assoication's leadership program with Cross Keys High School to create interactive Day of the Dead puppets. This project is part of GoSTEM, a larger effort at Georgia Tech to increase interest in science technology, engineering and mathmatics among Latin youth and to bring Latin culture to Georgia Tech. These animated puppet respond to one or more input(s) and produces visual, audio, and/or kinetic output.

Animating Human Dressing

Dressing is one of the most common activities in human society. Perfecting the skill of dressing can take an average child three to four years of daily practice. The challenge is primarily due to the combined difficulty of coordinating different body parts and manipulating soft and deformable objects (clothes). We present a technique to synthesize human dressing by controlling a human character to put on an article of simulated clothing. We identify a set of primitive actions which account for the vast majority of motions observed in human dressing. These primitive actions can be assembled into a variety of motion sequences for dressing different garments with different styles. 

Applying Design Studio Pedagogy in STEM Learning with Novel Presentation and Sensing Technologies

We use Augmented Reality presentation and sensing technologies to integrate design studio learning models into screen-based classrooms. The goal for this approach is to create STEM learning experiences that encourage creativity, innovation and help build strong peer learning environments. To accomplish this goal we implement room-scale augmented reality technology with projection-based presentation and sensing technologies -- projecting on surfaces and using depth sensing for unencumbered interaction (see http://research.microsoft.com/en-us/projects/roomalive/). This approach allows everyone in the space to participate in the experience, and the cost is fixed regardless of the number of participants. 


Two practices from the studio model for learning we build upon are:


– Pinups: In design studios, students will pin their work (completed parts, sketches, parts 

in-development) on a wall, and the teacher and students will walk the walls in order to 

comment on the pinned-up work. Pinups make both the artifacts and process of design 

work visible, and make it possible to compare and contrast approaches when all students 

work is pinned up at once.


– Meetups: Students working together in a design studio can look over to see what others 

are doing. Collaboration is fluid and at multiple levels. Sometimes, two students move 

their work near one another to work together (literally, “closely”). Sometimes, two 

students just look at each other’s work to share ideas.

AquaRium Tour: Georgia Aquarium tour experience design

This project integrates augmented reality to redesign the Georgia Aquarium tour experience. Based on the existing digital contents from Georgia Aquarium, AquaRium Tour features user-centered interaction to facilitate the aquarium tour experience, incorporating the functions of navigation, providing knowledge about aquatic life as well as sharing and other social features.


Atlanta has the reputation of being a “city in a forest”, with a large and varied tree population that provides shade for its residents, a habitat for wildlife, consumes carbon dioxide from the atmosphere, produces life-giving oxygen, in addition to many other benefits. In keeping with its context and commitments to environmental awareness and conservancy, the Georgia Tech campus contains hundreds of species of trees that cover the landscape.

The Imagine Lab is building an Augmented Reality application for mobile devices and tablets through which the myriad trees can be viewed interactively. In the app, the user can touch a tree and receive information from a vast database telling the user all the information about it: age, size, species. Not only can the user interact with the visible world, they are also capable of seeing projected tree growth for the next 10, 25, and 50 years, where newly planted trees will grow into shade-giving behemoths, all rendered in 3D on the screen. In addition, the user can experience the subterranean world, with interactive animations for the nearly unprecedented 1.4 million gallon cistern under Tech Green that provides water for the Clough Undergraduate Learning Center and local irrigation. The app allows for someone to receive an augmented view of the world in front of them, enhanced with illustrative educational information that will nurture the environmental consciousness of the user, making them aware of the tremendous benefits of having a green campus, how the systems work, and what it means for the future.


In a race against the clock, players embark on a dangerous adventure. Within moments, the journey goes haywire. Lost and alone, the player finds themselves stranded. In this VR interactive narrative, players fight to survive the dangerous landscape. Utilizing Oculus Rift, Unity, and unique interaction paradigms, Ares explores a wide range of new techniques in VR storytelling. This distinctive, immersive experience will test user’s survival skills and offer an exciting challenge.

Argon: AR-Enabled Web Browser

Argon is a mobile web browser designed to bridge the gap between Augmented Reality and The Web. Following in the tradition of web browsers like Chrome and Firefox, which differentiate themselves by providing custom functionality that is not yet standardized across all browsers, Argon exposes the core technologies needed to make AR possible. By making computer vision tracking (via the Qualcomm's Vuforia library) available to web pages, Argon provides a browser-based platform for rapid development of fully-interactive 2D/3D AR content & applications. The lab has developed tools to make rapid prototyping easier. The goal is to make it possible for designers and organizations with web app skills to create AR and MR (and even VR) applications. Come see projects & demos built using the Argon platform.

ARToss : Networked Argon3 Game Demo

ARToss is a multiplayer game built using Argon using a NodeJS web server with WebSockets.

Astro Pup: An interactive, musical, space adventure

This educational toy concept helps teach children visual-spatial cognitive skills and logical-mathematical reasoning through interactive music creation. Build from LegoMindstorms, the project explores the tool's applications in early stage interactive concept development for designers. 

Atlanta Beltline Exploration App

The Beltline Exploration App is a proposed location-based “walking tour” application aimed at increasing community engagement and participation on the Atlanta Beltline. The existing Atlanta Beltline app provides a wealth of information that can be improved with a more participatory interaction from the user and an element of user content creation. The goal is for the app to bring awareness to art, culture, and events along portions of the Atlanta Beltline that will introduce newcomers to the Beltline and promote repeat visits to the Beltline.

Auburn Avenue: Augmented Reality for Cultural Heritage

We are developing a suite of media experiences to introduce visitors to the rich cultural and economic history of Auburn Avenue. From about 1900 to 1960, Auburn Avenue was the center of African-American cultural and economic life in the city. The street also played a key role in the civil rights movement. From the 1960s on, the street suffered decline, and the local community disintegrated because of a range of social, economic, and urban planning factors. In recent years, however, the community has been the focus of revival efforts with attractive apartments and homes at its eastern end and increased economic activity along its more blighted corridor. In 2014 or 2015, a new streetcar line promises to bring even more tourists to its main attractions: the Martin Luther King Visitors Center, King’s birth home, the Ebenezer Baptist Church, and the King Memorial. Sweet Auburn was designated a National Historic Landmark in 1976. We are working in collaboration with Central Atlanta Progress and the History Preservation Division of the Dept of Natural Resources of the State of Georgia to bring this history to thousands of visitors and residents through an integrated integrated media strategy. Our media strategy centers on a prototype of a mobile app using the Argon browser. This will be supported by web applications that can run on other mobile devices as well as a web site.

The content types and features that we will explore include:
a. audio, images and text delivered on location at places of interest along the avenue.
b. panoramas and historical photographs to depict the visual history of Sweet Auburn.
c. informative texts to replace or complete existing physical signage;
d. forms of interaction that trigger the delivery of these images, audio, and text: for example, when users walk down the street, GPS tracking can tell the phone when to play certain audio or show certain images.
e. links to social media so that visitors can record their experience of the tour of the avenue for friends or for their own later use.

Our ultimate goal is to ensure the broadest possible class of visitors and web users to have a satisfying and informative experience of Auburn Avenue and make sure that the digital media application is a successful and sustainable informational companion that supports the preservation and revitalization efforts in this area.

Audio Lemonade Stand Game

This project helps teach STEM concepts with an audio-enabled version of the Lemonade Stand Game, in which visually impaired players (or any player that wants to experience a game that is sound dependent) need to manage their own stand while factoring in weather, local events, advertisement, and pricing in order to maximize profit for their business.

Auditory STEM: Math and Science Education for Students with Vision Impairment

The graphs and figures that are so prevalent in math and science education make those topics largely inaccessible to blind students. We are working on auditory graphs that can represent equations and data to those who cannot see a visual graph. A number of new areas we're starting research on is looking at teaching astronomy concepts through (like the Solar System) and the teaching and understanding of weather information through a combination of sonification and auditory description. Additionally we are working on making statistical output accessible for blind users to assist with higher level mathematics applications. We have a whole ecosystem of software and hardware solutions, both desktop and mobile, to help in this space. This project is in collaboration with the Georgia Academy for the Blind and the Center for the Visually Impaired of Atlanta.

Augmented Reality experiments on Support for Political Protest vs. Terrorism

In collaboration with policial science researchers from Georgia State University and University at Albany, we are developing augmented reality-based experiments to examine the impact of grievance, opportunity and risk as motivating factors when choosing to engage in political protest or terrorism.  Study participants assume the role of a fictional ethnic minority in a fictional country and engage in dialogs with virtual characters that attempt to persude the participant to join a peaceful student-led protest or join a violent resistance movement.  Participants wear a head-mounted video display instrumented with cameras that allow them to view computer graphics mixed with the physical space about them.  Sitting at a table, the participant can then see and hear virtual characters that appear to sit across from them, allowing the participant to experience a first-person point of view in dialogs with these virtual characters. 


Automated Driving Displays

Automated safety systems, a first step toward autonomous vehicles, are already available in many commercial vehicles. These are systems such as adaptive cruise control, which has the capability to slow down due to traffic, and automatic lane keeping, which maintains position within a lane without driver intervention. In order to ensure that these systems are properly used by drivers it is essential that they understand and appropriately trust the technology. We are currently investigating personal characteristics and driving environments that influence acceptance and use of automated safety systems and developing multimodal displays to increase situation awareness.


Automatically Generating Game Levels from Gameplay Videos

Check out videos of the system: here and here

Intelligent tools can ease the burden of game development. One approach to easing this burden is the use of co-creative, artificial agents, capable of helping a human developer by making suggestions or extending an initial design. However, agents capable of design have historically required a large amount of hand-authored design information—domain-specific rules, heuristic functions, or formal logic rules. Due to the time it takes to author this knowledge, such approaches do not remove the development burden, but shift it to the author of the agent. To solve this problem we present a demonstration of a level-authoring tool with a co-creative agent informed by knowledge learned from gameplay videos. The technique is demonstrated in the popular game, Super Mario Bros. We offer the experience of co-designing a level with a co-creative agent and then playing through the level yourself or with a friend.

Automod: Machine Learning-Based Approaches Toward Combatting Abusive Behavior in Online Communities

Since its earliest days, flaming, trolling, harassment and abuse have plagued the Internet. Our aim is to computationally model abusive online behavior to build tools that help counter it, with the goal of making the Internet a more welcoming place. In particular, we look at a novel approach to identify online verbal abuse using cross-community linguistic similarities between posts on different communities. This work will enable a transformative new class of automated and semi-automated applications that depend on computationally generated abuse predictions. 

B.B.C.S. - Bio-Behavioral Capture System

B.B.C.S. is a working system that can be easily deployed at homes, clinics, laboratories and therapy centers among others, in order to help its users collecting relevant behaviors of interest over long periods of time to get a deeper understanding of them.

This system captures behaviors of interest using multiple cameras alongside biological signals, such as heart rate, in a synchronized manner, allowing the user to analyze visible and invisible characteristics of behaviors. B.B.C.S. is intended to be an “everywhere / anywhere” system, so it allows the user to annotate, comment and control the system in situ.

Since B.B.C.S. can store weeks of data it was design that allows quickly browsing and filtering week’s worth of data to get to specific moments of interest.

As an example of a potentially interesting deployment scenario, we could mention the houses of families with individual(s) on the Autism Spectrum. Using this system would enable parents and researchers to obtain lots of data in their natural environment. We would be bringing the Lab home.

Ballet Hero: building a garment for memetic embodiment in dance learning

This project concerns the analysis and design of a wearable technology garment intended to aid with the instruction of ballet technique to adult beginners. A phenomenological framework is developed, and used to assess physiological training tools. Following this, a garment is developed that incorporates visual feedback inspired by animation techniques that more directly convey the essential movements of ballet. The garment design is presented, and a discussion is provided on the challenges in constructing an e-textile garment using contemporary materials and technique

Barcode Fitness

Barcode Fitness, a fitness application that helps you keep track of the details of your weightlifting workouts at the Campus Recreation Center at Georgia Tech. Ditch your clumsy notebook that you were using to write down all of your sets and repetitions in favor of Barcode Fitness. Barcode Fitness supports over 40 different exercises and allows for nearly instant selection of exercises by allowing you to scan the QR codes located on all supported exercise machines.

Being Nice on the Internet: Designing for the Coexistence of Diverse Opinions Online

Exposure to diverse opinions makes us more informed and engages society in a necessary deliberation process. Inwardly focused groups risk tunnel vision and an inability to challenge their own views. Technically, online we can connect to anyone in the world but social network analyses of blogs and Twitter have shown that we stay connected in groups of like-minded others. There is untapped potential here for online environments to go further towards giving more access to diverse views. Incivility has direct consequences for relationships with others of different opinions. It has been shown that in televised political debates, incivility increases negative feelings towards the other side. While disagreement is necessary in a healthy democracy, alienating arguments result in the current culture wars. I will present work I have done on encouraging civility on a platform like Facebook could alleviate frustrations such as the need to tune out when there is overwhelming disagreement.

BellSouth Black Calendar Interactive Installation

The goal of this project is to expand on the work done by Pamela Pryor and Jo Edwards (curators of the BellSouth calendar project), to remember, celebrate, and preserve stories of African Americans and Black-owned institutions in Georgia. The project consists of two parts an interactive exhibit and digital application. The exhibit and interactive installation will be located at the APEX (African-American Panoramic Experience) Museum in Atlanta, Georgia.

BellSouth Calendar Project: Interactive Installation Celebrating Black Voices

The BellSouth Calendar project was originally curated by Pamela Pryor and Jo Edwards. The purpose of the project is to expand on this work, remembering this narratives, and celebrate these individuals from the community. The project has two components: an interactive exhibit that will be on display at the APEX Museum(African-American Panoramic Experience) and the second is a website that will allow teachers, students, and community members to learn more. The purpose of these two platforms is they will allow more content such as images, videos, and interviews to be shared with visitors, with the purpose of encouraging individuals to learn and share these stories. The goal is that the project will encourage educators and students to research the individuals or institutions and discuss them within the classroom. Visitors and individuals from the community will also be impacted by the digital component by engaging them in conversations regarding African American culture and history.

Better Walk

In the U.S. alone, approximately 18 million people use crutches each year. The human body was not designed to bear its weight on the forearms and wrists, but all designs of the crutch force patients to do just this. In just a few steps with the underarm crutch, forearm fatigue sets in resulting in patients resting upon the underarm padding. The upward force from the padding leads to pain, chafing, blood vessel compression, nerve compression, and possible nerve damage. Forearm crutches, while avoiding the underarm area, are difficult to use and direct a large amount of torque to the shoulder, resulting in shoulder injuries, frequent imbalance, and falls. Not including the costs of treatments for the side effects of crutch use, patients spend over $800 million each year on these 5000 year old, inefficient devices.

The Better Walk crutch puts a patient’s mind at ease. The redesigned support system reduces the risk of underarm nerve damage, reduces forearm fatigue, and improves patient comfort resulting in increased compliance and a safer, more comfortable rehabilitation process.

Bicentric Diagrams: Design and Applications of a Graph-Based Relational Set Visualization Technique

Visualizations can help amplify human cognition. In an era where networks are becoming increasingly complex, the desirability of tools to compare and contrast sets, relationships, and reach is significant. Motivated by a practical need articulated by corporate decision makers, this research presents our journey in designing and implementing bicentric diagrams, a novel graph-based set visualization technique. A bicentric diagram enables simultaneous identification of sets, set relationships, and set member reach in integrated ego networks of two focal entities. Our technique builds on the well-established theory of tie strength to visually group and position nodes. We illustrate the broad applicability of bicentric diagrams with examples from four diverse sample domains: university collaboration, technology co-occurrence, health app purchases, and innovation ecosystems network. We assess the value of our technique using an expert-based value-driven evaluation approach. The paper concludes with implications and a discussion of opportunities for implementation in real-world settings.

Blending Liquids

We present a method for smoothly blending between existing liquid animations.

We introduce a semi-automatic method for matching two existing liquid animations, which we use to create new fluid motion that plausibly interpolates the input.
Our contributions include a new space-time non-rigid iterative closest point algorithm that incorporates user guidance, a subsampling technique for efficient registration of meshes with millions of vertices, and a fast surface extraction algorithm that produces 3D triangle meshes from a 4D space-time surface.
Our technique can be used to instantly create hundreds of new simulations, or to interactively explore complex parameter spaces.

Our method is guaranteed to produce output that does not deviate from the input animations, and it generalizes to multiple dimensions. Because our method runs at interactive rates after the initial precomputation step, it has potential applications in games and training simulations.

BlockParty: A Platform for Building Hyper-local Social Computing Applications on Residential Mesh Networks

Modern social media do a remarkable job of keeping friends and families connected—often across the globe. Yet, these same systems also overlook the communities and neighborhoods where we live our daily lives. In this paper, we present BlockParty, a platform for building hyper-local social computing applications aimed at neighborhoods. A key feature of our platform is that it runs on top of residential wireless routers via an underlying mesh network. Using BlockParty, people can socialize with their neighbors and share resources, without their data ever leaving their local community. The goal of BlockParty is to enable new forms of neighborhood-oriented social computing applications that encourage the creation of local ties and local social capital.

Bridging Cultural Differences

In the era of globalization, the ordinary viewer is exposed to cinematography from different countries and cultures, but does one understand the cultural context portrayed by the artists?
In this project I intend to use interactive television as a medium, that helps the viewer to gain a deeper understanding of a movie, by exposing him/her to its cultural layers.

Bringing Conversation Between: Web Narrative for Reproductive Justice Groups and Molecular Scientists

The advancement of medical research has freed us from many health related predicaments. However, despite the long history of our interest and studies in the field, US National Institutes of Health (NIH) has only recently passed the policy that ensures sex balanced research in preclinical studies, in 2014. This could have been accelerated if the two other ends, the reproductive justice advocates and the molecular biologists had come to a conversation along the way of preclinical studies. We seek to discover how these two different groups–reproductive justice groups and scientists–as well as the interested public perceive scientific information, surface common themes, and use the findings to inform the design of collaborative information space. The information space will function as an agora to produce new understandings of gender-balanced reproductive biology and health (contents). From designers' perspective, we want to ensure the following three questions are answered (form). What will attract the audience to visit the web narrative?; What will make sure that they stay more than 40 seconds?; What will make them to leave this website with motivation to collaborate?

Bringing Interactivity to Static Data Visualizations

Static visualizations have analytic and expressive value. However, many interactive tasks cannot be completed using static visualizations. As datasets grow in size and complexity, static visualizations start losing their analytic and expressive power for interactive data exploration. Despite this limitation of static visualizations, there are still many cases where visualizations are limited to being static (e.g., visualizations on presentation slides or posters). We believe in many of these cases, static visualizations will benefit from allowing users to perform interactive tasks on them. Inspired by the introduction of numerous commercial personal augmented reality (AR) devices, we propose a practical AR solution that allows interactive data exploration of datasets on static visualizations.

Bringing Interactivity to Static Data Visualizations through Augmented Reality

Static visualizations have analytic and expressive value. However, many interactive tasks cannot be completed using static visualizations. As datasets grow in size and complexity, static visualizations start losing their analytic and expressive power for interactive data exploration. Despite this limitation of static visualizations, there are still many cases where visualizations are limited to being static (e.g., visualizations on presentation slides or posters). We believe in many of these cases, static visualizations will benefit from allowing users to perform interactive tasks on them. Inspired by the introduction of numerous commercial personal augmented reality (AR) devices, we propose an AR solution that allows interactive data exploration of datasets on static visualizations.

Building and Testing a Developmental Milestone Tracking App with Parents and Childcare Providers

The earlier autism spectrum disorder (ASD) is detected, the earlier children can receive intervention services, resulting in improved social, cognitive, and adaptive skills. Birth to 5 years is an especially critical time for identifying potential signs of delayed or unusual development that may indicate ASD. Tracking children’s development can lead to an earlier diagnosis, and the Centers for Disease Control and Prevention (CDC) provides developmental monitoring tools for this through its “Learn the Signs. Act Early.” program.

The goal of this project is to develop and evaluate an Android app that makes CDC’s developmental monitoring tools more readily accessible to parents of young children, making it easier for parents to identify early signs of ASD or other developmental delays.

Came From Nothing: An Interactive Documentary

Came From Nothing is an interactive documentary that chronicles the incredible life story of Benjamin "Big Mouth Ben" Graham, an entrepreneur and motivational entertainer in Atlanta's Historic Sweet Auburn community. Big Mouth Ben overcame 17 years of addiction and homelessness on Auburn Avenue to open, together with his wife Tanya, a convenience store just two blocks from the same bridge under which he once slept. In the spring of 2017, as part of design and social justice project studio course titled "Sweet Auburn, Birthplace of Ideas," two students set out to capture Ben and Tanya's story in the narrative form of a feature-length documentary film. The film project was successful, premiering to over 100 people from the Sweet Auburn community on May 4th at The APEX Museum, and more recently inspiring hundreds of other people through subsequent screenings at Fulton County Drug Court. With this project, Nick Tippens, one of the documetary film's co-creators, aims to deepen and expand its potential impact by designing an interactive version of the narrative. This digital application creates a new experience for the story's audience, affording them agency over their interactions with its content. By engaging the audience with multimedia forms of this content, the interactive documentary highlights the nuances of the narrative that enrich their understanding of its complexities.

Campus Tour

Campus Tour is an augmented reality experience of Georgia Tech's campus. Once the channel is loaded in Argon, a standards-based Augmented Reality (AR) web browser developed by the Augmented Environments Lab.  The tour gives information to users through text, pictures and videos. Stops on the tour are panoramic images.  Within the panoramas are points of interests that once clicked reveal more information about their topic.  Campus Tour allows users to remotely enjoy the beauty of campus or to learn more about Tech while on campus. Campus Tour also lets you build your own expierences and tours.  Using our own custom web based editor you can choose which curated elements to use and build off of adding your own custom elements to create your own unique experiences.


College students encounter many challenges in the pursuit of their educational goals. When these challenges are prolonged, they can have drastic consequences on health and on personal, social, and academic life. Our multi-institution project, called CampusLife, conceptualizes the student body as a quantified community to quantify, assess, infer, and understand factors that impact well-being. Our goal is to develop privacy-honoring infrastructure and tools that can first sense lifestyle, moods, activities through active and passive techniques, and then utilize that information in the design of self-reflective tools that could make students more self-aware and pro-active toward improving their well-being

Captioning on Glass

Captioning on Glass is an on-going project creating an app for Google Glass with a companion Android phone app to assist the hard-of-hearing in everyday conversations. We are also working on another version of this app, "Translation on Glass", which will add the ability to translate between English and another language.

Care Process Graphs: Exploring Clinical Care Processes Using Visual and Data Analytics

Healthcare big data is being widely touted as a potential resource for curbing costs and improving outcomes. However, numerous challenges remain for leveraging this data to its full potential. In this position paper we identify the difficulties that characterize clinical data, based on our experiences working with pediatric asthma data from Children's Healthcare of Atlanta. The specific dataset we explored includes administrative items, medications, lab results, clinical respiratory scores (outcome), timestamps, and demographic information from 5,785 emergency department (ED) visits for asthma exacerbations. We argue that new data and visual analytic techniques are needed that are specifically tailored for solving challenges in healthcare, and we propose characteristics that these techniques should have and give our design rationale. To demonstrate how a tool that embodies these desirable features may be designed, we introduce CareProcessVis, a prototype interactive visual analytics tool that helps clinicians explore and understand the processes involved in pediatric asthma emergency department care.

CASE: Content Aggregation Systems for Elections

The Content Aggregation System for Election Observation (CASE) will aggregate real-time election observation data from formal observer missions and social media sources. Our new system, combining the power of crowdsourced data from social media with the precision of formal observers in the field, will create a first-ever fully integrated monitoring system. Simple technical interfaces will allow users to share particular information in real-time while still maintaining necessary data security and privacy. An integrated visual dashboard will allow all project participants to view, analyze and understand real-time data from social media fully integrated with real-time data from participating formal observer groups. The system will be test deployed in 2014 and fully deployed during the 2015 Nigerian national election.

Characterising the Mental Health Challenges of LGBTQ Populations Through Computational Analysis of Crisis Conversations

This research project aims to develop computational and analytic approaches that can utilize Crisis Text Line’s conversations to examine, quantify, and understand “minority stress” and thereafter glean meaningful insights into the unique mental health challenges and needs of LGBTQ (Lesbian, Gay, Bisexual, Transgender, Queer) individuals.

CHAT (Cetacean Hearing Augmentation and Telemetry) and UHURA (Unsupervised Harvesting and Utilitization of Recognizable Acoustics)

Working with Dr. Denise Herzing of the Wild Dolphin Project, we are creating wearable computers for conducting two-way communication experiments with cetaceans.  With CHAT, one researcher uses the waterproof system to broadcast a sound, associated with an object with which dolphin's like to play.  A second researcher, upon detecting the sound, passes the object to the first.  The researchers pass objects back and forth, further associating the sound with the object.  The goal is to see if the dolphins mimic the sound in order to "ask" for the play object.  The wearable computer uses pattern recognition technology to detect these mimicked sounds.  In a more long-term effort, UHURA uses pattern discovery techniques in an attempt to uncover fundamental units of dolphin vocalizations.

CHAT - A Dolphin Interaction Wearable

CHAT (Cetacean Hearing Augmentation & Telemetry) is a wearable underwater computer system, engineered to assist researchers in establishing two-way communication with dolphins. The project seeks to facilitate the study of marine mammal cognition by providing a waterproof mobile computing platform. An underwater speaker and keyboard enables the researchers to generate whistles. The system is equipped with a two channel hydrophone array used for localization and recognition of specific responses that are translated into audio feedback. The current system is the result of multiple field tests, guided by the researchers feedback and the environmental constraints.



CheckDroid is a service for Android development teams to test and support their applications on different devices. We are creating the next generation testing & debugging tools for mobile developers.
Testing mobile apps across different platforms is challenging because of the sheer number of device types -- 22 iOS devices & 18K Android devices. This is often referred to as the Fragmentation problem.

Our demo will present two tools:

1. App Mirror: This a capture-replay tool that allows a mobile developer to record their interactions with the app in one device and see the results of the same interaction across multiple devices.
It allows for both LIVE replay for manual testing and for reporting any differences or issues in an offline report.

2. Cloud Test: This is a web based environment that allows the developer to interactively write tests for their app and then run these tests on a test-bed of devices.

Cinematic Interfaces

Digital tools exist for creating practically every type of artistic, creative, or communicative digital artifact, including pictures, music, video, and computer animation. This project explores a combined AI-HCI approach to participatory intelligent agents that help amateurs create digital moving image media, such as machinima.

City Fables: An Argon Experience for the city of Malmö, Sweden

In collaboration with colleagues from Malmö University in Malmö Sweden, the AEL is helping to develop a mixed-reality experience that is a narrative of cultural moments from the first half of the twentieth century. The Swedish project, under the direction of Profs. Maria Engerberg and Per Linde, is called Stadsfabula. The AEL is helping to create and test an Argon application that will recognize historic photographs on the walls of a museum space and play the video and audio. This is an experiment in the use of the Argon-aframe platform to create a compelling multimedia experience that is also easy to program and to modify. 

Civic and Participatory Media (Sweet Auburn: Birthplace of Ideas)

This research initiative explores the potentials and challenges of civic and participatory media, investigating a set of research questions that probe the relationship between technology, place, storytelling, and community engagement. The aim is to investigate hybrid platforms that inform and engage local communities through the mediation of shared public spaces, digital media, mapping, and storytelling. For example, design experiments the context of Auburn Avenue seek to both highlight and preserve the important history of the neighborhood as a vital center of innovation, commerce, and community among African Americans and the center of the Civil Rights Movement during the era of segregation, as well as contribute to the current revitalization efforts within the neighborhood.


ClipLine—A social sharing mobile platform that helps users turn their favorite TV scenes into customized GIFs and instantly share them with their friends and the outside world. Voting up the best GIFs, re-clipping, and following other accounts will also be main features of ClipLine.

Clock Reader

Early detection of symptoms is of critical importance in diagnosing and treating cognitive dysfunction. One important instrument utilized for detecting early signs of cognitive dysfunction is the Clock-Drawing Test. In this test, patients are asked to draw a clock face at a certain time, and are evaluated on how well they perform this task. At present, analysts must individually administer and assess each test a person completes. Automating the process would grant many advantages: the patients could complete the clock test more often to measure improvement, stabilization or variation over time; the patients would receive immediate feedback on their results; the evaluation structure would become more standardized for broader assessment; and multiple evaluation tools could be utilized simultaneously. Toward these ends, the ClockReader project will seek to automate the administration and evaluation of Clock-Drawing Tests on tablet PCs. The ClockReader project will then be tested on both past Clock-Drawing Tests and new tests performed by new participants.

Collective Sensing: Building Better Human Networks

Collective sensing is a novel mobile technology which aims to build better human networks. It uses multiple informants to collect information regarding an individual in a variety of contexts with the goal of creating a more holistic story.

Community Historians

This project is developed through an ongoing collaboraton with the Historic Westside Cultural Arts Council. Through a series of design workshops and public events we are co-designing mobile and social technologies to help cultivate a shared community identity to support local civic engagement. By working directly with community members, we are able to build technology platforms suited to their specific needs and which amplify their values and concerns as the community goes through significant changes.

Computational Pretend Play

Pretend play helps children develop a wide range of cognitive skills and is therefore a critically important skill for kids to learn. Some children, such as those on the Autism Spectrum, have difficulties engaging in pretend play. This project seeks to understand and model what constitutes successful pretend play in order to design and implement technologies to support and facilitate highly engaging pretend play. The exact nature of that intervention is an open question, and we are exploring several exciting options including a robotic play partner and an immersive virtual play world. The first step in this initiative is building a cognitive model of play and then developing a computational framework that enables a artificial intelligence system to generate improvisational play behaviors based on our computational model of play. In our demo, we will show some early results from a study observing adult dyads engaged in play behavior as well as the first prototype of the immersive virtual play world.

Computer-supported Coorperative Praxis: Social Computing Technology in Support of Collective Action in a Grassroots Movement

Social movement organizing is becoming increasingly dependent on communication technologies. How can CSCW systems support grassroots organizations in facilitating collective action through democratic participation? In this paper, we study Science for the People-Atlanta, a social movement organization dedicated to build a grassroots movement around science activism. We used action research, both participating in the organization and studying it. Further, we interviewed ten active members of the organization. Extending the Freirian argument of participatory communication in movement building, we show how SftP-Atlanta uses sociotechnical systems to foster democratic participation that led the organization towards collective action. Inspired by Freire’s description of praxis as a combination of action and reflection in the process of movement building, we suggest Computer-Supported Cooperative Praxis (CSCP) as a model for grassroots organizing. Our research aims to contribute to the growing body of work around social-justice-oriented design by highlighting the importance of designing tools that aim to enhance the impact of CSCP.

Computer-supported Coorperative Praxis: Social Computing Technology in Support of Collective Action in a Grassroots Movement

Social movement organizing is becoming increasingly dependent on communication technologies. How can CSCW systems support grassroots organizations in facilitating collective action through democratic participation? In this paper, we study Science for the People-Atlanta, a social movement organization dedicated to build a grassroots movement around science activism. We used action research, both participating in the organization and studying it. Further, we interviewed ten active members of the organization. Extending the Freirian argument of participatory communication in movement building, we show how SftP-Atlanta uses sociotechnical systems to foster democratic participation that led the organization towards collective action. Inspired by Freire’s description of praxis as a combination of action and reflection in the process of movement building, we suggest Computer-Supported Cooperative Praxis (CSCP) as a model for grassroots organizing. Our research aims to contribute to the growing body of work around social-justice-oriented design by highlighting the importance of designing tools that aim to enhance the impact of CSCP.

Connected Living Experience

Connected living is the fast-growing intersection of mobile, wearable, home, community, car and other technologies to assist individuals in accomplishing more seamless interactions and goals in daily life. The Aware Home is the perfect environment for exploring how smarthome systems may be advanced in the future. Student design projects are an opportunity for students to research the preferences of the end users and design prototype systems that will inform this future.

Connected Living Research Initiative

Connected living is the fast-growing intersection of mobile, wearable, home, community, car and other technologies to assist individuals in accomplishing more seamless interactions and goals in daily life. Mobility and cloud computing are two pillars of growth that has brought about significant changes in industry. Cloud computing, big data, mobility and low-cost sensors are driving the internet of things and connected industries, and the internet of things is forcing transformation and innovation across the connected home, connected workplace and connected city. It is estimated that the Connected Living market will reach 730 Billion USD by 2020.

We are in the process of defining the Connected Living Research Initiative (CLRI) to bring together industry stakeholders, academic/research faculty and civic partners in defining the future of the connected life. CLRI is currently on boarding partners to delineate research goals that include (but is not limited to) the future impact of big data, improved user experience in daily activities, and data security and privacy in this ever more connected daily experience.

For more information contact: Brian Jones or Siva Jayaraman


Convergence Innovation Competition

The Convergence Innovation Competition (CIC) is a unique competition open to all Georgia Tech students and is run in both the Fall and Spring semesters. Each year the categories in the CIC are defined by our Industry partners who provide mentorship, judging, and category specific resources which are often available exclusively to CIC competitors. While the competition is not tied to any specific course, competitors are often able to take advantage of class partnerships where lecture and lab content, guest lectures, and projects are aligned with competition categories. CIC Competitors are supported by GT-RNOC research assistants who provide technical support and shepherd teams through the competition process. The overarching goal of the CIC is to create innovative and viable products and experiences including a strong user experience and a business case. Winning entries will include a working end-to-end prototype which operates on converged services, media, networks, services, and platforms. CIC winners go on to commercialization, other competitions, as well as internship and job opportunities strengthened by their competition experience.

Conversational Media: Designing a Decision Aid for Diabetes Medication Choice

Over 29 million people in the U.S. live with type II Diabetes. There are many types of medications available to help manage Diabetes, and these medications impact patients' lives in unique ways. Following tenets of evidence-based medicine, participatory design and shared decision making, design researchers at the Mayo Clinic have created a set of cards for use in patient-physician conversations, to help both parties reach a decision on diabetes medication choice. I'm working on an updated digital version of this decision aid, which offers opportunities for tailored content and easier-to-update information while aiming to maintain the flexible, accessible spirit of the original tool. 


This project involves the design and evaluation of an interactive computer game that allows deaf children to practice their American Sign Language skills. The game includes an automatic sign language recognition component utilizing computer vision and wireless accelerometers. The project is a collaboration with Dr. Harley Hamilton at the Atlanta Area School for the Deaf.

CopyCat: Helping Young Deaf Children Acquire Language Skills Using Sign Language Recognition

CopyCat and PopSign are two games that help deaf children and their parents acquire language skills in American Sign Language.  95% of deaf children are born to hearing parents, and most of those parents never learn enough sign language to teach their children.  As short term memory skills are learned from acquiring a language, many deaf children enter school with short term memory of less than 3 items, much less than hearing children of hearing parents or Deaf children of Deaf parents.  Our systems address this problem directly.  Even though they are still under development our games have been shown to be effective in multiple user studies.

Copyright and Social Norms in Online Creative Communities

Every day, ordinary Internet users engage with complex copyright laws. Particularly in the context of creative work and appropriation, they are making decisions related to legal areas that are notoriously gray. Where legal knowledge is imperfect, social norms and ethical intuitions fill in the gaps. This research attempts to understand how these decisions are made, how norms and knowledge differ in different creative communities, and what lessons can be derived for online community management and design.

COSMOS: COmputational Skins for Multi-functional Objects and Systems

COSMOS (COmputational Skins for Multi-functional Objects and Systems) is an interdisciplinary collaborative project to design, manufacture, fabricate, and apply "computational skins". COSMOS consist of dense, high-performance, seamlessly-networked, ambiently-powered computational nodes in the form of 2D flexible surfaces that can process, store, and communicate sensor data. Achieving this vision will redefine the basis of human-environment interactions by creating a world in which everyday objects and information technology become inextricably entangled. This will also enable alternative and neuromorphic computing that can change the foundation of computing today.

Coupling Cloth and Rigid Body for Dexterous Manipulation

This project introduces a new simulation technique to enable detailed dexterous manipulation of cloth. Without reimplementation or substantial modification, existing cloth simulators can only be used to approximate limited interaction between cloth and rigid bodies due to the incorrect computation of contact forces. For example, a simple scenario of two fingers pinching a piece of cloth often results in the cloth slipping out of the hand. Our technique provides a simple solution to cloth-rigid coupling using existing cloth and rigid body simulators as-is. We develop a light-weight interface so that the rigid body and cloth simulators communicate on a demand-driven manner to achieve two main goals: allow the rigid bodies to impart friction forces to the cloth and avoid unsolvable collision situations between the rigid bodies and the cloth. We demonstrate a set of basic manipulation skills including gripping, pinching, and pressing, that are frequently seen in daily activities such as dressing and folding clothes.


CREDBANK: A Large-scale Social Media Corpus With Associated Credibility Annotations

Social media has quickly risen to prominence as a news source, yet lingering doubts remain about its ability to spread rumor and misinformation. Systematically studying this phenomenon, however, has been difficult due to the need to collect large-scale, unbiased data along with in-situ judgements of its accuracy. In this paper we present CREDBANK, a corpus designed to bridge this gap by systematically combining machine and human computation. Specifically, CREDBANK is a corpus of tweets, topics, events and associated human credibility judgements. It is based on the real-time tracking of more than 1 billion streaming tweets over a period of more than three months, computational summarizations of those tweets, and intelligent routings of the tweet streams to human annotators—within a few hours of those events unfolding on Twitter. In total CREDBANK comprises more than 60 million tweets grouped into 1049 real-world events, each annotated by 30 human annotators. As an example, with CREDBANK one can quickly calculate that roughly 24% of the events in the global tweet stream are not perceived as credible. We have made CREDBANK publicly available, and hope it will enable new research questions related to online information credibility in fields such as social science, data mining and health.

Crowdsourcing UN 2015 Millennium Development Goals

In 2000 the United Nations announced the Millennium Development Goals, a set of development targets and objectives to reduce poverty and improve health, education, and the environment. These goals are set to be completed by 2015. The system of United Nations organizations is currently formulating a new set of development goals for beyond 2015. To create a more participatory process, the International Telecommunication Union uses an online platform to crowdsource the ideas and comments of youth around the world. The ITU requested the assistance of the TID lab to provide interpretation and textual analysis of the youth’s priorities based on the crowdsourced data. We are developing new visualizations and analysis of this unique dataset. This analysis will help inform the post 2015 UN development agenda.

CSLearning4U: Creating Electronic Books for Teacher CS Learning

A key idea in CSLearning4U is that we can design CS learning opportunities. Simply wrestling an interpreter or compiler can't be the best way to learn about computer science. Throwing people into the deep end of the pool can teach people to swim, but there are better ways. We want to do better than a book for CS learning, and we want to design the phonics of computing education to integrate with the "whole language learning" of programming.

We are creating a new distance-learning medium for computing education especially for in-service high school teachers based on ideas from instructional design and educational psychology. In-service high school teachers are particularly time-constrained (and thus need efficiency) and they are more metacognitively aware than other students (and thus able to better inform the project design). The new medium will combine multiple modalities, worked examples, and structure based on cognitive models of designers' knowledge. The research questions are that (1) the teachers will learn CS knowledge in the on-line setting, (2) the teachers will be more efficient at programming tasks, and (3) the teachers will find the materials useful and satisfying. Because of its focus on teachers, the project can potentially have broad impact, in particular on the strategies for training the 10,000 teachers envisioned in the CS 10K Project. The project will establish models and design guidelines that can be used for the creation of other learning materials, including materials for students in, for example, the proposed new CS Principles AP course.


CulturEat is an application that bridges the skilled home chefs or cooks with urban diners who are looking for authentic and affordable cultural meals. It gives the cooks the ability to upload their masterpieces and sell them to earn extra income while sharing cultural heritage/stories of the dish with diners. It helps to restructure the current societal food sharing system and promote positive cultural impacts on social cohesion via food.

Curve Averaging

We present our work on computing an average curve given a set of planar input curves, with select applications. This work, to be soon presented at the Symposium on Geometric and Physical Modeling, provides a mathematical formulation and a fast algorithm for the problem of finding an average curve, given a set of input curves. Applications in the field of animation and statistical analysis are highlighted.

Cycle Atlanta

Fifty percent of all trips are 3 miles or less, yet only 1.8% of those trips are biked.  Meanwhile, 35.7% of US adults are obese and the transportation sector accounts for 32% of US greenhouse gases.   One of the main reasons citizens do not use the healthier mode of cycling is due to a lack of safe infrastructure—dedicated bicycle routes, roads with bicycle lanes, and other designated bicycle facilities.  The City of Atlanta has a desire to put proper cycling infrastructure in place but needs better information from citizens about where they currently and would like to cycle.  Therefore, the initial goal of the Crowd-sourced Bicycle Route Desirability project is to modify the open-source CycleTracks application (previously adopted in San Francisco, CA, and Austin, TX.) for use in Atlanta.  CycleTracks tracks the existing routes of cyclists using their smart phones and allows comparison of these routes to the quickest path from origin to destination.  This allows us to begin to make appropriate infrastructure improvements to the most traveled routes in a study area by seeing logical paths that cyclists avoid.  A second phase of the project would develop applications allowing riders to express their desired bike routes even if they currently do not cycle because of lack of adequate facilities.

Data Artifacts

The term “artifact” has at least two meanings. From a technical perspective, an artifact is an unintentional pattern in data, arising from processes of collection and management. From a cultural perspective, an artifact is a designed object, with a social and material history. At metaLAB, which is grounded in both technical and cultural methods, we are examining digital artifacts with both meanings in mind. In Data Artifacts, we are developing visual methods of revealing the often-unacknowledged patterns in digital data that speak to the social and material history of its accumulation. Never raw, all data carries traces of human labor, intentions and values. Data Artifacts is an inquiry into the deep history of digital collections. Digital cultures, which devote vast resources to the harvesting and handling of data sets, can be understood in part through the particular ways in which they pattern data. Artists and designers with knowledge of computing are poised to uncover such data artifacts through visualization. However, most formal approaches to visualization call for data to be filtered and standardized at the outset. In contrast, we focus on the heterogeneity inherent in human-made data. The messiness of data sets can tell us much about the history of their production. The ambition of Data Artifacts is to develop new tools to contemplate such large-scale collection processes and enable richer discussions about their technical and cultural significance.

Data Documentary

Understanding and visualizing pedestrian accidents on Buford Highway

Data Illustrator

Data Illustrator is a system that helps graphic designers created data-driven visualizations and infographics without the need to do any programming. More specifically, it is a vector editing tool for creating data visualizations and infographics. Graphic designers can use Data Illustrator to craft their own visualizations by repeating and styling shapes with data-driven rules. The tool supports the creation of expressive, flexible, and parametrically defined visualizations without the need to program them.

Data-driven Connected Home

Using the Z-wave protocol stack, we are building a controller for the Aware Home using a Raspberry Pi that will allow users to control and query device data on a dashboard. This collected data will then be used to predict usage patterns and serve tips for power saving. Finally, a user-friendly rules engine enables users to create certain rules using sensor data.

DataToMusic Web API

DTM API is a musical data sonification toolset for rapid development and experimentation of web-based audio applications. The API offers a data-agnostic, adaptive, and highly interactive real-time system, with reusable and extendable musical structure models to represent data in various ways. The API is being used in several projects, including in Beltline Social Dashboard, Decatur Civic Sonification with Sonic Generator performance, which is presented at Atlanta Science Festival 2015, in collaboration with GTRI Configurable Lab.


Dear Games

Dear Games is an educational program collaboration between Charis Circle, members of the GA Tech Game Studio and Different Games Collective. We offer inclusive events to support diverse participation in videogame developement and culture at the South's oldest independent feminist bookstore, Charis Books and More, with consideration to the ways that longstanding feminist community organizations can inform contemporary efforts to increase diversity in STEM.

Debate Slates

Debate Slates is a second screen application experience designed to facilitate discussion of theories and future plot developments of long-form narrative television (e.g. Game of Thrones, True Detective and Fringe); Debate Slates also hopes to facilitate discussions on current events, focusing on televised reportage of ISIS and the developing situation in Iraq and Syria.

Defining Digital Self Harm

This project aims to define the concpet of digital self-harm for the HCI community. In this project we have explored the limited HCI scholarship related to self-harm within a social computing context. We offer the community an operatlonalized defintion of digital self-harm and propose a theoretical base to orientate related research questions into actionable activities. We also describe a research agenda for digital self-harm, highlighting how the HCI community can contribute to the understanding and designing of technologie sfor self-harm prevention, mitigation, and treatment.

Design for Mindfulness

We are living in a multitasking society. We are experiencing an unprecedented level of sensory and cognitive overload, in which we have too many things going on at once, making us more likely to be absentminded. How to involve technology in promoting mindfulness and making it part of the process of achieving it is the question we need to answer in this project.

Design Guidelines for Parent-School Technologies to Support the Ecology of Parental Engagement

Parents’ engagement in their children’s education is key to children’s academic success and social development. For many parents in the U.S., engagement is still a struggle partly due to a lack of communication and community-building tools
that support the broader ecology of parenting, or parental ecology. Although current technologies have the potential to create opportunities to improve parental engagement, little is known about the impact of existing technology’s design on
the parental ecology. We present findings from 63 interviews with parents and an observation of existing technologies that support parent-school interactions. We found four critical issues that the design of current technologies need to address:
(1) inflexibility in the boundaries of digital spaces, (2) inequality, (3) fragmentation and inconsistency of information, and (4) lack of relevant non-academic information. As a result, we propose design guidelines for technologies to support the parental ecology, and reflect on design issues that require further research.

Designing Adaptive Technology to Provide Personalized Support to Cancer Patients

We design, deploy, and evaluate mobile health tools that support and meet patients needs over time from diagnosis of a chronic disease, through treatment and into survivorship. Our research explores the ability for personalized, adaptable, mobile tools to support patients over the course of their individual breast cancer journeys. Our technology needs to anticipate and recognize barriers to care that occur at various points in a cancer journey, adapt with the patient as they navigate these barriers, and successfully provide patients with the tools and resources they need to manage and mitigate such barriers. The goal of our work is to improve patient health outcomes by supporting patients' outside of the clinic by helping them to learn about, engage with, and manage their disease alongside the demands of daily life.

Designing for Equity in Digital Access through Search Engine Optimization

The field of Human-Computer Interaction has long engaged in creating and studying online tools that serve as information resources. However, few studies explore how people, particularly underserved populations, find these tools. This study examines the role that online search plays in the access to online information resources using access to online Computer Science (CS) education as a case study. First, we conduct a survey with U.S. parents to identify terms used to search for online CS education. Second, we look at the search results for those terms. Third, we analyze the current on-site search engine optimization (SEO) trends among CS education tools. Then we compare the search results and trends to find opportunities to bridge the access gap and provide SEO suggestions to better reach underserved populations. This is a unique research method, leveraging SEO to improve access to information technology, that can be applied to many domains 

Designing the front-end of a tool that facilitates bypassing censorship on Sina Weibo

Like traditional media, social media in China is subject to censorship. However, in limited cases, activists have employed homophones of censored keywords to avoid detection by keyword matching algorithms. This project focusses on designing an interactive, client-side tool that promotes free speech. An iterative design process, involving the inputs of end-users will deliver a final design. In the evaluation of the design we will target the following research questions:

RQ 1. Does the UI workflow fit into context of use of the users?
RQ 2. What the user preferences in terms of providing input and receiving output?
RQ 3. Do users prefer a desktop version or mobile or both?
RQ 4. Does the design feel familiar and similar to the UIs in China?
RQ 5. Does the UI feel trust-able ( does the UI breed and draw trust from the users? )

For the info on the algorithm, see project: Algorithmically Bypassing Censorship on Sina Weibo with Nondeterministic Homophone Substitutions

Designing the front-end of a tool that facilitates bypassing censorship on Sina Weibo


Like traditional media, social media in China is subject to censorship. However, in limited cases, activists have employed homophones of censored keywords to avoid detection by keyword matching algorithms. This project focusses on designing an interactive, client-side tool that promotes free speech. An iterative design process, involving the inputs of end-users will deliver a final design. In the evaluation of the design we will target the following research questions:

RQ 1. Does the UI workflow fit into context of use of the users?
RQ 2. What the user preferences in terms of providing input and receiving output?
RQ 3. Do users prefer a desktop version or mobile or both?
RQ 4. Does the design feel familiar and similar to the UIs in China?
RQ 5. Does the UI feel trust-able ( does the UI breed and draw trust from the users? )

For the info on the algorithm, see project: Algorithmically Bypassing Censorship on Sina Weibo with Nondeterministic Homophone Substitutions

Designing Virtual Character Appearance in Informal Learning Environments for Children

The rise of ubiquitous technology has resulted in opportunities for the design of new interactive museum exhibits that can be customized to families. Children’s museums can be engaging, informal settings in which children learn fundamental science, technology, engineering, and math (STEM) concepts through hands-on experiences. In order to optimize and personalize learning experiences in such informal environments, we propose the concept of a virtual buddy that uses personal, physical, and social context knowledge regarding the child to facilitate new opportunities for STEM learning. To understand how children choose, perceive and interact with a virtual buddy and how that may impact STEM learning, we conducted participatory design activities with 18 children in a local museum. The goal of this project is to inform the design of a Virtual STEM Buddy (VSB) that could provide contextualized explanations, to seed parents contextualized explanations and to bridge the museum experience to other informal learning experiences.

Designs for Foraging


Designs for Foraging is a design project that explores the use of IoT technologies in support of urban foraging. Through this project we are developing use-cases; prototyping hardwares, software and user interfaces; and exploring the use of open technologies for image capture and analysis. The underlying motivation for this project is to use design as a means of investigating future practices and to provide the basis for near-term open innovation with IoT in support of alternative practices of agriculture.

design[ED] Lab

Students solved these problems in design[ED] Lab (“Design Education Lab”), a user experience workshop that introduced teenagers and pre-professional adults to design-thinking, to encourage problem solving and critical thinking skill development. This workshop was in partnership with The Bridge Academy (College Park, GA), a full-time High School Diploma and GED Prep program offering a nontraditional path for students. Students used a design-thinking approach to respond to problems based on the College Park Comprehensive Plan (2011 – 2031), by defining the problem, brainstorming solutions, thinking empathetically, iterating on the prototype, and critiquing the work. design[ED] Lab aims to expose underrepresented minorities to design-thinking as a method to solve important problems within their community. Empowered with the tools to make a difference, we hope to inspire the minds that will change the world.

design[ED] Lab is a research project created and facilitated by Monet Spells, a Master’s student at Georgia Institute of Technology studying Human-Computer Interaction.


Developing a Quick-Start Guide to Aid Older Adults with Gesture Performance

Exergames, or exertion video games, are interactive, exercise based video games that are a promising in-home approach to physical activity, therapeutic and rehabilitation training, and social interaction for older adults. Research shows that older users have difficulty using exergames due to overly complex interfaces, difficult gestures, and an overall lack of training and familiarity with these systems. To alleviate these usability challenges older adults experience with the use of Kinect-based exergames, we developed a Quick-Start Guide (QSG) as a form of instructional guide to display gesture interactions and trouble-shooting techniques to aid system use. Our current study evaluates three different formats of QSG to assess the most effective in helping older adults use these systems. Findings will provide insights into the best methods of constructing quick-start guides for interactive technologies for the older adult population. 

Digital Block and Box Test

Technology is changing the scope and quality of healthcare through applications such as telemedicine and home health technology by offering a cost-effective and accessible means to manage chronic disease. People are increasingly taking a proactive role in monitoring and maintaining their health, e.g., monitoring blood pressure to prevent stroke, or measuring blood sugar levels to regulate diabetes. One of the most pressing health issues we face today is stroke. Statistics from Centers for Disease Control and Prevention indicates that stroke is the leading cause of serious, long-term disability in the United States. While more and more stroke rehabilitation therapy are conducted in patient’s home, care providers still requires patients to visit the clinic to perform the clinical assessments.

We investigate a computational tool – the Digital Box and Block Test (DBBT) – that can help medical professionals record and assess rehabilitation progress of stroke patients with easy setup. Embedding this technology in the residential spaces could also help patients to relearn and recall how to use their arms, hands and fingers. With the system, care providers would be able to more precisely detect, track, and monitor patient’s post-stroke functional motor improvements remotely.

Digital Enrichment for Orangutans

With their high cognition, engineer-like curiosity, and close relation to humans, orangutans are an extraordinary user to study. The project aims to provide animal care staff and organizations new methods in enriching the lives of animals in their care by creating applications with the Kinect for interactive projections.

orangutan enrichment scenario

Digital Icon-Based Asthma Action Plan

Currently 6.8 million children in America have asthma, a disease of the respiratory system that causes inflammation of the airways. An asthma action plan is an individualized health management plan that doctors give to their patients to help control their condition. It functions by illustrating what actions to take at different levels of symptom severity from day-to-day medication use to emergency situations. A problem arises for the caregivers of asthmatic children who may not have the educational background to understand the information in an action plan. These children may be in danger if their caregivers are unsure of the proper actions to take to treat their symptoms. The asthma action plan also serves as a partnership between the caregiver and physician. An action plan that is difficult to understand may degrade this partnership, however, research indicates that better communication between caregivers and physicians can lead to better medication adherence. Our solution is to develop a digital icon-based asthma action plan (I-BAAP) that can be integrated into patient's electronic medical records. The system is composed of a physician portal in which doctors input information relevant to a patient. The portal outputs a link to a responsive web application consisting of the I-BAAP and other features that augment communication between caregivers and physicians. The caregivers can access the web app on their phone, increasing possession of an action plan as compared to paper-based plans which are often lost or misplaced.

Digital Naturalism

Digital Naturalism investigates the role that Digital Media can play for Biological Field Work. It looks to uphold the naturalistic values of wilderness exploration, while investigating the new abilities offered by digital technology. Digital Naturalism unites biologists, designers, engineers, and artists to build and analyze new devices. It focuses on crafting DIY technology and interacting with animals in new ways.In particular, Digital Naturalism looks at how digital media can be used to explore animal behaviors situated in their natural context. Most recently, this research has been carried out directly in the field in the form of Hiking Hackathons.This research originally comes from Andrew Quitmeyer‘s PhD research at Georgia Institute of Technology. It now forms a lifelong project and multiple cross-disciplinary collaborations all pursuing the many aspects of Digital Naturalism.


Digital Self-Harm - A case of eating disorders

Self-harm is the infliction of pain or injury onto oneself. Though historically these behaviors were relegated to the fringes of traditional communities, information technology now enables new ways to foster and encourage these dangerous activities. This research strives to understand these online behaviors and activities and how that impacts aspects of an individual's disease. This research aims to do the followiong:

  • Define the concept of digital self-harm as the online communication and activity that leads to, supports, or exacerbates, non-suicidal yet intentional harm or impairment of an individual’s physical wellbeing. 
  • Ground-truth public data analysis of online eating disorder content with actual patients
  • Understand how patients and clinicians percieve the influence of social media with respect to eating disordered behaviors and activities.
  • Explore correlations between online activity and offline disease presentation.

Don't Open That Door

“Don’t Open That Door” is a gesture-based interactive narrative project set in the universe of the TV show Supernatural. This project creates dramatic agency for the interactor by leveraging expec- tations of the horror genre within a seamless scenario that elicits expressive actions and provides a dramatically satisfying response.

We match interaction and narrative elements to support the following design goals: Story-driven Physical Reactions, Persistent and Uninterrupted Narrative, Scripting of the Interactor by Narrative.

dotlink360: Visual Business Ecosystem Intelligence

Business ecosystems are characterized by large, complex, and global networks of firms, often from many different market segments, all collaborating, partnering, and competing to create and deliver new products and services. Given the rapidly increasing scale, complexity, and rate of change of business ecosystems, as well as economic and competitive pressures, analysts are faced with the formidable task of quickly understanding the fundamental characteristics of these interfirm networks. Existing tools, however, are predominantly query- or list-centric with limited interactive, exploratory capabilities. We have designed and implemented dotlink360, a web-based interactive visualization system that provides capabilities to gain systemic insight into the compositional, temporal, and connective characteristics of business ecosystems. dotlink360 consists of novel, multiple connected views enabling the analyst to explore, discover, and understand interfirm networks for a focal firm, specific market segments or countries, and the entire business ecosystem.

Drawing Apprentice: Co-Creative Drawing Partner

Collaboration is known to push creative boundaries and help individuals sustain creative engagement, explore a more diverse conceptual space, and synthesize new ideas. While the benefits of human collaboration may seem obvious, the cognitive mechanism and processes involved in open-ended improvisational collaboration are active areas of research. Our research group has developed a co-creative drawing partner called the Drawing Apprentice to investigate creative collaboration in the domain of abstract drawing. The Drawing Apprentice draws with users in real time by analyzing their input lines and responding with lines of its own. With this prototype, we study the interaction dynamics of artistic collaboration and explore how a co-creative agent might be designed to effectively collaborate with both novices and expert artists. The prototype serves as a technical probe to investigate new human-computer interaction concepts in this new domain of human-computer collaboration, such as methods of feedback to facilitate learning and coordination (for both the user and system), turn taking patterns, and the role control and ambiguity plays in effective collaboration.

Dressing Humans

We propose a general framework for character self-dressing interactions with simulated clothing. We show that by breaking the process of dressing into sub goals, we can design specific action controllers which, when combined allow a character to put on a garment via a user defined style.

Driving Georgia Tech: Creating a Driving Simulation of Georgia Tech's Campus

Applying driving simulators for in-vehicle research allows for a wide range of studies to be performed particularly when investigating cognitive demand and distraction caused by devices in the car. By using simulations, researchers can investigate driving behaviors in high-risk situations without putting participants or others in harmful way. Currently being conducted within the School of Psychology at Georgia Tech, in-vehicle research could provide more insight into behavior and increase in applicability if participants were able to drive in areas that they are familiar with. Specifically, research being done in coordination with the Atlanta Shepherd Center investigating the use of in-vehicle technologies to assist individuals who have had a Traumatic Brain Injury could benefit largely through these real location maps. The Georgia Tech School of Architecture coincidentally has already developed a 3D model of the Georgia Tech campus and some of the surrounding areas including the Peachtree corridor (26 miles along Peachtree Street). However, in order to make this model usable within the simulator, it must be optimized and converted in a compatible format. Researchers in the School of Architecture and School of Psychology will be working on creating methods and conversion processes that will allow any 3D model to be integrated into the simulator. Development of this process of conversion will allow Georgia Tech to offer documentation and map-creation services to other researchers around the world assisting in increasing the applicability of in-vehicle research.

Dynamic Difficulty Adjustment for Computer Games

Part of the fun of computer games is to master the skills necessary to complete the game. Challenge tailoring is the problem of matching the difficulty of skill-based events over the course of a game to a specific player’s abilities. We have devised a data-driven approach to predict changes in players’ skill mastery over time. By modeling players’ skill mastery, we are able to dynamically select game content that challenges individual players at the ideal level, avoiding frustration and boredom.

EarBit: Using Wearable Sensors to Detect Eating Episodes

We demonstrate a wearable system that detects eating instances in real time.

EarSketch Viz

EarSketch plus visualization and real-time interactivity.

EarSketch: Teaching Computer Science through Music Composition

Computational remixing of hip hop (i.e. using code to control loops and beats to compose music) can be used as a tool for the cultural engagement in computing of underrepresented populations. EarSketch is a digital audio workstation environment, with an accompanying curriculum, that will allow high school and summer workshop students to create their own computational remixes through learning computing principles.

Eating Disorders and Social Media - Characterizing the Presentation of Eating Disorders Online

Within the computing field, little has been done to systematically analyze online eating disorder (ED) communities. This research project focuses on understanding how individuals use social media platforms to promote and share their eating disorders with their networks and with the world. We use social computing techniques to identify and anzlye content generated across several popular social media platforms. Through this characterization of eating disorder activities online, we draw attention to the increasingly important role that technologists play in understanding how the platforms and technologies that we create are used and misappropriated for negative health purposes. CAUTION: This project includes media that could potentially be a trigger to those dealing with an eating disorder or with other self-injury illnesses. 

eCoach: Avatar-Guided Decision Aid for Prostate Cancer

This study employs gaming technologies and techniques to create an intelligent encapsulated conversational agent (ECA) to act as a virtual coach who will lower the cognitive effort required by prostate cancer patients to understand key aspects of decision-making, provide more appropriate reference points from which patients more accurately interpret personal risk, and frame information to optimize the patient’s chances of applying his own preferences and values to the decision at hand. A stylized, animated ECA will have a brief, focused conversation with a patient in order to explain, in layman's terms, the various treatment options and their risks and benefits and ask questions to assess the patient's medical literacy and values preferences, for example, the patient may value interventions with lower risk of side effects over being cancer-free.

The eCoach ECA is being developed with the Unity3D game engine and uses gaming AI tools such as behavior trees to model a dialog and ECA behavior. The patient will respond to each ECA question by selecting from among several predetermined answers and the history of patient answers will determine how the conversation unfolds. For example, if the ECA determines that the patient is not sure about the risks and benefits of the various treatment options, it will spend more time explaining what these are as well as ask questions to assess knowledge of them afterward.

This study represents a multidisciplinary collaboration between Emory University’s School of Medicine, the College of Computing and the Interactive Media Technology Center (IMTC) at the Georgia Institute of Technology.

ecoxight: Visual Business Ecosystem Intelligence

Business ecosystems are characterized by large, complex, and global networks of firms, often from many different market segments, all collaborating, partnering, and competing to create and deliver new products and services. Given the rapidly increasing scale, complexity, and rate of change of business ecosystems, as well as economic and competitive pressures, analysts are faced with the formidable task of quickly understanding the fundamental characteristics of these interfirm networks. Existing tools, however, are predominantly query- or list-centric with limited interactive, exploratory capabilities. We have designed and implemented ecoxight, a web-based interactive visualization system that provides capabilities to gain systemic insight into the compositional, temporal, and connective characteristics of business ecosystems. dotlink360 consists of novel, multiple connected views enabling the analyst to explore, discover, and understand interfirm networks for a focal firm, specific market segments or countries, and the entire business ecosystem.

Effects of Multimedia Interactivity on Spatial Task Learning Outcomes

Prior research has produced mixed results regarding the usefulness of interactivity in multimedia learning. In this study, participants learned to solve part of a Rubik’s Cube using either a tutorial with interactive features or a passive (video-based) tutorial. Participants with low spatial ability benefited more from interactivity than those with high ability, though no performance main effects were found between the tutorials. Targeted use of interactivity could be effective in engaging students and helping them learn.

Emotional Prosthetics

People with severe motor disabilities such as ALS may not be able to move their facial muscles to communicate. This study is examining the salient features of facial expressions in order to create "emotional prosthetics" - ways for people with disabilities to express emotion. The resulting prosthetics will be controlled by voluntary and involuntary brain signals.

Encounter - A Stat Trek Story

You are on a mission to foreign planets with the company of your captain. While landing on the planet in a transporter, you need to show hospitality to the alien representative with gestures since no translators to the alien culture are present. You need to mimic the gestures of the alien representative and establish a diplomatic relationship.

Enhanced In-Vehicle Technologies: Novel Interfaces and Advanced Auditory Cues to Decrease Driver Distraction

In-vehicle technologies such as modern radios, GPS devices, eco-driving displays, and smartphones require users to interact with multiple types of visual-based menus and lists while driving. Modern technologies require users to navigate these screens using physical buttons and touch screens, although recent advances have included the use of steering wheel buttons, turn wheels, Head Up Displays (HUDs) and others. Through design and prototyping of novel menu system interfaces through innovative visual display methods, interaction techniques, and the application of advanced auditory cues to old designs and these novel interfaces, we can attempt to decrease driver distraction, therefore allowing for better driving performance, while also improving search times and decreasing cognitive load on the driver.

Enhancements on A Tongue-Operated Robotic Rehabilitation System

Patients suffering from traumatic brain or spinal cord injuries may benefit from neuroplasticity guided and reinforced by motor learning feedback through reorganization of the neural pathways in intact parts of the brain and spinal cord. An enhanced version of a tongue-operated robotic rehabilitation system is presented for accelerating the rate of improvement in the upper extremity motor functions for patients with severe hemiparesis following stroke. A new rehabilitation robot, called Hand Mentor ProTM (HM) was utilized by reading its pressure and joint angle sensors and combining them with control commands from the Tongue Drive System (TDS) to enable both isometric and isotonic target-tracking tasks in a coordinated tongue-hand rehabilitation paradigm.

Enhancing the effectiveness of Impossible Spaces in VR

Impossible Spaces is a technique that uses self-overlapping architecture to incorporate natural walking in virtual environments without the use of any other movement techniques like teleportation or portals. I am showcasing some design interventions that when applied to self overlapping architecture enhance the believability of the space and might even lower the threshold of detection of the architectural manipulation. These design techniques can then be used by VR narrative developers to futher enhance the believability of their VR narratives. 

EOB Reader: Demystifying Explanation of Benefits

The United States’ medical billing system is exceptionally complex. Medical bills and Explanation of Benefits (EOB) statements are undecipherable and incomprehensible even for experts to understand. In addition, a 2015 survey conducted by TransUnion Healthcare found that 55% of American patients were either sometimes or always confused about their medical bills and that 61% of patients were either sometimes or always surprised about their out-of-pocket costs. Furthermore, forms of healthcare fraud including medical identity theft is one of the fastest growing crimes in the United States, costing the the nation approximately $30 billion a year. The goal of this research is to identify ways to simplify complexities within the EOB. Even though there are major problems which exist on the clinical and payer side of financial transactions including human error in medical coding, the issue considered in this project is empowering the end user to deal with the confusion and frustration involved in understanding one’s medical services. The proposed project aims to leverage principles of participatory design to build a mobile application which takes advantage of the latest OCR technology solution and prevent the relevant information via an easy-to-understand interface.

Epheno: Visual Sequencing of Enterprise Alliance Portfolios

In an increasingly global and competitive business landscape, firms must collaborate and partner with other firms to ensure survival, growth, and innovation. Understanding the evolutionary composition of a firm’s relationship portfolio and the underlying formation strategy is a difficult task given the multi­dimensional, temporal nature of the data. In collaboration with senior executives, we have designed and implemented an interactive visualization system that enables decision makers to gain both systemic (macro) and detailed (micro) insights into a firm’s relationship activities and discover patterns of multi­dimensional relationship formation. Our system provides sequential/temporal representation modes, a rich set of additive cross­linked filters, the ability to stack multiple enterprise genomes, and a dynamically updated Markov model visualization to inform decision makers of past and likely future strategy moves.

Epilepsy - Everyday epilepsy self-management tools for patients and families

Many patients and caregivers struggle to complete everyday epilepsy self-management practices: remembering to take daily medications, reporting seizure events and self-regulating behaviors such as getting enough sleep.

Jon Bidwell and Beth Mynatt are working with adolescents patients (11-18 years old), caregivers and clinicians from the Children's Healthcare of Atlanta (CHOA) to investigate how mobile and wearable health tracking technologies can support these everyday self-management needs.

    In the coming months we will be providing patients and caregivers with a number of mobile and wearable tools that include:

    • a mobile phone app for reporting seizures and health information,
    • a smartphone for detecting medication adherence,
    • a wristband for measuring seizures and sleep at night and a
    • a wristband for measuring daily activities and stress throughout the day

    The research will include four experimental conditions to investigate:

    • The use of mobile phones for collecting twice daily survey information and seizure reports,
    • The impact of "smart", context-sensitive reminders for completing daily surveys,
    • The impact of health tracking devices and health dashboards on survey response rates and
    • The impact of goal setting and daily financial rewards on survey response rates

    If successful this work will contribute to technology design implications for greatly improving upon current epilepsy self-management tools that are available to patients and families.


    Epilepsy - Health dashboard for remote patient outreach

    Health dashboards stand to help clinicians to identify patient challenges and contact patients between appointments. Many patients and caregivers struggle to complete epilepsy self-management practices: remembering to take daily medications, reporting seizure events and self-regulating behaviors such as getting enough sleep.

    Jon Bidwell and Beth Mynatt are working with attendings at the Children's Healthcare of Atlanta (CHOA) to develop a health dashboard for clinicians. The proposed health dashboard aims to help nurse practitioners review patient and caregiver collected health data, evaluate how well patients and families are keeping up with daily self-management practices and prioritize phone call follow-ups.

    In the coming months, patients and families will be given a range of mobile and wearable health tracking technologies. These technologies include:

    • a mobile phone app for reporting seizures and health information,
    • a smartphone for detecting medication adherence,
    • a wristband for measuring seizures and sleep at night and a
    • a wristband for measuring daily activities and stress throughout the day

    Healthcare professionals are using technologies to stay increasingly connected with patients and caregivers between appointments. This research seeks to help a small number of clinicians to reach a much larger group of patients.

    Epilepsy - Investigating neurocognitive clinician self-reporting needs

    Healthcare professionals rely heavily on patients and caregivers to self-report important health information during treatment. However, in practice, these self-reports are often inaccurate, incomplete and can even be misleading. Mobile and wearable technologies stand to help patients and caregivers to collect more accurate, consistent and reliable data.

    In this study, we investigated clinical self-reporting needs three neurocognitive fields of medicine: neurology, psychiatry and sleep medicine. In-person expert panel sessions were conducted with 14 clinicians (five epilepsy, four psychiatry, and five sleep medicine specialists) to establish the priority of different types of patient-reported data during diagnosis and treatment, respectively. Then we conducted online surveys with clinicians from the same specialty areas for further assessing the availability and quality of current patient and caregiver self-reporting data being collected.

    The results highlight several important yet underexplored data collection and design opportunities for supporting diagnoses, treatment and self-management within these three fields as well as expose gaps between clinical data needs and patient practices. The resulting findings stand to benefit from the development of technological tools that support patient data collection activities and shared decision making between patients and providers.

    Epilepsy - Retrospective video review for more accurate wearable seizure reporting

    Epilepsy treatment requires accurate seizure accounts between appointments for adjusting medications, however, this information is often either unavailable or inaccurate.

    • Most patients are unable to recognize seizures at night and therefore under-report seizures [1] and by contrast
    • Mobile and wearable seizure detection devices often over-report seizures due to high numbers of false alarms [2].

    In this study we're investigating patient and caregiver video review as an approach for addressing the shortcomings wearable technologies that may otherwise be applicable for long-term use in the home.

    • The study involves 16 pediatric and 16 adult patients who are being monitored at a hospital Epilepsy Monitoring Unit (EMU).
    • The patients are video recorded and wear a pair of seizure detection wristbands that detect possible seizure events.
    • The patient and caregivers then review video of these events and dismissing false alarms (e.g. video of head scratching or text messaging while in bed)   

    The results suggest that a video review can indeed improve the performance of current the wearable seizure reporting as a "second pass". To date we've seen near perfect agreement between patients/caregiver and electroencephalogram technicians.


    1. Hoppe, C., Poepel, A., and Elger, C.E. Epilepsy: accuracy of patient seizure counts. Archives of neurology 64, 11 (2007), 1595–1599.

    2. Van de Vel, Anouk, Kris Cuppens, Bert Bonroy, Milica Milosevic, Katrien Jansen, Sabine Van Huffel, Bart Vanrumste, Lieven Lagae, and Berten Ceulemans. “Non-EEG Seizure-Detection Systems and Potential SUDEP Prevention: State of the Art.” Seizure 22, no. 5 (June 2013): 345–55. doi:10.1016/j.seizure.2013.02.012.



    Epilepsy - Reviewing mobile apps for pediatric epilepsy self-management

    Health reporting plays an essential role in the diagnosis and treatment of epilepsy. Healthcare professionals currently rely on patients and caregivers to document a range of patient seizure symptoms and health behaviors; however, studies have shown that patients and caregivers struggle with these responsibilities. Inaccurate, incomplete or inconsistent information can impact clinical decision making and increase the time required to find an effective seizure control medication.

    Mobile and wearable technologies stand to help address these challenges by helping patients and caregivers with tools for collecting the types of health indicators. In this study, we surveyed clinicians and reviewed and compared the performance of existing seizure detection technologies to current patient self-reporting.

    The results from our survey showed that

    • Low-cost video could help clinicians during initial epilepsy diagnosis
    • Existing seizure detection devices work best for GTCs (only 30% of seizures)
    • Existing seizure detection devices are best suited for nighttime use when patients are less able to report seizures

    These findings helped to shape our current research efforts. Bidwell and Mynatt are developing mobile and wearable tools aimed at supporting everyday data collection for patients with epilepsy. 


    Bidwell, Jonathan, et al. "Seizure reporting technologies for epilepsy treatment: A review of clinical information needs and supporting technologies.Seizure 32 (2015): 109-117.


    Escape Room VR

    Moving from 2D and digital to 3D and virtual, Escape Room VR explores the opportunities for computers to communicate with humans more effectively in the medium of virtual reality. This is a short demo that will ignite your curiosity of your surroundings and encourage the discovery of playful interactions. Real-time, 3D, and highly interactive, are you ready to escape the room?

    Ethical Challenges in Algorithmic Inference of Mental Illness with Large-Scale Social Data

    Facebook recently unveiled a tool that assesses whether someone might be contemplating suicide or self injury – these predictions are made using an individual’s Facebook activity. As these new technologies and potentials emerge, questions also emerge about the methodologies these studies use as well as the ethical implications of such work. How should researchers provision social media data to make predictions? What methods and measurements are appropriate to use when creating these predictive algorithms? What rights might users have in these studies, and what other ethical implications arise? 

    This project explores the ethical implications for making individualized predictions on mental wellness using social data. We find three areas of unresolved ethical contention. First, we look at research design and ethics approval, addressing how ethic committees and IRBs handle such studies and how we define individuals in these studies. Second, we look at data collection, analysis, and methods in this field. Finally, we look at the implications and outcomes of these algorithms when applied to clinical contexts. We consider how these methods impact clinicians, researchers, and the users/patients themselves. Finally, we conclude with three calls for action in the field to set standards and propose strategies to address these ethical dilemmas.

    By understanding the intricacies of the ethical challenges of designing, developing, and deploying these algorithms from start to finish, we hope that our analyses and calls to action invite deeper conversations. We hope this discussion prompts principled actions from a diverse set of stakeholders to be taken to address these issues in this emergent field.

    Evaluating the Effectiveness of a new Lecture Aid

    This research project is a MSCHI 2nd year Masters project that attempts to design a wearable device that will reduce distraction in classrooms by making it easier for professors to deal with technology issues that may occur (e.x. The wifi cutting out) in a way that will help them maintain focus on the subject matter of the class.

    Examining behavioral markers leading to social media disclosures on schizophrenia

    Mental illness such as psychosis and schizophrenia are serious public health concerns. However, timely detection of an episode of psychosis is often difficult due to several reasons such as social stigma, lack of mental health awareness and literacy, and the retrospective nature of clinical therapy. We examine the potential of leveraging social media disclosures as a new kind of lens in characterizing and predicting experiences leading up to a psychotic episode. In contrast to self-report methodology, where responses typically comprise of recollection of (subjective) health facts, social media captures behavior and language in a naturalistic setting. This gives us access to real-time activity and psychological states that can be analyzed to discover and predict behavioral markers associated with a psychotic episode. With an initial dataset of 11,000 tweets which disclose symptoms of psychosis such as hearing voices, having delusions, schizophrenia etc., we develop a computational method to identify behavioral and linguistic markers that attribute to an episode of psychosis. Further, in collaboration with clinical psychologists, we examine specific user timelines that include mentions of relapse or hospitalization. Based on the data analysis, we aim at building a prediction model to identify prospective behavioral markers leading to an episode. We believe information derived from our prediction model can be valuable to clinical psychiatrists in facilitating timely diagnosis.

    Exergames for Older Adults

    Exergames -- video games played by engaging in physical activity -- could help older adults become more physically active. However, most exergames on the market are not developed with consideration of older adult users’ physical and cognitive abilities. Our current study is evaluating the usability of commercially available exergames for this population by testing two exergames for Microsoft Xbox 360 with Kinect with participants aged 60 to 79. These findings will be leveraged to develop guidelines for designing a tutorial to teach older adults to use exergames.

    Explainable AI

    In the near future, autonomous and semi-autonomous systems will interact with us with greater frequency. When they fail or perform unexpected behaviors, non-experts must be able to determine what went wrong. We introduce “rationalization”, a technique for automatically generating natural language explanations as if another human were describing what the autonomous system was doing. We demonstrate rationalization in the test-bed domain of the Frogger game. 
    Video: https://www.youtube.com/watch?v=vXcuLEBwXsQ

    Exploring Complex Systems Through Interactive Games by Leveraging the Diseases of Chronically Ill Children

    We developed a suite of educational digital games for chronically ill children that help us build upon the learning science theory of islands of expertise and contributes to educational game studies by exploring a interrelated core set of games that can teach specific science, technology, engineering and mathematics (STEM) content in the context of complex systems. In this study, we interviewed 20 children with chronic disease and gathered feedback about the mini games developed

    Fan Funhouse

    Fan Funhouse is a browser-based video remixing application that allows users to edit webcam videos using a palette of effects inspired by a pop culture franchise. The increasing prevalence of user-generated media production in apps and on the web has coincided with pop culture brands, ranging from The Powerpuff Girls to Peanuts, providing fans with opportunities to quickly create and share personalized "fan media" in their web browsers. While most of these fan media experiences have involved the production of images or GIFs, Fan Funhouse gives users creative agency to easily remix short video clips in the style of brands they love. The test scenario for Fan Funhouse features the Adult Swim comedy duo Tim & Eric, who are known for their retro, lo-fi aesthetic. In the Tim & Eric Fan Funhouse demo, users can apply whimsical effects to make their own videos look like Tim & Eric sketches.

    FIDO - Facilitating Interactions for Dogs with Occupations

    The FIDO Sensors team is creating wearable technology to allow working dogs to communicate. Assistance dogs can tell their owners with hearing impairments what sounds they have heard; guide dogs can tell their owners if there is something in their path that must be avoided. We will be demonstrating a variety of scenarios with five wearable sensors designed for dogs to activate.

    Flex: Connected Home Integration

    In 2014, Flextronics (now Flex) came to Georgia Tech with an interest in integrating and testing devices in our authentic home environment (The Aware Home). They were at a stage in their development of the Wink Hub where they needed a home environment to test ranges and reliability, as well as show clients how their products would integrate into a home environment. The Wink Hub is now available as a Do It Yourself solutions for the connected home, enabling  transfer of messages between in-home devices and the Wink cloud. Devices from different manufacturers with their own dedicated app, could now be integrated with the Wink app to provide a more connected consumer experience e.g. locking the front door lock could trigger light switches to turn off and blinds to close. During this early phase, Georgia Tech students helped with testing the Wink Hub in various locations around the Aware Home to ensure reliability of adding, removing and controlling devices. Since this time, Flex has continued to expand their expertise in designing and manufacturing connected home/living products.

    Aware Home researchers have collaborated with Flex to consider the future of the connected home environment and helped to educate their clients on how connected living solutions with greater data intelligence could improve the lives of residents, including solutions that targeted at energy resource management, independence, health and home management.

    Flow Medtech

    Currently, 4.4 million Americans have been diagnosed with atrial fibrillation (AF), in which the heart beats in an irregular rhythmic pattern. That number is estimated to reach 12-16 million by the year 2050. Patients with atrial fibrillation have over a fivefold increase in the chance of stroke. Due to complications from the current standard of care, anticoagulants (i.e. blood thinning drugs), to treat resulting thromboembolism (i.e. clotting) from AF, alternative treatments are actively being sought out to decrease complications and risk of stroke. With 90% of clots found in the brain originating from the left atrial appendage (LAA), a fingerlike projection off the heart that rarely contracts in AF patients, LAA closure devices have become an increasingly attractive option, but current options undergoing FDA approval are considered by cardiologists to be first generation devices that need to be improved upon due to limited functionality. Flow Medtech is currently developing occlusal technology that fully blocks the LAA from thromboembolism, features customizability to conform to the unique shapes and sizes of individual LAAs, and uses a secure anchoring system to prevent migration. These features will prohibit thromboembolism in the LAA, and thus, will significantly reduce the risk of stroke in AF patients.

    Food For Thought: Developing a Cognitive Training Game for Older Adults

    Over the past 2 years, we have performed experiments to understand what activities within a video game context result in cognitive gains (and which do not). From these findings, we have developed a custom cognitive game called "Food for Thought."

    The specific goals of this research program are to: understand how video games can contribute to improvements in cognition, what properties of the gaming environment (novelty, active attention, and/or social interaction) are critical for cognitive improvement, create an older adult specific game that leverages the critical properties identified empirically, and test the efficacy of this theoretically designed game to produce the largest gains in the cognitive performance of older adults.

    Function of Educational Technology

    Much of the research on educational technology (e.g., MOOCs and adaptive learning systems) has been driven by the capabilities of technology instead of the pedagogy and cognition of learners. Our research takes the opposite approach. A review of the literature on educational technology and instructional methods for teaching STEM courses was used to identify the strengths of technology in education. These findings being used to develop educational technology and provide heuristics and guidelines for developing effective STEM courses that optimally support learning.

    G.L.I.M. - Glass Live Interaction Monitor

    Successful social interactions are essential to enhance our quality of life. Being aware of our own internal emotional state as we interact with other individuals maximizes our chances of effectively co-regulating with them, and therefore enhancing the quality of our interactions. The challenge we face regarding self-regulation lies in the fact that in today’s busy world often times we become so overwhelmed that we lose the ability to read our own internal emotional states hindering our ability of self – regulate potentially hurting the quality of our interactions.

    Given that strong evidence indicates that in successful social interactions, synchrony occurs at the physiological level between individuals as they interact, we are introducing G.L.I.M. (Glass Live Interaction Monitor), a system that helps the user self-regulate in situ by leveraging the combination of 3 comfortable, wireless, non-obtrusive and wearable devices.

    G.L.I.M. components are: a) Google Glass, b) Electrodermal Activity (EDA) / Galvanic Skin Response (GSR) , c) Heart Rate Monitor, and d) Laptop application for offline analysis and self-reflection.

    Case Study: Given that interacting with Children with Autism (CWA) can probe to be challenging, and that their primary bridges to the world are their parents and therapists, it is essential for them to be adequately self-regulated in order to maximize the quality of their social interactions. G.L.I.M. offers the possibility of optimizing parental and therapist internal self-awareness while interacting in their natural environment. In a second stage we are also planning to instrument the CWA so we can monitor both the caregiver and CWA internal states in situ, providing a even deeper insight on how the interaction flows. Gaining knowledge of both interacting partners invisible physiological signal can probe to be essential in providing strategies oriented to maximize the quality of their relationship.

    Game Based CAPTCHA

    A CAPTCHA is a challenge-response test used on the Internet to prevent bots from accessing web services that are designed for humans. We are investigating Automatic Game based CAPTCHA Generation (AGCG), in which an AI system generates games that, when played, distinguish between humans and bots. The game based CAPTCHA takes advantage of not only the bots' difficulty performing pattern/object recognition, but also their lack of commonsense knowledge. Thus it is more secure but remains easy and fun for humans, compared to traditional visual based CAPTCHAs. Furthermore, our AGCG system is capable of learning new commonsense knowledge based on users' response in the game based CAPTCHAs.

    Game Mechanics for Games With a Purpose

    Games with a purpose (GWAPs) have proven to be effective solutions to solving difficult problems, labeling data, and collecting commonsense knowledge. Unlike traditional games, GWAPs must balance between acquiring accurate solutions or data and maintaining player engagement. However, when it comes to designing GWAPs, the effects of different game mechanics on accuracy and engagement are not well understood. We have conducted two studies to understand the way different choices of game mechanics and their affect on player behavior. The first study (Cabbage Quest) compares cooperative and collaborative game mechanics. The second study (Gwappy Bird) compares different difficulty levels.

    Game of Game of Thrones

    The complexity of television shows has been increasing.  In order to follow a story, viewers might be expected to stay abreast of more plot threads, remember more characters, and retain information introduced in earlier seasons.  Media technology has made the job easier by allowing viewers to review in various ways; they may replay a scene or entire episodes, they may visit an online forum for fans, or they may play a video game that is related to that story.

    Game of Game of Thrones is a video game design intended to explore how a video game could enhance the viewing of a television series.  Specifically, what could be accomplished by an episodic game whose episodes are interleaved with series episodes?  Our design was guided by two goals: 1) help viewers cope with increasing information requirements and 2) offer an additional dramatic layer to the series, one that could be harmlessly eschewed by viewers who don't care to play a game.  

    Game of Thrones Companion

    The transition from novel to television creates the issue of compressing story into episodes limited by time and budgets. HBO’s translation of Game of Thrones is a rich tapestry of characters and narratives; however, the viewer can lack back story, geographic awareness, and an understanding of character relationships. This second-screen companion app orients viewers to the world of Westeros by mapping families throughout episodes. Greater character understanding is achieved by the mapping of character relationships, both during characters present in each scene or within the episode.

    Games for Assessment

    We have a multi-year project exploring how game performance and player behavior can be used to perform scientifically valid cognitive, personality, skill, and behavioral measures. This project involves hypothesizing about how game mechanics, levels, situations etc. could assess aspects of player that are currently measured via validated traditional tests/activities/interviews, designing games around these hypotheses, and running user studies. Another aspect of this work is exploring how theming, feedback, game type influence the assessment validity and the players' desire to play the game.

    Garment Based Body-Position Monitoring

    Georgia Tech and the Human Interface Branch of NASA partnered together to find a way to detect astronauts’ body positions in space. In the zero gravity space environment it becomes difficult to monitor tasks that lead to repetitive stress injuries or fatigue. Monitoring movement would help NASA pinpoint high stress actions and make adjustments to corresponding mission tasks.We developed an unobtrusive, textile based system to monitor astronauts’ arm position in real time, in zero gravity, and without the constraints of camera based motion-input devices.

    Georgia Brew Finder

    A textual visualization that analyzes and presents beer review data to allow users to find beer that meets their tastes.

    Getting it Right the First Time - Performance Guarantees for Robot Behavior

    Robotics has been considered as one of the five key technology areas for defense against attacks with weapons of mass destruction (WMD). However, due to the mass impact nature of WMD, failures of counter-WMD (C-WMD) missions can have catastrophic consequences. To ensure robots’ success in carrying out C-WMD missions, we have developed a novel verification framework in providing performance guarantees for behavior-based and probabilistic robot algorithms in complex real-world environments. We cannot assume the luxury of a do-over; we must get it right the first time.

    Giants in the Sky: The life of stars

    Giants in the sky is a Tangible User Interface (TUI) that explores the role of mass and gravity in the life and death of exosolar systems. With the use of various tangibles with different physical attributes, this TUI aims to teach basic concepts of astronomy in science museums. These tangibles allow users to create and manipulate digital celestial objects in a sandbox simulation.

    Gleaning in Atlanta

    Gleaning is the practice of salvaging food left over from its intended use. Our research delved into the activities of gleaning with an emphasis on the tools used in gleaning. From this research we identified a series of design opportunities. Perhaps the most fertile opportunities are related to socio-technical networking: the processes and infrastructures for providing information about the availability of food for gleaning and access to the actors who can move and store gleaned food. 

    GloHood: An interactive toolkit for dance

    The gloHood is a wearable technology garment that amplifies and augments the expressive movement of a dancer. It provides the novice audience with an available affordance to better appreciate and understand modern dance, and the dancers new tools to better communicate with the audience, with each other, and with themselves. The garment provides the dancer with a gesture control interface through embedded RFID tags, and a motion control interface through accelerometers sewn into the garment.Each of these can trigger playback of animated light patterns on an array for LEDs arranged over the neck and shoulders. This allows the dancer control over the garment while in use, and the ability to enhance his movement.This garment was designed and tested in collaboration with local dance troupe gloATL.

    GloSkirt: An interactive garment for dance performance

    The gloSkirt is a wearable technology garment design for Mary Jane Pennington of dance troupe gloATL. The team wanted to give her an experimental tool to challenger her own movement style and better engage audiences new to dance. A base layer of LEDs responds to resistive sensors embedded within layer of the skirt, causing the garment to ‘pulse’ and ‘breathe’ as the dancer crushes and separates with her movements.

    Goodbye Text, Hello Emoji

    Lab: TanDEm

    This project is a qualitative study of non-textual mobile communication practices in Southern China. Examining the rapid proliferation of emoji in WeChat use, we attend to the lessening dependence on text. We use interview and observation data from 30 participants to investigate how rural, small town, and urban Chinese adults creatively and innovatively balance the use of emoji and text in their communication, as we envision the evolution of emoji into a modality of its own. We look into various possibilities for future work to explore circumventing the prerequisite of print literacy for mobile communication, especially for low-literate populations.

    Graphiti: Interactive Specification of Attribute-based Edges for Network Modeling and Visualization

    Network visualizations, often in the form of node-link diagrams, are an effective means to understand relationships between entities, discover entities with interesting characteristics, and to identify clusters. While several existing tools allow users to visualize pre-defined networks, creating these networks from raw data remains a challenging task, often requiring users to program custom scripts or write complex SQL commands. Some existing tools also allow users to both visualize and model networks. Interaction techniques adopted by these tools often assume users know the exact conditions for defining edges in the resulting networks. This assumption may not always hold true, however. In cases where users do not know much about attributes in the dataset or when there are several attributes to choose from, users may not know which attributes they could use to formulate linking conditions. We propose an alternate interaction technique to model networks that allows users to demonstrate to the system a subset of nodes and links they wish to see in the resulting network. The system, in response, recommends conditions that can be used to model networks based on the specified nodes and links. In this paper, we show how such a demonstration-based interaction technique can be used to model networks by employing it in a prototype tool, Graphiti. Through multiple usage scenarios, we show how Graphiti not only allows users to model networks from a tabular dataset but also facilitates updating a pre-defined network with additional edge types.

    Grocery Pool - A ride sharing system to help students in food deserts get better access to food

    Despite growing awareness of the term food desert millions of people still have poor access to healthy food. The focus of the research is to help students living in food deserts get better access to grocery stores. Grocery Pool is a mobile application that students can use to collaborate and plan trips to grocery stores. 

    GT Art Crawl

    The annual Clough Commons Art Crawl serves as a unique opportunity for Georgia Tech students to close their books, catch their breath, and enjoy the therapeutic effects of art. The blank walls of the Clough Commons will once again be transformed into a make-shift gallery, all centered around the artistic work of Georgia Tech students.

    The RNOC has built the companion app for the Art Crawl utilizing Augmented Reality technologies and the RNOC's Dev Hub platform

    GT Journey

    GTJourney is an opportunity for all members of the Georgia Tech community to collaborate on applications and solutions that benefit the campus. It is a virtual focal point for students, faculty, and staff to develop ideas and solutions, find technical support and resources, advertise and access campus data, and share applications and experiences.


    GTMobile is a web portal, built and maintained by the GT-RNOC, for the
    deployment of web applications. GTMobile is meant to be a resource that
    benefits the Georgia Tech community by providing a place where any
    student, staff, alumni & faculty can host their application or service.
    GTMobile features capabilities such as integration with Campus
    authentication and authorization to ensure applications and services can
    be differentiated and offered to the active GT community or the public.
    GTMobile is also the showcase for the winning entries of Georgia Tech’s
    Fall Convergence Innovation Competition (cic.gatech.edu).
    GTMobile is open to the entire GT community and all are encouraged to host
    their applications on this portal and ensure that GTMobile is the
    continued singular web point of presence for GT based services.

    GTMobile on Glass

    This glassware is designed for the Georgia Tech campus community and visitors. It uses your location information to help you know what buildings are nearby as well as find the nearest bus stop. This demos how easy it is to leverage our existing APIs and resources in order to support new platforms and development.

    Gundam VR: Mobile Suit Agency

    Gundam VR is a virtual reality adaptation of the Japanese animated television show Mobile Suit Gundam: Iron-Blooded Orphans. In this virtual reality experience, you will take the role of protagonist Mikazuki Augus, a young soldier that pilots a giant robot, known as a Gundam, in battle as a mercenary. Left paralyzed on his right side due to the physical strain from piloting his Gundam, Mikazuki is only able to control his entire body when plugged into his robot, the Barbatos. The scenario has Mikazuki transport himself to his Gundam through a hangar while still paralyzed on his left side only to regain his bodily autonomy once he is plugged into the Barbatos and ready for combat. This virtual reality experience asks the question of how the giving and taking away of agency in VR can be used to simulate physical impairment.

    GVU Prototyping Lab

    Come see the tools that we use to create one-of-a-kind research prototypes. We have everything from laser cutters and 3D printers to table saws and soldering irons, and we use them to create many of the custom electronics, cases, and wearable prototypes you see in our demos.

    The Prototyping Lab is located in the basement of the building, so just look for signs by the elevators to go down there, or meet by the elevators on the 2nd floor every quarter hour on the quarter hour to get a tour.

    Haptic Mirror Therapy Gloves: Aiding the treatment of a paretic lib after a stroke

    “Haptic Mirror Therapy Glove” is an interactive mirror therapy glove for the treatment of a paretic limb following a stroke.  It allows the user to stimulate the fingertips of their effected hand by tapping the fingers of their unaffected hand using force sensing resistors to trigger linear resonance actuators on the corresponding fingers.  The glove may potentially be useful to stroke survivors and their therapists by encouraging the development of new multi-sensory rehabilitation exercises, which might better help recover lost sensation and strength in their fingers.This project was selected as the “Best Functional Design” at the 2013 International Symposium on Wearable Computing in Zurich, Switzerland.

    Harry PottAR

    An augmented reality mobile application that brings the wizarding world of Harry Potter to the real world for the purpose of answering the following research question: How do the following factors - timers, audio, interacting with virtual objects in the real world, interacting with real objects in the virtual world - increase or decrease a user's motivation to follow an interactive location-based narrative? This project will inform a set of design guidelines for motivating users to follow an interactive location-based narrative.

    Hashtag Adoption in the Pro-Eating Disorder Community on Instagram

    It is true that social networking has been a powerful force for good; however, these sites have also enabled sharing and connectivity for more nefarious purposes. Specifically, the Internet connects people in ways that can enable and amplify the destructive power of eating disorders (EDs). Some pro-ED communities have emerged that support users' choices of self harm as a reasonable lifestyle alternative. These communities are not only dangerous for those with EDs but also for potential contagious effects of these communities on those who don’t already have these behaviors. Instagram, the photo and video sharing site, has taken proactive steps to block hashtags associated with eating disorders, yet the pro-ED community works around these bans by create new hashtags with lexical permutations to congrea

    My research examines the formation of the pro-ED community on Instagram around these hashtags and the life cycles for these hashtags. I hope to examine questions such as: what categories of lexical permutations are created for banned hashtags? What characteristics of a hashtag make it "stick" and used around the network? Can we predict what lexical characteristics make a hashtag better or worse at avoiding detection and connecting the pro-ED community together?



    Heads-Up is a Google Glass prototype that functions as a translucent second screen over the television while re-watching a favorite show. Our demo will focus on HBO’s highly acclaimed series, Game of Thrones. The intention of the design is to allow the user to continue viewing the TV screen while receiving synchronized commentary through Google Glass rather than be distracted away from the screen by a computer, phone, or tablet.

    Health Observatory in Atlanta

    This project visualizes health data within the Atlanta metro region. Although some research about health inequities among this region has occurred, it typically is based on county-level data. In order to have a better understanding of health inequities and disparities in our home area, a city profile for Atlanta should be established. This project has created an interactive visualization of data such as rates of teen pregnancies, low birthweight babies, etc. The system allows the viewer to explore correlations among the different variables.

    HealthSmart: A Mobile Personal Health Record for Behavioral Health Homes

    Poor quality of medical care is a major contributor to excess medical morbidity and premature mortality in persons with serious mental illnesses (SMI). To address this problem, community mental health providers are increasingly partnering with safety net medical providers to develop behavioral health homes, integrated clinics in which persons with SMI receive coordinated medical and mental health care. However, behavioral health homes have faced logistical and privacy challenges in integrating electronic medical records across organizations.

    This application proposes to develop and test a mobile Personal Health Record (mPHR) to overcome this problem while more fully engaging patients in their health care. The mPHR will have the capability to access medical and mental health medication and lab data in real time; to help clients set and maintain health and lifestyle goals; to provide medication and appointment prompts and reminders; and to facilitate communication with providers via asynchronous communication with the EHRs.

    This project is a collaboration with Emory University's Center for Behavioral Health Policy Studies.

    How Cultural Background Influences Western and Eastern MMOG Players in World of Warcraft

    The purpose of this study is to investigate how cultural background influences Western and Eastern MMOG players in the case of WOW.
    This thesis explores the influence of culture on MMOG players in three different cultural contexts: United States (US) servers, Chinese (CN) servers, and Taiwanese (TW) servers. This comparison will allow a comparison of Western vs. Eastern players as well as two similar Eastern cultures that play slightly different versions of the game. This comparison will also show a distinction among the similarities and differences of culturally-influenced behaviors as well as behaviors that arise out of specific game features. This paper will specifically focus on identifying the differences among these three cultures and the unique aspects of players of different cultural backgrounds that alter the atmosphere of the game. In addition, this study will also examine how Chinese WOW players who have virtually “immigrated” from Chinese servers to Taiwanese servers have influenced the local game culture on Taiwanese servers. This study will cover the following research questions:
    Does real world culture influence the form of the virtual world culture?
    Sub-questions of this main question are as follows:
    What aspects of their cultures do players bring from their own lives and how do they incorporate them into their game behavior?
    Does the behavior of players in different cultures reveal different values and attitudes in the game?
    When players immigrate to other servers, what are some habits and behaviors from their original servers do they bring to the new homeland servers?

    How does social network help people adhere to their fitness regime

    More and more people today are using activity trackers like Fitbit and connecting them to their social media like Twitter. And what more, some people make their daily quantified activity time series public. So on one hand, we have their entire twitter network in the form of timelines, friends, followers and their entire network and on the other hand, we have their entire workout data. We are trying to answer some interesting question from these two sets of data. What is the effect of social network on one health regime? Is there any correlation between the number of times a person posts about health and the average workout the person does. Previous research has shown that having friends who are also health conscious actually increases one's tendency to adhere to health regimes. We are trying answer how does weak and strong ties of health conscious and non-health conscious friends and followers affect a person's adherence to health regimens. Previous research shows support in the form of retweets, comments, loves of a users' health tweet actually motivates him to continue using quantified health devices, but now with the exact data of a person's workout, we are trying to quantify this motivation.

    Human body mediated sensing

    We explore how to use human body as a sensing medium and what novel applications can be developed. 

    Human-guided Task Transfer in Interactive Robots

    As robots become more commonplace, they will need to address a wide variety of problems. Since a robot cannot be programmed to complete every task, it is necessary for robots to learn new tasks by interacting with a human teacher. Current methods require that the robot receive many demonstrations of a task, or they are limited to completing tasks which are nearly identical to previous demonstrations. We are developing a cognitive system based on case-based analogical learning that may enable a robot to collaborate with a human teacher to transfer task knowledge to a range of target problems.

    IAL - Interactive Analytics Library

    User interaction is central to the data analysis process fostered by interactive visual analytic interfaces. However, in many current systems, user interaction is represented as an ephemeral action taken by a user that moves the system from one state to another. User interactions are quantitative bits of the analytic dialog between people, the system, and the data - and when modeled - can be tactfully integrated into visual analytic systems. We propose a library to help researchers and developers capture, interpret, and model interactions in web-based visual analytic tools. We introduce Interactive Analytics Library, a JavaScript library which enables developers create data models of a user's interest based on their interactions with the system. By encapsulating interaction as an attribute of the data, managing weight vectors, and providing analytical models pre-tuned to generate results tailored to user interest, Interactive Analytics Library offloads responsibilities from developers of visual analytics so that they can focus more on the data representation and other front end system components.

    ICTs for HIV outreach work in India

    Lab: TanDEm

    HIV in India is a widely prevalent and complex disease. Treatment is an issue for the economically disadvantaged for financial and social reasons. When designing technology for this context it is important to remember not just the technological requirements but also the effects of stigma and other such contextual factors. NGOs play a key role in the dissemination of awareness and treatment. Thus the outreach workers at the NGO are an important aspect of connecting with the PLHIV.

    This project is the result of ethnographic research work done over summer '16 and spring '17. Based on the research findings, we are designing technology that can be used effectively by the outreach workers.




    Identifying Human Trafficking Online

    We are looking at identifying instances of Human Trafficking on online marketplaces and review sites. Using textual and visual cues, we are training a classifier to predict likelihood of being trafficked. 

    In-context Motion Gesture Design

    Motion gestures can be expressive, fast to access and perform, and facilitated by ubiquitous inertial sensors. However, implementing a gesture recognizer requires substantial programming and pattern recognition expertise. Although several graphical desktop-based tools lower the threshold of development, they do not support ad-hoc development in naturalistic settings. We present a mobile tool for in-context motion gesture design. Our tool allows interaction designers to create and test motion gestures using inertial sensors in commodity and custom devices. Therefore, our tool encourages development of gestures with common as well as atypical body parts. Moreover, the data collection, design, and evaluation of envisioned gestural interactions can now occur within the context of its use. 

    In-Vehicle Assistive Technologies

    There are many populations who need assistive technologies while driving such as the millions of Americans suffer traumatic brain injuries each year, and the majority of them return to driving at some point following their recovery. However, the residual effects of TBIs can affect perception, cognition, emotion, and motor abilities. In collaboration with the Shepherd Center we are developing software that can help improve the attention and abilities of drivers post-TBI. The system could help all kinds of drivers who may have attention lapses, cognitive processing issues, or other issues that impact driving. Similar types of applications could be built for many other types of issues as well (e.g., novice drivers, aging adults, & quote stressed out drivers).


    The {egg} Incubator group focuses on research, analysis, and rapid prototyping of emergent gameplay. Several, exhiting prototypes were build to test the theories of emergent gameplay.

    Information Seeking Practices of Parents: Exploring Social Networks, Fears and Challenges

    Through findings from over 60 interviews and a national online survey with 978 with diverse groups of parents, we explored parents’ ability to find learning opportunities, we identify differences in parents’ use of online social networks in finding learning opportunities for their children across different socioeconomics.

    Information Visualization on Tablets and Mobile Platforms

    Visualization has an important role in science and technology. People rely on visualizations to better understand problems they have to solve. Information visualization has recently increased its domain, from being used for representations of business data, to more political and social uses via groups like visualizing.org and infosthetics.com. In parallel with this growth we have seen the widespread acceptance of mobile technology by masses. Mobile phones, today, are being used for everything from email to ticketing and web browsing to watching videos. As society becomes more mobile, it is important to consider the application of information visualization on mobile and other touch based devices. The aim of this project is to understand if and how traditional information visualization techniques like line charts, bar graphs, and treemaps can be useful in a mobile environment and what the best style of interaction with those charts should be.

    Input for Virtual Reality Environments

    A series of demonstrations of novel interactions with virtual reality systems.

    Insights into Psychological Wellbeing and Urban Crime Via Social Media

    This ongoing study investigates the effect that proximate criminal activity has on emotional expression in social media. Proximity to crime and well constant fear of crime can have great negative psychological effects on individuals. Social media currently being one of the most popular means of publicly expressing personal opinions and emotions, we expect to find an effect of temporal and spatial proximity to crime on social media mood expression and other patters of online communication. Moreover, we expect that the use of certain terms related to crime will have different emotional connotations that correlate with the baseline criminal level of activity in the area.

    Instagram Content Moderation in Pro-Eating Disorder Communities

    The existence of pro-eating disorder (pro-ED) communities has challenged many social media platforms, such as Instagram. These communities promote the adoption and progression of eating disorders, which are known to have negative impacts on health. Instagram has reacted by banning searches on several pro-ED tags as well as issuing content advisories on others. In response, the pro-ED community has adopted non-standard lexical variations of these moderated tags to circumvent restrictions. This research investigates the impacts of Instagram banning tags on the community. Our work analyzes how the pro-ED community changes what tags it uses to avoid detection, what topics are discussed before and after banning, and what intervention and design strategies can be taken to assist these populations.

    InSync - Companion App for Live Sports

    Exploring experiences for a real-time companion device application to enhance live sports-watching experiences. The application facilitates for active social engagement typically of sports and assists in understanding the dynamics of the match more efficiently.

    Intelligent Biologically Inspired Design

    Engineers are interested in solving some practical problem, such as capturing water vapor in arid regions. They intend to use biological analogs as a source of inspiration. Doing so, however, raises several problems. First, the biological literature is immense, and keyword-based search produce voluminous hits, but most of the retrieved documents are of no use. That is, there is high recall but low precision. The second problem is that even when a document looks like it might be relevant, it is still burdensome for the engineer to make a determination. There are several reasons for this sub-problem. One is that the vocabularies used by biologists and engineers are quite different, even when the actual problem being solved is the same. Also, a significant part of biology articles is devoted to descriptions of experimental procedures, theoretical justification and related work, rather than specific to the biological process being described. It is therefore doubly difficult for the engineers to derive what they need from the papers. The third difficulty with a keyword-based approach is that what is really being searched for is often only analogically related to the problem being solved rather than being directly expressed in the biology article. For example, the archetypical biological inspiration is Velcro, in which the engineering problem being solved is attachment, whereas the biological problem being solved by the burr is seed propagation. Keyword-based search is ineffectual where analogies are involved.

    IBID directly addresses these three problems. It uses off-the-shelf NLP tools to parse documents, but goes further by semantically analyzing them. In particular, both the query and target documents are mapped into a common vocabulary, using a taxonomy of function concepts we have developed. That is, we have developed a taxonomy of concepts related to the functioning of complex systems, including both biological and mechanical ones. The ontology serves to bridge the vocabulary gap between the biologist and the engineer.

    Interaction Techniques for Children's AR Education

    Augmented-reality is a technology that can revolutionize children's education and entertainment. In studies of adolescents and adults, it has been shown to have measurable benefits for advancing STEM education through situated 3D simulations, providing entertainment through whole-body interaction, and enhancing physical & cognitive rehabilitation through motivational engagement. 

    We are interested in bringing such experiences into the hands of elementary-school children. In this project we are studying young children's ability to effectively use various types of handheld-AR interfaces. Handheld-AR interfaces are different from more traditional interfaces, by being small portable windows into physical spaces augmented with digital content, and their use may require more complex motor and cognitive skills than compared to traditional interfaces. Due to the novelty of handheld-AR technology, there are no standard interaction techniques for handheld AR, and little is known about children's ability to use these interfaces. 

    Through this research we are generating guidelines for technology designers, answering questions such as: What kinds of handheld-AR interaction techniques are suitable for young children? To what degree does age influence children's ability to interact with handheld-AR interfaces? What are best practices for designing handheld-AR interfaces for children ?

    Interactive Improv Storytelling AI

    Improvisational storytelling involves one or more people interacting in real-time to create a story without advanced notice of topic or theme. Human improvisation occurs in an open-world that can be in any state and characters can perform any behaviors expressible through natural language. In this project, we strive toward the grand challenge of computational improvisational storytelling in open-world domains. The goal is to develop an intelligent agent that can sensibly co-create a story with one or more humans through natural language.

    Interesting Details in Science Videos: Seductive Detail or Learning Aid?

    Scientists disagree about the effect of adding emotionally interesting details to learning materials. While some argue that interesting information enhances learning, others contend that interesting information is distracting. However, the issue might not be the interestingness of the information, but rather the relevance of the details to the main idea.

    Intersectional HCI

    Understanding users becomes increasingly complicated when we grapple with various overlapping attributes of an individual’s identity. As a term, the user now represents an expanding, diverse set of people. In this work we introduce intersectionality as a framework for engaging with the complexity of users’—and authors’—identities, and situating these identities in relation to their contextual surroundings. We conducted a meta-review of identity representation in the CHI proceedings, collecting a corpus of 140 manuscripts on gender, ethnicity, race, class, and sexuality published between 1982-2016. Drawing on this corpus, we analyze how identity is constructed and represented in CHI research to examine intersectionality in a human-computer interaction (HCI) context.

    We chose intersectionality, a framework that focuses on how various dimensions of identity (e.g., gender, race, and class) coalesce inseparably and relate to the conditions of one’s surroundings, because it supports efforts to situate the relationship between technology and social systems. In situating these relationships, we believe this work can help HCI’s broader agenda to do the right thing, within and outside third wave research. Our goal is to provide HCI researchers with empirical insight into current identity representation practices in CHI as well as to develop principled insights and recommendations for advancing the representation of identity in HCI.

    We find that previous identity-focused research tends to analyze one facet of identity at a time. Further, research on ethnicity and race lags behind research on gender and socio-economic class. From these findings, we developed recommendations for incorporating intersectionality in HCI research broadly, encouraging clear reporting of context and demographic information, inclusion of author disclosures, and deeper engagement with identity complexities.

    Introducing inspirit: Virtual Reality for Learning

    Lab: TanDEm

    Our research examines the role that low-cost virtual reality technology could play in supporting learning in low-resource contexts. Specifically, we propose to study the potential of creating affordable virtual reality-based learning experiences for children in these contexts. There has been a rising penetration of low-cost mobile technologies and internet connectivity in under-resourced communities, and this motivates us to explore the feasibility of virtual reality as a medium to enhance learning experiences for low-resource contexts. Keeping this in mind, we introduce inspirit - a free mobile platform for hosting VR-based learning content for the classroom. Please visit us at www.inspiritvr.org and download our mobile application from the Google Play Store. 

    Intuition, Rational Thinking & Creativity

    This research explored the effect of intuitive versus rational thinking on creativity. Our objective was to investigate this relationship through design tasks with undergraduate industrial design students. Students performed nine separate design tasks across three conditions. Their work was scored for novelty and feasibility, and we analyzed this performance data in conjunction with self-reported mood and information processing assessments. Our results show numerous statistically significant differences. Based on our analysis, we identified a variety of simple, actionable suggestions for design educators to integrate with their teaching, as well as additional thought provoking considerations. 

    Intuitive Pickups

    We will demonstrate our acoustic guitar pickup system. Our demo will consist live acoustic guitar playing with comparisons of our technology to existing technology.

    Investigating the abandonment of health tracking technologies

    Personal health-tracking technologies have become a part of mainstream culture. Their growing popularity and widespread adoption present an opportunity for the design of new interventions to improve wellness and health. However, there is an increasing concern that these technologies are failing to inspire long-term adoption. In order to understand why users abandon personal health-tracking technologies, we analyzed advertisements of secondary sales of such technologies on Craigslist. We conducted iterative inductive and deductive analyses of approximately 1600 advertisements of personal health-tracking technologies posted over the course of one month across the US. We identify health motivations and rationales for abandonment and present a set of design implications. We call for improved theories that help translate between existing theories designed to explain psychological effects of health behavior change and the technologies that help people make those changes.

    Invisible Cities: Multi-user AR Public Monuments

    Invisible Cities is a mobile dashboard for collaboratively developing AR monuments and expansive artworks in public squares. Users are able to create, share, and maintain public AR works that can be shared with a small community or the entire city. To get started, users need only download the Georgia Tech's Argon AR browser and be in a public space with AR content. From there, they can engage in creating a socially engaged AR with their fellow citizens. In an age of post-truth and fake news, AR public monuments serve as a touchstone for cultural heritage and expression. Affording communities the ability to develop AR worlds that reflect their sentiments, is a key aspect of this project. The research does take a cue from Italo Calvino's wonderful book, Invisible Cities. 


    The Internet of Things (IoT) will soon touch nearly all of the interactions we have with our world and with the things around us, and the interaction of those things with each other. GT-RNOC is developing a number of IoT-related projects that help students demonstrate and better understand some of the complexity and range of applications that the IoT encompasses.

    iPad Roadside Sobriety Test

    There are specific cognitive functions necessary to safely drive a vehicle that can be impacted by alcohol, drugs, fatigue, or other sources. There is currently not a roadside assessment that can detect driver impairment regardless of the source. The iPad Roadside Sobriety Test project is exploring the use of cognitive tasks as an assessment tool of driver impairment to resolve this problem. We are currently investigating the diagnostic abilitiy, reliability, and validity of different cognitive tasks for the impairment of cognitive functions essential for driving. 

    Isola VR

    Isola is a VR experience that takes place in a fantasy world consisted of many floating islands. Special pieces representing forgotten dreams are scattered in the space. The player has to find his lost pieces to become complete again.

    During the Journey, a special vehicle will be available for navigation and interaction.  With the vehicle, the player can sail among the floating islands. He/she will overcome bad weather, throw a ring to attract giant fish, and at the end, combine two broken pieces collected along the way to form a full star. 

    The entire story is accompanied by a little bird which guides the player’s attention and cues for possible interactions.

    This project explores the possibility of using a non-playable companion character to diegetically inform the player how to interact within a virtual space

    Jeli: The interactive, wearable pet

    Jeli is a wearable pet with personality. Just pin him to your shirt or jacket and take him along wherever you go! Jeli is the perfect travelling buddy for those who want a little company during their busy day.Pet Jeli on the head, squeeze his head, or scare him with a loud noise to make him come to life.
    Jeli served as an exploration in soft circuit design. The product is controlled by a Flora microcontroller, as it is well-suited to sewn circuitry and wearable products. Our system also incorporates homemade sensors, such as a potentiometer and a force sensor. We built these sensors from copper tape, felted conductive fibers, and velostat, and connected them all with conductive thread. In addtion to these soft sensors, Jeli also uses a microphone, neopixels, and a piezo buzzer to give him life and personality during interactions.

    Jigsaw: Visual Analytics for Text Document Collections

    Many types of investigators routinely perform analysis that involves large collections of documents. The Jigsaw system helps investigative analysts with reasoning and sense-making in such scenarios. Jigsaw acts like a visual index
    onto a document collection. It first analyzes the documents, identifies
    entities, clusters related documents, analyzes sentiment, and summarizes each document. Next, it provides multiple visualizations of the documents, entities within, and the analysis results. We have used Jigsaw to explore a wide variety of domains and document collections including academic papers, grants, product reviews, business press releases, news articles, intelligence and police reports, statutes, and even books such as the Bible.

    Knowledge Ecosystem Reaction to Platform Change

    Given the importance of developers for the success of mobile platforms, it is critical for vendors to understand how platform innovations impact developer interaction activity and what issues and topics are discussed. An understanding of these issues can help providers improve their release strategies, manage developer expectations, and avoid negative reputation effects. To facilitate this understanding, we are analyzing knowledge ecosystem reactions to change in mobile software development platforms. As part of this work, we have developed a method for gathering information about change events from two sources: endogenous information derived from traces of user interactions within knowledge ecosystems, and exogenous information harvested from official documentation, press releases, and news reports. The method is being applied to data describing interactions on Stack Overflow, the world’s most popular social information seeking community for developers. By demonstrating how such data can be processed to highlight periods of rapid change, and how this evidence can be combined with external indicators of change events, we are contributing a new technique to supplement approaches based on direct consultation of system participants.

    Learning a Universal Policy with Online System Identification

    We present a new method of learning control policies that successfully operate under unknown dynamic models. We create such policies by leveraging a large number of training examples that are generated using a physical simulator. Our system is made of two components: a Universal Policy (UP) and a function for Online System Identification (OSI). We describe our control policy as universal because it is trained over a wide array of dynamic models. These variations in the dynamic model may include differences in mass and inertia of the robots' components, variable friction coefficients, or unknown mass of an object to be manipulated. By training the Universal Policy with this variation, the control policy is prepared for a wider array of possible conditions when executed in an unknown environment. The second part of our system uses the recent state and action history of the system to predict the dynamics model parameters mu. The value of mu from the Online System Identification is then provided as input to the control policy (along with the system state). Together, UP-OSI is a robust control policy that can be used across a wide range of dynamic models, and that is also responsive to sudden changes in the environment. We have evaluated the performance of this system on a variety of tasks, including the problem of cart-pole swing-up, the double inverted pendulum, locomotion of a hopper, and block-throwing of a manipulator. UP-OSI is effective at these tasks across a wide range of dynamic models. Moreover, when tested with dynamic models outside of the training range, UP-OSI outperforms the Universal Policy alone, even when UP is given the actual value of the model dynamics. In addition to the benefits of creating more robust controllers, UP-OSI also holds out promise of narrowing the Reality Gap between simulated and real physical systems.

    Learning about Teaching in Low-Resource Environments

    Lab: TanDEm

    While there is a growing focus on leveraging technology use for learning gains across the world, this focus is yet to extend to infrastructurally limited environments in India, among other countries. We draw on qualitative research conducted in the Indian states of Tamil Nadu, Maharashtra, and West Bengal to highlight the challenges of designing educational technologies for "low-resource'' contexts, particularly when they are "low-resource'' along different dimensions. We also present findings from a survey of online educational technology providers in India to highlight the gaps that must be addressed before these can target socioeconomically disadvantaged populations. Taken together, our research provides a deeper understanding of the nuances that accompany "low-resource'' and how a careful assessment of these might inform appropriate design of educational technology interventions in the field of HCI for Development (HCI4D).

    Learning Bicycle Stunts

    We present a general approach to simulate and control a human character riding a bicycle. The rider not only learns to steer and to balance in normal riding situations, but also learns to perform a wide variety of stunts, including wheelie, endo, bunny hop, front wheel pivot and back hop.

    Learning Robot Behavior from Stories

    The Quixote system is an artificial intelligence technique for teaching robots and artificial virtual agents how to do things by telling them stories. Stories present a natural means of communicating complicated, tacit procedural knowledge. Quixote thus reads in natural language stories and learns to emulate the behaviors of the characters in the stories. The long term goal of the project is to make AI programming accessible to non-programmers and non-AI experts. 

    We have also shown that stories can be an effective means of demonstrating ethical behavior to robots and AIs.

    Living Online - The Curious Lives of Digitally Connected Teens

    This project discuses the findings from a 4-year study of 10-15 year old students from a large Metro-Atlanta school district. Over the course of the project 164 students took surveys and participated in focus groups and interviews regarding the amount of connectivity they experience, where they are going online, and what behavioral issues that are pervasive for this demographic as it relates to their online peer interactions. This work demonstrates how social computing influences communication patterns within this population as well as how social computing influences the everyday behavioral and emotional health and wellness of digitally connected teens.

    Lonely Mountain

    Lonely Mountain is a virtual reality adaptation of the movie The Hobbit: The Battle of the Five Armies. In this VR experience, Lonely Mountain has fallen into the claws of Smaug the Terrible. You will take the role of the Hobbit, Bilbo Baggins. Your mission is to find and recover the Arkenstone, and unite the dwarf realms once more under the same banner to save Lonley Mountain. The scenario has Bilbo reaches the treasure room and picks up a tool to grab the Arkenstone from the claws of Smaug without waking up Smaug. This virtual reality experience asks the question of how can diegetic elements be used to show the current state of an NPC.

    LuminAI: An Exploration of Human-AI Movement Improvisation

    LuminAI is an interactive art installation that explores the improvisation of proto-narrative movement between humans and virtual AI agents using full body, expressive, movement-based interaction. Interactors can co-create movement with an autonomous virtual agent that learns movement, response, and improvisation directly from interacting with human teachers. It analyses their movement using Viewpoints movement theory.

    MAGIC Summoning: Towards Automatic Suggesting and Testing of Gestures With Low Probability of False Positives During Use

    Gestures for interfaces should be short, pleasing, intuitive, and easily recognized by a computer. However, it is a challenge for interface designers to create gestures easily distinguishable from users' normal movements. Our tool MAGIC Summoning addresses this problem. Given a specific platform and task, we gather a large database of unlabeled sensor data captured in the environments in which the system will be used (an "Everyday Gesture Library" or EGL). MAGIC can output synthetic examples of the gesture to train a chosen classifier.

    Magic Window

    Magic Window supports immersive augmented video experiences allowing viewers to change perspective, as if they are looking through a real window.
    A rich set of collaborative interactions with live and pre-recorded media content as well as connected devices are possible through gesture-based controls.

    Makerspaces and Makerplaces: Collaboration, Togetherness, and Learning in Maker Communities

    Interviews with 61 makers, along with observations in several maker communities, provide empirical insight on the nuances between different types of communities and how these differences are influenced by the space and place of the makerspaces. Our exploration led to the identification of five prototypical maker communities; closed and regulated, open and messy, hybrid, online large-scale, and online small-scale.

    Making Smarter Transportation Choices

    Driving is the second highest expense for the average American household -- more than food or healthcare, and behind only housing. Yet most people do not understand the total cost of owning and operating their vehicles, and they cannot accurately estimate the cost of a common driving trip (such as a commute from home to work). That’s because the costs of owning and operating a vehicle are spread over many expenses incurred at different times. For example, you may fill up the gas tank once a week, make a monthly car payment, and pay insurance twice a year. Depreciation is a significant invisible expense of driving.<br><br>

    We have developed a trip cost meter that makes the total cost of each driving trip visible to the user. We are exploring how this tool can help people make better informed personal transportation decisions, including choice of vehicle and choice of alternate modes of transportation (e.g., Uber, transit, ridesharing, or walking/biking).

    Mapping iThemba

    Mapping iThemba draws on ethnographic research that Professor Anne Pollock began in 2010 at iThemba Pharmaceuticals (pronounced ee-TEM-ba), a small start-up pharmaceutical company in the outskirts of Johannesburg that was founded in 2009 with the mission of drug discovery for TB, HIV, and malaria. The synthetic chemistry research that scientists do at iThemba is no different than what might be done in a well-equipped lab anywhere in the world. Yet, place matters. The interactive map is an opportunity to explore how. 

    Mapping iThemba has been made possible by a grant from the National Science Foundation program for Science, Technology, and Society (Award #1331049). Professor Anne Pollock did the research and wrote the text for this site, new media artist Katherine Behar conceived the interactive map, and Digital Media master's student Russell Huffman designed, illustrated, and programmed it.

    This site provides only one small window into the project. More is available in an article that Anne Pollock published in Social Studies of Science: "Places of pharmaceutical knowledge-making: Global health, postcolonial science, and hope in South African drug discovery." Email apollock@gatech.edu if you would like to request a copy. Currently, she is writing a book manuscript on the project with the provisional title Synthesizing Hope: Global Health, Postcolonial Science, and South African Drug Discovery. For updates on publications from the project, see her website at Georgia Tech.

    Mapping Place

    As part of the exhibit, Mapping Place: Africa Beyond Paper, which contrasts western concepts of mapping (i.e. Cartesian plots of locations) with other traditional practices, Synlab students created an interactive tabletop installation that lets participants tell their own stories by creating a digital Lukasa, a mnemonic device used by the Luba people of central Africa to record genealogy and history. The exhibition was at the Robert C. Williams Paper Museum from February 27 to June 6, 2014.

    Marlin: A wearable swim coach

    An ever increasing number of smart technologies are being developed all over the world to coach people on healthier, as well as more responsible behaviors, providing them timely and ubiquitously with personalised information and support.
    Marlin is one such wearable swim coach, specifically for distance swimmers, which constantly monitors a swimmer’s performance and provides them necessary real time feedback through sonification while swimming. It is tool for coaches to plan a detailed training program, set new targets and sync them to the swimmers’ device. It allows both the coaches and swimmers to analyze their performance by tracking their progress and giving them guidance immediately. In this project, we evaluate the usability of the interface for the coaches and the swimmers, and study if they modify their behavior according to the feedback.

    MD2K: Engagement Wrapper - Keeping Users Engaged Through Mobile Technology

    HCC professionals, psychologists, and many other researchers are interested in understanding how to better influence participant engagement in interventions that involve technology. This is especially true in instances when researchers are not able to provide monetary incentives over an extended period of time. Georiga Tech along with a team at the University of Michigan (working under an NIH-funded MD2K project) are exploring how to keep individuals motivated in such interventions through mobile technology and gamification.

    Measuring the World’s Digital Natives

    We offer a ?rst attempt to measure the global digital native population with a model for calculating the number of digital natives in each country of the world. We have calculated the size of the digital native population by country, by region and by income level and have related the presence of digital natives to education and literacy levels, and ultimately to policy-making. According to the model, in 2012 there were around 363 million digital natives out of a world population of around 7 billion – or 5.2 per cent. De?ning “youth” as young people aged 15 to 24, this means that 30 per cent of the world’s youth have been active online for at least ?ve years. While it follows that fewer than a third of the world’s young people today are digital natives, this group nonetheless plays an important role: ?rst, because where the online population is concerned, youth are clearly overrepresented, and second, because digital natives are key drivers when it comes to ICT uptake, innovation and impact.

    Medication Management Tool Design for Students

    A tool which provide bran-new medication management experience to students as well as consider international students special pain points when using the US. medication system.

    Medium Probe: A Method for Seeding Dialogue to Explore the Suitable Medium of Communication in Design

    A method for engaging community members in productive discussions about technology with technology designers.&nbsp;Medium probes offer designers and participants&nbsp;a chance for open-ended exploration of design issues related to participants&rsquo; practices with technology, cultural values, skills, and&nbsp;access level.

    Mermaids MMOG

    Mermaids is a massively multiplayer online game set in an underwater world in which players take the roles of hatchlings coming to life in the ruins of a long-extinct mermaid culture. The over-arching goal and storyline is to rebuild the lost Mermaid culture and reclaim their various skills and cultural practices, while at the same time trying to avoiding the mistakes that caused the extinction of their ancestors. Mermaids is designed as an experiment in emergent game play, with specific affordances designed to promote social emergence. This presentation will include a live demo of the game, plus a poster on the modular mermaid construction system currently being developed by an undergraduate student research team.

    MICE Learning Tools

    Multisensory Interactive Children's Electronic (MICE) learning tools provide a new format for electronic dictionaries and books utilizing text, images, speech, and video to maximize learning and comprehension for children.

    Midtown Buzz

    Midtown Buzz

    Midtown Buzz is an experiment in mobile innovation focused on engaging urban communities. It includes mobile platform and app development, open-source data curation, contextually aware environments, social navigation, developer workshops, hackathons, trials, needs assessments and the creation of a Live-Work-Play “Laboratory” for exploring the potential of media technologies in creating a climate for innovation.

    Be sure to check out the innovative Buzz projects, such as Storyoke and Auggy! Please visit www.midtownbuzz. org for more information.

    MoodChat: Emotion Communication Wearable System

    Sharing emotions is a way to connect people to one another, this project uses a wearable system to recognize, share & connect people via emotions. MoodChat is a wearable system that automatically recognize human emotion and allows people to share their feelings and emotions through simple interaction. The system includes a wristband and a mobile app. 

    Motivational Glanceable Reminders - Designing a Better Medication Reminder App for Pediatric Asthma

    This research aims to explore the use of glanceable reminders with a motivational component to support medication adherence. The healthcare industry has begun to focus on mobile health (mHealth) to improve medication adherence through the use of medication reminders. To date, mHealth apps have provided reminders that are text-based and purely informational in nature. The goal of using motivational glanceable reminders is to provide reminders that appeal to the emotional side of a person's decision making process and can be interpreted at a glance without the need to read, or even be literate. The research focuses on the pediatric asthma population. This research uncovers insights that can inform the design of future medication reminder mHealth apps that seek to integrate motivational glanceable reminders.

    MR ED - Measurement & Recording Equine Device

    Research, on current equine technologies, shows that limited exploration is being done in the space of wearable technology. MR ED is a research project focused on designing wearable tech for horses & riders.

    MS Projects

    A collection of projects that explore the convergence of entertainment formats and computation, with focus on HCI design and research methods.

    Sanat Rath: Giggles, an application to help viewers relive moments from their favorite sitcoms.
    Sruthi Padala: A second screen application for the popular TV show 'The Voice'.
    Vipul Thakur: Talkista, an application that serves as your information resource, companion in conferences, meetups and classrooms.
    Amrutha Krishnan: Newspad, design of a second screen application for news that enables viewers to understand the news better by providing them the required context as well as supplementary information.

    Multi-touch Dust and Magnet

    This demo shows a system called Dust and Magnet (DnM) that is a general purpose data visualization system. DnM represents data items as iron dust. Each attribute of the data then is a magnet. The system is implemented on a large multi-touch display where the analyst can deploy magnets and drag them around the view. Data points will then be attracted more strongly or weakly depending on that data item's value of the attribute represented by each magnet.  This system provides a very hands-on, visceral data exploration experience.

    Multifunctionality In Biologically Inspired Design

    Biological systems in general are multifunctional and environmentally sustainable.

    Thus, biologically inspired design is posited as leading to multifunctional and environmentally

    sustainable designs. Design in general is characterized as a problem-driven process. However,

    biologically inspired design also entails the twin process of solution-based design. Previous work

    has postulated that the solution-based design process is prone to design fixation but leads to more

    multifunctional designs. Design Study Library (DSL) is a digital library of eighty-three cases of

    biologically inspired design. We present a preliminary analysis of the DSL case studies to examine

    two hypotheses. (1) The process of solution-based design results in more multifunctional designs

    than the problem-driven design process. (2) The process of solution-based design is more prone to

    fixation than the problem-driven design process. We find strong evidence in favor of the first


    Multimodal Speech Capture System

    The motion of our articulators is responsible for generating speech. To re-learn how to speak intelligibly after a brain injury, practicing enunciation when learning a new language, or change our native accent to fit in a new place or required as part of our job, accessing and correcting our articulators’ motion is crucial. Few practical solutions exist in the market, and our system is filling this gap by providing an affordable, unobtrusive, portable and user-friendly solution to visualize and generate feedback to the user about their speech performance (e.g., tongue and lips gesture).

    Multisensory Prayer Nuts

    We present three prototypes designed for a hypothetical museum exhibit that elicit historical and experiential qualities of early 16th century prayer-nuts. As personal religious experiences included a “dependence of spirituality on material objects” during the 16th century, we believe that digitally-enhanced multisensory interactions can help situate the artifact in its historical context. The 3D printed interactive prayer nuts augmented with audio-visual effects support the visual voyage, experience of spirituality, and scents of power. The tactile, aural, visual, olfactory sensory interactions are mapped meaningfully to incorporate some of the original sensory aspects of the artifact and related practices. Our research provides insight on how multisensory interactions can provide museum visitors with the opportunity to experientially engage in content related to an artifact’s history and original use. 

    Mumerize: Educative Music Game for Music Learner

    Mumerize is an educative music game to help music learners to memorize music intervals and learn the structure of music. This game provides the player with a scene like a platform game where the platforms are created by dividing a melody into music notes. The player should determine the position of the next platform by hearing and selecting an answer of intervals. The player should survive or die after each jump according to whether the answer is right or wrong

    Mutual Gaze Eye Contact

    Real-Time Eye Contact Using SMI Gaze Tracking Glasses

    Mwangaza Project

    The Mwangaza Project is a collaboration among the Sonification Lab, inAble, and Kenyatta University to develop and deploy accessible STEM educational resources to schools for the blind throughout Kenya. Projects that we are working on include accessible weather and climate education, math software for accessing graphing and number lines, and renewable energy as a component of STEM education and support for educational technologies.

    My Journey Compass: Examining PHR Usage Among Breast Cancer Patients

    Health information management for cancer care is a challenging and personal process that changes over time based on one’s needs, goals, and health status. While technologies supporting health information management appear promising, we do not fully understand how health information tools fit into patients’ daily lives. To better understand the opportunities and usage barriers of these tools, we designed and deployed a mobile, tablet-based health management aid: My Journey Compass. We found that developing a tool that was customizable, mobile, and integrated into the patients’ healthcare system resulted in a set of surprising uses by breast cancer patients for a wide variety of tasks. Our study demonstrates the potential for health management tools to improve the cancer care experience and for HCI research to influence existing healthcare systems.


    A Beginner's Guide Empowering Cocktail Making

    Navigating Media Use: Chinese Parents and Their Overseas Adolescent Children on WeChat

    In many Chinese families, adolescent children are moving abroad to pursue higher education. Due to cultural issues and China’s One-Child Policy, parents have high expectations for their children and are willing to invest time and money to help them build a successful future. In this study, we interviewed both Chinese students in the US and Chinese parents in China whose children are studying abroad about their family communications. Parents and students used a variety of media, and families negotiated how each medium was used. Most families eventually migrated to communicate through WeChat, the ubiquitous Chinese based cross-platform instant messaging service. The negotiation of what medium to use, how to use and when to use specific media was tied the students’ and parents’ goals to construct a positive presentation of self. We offer design considerations for cross-cultural, family and intimate long-distance communication media.

    Navis: A College Orientation Game

    Navis is a college orientation game designed by graduating Digital Media Master's student Laura Schluckebier. Navis is designed as a campus-side scavenger hunt with team building challenges. Upon arriving to campus for their orientation session, first years work with their teammates to discover clues around campus and to compete in team building challenges. Completing these challenges and earns them points.

    Navis' features include 1) a variety of challenges types that teach students through game actions rather than content 2) fluid play groups 3) and an overarching structures that encourages individual intiative.

    The game also features a Framework that allows university orientation staff from a variety of college campuses to customize and deploy Navis during their orientation session.

    New Media Nollywood

    The Nigerian film industry – colloquially known as Nollywood – is enormous, innovative, and digital. We are working to extend its capacities with new media technologies such as games, mobile and social media. In addition, we are developing social messaging campaigns within Nollywood films in particular around health issues. Come see emerging technologies and film content, including the feature length Nollywood film we produced and are premiering.

    Non-Invasive Diagnosis of Airway Resistance in Young Children Using Microsoft Kinect

    Respiratory syncytial virus (RSV) is a virus that causes respiratory tract nfections especially in young children. This infection increases the airway resistance and makes it harder to breathe because more pressure has to be generated in the lungs to the extent that the respiratory muscles may get so tired that the patient stops breathing. In the U.S., by the end of 2-3 years of age, nearly all of the children are going to be infected with RSV at least once. Among them, 2-3% will develop bronchiolitis and need to be hospitalized. For this disease, similar to the most of the medical complications, the best strategy is prevention. RSV is a virus, thus a vaccine would be the best answer. Unfortunately, at present no RSV vaccine exists. The last attempt was a trial in 1960s that failed! On the other hand, some symptoms (e.g. temporary difficulty in breathing especially in infants) can easily be mistaken with RSV infection and cause a lot of unnecessary visits to the hospitals/emergency rooms (very high false positive rate).

    In this research, we aim to quantify airway resistance through a simple, non-invasive measurement of the chest volume changes over time that can act as a surrogate measure of chest pressure and volumetric airflow. Our approach uses signal and image processing techniques to infer airway resistance using a commercially available infrared depth-sensor, Microsoft Kinect. We envision in the case of commercialization, at a similar price to baby monitors, this technology would have the potential to greatly improve the management of the infant obstructive pulmonary diseases and reduce unnecessary hospital visits. 


    NotifiVR- A VR notification design framework

    Due to the lack of standardized notification systems in virtual reality (VR), an immersed user can face different problems like bumping into walls, tripping over pets, losing track of time, missing incoming calls, getting late for scheduled appointments, etc. In this paper we present a study of common interruptions in a VR context and explore methods of representing them in an abstract way in the VR world. We further present NotifiVR, a Unity based notification framework that allows developers and designers to create, integrate, and customize auditory, visual, and haptic notifications in a VR scene.

    Novel Interactions with Wearable Devices

    A series of demonstrations of novel interactions with wearable devices, from smartwatches to head-mounted displays.


    Nutshell is a video editing tool that allows users to isolate clips from their favorite TV shows and create "supercuts" that summarize complex plot arcs and compile recurring inside jokes.

    Offline Mobile Media Sharing

    Lab: TanDEm

    This study examines different aspects of media sharing within Mobile Vaani - an Android app that displays crowdsourced news recordings from rural communities in many states in India. We have developed a feature that facilitiates offline sharing of these news recordings, in light of the relatively high cost of mobile data in the areas where Mobile Vaani is most used. As Mobile Vaani and offline sharing is launched, we propose a study of the learning processes and sharing practices among users of the app in order to further understand why and how people share mobile media content within a community.


    One Block: A Local Parent Community Bridging the Gap in Digital Inequality

    Parents have an important role in improving their children's educational experience by providing them with learning opportunities Free online informal learning resources are available more than ever; however, research shows that people who may need these resources the most, are not able to find them. In this study, we go beyond a binary definition of access to examine technology practices and level of access to information technologies among African-American parents in financially depressed communities in westside Atlanta. We propose a hyper-local community for parents to be able to engage in neighborhood discussions and communicate available learning opportunities within the community.


    The OneBusAway transit traveler information system gives users information about transit vehicle arrival times including real-time arrivals and schedule information. It is comprised of multiple interfaces to access information, including a website, a mobile-optimized website, and native applications for iPhone, Android and Windows platforms (see http://onebusaway.org). OneBusAway was developed under multiple federal grants as an open-source system allowing other transit agencies to adapt the code for their own systems. The initial development took place at the University of Washington and the Seattle instance still serves over 100,000 unique weekly users. The platform is also now used in Tampa and Washington DC and as the backbone of MTA New York's BusTime.
    Here in Atlanta, the Urban Transportation Information Lab (UTIL) has worked to create a local version of OneBusAway using funding from Georgia Tech’s GVU Center, the Institute for People and Technology (IPAT) and the National Center for Transportation Systems Productivity and Management (NCTSPM). We have integrated all of MARTA’s train and bus real-time information, Georgia Tech’s trolley and shuttle real-time information and schedule data from multiple area agencies (see http://atlanta.onebusaway.org). Additional transit services are being added over time. We are using the platform to conduct an evaluation of its impacts on transit riders, specifically using smartcard data to quantify the increase in trips. Previous studies of real-time transit information have demonstrated a number of user benefits, including decreases in perceived and actual waiting times, as well as increases in frequency of travel and customer satisfaction.

    Online Support for Mental Health

    Mental health issues are considered to be socially stigmatized in the society. Consequently, to many individuals, finding a trusted individual whom they can confide to regarding their mental illness is a grave challenge. It has been established that online communities provide a powerful platform of candid disclosures and support seeking around stigmatized concerns and experiences. Online health support groups in Reddit, in particular, due to their semi-anonymity feature, have established an extensive support platform for people with social inhibitions seeking help. These communities can provide support in various forms, ranging from informational to emotional support.. Informational support may be helpful for short term challenges in contrast to the emotional support. However, information support can also present itself with credibility and trust issues. On the other hand, emotional support is deemed more helpful to people suffering from mental illness. How do these different types of support impact an individual’s perceived sense of mental well-being in an online community? How do support types relate to an online community’s helpfulness, efficacy, and survival over time? Through this project, we will show some preliminary quantitative findings that addressed these questions.

    Open Science in the Public Interest

    In our research, we leverage ideas from citizen science and the open science movement to create an infrastructure for facilitating partnerships between volunteer STEM workers and social movement organizations. Towards this goal, we will do action research with the nonprofit organization Science for the People (SP). Science for the People engages in research, activism, and science communication for the betterment of society and the environment. Members of Science for the People are STEM workers, educators, and activists who are socially and ethically focused, and believe that science should be a positive force for humanity and the planet. We work with members of the Atlanta chapter of the organization, assisting them in building partnerships between non-governmental organizations (NGOs) and STEM volunteers, and at the same time studying the process of how social movements can best use scientific knowledge of the volunteers.

    Optimal Display Placement for Wearable Computers

    Where should a head worn display (HWD) be placed for optimal viewing?  Does the optimal position change between reading, working on a physical task like order picking, or social conversation? How do we test such issues?  Our ongoing studies on user comfort and performance suggest the optimal placement is somewhere between 10 and 20 degrees off-center toward the ear.

    Order Picking with Wearable Computers

    Warehouses throughout the world distribute approximately $1 trillion in goods per year from nearly a million warehouses. Order Picking is the process of collecting items from inventory and sorting them into orders for distribution. It represents one of the main activities performed in warehouses. About 60% of the total operational costs of these warehouses is order picking. Most are still picked by hand, often using paper pick lists. Our objective is to implement and compare various order-picking systems, including:

    • Pick-By-Paper list
    • Pick-By-Light
    • Pick-By-Tablet
    • Pick-By-HUD (Heads-Up Display).

    Order Up!: Mobile Gaming To Promote Healthier Diet Choices

    OrderUp! takes health-related gaming in a new direction and seeks to educate players about how to make healthy eating choices in situations nearly everyone encounters regularly in their lives. By casting players as virtual restaurant servers, OrderUp! forces players to make healthy—and fast—menu decisions for a group of demanding, impatient customers. OrderUp! was originally developed as a simple, casual game on Nokia N95 mobile phones. We are building on research findings from testing this first version of the game to develop a new version with higher fidelity graphics and more sophisticated game play. This new version will run on modern, Android smart phones and will incorporate features intended to promote cognitive flow, an increased level of engagement and fun with the game, such as progressive increases in game play difficulty and better performance and scoring feedback. The revised OrderUp! game will be tested with a population of largely African Americans who are being treated for mental health issues. As such, OrderUp! is designed with contextually relevant motifs and with relevant data and personas.

    Orko: Multimodal Interaction for Exploring Network Visualizations

    Data visualization systems have predominantly been developed for WIMP-based direct manipulation interfaces. Only recently have other forms of interaction begun to appear, such as natural language or touch-based interaction, though usually operating only independently. Prior evaluations of natural language interfaces for visualization have indicated potential value in combining direct manipulation and natural language as complementary interaction techniques. Unfortunately, however, little work has been done in exploring such multimodal visualization interfaces. Orko is a visualization we have created to allow people to explore data using speech and touch-based multimodal interaction.  With Orko, we try to bridge the gap between existing direct manipulation based interfaces and hypothesized futuristic natural user interfaces for data analysis with visualization.

    Our Driverless Futures: Speculating Moral Dilemmas of Self-Driving Cars

    “Our Driverless Futures” is a web-based interactive narrative that critically examines the ethical implications of self-driving cars. In public and scholarly discourse, self-driving cars are often positioned as safer and more efficient transportation alternatives, with one prominent debate remaining in their programming: how should they react in an inevitable fatal accident, and whose lives should they prioritize to save? However, this focus on “kill decisions” has limited the terms of discourse to utilitarian ethics and normative classifications of people. Moreover, as discussed by scholars in critical algorithm studies, relegating moral decision-making to algorithms introduces yet to be seen issues of discrimination, transparency, accountability, and surveillance. Informed by STS and feminist discourses, “Our Driverless Futures,”  which won the 2017 Making and Doing Award at the Society for Social Studies of Conference, advances a critical reflection on algorithmic morality and its consequences.

    Over The Garden Wall VR

    Alone in a strange and dark forest, you find a lantern, your sole source of light and comfort. But soon you are joined by two characters, and each makes a compelling argument as to why you should give the lantern to them. Who do you trust? You decide!

    This virtual reality experience places you at the penultimate moment of Cartoon Network's Emmy-award winning "Over the Garden Wall" mini-series.    

    Parent Pedagogical Perspectives: A Framework for Design of Museum Exhibits

    The rise of ubiquitous technology has resulted in exponential growth in potential options for the design of interactive museum exhibits. We posit that using a framework of parental beliefs about learning and teaching for design can be useful to the HCI community when creating museum exhibits to facilitate learning. Our study builds upon Swartz and Crowley’s framework of parental pedagogical approaches through analysis of 118 observations of social interactions between parents and children at museum exhibits. We classify our observations of parent-child interactions into the framework’s five categories: Fun Exploration, Individual Discovery, Basic Knowledge, Parent Engaged Learning, and Contextualized Explanations. We identify how these categories can be applied to evaluate and guide design for learning in the context of museums and other informal learning environments.

    Participatory Multi-User Approach in Smart Home Systems

    Lab: App Lab

    Designing a participative approach for multiple users to control their smart home.

    Particle in a Box (An Experiential Approach to Quantum Mechanics Education)

    Theories of Quantum Mechanics(QM) have been central to the philosophical and technological advances in physics and related fields. Some of the most important aspects of these theories are outside the bounds of human experience, predominantly explained and taught drawing on abstract mathematical formulas. How can we advance experience-based learning of abstract concepts such as QM so students develop the in-depth understanding needed to further advance these theories by generating and testing new hypotheses? This research project addresses this question through a series experimentations with digital media (e.g., by designing interactive games based on the rules of QM) engaging whether and how digital media could serve as the basis for an experiential understanding of QM concepts. For more information and to play the latest version of the game please visit, http://learnqm.gatech.edu 

    Passive Haptic Learning

    Passive Haptic Learning (PHL) is the acquisition of sensorimotor skills without active attention to learning. Vibrations are used to passively ‘teach’ the motor skill and are typically delivered by a wearable, tactile interface. Our group has previously demonstrated Passive Haptic Learning of piano melodies and of typing skills for text entry on a unique 10-key keyboard. We now investigate whether Passive Haptic instruction facilitated by wearable computers may be a feasible method of teaching Braille typing.

    Passive Haptic Learning: Learn to Type or Play Piano Without Attention Using Wearables

    Our Passive Haptic Learning gloves teach the "muscle memory" of how to play piano melodies without the learner's active attention.   These gloves can also help wearers recover sensation in their hands after a traumatic event, such as a partial spinal cord injury.  The PHL gloves are fingerless gloves equipped with vibrators at each knuckle.  As a mobile MP3 player plays each note ofa song, the gloves tap the finger that corresponds to the respective piano key.  In our testing, users can learn the first 45 notes of simplesongs like "Amazing Grace" in 30 minutes while concentrating on reading comprehension exams.  Similar gloves can passively teach a wearer how to type and read Braille in four hours without active attention.  We have extended the work to Passive Haptic Rehabilitation: helping people with tetraplegia due to partial spinal cord injury improve sensation and dexterity in their affected hands.  In  future we hope to extend the gloves to helping with other conditions such as stroke, MS, ALS, and Parkinsons.

    Passive Haptic Rehabilitation: Improve Sensation and Dexterity after Traumatic Injury Using Wearables

    Our Passive Haptic Learning gloves teach the "muscle memory" of how to play piano melodies without the learner's active attention.   These gloves can also help wearers recover sensation in their hands after a traumatic event, such as a partial spinal cord injury.  The PHL gloves are fingerless gloves equipped with vibrators at each knuckle.  As a mobile MP3 player plays each note ofa song, the gloves tap the finger that corresponds to the respective piano key.  In our testing, users can learn the first 45 notes of simplesongs like "Amazing Grace" in 30 minutes while concentrating on reading comprehension exams.  Similar gloves can passively teach a wearer how to type and read Braille in four hours without active attention.  We have extended the work to Passive Haptic Rehabilitation: helping people with tetraplegia due to partial spinal cord injury improve sensation and dexterity in their affected hands.  In  future we hope to extend the gloves to helping with other conditions such as stroke, MS, ALS, and Parkinsons.

    Peer Feedback

    Peer assessment is a popular approach for grading assignments that cannot be simplified to multiple choice questions in Massive Open Online Courses (MOOCs). How can we take peer assessment and transform it into an engaging and enjoyable experience which enhances the learning outcomes for students?

    Persimmons: (De)constructing Cultural Identity through Interactive Fruit Foodway Narratives

    Food plays a huge role in shaping our identity. We engage in food practices daily, and through these engagements, we associate meaning with these specific acts. Our research explored this theme by deconstructing cultural identity through food narratives. How can we use narrative to make sense of and frame our identities around food within cultures and across different cultures? From this research, we gathered multiple stories around a single fruit - the persimmon. Drawing from food journalism, food field guides, and digital media, we worked within this intersection to not only share these narratives but provide us a means to frame our own identities around them.  

    Personalized Augmented Reality Lenses for STEM Education

    Augmented reality (AR) has long been explored as a tool for education, from textbooks that come to life, to plant identification training via the augmentation of actual flora, and a tangible molecular modeling tool. The power of AR (overlaying virtual content on the physical world) is that it can be used to show the “unseen” and the “hidden” information in the world. While this can involve showing representations of occluded objects (such as pipes underneath the ground), it can also be used to visually represent data or properties of the physical world that you would not normally see (such as the forces acting on an object). There is considerable research in this area of situated visualization, defined as “the visual representation of data presented in its spatial and semantic context”. This technique addresses the need in certain contexts to convey to the user the relationships between physical objects and virtual data.

    We are demoing our initial prototypes that explore how to use AR and the concept of situated visualization to create a combination physical and virtual “exploration kit” for students that allows them to construct simple static and dynamic systems with physical building components. AR will allow the students to see virtual visualizations of the physics properties and concepts (e.g. velocity, acceleration, forces, friction, elasticity), which control the system, in real-time and overlaid on the real objects.

    Funded by the Verizon Foundation

    Podium: Ranking Data Using Mixed-Initiative Visual Analytics

    People often rank and order data items as a vital part of making decisions. Multi-attribute ranking systems are a common tool used to make such data-driven decisions. These systems are often table-based tools that can produce rankings based on numerical weights that a user assigns to each attribute, where the weight represents how important the user believes an attribute is to their decision. These systems assume that users are able to quantify their conceptual understanding of how important particular attributes are; however, this is not always the case. Users often have a more holistic understanding of the data. They form opinions that data point A is better than data point B but do not necessarily know what attributes are important. 

    To address these challenges, we present an application of SVM to infer attribute weights based on a user's interactions for the purpose of ranking multi-attribute data. We developed a prototype system, Podium, that allows users to drag rows in the table indicating where they think data items belong based on their knowledge or preferences. Our system then infers a weighting model using SVM that satisfies the user's preferences as closely as possible. Whereas past systems help users understand the relationships between data items based on changes to attribute weights, our approach helps users to understand the attributes that might inform their understanding of the data. Our approach allows users to explore their preferences and expose their biases.

    PopSign: Teaching American Sign Language Using Mobile Games

    CopyCat and PopSign are two games that help deaf children and their parents acquire language skills in American Sign Language.  95% of deaf children are born to hearing parents, and most of those parents never learn enough sign language to teach their children.  As short term memory skills are learned from acquiring a language, many deaf children enter school with short term memory of less than 3 items, much less than hearing children of hearing parents or Deaf children of Deaf parents.  Our systems address this problem directly.  Even though they are still under development our games have been shown to be effective in multiple user studies.

    POSTER -- The Science Behind Getting (and Keeping) Followers on Twitter

    Followers are Twitter's most basic currency. Building an audience of followers can create access to a network of social ties, resources, and influence. Yet, little is understood about how to grow such an audience. We examine multiple factors that affect tie formation and dissolution over time on the social media service Twitter. We collected behavioral, content, and network data approximately every three months for fifteen months and compared the relative contributions of 22 variables from each perspective for predicting link formations in online social networks.

    POSTER presentation -- The Language that gets People to Give - Phrases that Predict Success on Kickstarter

    Crowdfunding sites like Kickstarter—where entrepreneurs and artists look to the internet for funding—have quickly risen to prominence. However, we know very little about the factors driving the “crowd” to take projects to their funding goal. In this paper we explore the factors which lead to successfully funding a crowdfunding project. We study a corpus of 45K crowdfunded projects, analyzing 9M phrases and 59 other variables commonly present on crowdfunding sites. The language used in the project has surprising predictive power— accounting for 58.56% of the variance around successful funding. A closer look at the phrases shows they exhibit general persuasion principles. For example, also receive two reflects the principle of Reciprocity and is one of the top predictors of successful funding. We conclude this paper by announcing the release of the predictive phrases along with the control variables as a public dataset, hoping that our work can enable new features on crowdfunding sites—tools to help both backers and project creators make the best use of their time and money.

    Poster: Careflow Analytics and Visualization

    Healthcare delivery processes are complex activities that span organizational, spatial, and temporal boundaries. Systemic insights, redesign, and improvements are consequently difficult to achieve. Using existing digital healthcare data, we are developing a data-driven methodology, fusing computational systems modeling, data mining, and interactive visualization, to identify, describe, and visualize healthcare delivery processes. Our system will help providers (e.g. physicians, nurses, etc.) and strategic decision makers (e.g. CIOs, CTOs, CFOs) interactively discover, analyze, and visualize processes, determine variations in care quality, cost, and outcomes stratified by physician and patient population, and provide an important evidence-based foundation for healthcare delivery improvement.

    Poster: Computational Analysis and Visualization of Global Supply Network Risks

    Management of supply network risks is a critical competency for today’s global enterprises. Current practices and tools, however, have limited capabilities and do not allow for systemic exploration of alternate risk strategies. We develop a
    computational model of risk diffusion in global supply networks that is grounded in techniques from complex systems, network analysis, and epidemiological risk modeling. We draw on a unique, curated dataset of firms, their supply networks, and financial risk in the global electronics industry. Specifically, we assess and visualize the impact of network structure on risk diffusion and supply network health, and determine the impact of visibility on reduction and potential mitigation of cascading risks. Our approach enables decision makers to identify risks and determine potential paths of their diffusion. In doing so, we advance our understanding of the design and development of computational risk management tools in a global supply network context.

    Poster: CSCW in the Healthcare Enterprise: A Knowledge Domain Visualization

    Designing effective CSCW systems in healthcare requires a careful consideration of the entire enterprise. This study uses computational text analysis and network visualization of topical terms and keywords to map the extant knowledge domain of CSCW in healthcare. The results are framed using a multi-level enterprise model, comprised of people, technology, process, and organization. Emerging trends and prominent patterns are identified. The study contributes to a broader understanding of CSCW research in healthcare and demonstrates the value of adapting an enterprise (as a) system lens.

    Poster: Health Policy Flight Simulator

    Computer simulations are effective tools for addressing enterprise transformation in terms of alternative organizational policies, operating procedures, and allocations of resources. We present a multilevel approach to computationally model health delivery enterprises. This approach is illustrated by its application to an employer-based prevention and wellness program. The decision of interest in this application concerns the design of prevention and wellness programs that are self-sustaining and provide a positive return on investment for the overall enterprise. The nature of this decision is shown to have enormous implications for how delivery services are organized.

    Poster: Knowledge Discovery of Competitive Strategies in Converging Ecosystems

    With the remarkable increase of data, novel tools and metrics are needed for comprehensive and systemic analyses of converging business ecosystems. Knowledge discovery is the computational process of identifying valid, novel, interesting, potentially useful and ultimately understandable patterns in data. The objective of our study is twofold. First, we introduce the emerging domain of "big" business ecosystem data. Second, we describe the success and challenges that we encountered in analyzing this data using state of the art analytics and visualization techniques. Specifically, we illustrate our computational knowledge discovery approach with case studies of innovation, coopetition, and convergence in the mobile ecosystem. We conclude with theoretical and managerial implications and identify opportunities for future research in the emerging domain of computational enterprise science.

    Poster: Product Evolution and Platform Strategy: A Study of Smartphones

    We study how operation and platform strategy affect product- and firm-level innovativeness in the smartphone industry. Our study is based on the technological evolution of smartphones including physical characteristics, performance characteristics, and features over the past decade. Our longitudinal analysis of a dataset consisting of 1,171 smartphones by 79 device manufacturers and 12 different platforms highlights the significant transformational changes that have occurred. We propose an index to measure innovativeness of a smartphone upon which we test our main hypotheses. Our findings are threefold. First, we find the rate of product launch is negatively associated with product innovativeness. Second, we find the number of product lines is positively associated with product innovativeness. Lastly, we find a nonlinear pattern between platform strategy and product innovativeness in that a mixed platform strategy is associated with higher product innovativeness than a pure platform strategy.

    POSTRE: Faces Engage Us: Photos with Faces Attract More Likes and Comments

    Photos are becoming prominent means of communication online. Despite photos' pervasive presence in social media and online world, we know little about how people interact and engage with their content. Understanding how photo content might signify engagement, can impact both science and design, influencing
    production and distribution. One common type of photo content that is shared on social media, is the photos of people.
    From studies of offline behavior, we know that human faces are powerful channels of non-verbal communication. In this paper, we study this behavioral phenomena online. We ask how
    presence of a face, it's age and gender might impact social engagement on the
    photo. We use a corpus of 1 million Instagram images and organize our study around two
    social engagement feedback factors, likes and comments. Our results show that photos with faces are 38% more likely to
    receive likes and 32% more likely to receive comments, even after
    controlling for social network reach and activity. We find, however,
    that the number of faces, their age and gender do not have an
    effect. This work presents the first results on how photos with human faces relate to engagement on large scale image
    sharing communities. In addition to contributing to the research
    around online user behavior, our findings offer a new line of future
    work using visual analysis.

    Power Puppets

    The Power Puppet project combines the creation of a mechanical puppet with simple circuit building. Its goal is to teach middle school students basic circuit building in the setting of a puppet building workshop. As students build their puppets, including control mechanisms (like rods and strings) and expressive elements (like joints and materials), they also create basic circuits that operate in combination with the puppet that houses them.

    Procedurally Generated Augmented Reality Games

    Augmented Reality gaming promises new ways for humans to engage with their physical environment by overlaying gameplay elements via a head-mounted display. We present an artificial intelligence technique to automatically generate novel gameplay content for mixed-reality environments. We demonstrate the technique with a game we call "Augmented Reality Lemmings", a platform game in which the level content was procedurally generated.

    Projet: A Panorama Narrative

    This is the first is a series of experiments in locative narrative. In this version Projet is entirely screen-based. Later applications in this series will use Argon to create true location-based narratives. Projet itself can be experienced on a laptop or desktop screen, but there is greater intimacy and effect when experienced on a tablet or phone.

    A woman finds herself in a village in the French countryside, reflects on her present life, and recalls her past. Projet reduces the viewer’s interaction to the barest minimum. The viewer intervenes by touching the screen at various moments to reanimate the narrative—to set a panorama or slideshow in motion like a music box. She can also scroll to explore the panoramas will the audio plays.

    Idea, text and images: Maria Engberg. Experience design and implementation: Maria Engberg and Jay Bolter. Voice Actor: Tess Kincaid.

    Prototyping Puppets - Teaching Circuitry

    We combine craft and performance art to teach early middle school students basic prototyping skills. We develop informal STEM workshops for puppetry that combine narrative framing, craft-inspired building, and performance. This key approach combines craft, art, and basic hardware prototyping to attract new audiences to STEM. It is a collaboration between Georgia Tech and at the Center for Puppetry Arts funded by the NSF.

    PUNGA: Provenance-supported Undirected Node Graph Analytics

    PUNGA (Provenance-supported Undirected Node Graph Analytics) is a tool for intelligence analysts. PUNGA assists analysts in making sense of a large textual-based dataset by supporting data processing (Named Entity Recognition), data cleaning, data analysis, and analytic provencance.

    PUNGA provides users the ability to combine, format, clean the data as per their convenience before and during analysis with the Entity View. PUNGA also facilitates user interaction with the data sets in a number of linked views. These visualizations include the Document Viewer, the Node Graph View, and the Calendar View. Finally, PUNGA provides a Provenance View that displays quantitative values that summarize the analysis session and more importantly help in analytic provenance.

    Quick-Cut Video Editor

    Videos are great tools for teaching, learning and communication; however, because video editing is time consuming and tedious there are few people who take on the task. Utilizing the advances that have been made in video editing tools and algorithms we could help to ease this burden. Quick-Cut is a video editing tool focused on narration videos. It attempts to cut down the time it takes to edit videos together, while still producing a quality final cut. The tool specifically targets improving the time for: loading videos, aligning videos to specific points in the audio, and creating optimal cuts with optimal video quality.

    R&D at the Interactive Media Technology Center

    We will be showcasing a variety of projects that highlight our applied research and development at the intersections of wearable computing, machine learning, smart textiles, internet of things, virtual/augmented reality, health and wellness, games with a purpose, educational technologies, assistive technology, connected living, and the arts.

    Rapport: Pediatric Patient and Family Oriented Radiology Report

    Diagnostic radiology reports are increasingly being made available to patients and their family members. However, these reports are not typically comprehensible to lay recipients, impeding effective communication about report findings. Rapport is a prototype system that aims to facilitate communication about radiology imaging findings among pediatric patients, their family members and clinicians in the clinical setting.

    Re:Activism Atlanta

    The Pervasive Games Group of the {egg} presents their new design of the big urban game Re:Activism, originally developed by the PETLab at Parsons The New School. This modular design of Re:Activism gives players a condensed experience of activist history in Atlanta. Players travel around to downtown sites that are tied to historical moments of activism and complete challenges related to those events.

    Real Time Aggregation of Keywords from Censored Posts on Sina Weibo using Hadoop and Cron

    With the increasing presence of censorship on Chinese social media, it is imperative to provide the users of platforms such as Sina Weibo a way to freely share information without alerting the censors and systems of surveillance on social media. The aim of this project is to implement a Real-Time Keyword Aggregator that collects keywords that have most likely resulted in censorship of posts from various publicly available archives of censored sina weibo posts. In this work, utilize a Distributed Computing based technique to identify additional possible keywords from the posts using a TF-IDF based technique. The result of this project will be a large, continuously populated and curated homophone dictionary for currently censored keywords on Sina Weibo.

    Real-Time Dispatching on Atlanta Streetcar


    The streetcars run in the heart of downtown. They are subject to unstable operating conditions caused by traffic congestion, basketball games, and obstructed right-of-way. These perturbations make the Atlanta Streetcar prone to streetcar-bunching, which causes undue passenger wait and crowding. We have developped a real-time dispatching method that considers every streetcar on the route to dispatch them with even headways, while maintaining a high frequency of service. The dispatching method will replace the current schedule on the Atlanta Streetcar as a case study for several weeks. We will evaluate the impact of real-time dispatching on operations, driver behavior, and passenger waiting time.

    Real-time Eye Contact Detection

    Using Google Glass to detect eye contact in real-time.

    Reconciling History

    The Spring 2016 television miniseries, The People vs. OJ Simpson: American Crime Story, captured imaginations and stirred memories. The series was based on the murder trial of football legend OJ Simpson and the resulting media circus. While watching, either viewers were remembering the events of the trial or enjoying the show’s fictionalized narrative since they were too young or born after the trial. Since viewers are either re-living or “experiencing” these events for the first time through the series, can first-hand accounts and archived media be used as insights in a multimdia companion to dive deeper?

    Redesigning Internet Access from the Ground-Up in Bangladesh

    Lab: TanDEm

    The internet, mobile, and social media technologies have been successful in connecting large segments of the world's population, generating wide access to global networks of information. There are several clusters of unconnected/under-connected populations across the world, however, who do not have access due to financial, social, or political limitations. In response, state, industry, and grassroots initiatives are being put in place with the aim of transforming the current state of access so as to advance towards a global free-market economy, increased digital literacy, and more equitable online participation overall. Multinational technology corporations such as Facebook, Google, and Amazon have invested towards providing socioeconomically disadvantaged communities with increased, free of charge access to information and communication services. In this research, we look at the unexplored media and information sharing practices in the rural regions of Bangladesh, how these unconnected/under-connected populations find means to get online. We also argue that the "offline" practices for media sharing should be factored in the design of internet access.

    Redesigning the Career Fair Experience

    As students, many of us regard career fairs as vital events for securing job and internship opportunities. However, there are several frustrating aspects of career fairs, which can make the experience less enjoyable and less efficient. We aim to improve the overall experience of career fairs while considering the perspectives of multiple stakeholders, through service design.

    Reducing Cognitive Load to Improve Learning to Program

    Cognitive Load is the amount of "processing" your brain does when learning something new. This project investigates ways to lower the cognitive load while learning to program.

    Requirements Engineering for Privacy, Security and Compliance in Data Science Research Projects

    This research addresses the privacy, security, and compliance challenges faced by university researchers and ethics review boards when working on data science projects. Due to the emergent properties of big data, researchers regularly re-evaluate and modify their goals. These changes must be reflected in the project’s governing documents, including research protocols, consent forms, privacy and security policies, and data-use agreements. These documents must be consistent, must cater to diverse and sometimes conflicting stakeholder needs, must be compliant in a complex regulatory landscape, and must ensure the privacy and security of research participants. Consistent involvement by a privacy and security expert in every research project, although effective, is not a feasible solution. The goal of this project is to explore whether requirements engineering can be leveraged as a potential solution to these challenges. Requirements engineering can not only help align stakeholder goals with a project’s governing documents but can be used to develop tools to enable researchers and ethics review boards to better address privacy, security and compliance in research protocols.

    RERC TechSAge: A Mobile Application to Measure Gait Speed

    Multiple studies have shown a consistently strong association between gait speed of frail older adults and negative functional (e.g., survival) and activity outcomes. However, health care professionals have been slow to measure this physiologic parameter, largely due to the lack of a simple, standardized way of measuring it. The purpose of this project is to develop a reliable, simple, and cost-effective mobile app to measure gait speed and demonstrate the feasibility of this measure as a predictive tool to identify risk of functional decline and activity limitation in frail elders who are aging with ambulatory disability.

    RERC TechSAge: ALIGN - Application for Locational Intelligence and Geospatial Navigation

    This project will bring together the static (e.g., sidewalk condition, street networks, characteristics of land use) and dynamic (e.g., traffic volumes, weather, and light quality) characteristics of the environment that are monitored close to real time into one decision support system for route planning. The evidence-based app will enhance community mobility by providing up-to-date information about the critical environmental attributes that could affect community mobility and allowing the user to provide information about their own abilities and personal preferences to determine a route that optimizes those preferences. Having such a tool can support successful aging through maintaining community mobility even as functional limitations result in new and pervasive environmental barriers. The long-term goal is to refine and expand the app to provide broader applicability to a wider range of users.

    RERC TechSAge: GatePal - A context-aware airport guide app for all

    Lacking of information provided about airport environments and route conditions makes it difficult for air travelers to plan indoor trips at airports and anticipate possible environmental barriers, which could lead to navigation difficulties. An application that provides real-time environmental information has been designed to facilitate trip planning at airports for travelers, especially those with functional limitations. In this project, we conducted a set of studies to define and further understand the problem of airport navigation and developed a mobile application as the solution. The application will be tested to evaluate its usefulness and usability.

    RERC TechSAge: MS Assistant - A Health and Wellness Self-Management App for Individuals Aging With Multiple Sclerosis

    A majority of individuals diagnosed with multiple sclerosis (MS) experience a significant decline in their abilities due to the progression of MS after five years post-diagnosis. Following this period, they need to learn how to cope with the functional limitations caused by the disease and how to age with MS. They must manage effects of the disease on their lives on a daily basis. Self-management activities can help with this challenge.

    MS Assistant is a health and wellness self-management app that assists individuals aging with MS to understand their disease by monitoring all the factors that possibly contribute to the symptoms and keeping track of the changes and what causes them in order to identify triggers and patterns. All that data is available to their healthcare providers with the alerts sent to them in a case of emergency or a special need. In addition, the app integrates storing all the relevant health and wellness data in one place, communicating with other individuals with MS to get the information and support from others who understand what they are going through, setting up the goals and finding the needed MS-related information, playing the games to improve their cognition and balance, remote health monitoring, and calling emergency contacts. This evidence-based app provides their users with the complete health and wellness picture based on its input data, and a way to self-manage MS. It presents a holistic and comprehensive self-management tool for its end-users. The long-term goal is to develop the version of this app for the healthcare providers, caregivers, and family members, and to apply this self-management tool to other chronic diseases. 

    RERC TechSAge: SmartBathroom

    The needs and abilities of people who are aging with progressive chronic conditions, such as MS, Parkinson's, ALS and Arthritis fluctuate from day to day. Yet, even when they have supportive AT, such as grab bars, to compensate for functional limitations, those features are fixed, only able to support some abilities, some of the time. The purpose of this project is to develop a SmartBathroom environment capable of assessing an individual's abilities at any point in time and spontaneously adjusting supportive environmental features to accommodate those abilities.


    Juvenile idiopathic arthritis (JIA) is the most common cause of chronic childhood arthritis, affecting ~1 in 1000 children. Patients accurately reporting the number of joints with pain, swelling and limited mobility, and global assessment of their disease, is vital for assessment of affected joints, accurate diagnosis, decision making and management by providers, evaluation of effectiveness of intervention, change or escalate therapy based on the American College of Rheumatology guidelines, outcomes and comparative effectiveness studies and many disease activity scoring systems.

    We have developed an IOS application to capture joint health information and disease progression with a focus on using game design to promote patient engagement.

    This project was supported by funding from the 2013-2014 center Grant from Institute for People and Technology (IPAT), cosponsored Children's Healthcare of Atlanta, Emory Department of Pediatrics and Georgia Institute of Technology

    Robotic Co-Mediators for Parkinson’s Patients

    This project aims to ensure that the dignity of a Parkinson’s patient is maintained during interactions with caregivers using a robot as a co-mediator. The mediator will hold a mental representation of the patient’s and caregiver’s emotional state and react accordingly. The co-mediator will express the patient's internal emotional state through spatial and emotive gestures.

    Room-scale Video mixed Augmented Reality

    We are creating a platform for experiencing room-scale augmented reality through head-mounted displays. Prototypes will involve various tracking methods to interpret the user's gestures and movements. This project will be a basis for our research in the following topics:
    - Developing a collaborative workspace
    - Solving multi-content issues
    - Creating content for AR

    ROSS: Responsive Objects, Surfaces, and Spaces

    The Responsive Objects, Surfaces, and Spaces (ROSS) API is a way for tangible applications to operate seamlessly across a variety of tangible input devices and platforms. It allows applications to exchange information about the devices they are running on and obtain real-time data about tangible and touch interactions from other devices. In a ROSS world, you can use your mobile phone as a controller to play games on the digital coffee table in your living room; and your guests can join in with their phones too.


    In 2008, Pakistan took down YouTube from entire Internet for nearly three hours. They did this using BGP, the border gateway protocol, the IP-to-network address book protocol of the Internet. They announced that they owned  YouTube's IP addresses, even though they did not own them. RPKI, the resource public key infrastructure, is the first step in addressing this issue to only allow owners of IP address to announce where they are located. Unfortunately, implementing RPKI is not a trivial task, and we are working on making a "cookbook" on how to properly deploy RPKI on university campus routers.

    RSketch: Streamlined Mars Rover Path Planning

    Large, multisensor datasets are available covering a large portion of Mars. Analysis and display of these datasets are currently in use for path planning tools that provide a precise, low-level visualization that fosters precision planning for Rover Planners at NASA Jet Propulsion Laboratory. However, these visualizations do not foster path planning at a higher level of abstraction. In addition, planning a path uses a non-intuitive process of generating rover commands, simulating them, visualizing the results, and then tweaking the commands until the path looks correct.

    RSketch expedites path planning for JPL Rover Planners  by allowing them to intuitively generate and assess rover paths. The path generation process in RSketch must be influenced by traversability measures, incorporate and visualize multisensor data used for the traversability analysis, enable rapid path generation and comparison between alternatives, and export generated paths to mission operations tools.

    The prototyped RSketch system aims to provide a streamlined path generation and analysis tool for Rover Planners. The tool uses processed data, raster images, and rover state files as the basic dataset. RSketch uses this dataset to provide the Rover Planner with situational awareness of the rover’s state and the surrounding environment in a 2-dimensional map space. The Rover Planner can select and modify the display of the overlaid data on the map, as well as view annotations such as the long-term path.

    In order to plan out a path, RSketch provides Rover Planners a simple sketching capability to drop and modify waypoints that define a driving path. These waypoints can later be imported into the rest of the Rover Planner’s system as localized waypoints – other parts of the Rover Planner system then generate low-level rover commands from the exported path in RSketch. This export feature allows Rover Planners the ability to integrate with the rest of their system.

    When sketching out a path, Rover Planners are able to visualize data along the entirety of the rover path. The data is visualized in two different forms: directly on the path and along a slideout panel. Both of these visual forms afford varying analysis modalities. The data along the path, coupled with path sketching affords the ability to make informed decisions on how a modification affects the rover’s planned traversal. The slide-out graphs allow Rover Planners to conduct summative analysis at different chronological points of the path planning process by visualizing all data parameters at once.

    SayWhyPoll & Tangible Anchoring

    Recently featured in Georgia County Government magazine, the SayWhyPoll mobile app enables elected officials, civic leaders, and media producers to engage with constituents and audiences either remotely or face-to-face using surveys that tightly couple close-ended survey items with rich media, such as video. The SayWhyPoll is designed to increase opportunities for public debate on civic issues, but is also suitable for pure entertainment topics, such as sports and lifestyle. Once people contribute their viewpoints, Tangible Anchoring enables the results of this experimental survey technique to be easily explored and discussed via interactive tabletops using information visualizations designed for television programs and public meetings.

    Location: Synlab, TSRB 209

    SCADE - Supportive Computational Analysis, Discovery and Exploration

    SCADE is a visual text analytic tool. The goal of the project is to help analysts make sense of a larger number of text document while tracking the analyst's provenance.

    Scheherazade Story Generator

    Story generation is the problem of automatically selecting a sequence of events that meet a set of criteria and can be told as a story. Story generation is knowledge-intensive; traditional story generators rely on a priori defined domain models about fictional worlds, including characters, places, and actions that can be performed. Manually authoring the domain models is costly and thus not scalable.

    We present a novel class of story generation system--called an Open Story Generator--that can generate stories about any topic. Our system, Scheherazade, generates plausbile sounding, but fictional stories about real world situations. It automatically learns a domain model by crowdsourcing a corpus of narrative examples and generates stories by sampling from the space defined by the domain model.

    Scheherazade can also be used to create interactive narratives in which a player gets to choose the actions for a particular character in the crowdsourced story world. See a video of the system in action: https://www.youtube.com/v/znqw17aOrCs


    Sketching plays an important role in learning in the sciences. The process of sketching can help students think about and better understand scientific concepts. By sketching collaboratively, students can also compare their mental models with each other and share them with instructors in order to further enhance their understanding. What if these sketches could come to life so that students could experimentally test out and iteratively refine their models of natural phenomena and systems? We are designing SciSketch, a tabletop tool for sketch-based problem-driven collaborative learning in the sciences. The system tracks multiple pen inputs on a tabletop display surface and can transmit sketch data to a remote computer. The first prototype provides basic functionality of digital sketching tools, such as copy, paste, and playback. We study how such a tool could be incorporated into the classroom environment for undergraduate courses in biomedical engineering.

    Scraping big data without API

    Using scrapy a python framework to scrape Sino Weibo without the API. 

    SDN orchestrator driven traffic engineering for adaptive bit rate video streaming

    We will demonstrate a Software-defined networking (SDN) controller that orchestrates traffic engineering for adaptive bit rate video streaming. An OpenFlow-enabled SDN architecture in combination with MPEG-DASH constitutes an optimized and seamless video streaming platform.

    The GENI infrastructure was utilized to evaluate video traffic over current network architecture and SDN architecture.

    Search and Rescue Drone Interfaces

    When a person goes missing, the ability to quickly canvas a large area is crucial.  However, many search and rescue missions occur in rough terrain where it can be difficult, slow, and even dangerous to cover on foot.  Drones have become a cost- and time-effective solution, able to quickly fly over areas of interest and provide video feed from a mounted camera.  A pilot and spotter monitor the video feed through an app on a tablet or phone and watch for clues which might indicate where the missing person is located.  There are several apps already available which cover various tasks involved, but none are considered to be complete.  Through interviews and participatory design, a new mobile interface is created to cover all user needs in streamlining missions.  

    Seed: Sensors and electronics educational database

    Seed is a sensor and electronics educational database developed for use at the Interactive Product Design Lab. The lab teaches designers electronic prototyping skills-- Seed assists in this mission my providing information on electronic components through an online database, physical RFID card library, and RFID sensing unit. Parts are categorized into 6 groups: logic, power, input, output, tools, and projects. When a student needs information about a component, they can select a card and place it on the RFID reader. This brings up an information sheet on that component, complete with step-by-step start guides, wiring diagrams, sample code, and even past projects. 

    SentenTree: visualizing large-scale social media text

    The growing popularity of social media makes it increasingly difficult to keep up with the huge volumes of information they produce. We present SentenTree, a novel visualization technique that helps people gain a quick understanding of the key concepts and opinions expressed in a given social media text set. SentenTree can be used by both casual social media users and professional analysts.

    Silent Speech Recognition

    In this study, we address the problem of performing continuous speech recognition where audio is not available (e.g.,due to a medical condition) or is highly noisy (e.g. during ?re?ghting or combat). Our Tongue Magnet Interface (TMI) uses 3-axis magnetometers to measure the movement of a small magnet glued to the user’s tongue. Tongue movement corresponding to speech is isolated from the continuous data by comparing the variance of a sliding window of data to the variance of signal corresponding to silence. Recognition relied on hidden Markov model (HMM) based techniques. Using a custom headset with four magnetometers placed close to the cheeks of the participant, a maximum user dependent recognition rate of 99.8% is achieved for a ?xed phrase set of 12 sentences spoken by able-bodied participants. The average accuracy across four users is 95.9%. Using the single magnetometer aboard Google Glass, a commercial wearable computing device worn at eye level, one of 12 phrases could be selected with 93.8% average accuracy. To improve the latter recognition result we introduced a new interface, known as the Outer Ear Interface (OEI), which captures the lower jaw movements by measuring the deformation it causes in the ear canal. This measurement is done using a pair of infrared proximity sensors, one in each ear. We hypothesize that combining features from both interfaces will improve accuracy results significantly.

    Situated Anonymity: Anonymity, Ephemerality, and Hyper-Locality on Social Media

    Anonymity, ephemerality, and hyper-locality are an uncommon set of features in the design of online communities. However, these features were key to Yik Yak's initial success and popularity. In an interview-based study, we found that these three features deeply affected the identity of the community as a whole, the patterns of use, and the ways users committed to this community. We conducted interviews with 18 Yik Yak users on an urban American university campus and found that these three focal design features contributed to casual commitment, transitory use, and emergent community identity. We describe situated anonymity, which is the result of anonymity, ephemerality, and hyper-locality coexisting as focal design features of an online community.

    Typically, commitment to an online community has been characterized as either bond-based commitment, meaning attachment to specific users within the community (like on Facebook), or identity-based commitment, meaning attachment to a topic or characteristic—such as home-town pride, surviving cancer, or anime fandom (like 4chan). However, when a community deviates from established practices and thrives, it introduces a compelling opportunity to probe standard understanding. This work extends our understanding of use and identity-versus-bond based commitment, which has implications for the design and study of other atypical online communities.

    Situated healthcare - The case of neighborhood clinics

    Lab: TanDEm

    Our group explores the concept of place as a determining factor in the design of healthcare interventions that seek to target specific underserved communities. Our ethnographic study on healthcare access in marginalized communities demonstrates that the prevalent medical conditions, informedness about healthy behaviors, medical practice, and perceptions around health are highly situated and influence how and where people seek healthcare. We use our findings to guide the design of interventions that take into account this situated nature of healthcare.

    Sketching to understand the notional machine

    Foundational math, physics, biology, and chemistry instruction all have established traditions of using sketching and drawing to solve problems. From long division to Punnett squares, from free-body diagrams to molecular structures, students create diagrams and manipulate them to problem-solve. Introductory computer science and programming classes lack such well-developed and widely-used methods, even as computer science classes grow in popularity.

    Sketching and drawing may be especially useful for learning programming, since students face many difficulties understanding the process of code execution, termed the notional machine. Students often have an incomplete mental model of the notional machine, and this misunderstanding of programming language semantics leads to errors in both reading and writing code.

    If students sketch the process of code execution, they may be able to better track the process of the notional machine. We have found that creating certain types of sketches is correlated with success on code reading problems. We are investigating the following questions:

    • What methods do students use to sketch?
    • Why do students choose to use a certain technique?
    • Does sketching improve student understanding of the notional machine?
    • Can sketches allow instructors to identify where students misunderstand the notional machine?


    Smart Ballet Shoe: Transforming adult ballet learning with wearable tech

    The project seeks to improve the current model of ballet training with wearable technology integrated into existing dance garments-- socks, soft ballet shoes, and legwarmers that sense movement and give real-time feedback to the dancer.Long term goals of this project include 1) preventing injury by using garment calibration to teach dancers about their body’s natural limits, and give feedback when they are not dancing safely, 2) speeding the development of ballet technique basics by monitoring proper weight distribution, balance, and pointing of the toes, and 3) providing more individual guidance than encouraging students to return to class, answering questions, and building confidence. To date, the sock is the sensing layer- embedded with pressure sensors and an IMU to detect balance and positioning of the leg. The ballet shoes provide feedback- directing the dancer’s movement with lights and haptics.. The legwarmer provides unobtrusive mounting of the microcontroller and battery while connecting all parts of the system through I2C connections.

    Smart Cities - Metro SDX

    Smart communities have many different network providers, often sharing underground conduits or telephone poles. Their proximity cannot be closer in many cases. Unfortunately, in order for one network to send traffic to the other, providers often much connect through a third party, or must send their nearby city to exchange traffic. Further, a network may fail, and there is no alternative for the end user other than to wait for their provider to fix the failure. In order to improve reliability, we introduce the MetroSDX network design with leads to better network resiliency through "roaming on the wire" and lower inter-network latency by interconnecting networks opportunistically all over the city.

    Smart Cities and Active Citizens

    A project exploring how people who run in Atlanta might use data generated by city sensor nodes and other sources. 

    Smart Cities: Design and Human Values

    Recent discourses on smart cities have been primarily focused on the deployment of technical infrastructure such as sensor installation, data collection, and security measures. These practices, however, are accompanied with tacit and explicit ideas about ideal cities and human values. This paper presents the preliminary results of an ethnographic study that looks closely at the North Avenue Smart Corridor in Atlanta, Georgia aimed at unpacking the driving ideas behind smart cities initiatives and critically engaging its key assumptions of progress and efficiency. By understanding the narratives surrounding smart city development in Atlanta, we can begin to broaden the conversation around smart cities and close the gap between technologists, designers and the public.


    This project involves the development and evaluation of a mobile content
    delivery system. Using small, unplanned moments throughout the day, we endeavor to increase the ability of hearing parents with deaf children to recognize and produce American Sign Language vocabulary.

    SMILE (Systems Using the Mind for Latent Expression)

    SMILE (Systems using the Mind for Latent Expression) is developing BCI-controlled wearable technology for expressing emotion

    SMSPress: An SMS Research Study Management System

    Text messaging is a functionality that has been widely used in user studies due to its presence in smart phones and feature phones alike. However the ability to use cell phones for data collection has been limited to researchers that have programming abilities. We present SMSPress, a web-based SMS management system that allows non-technical users to configure SMS studies, and schedule text messages for delivery to participants via a web-interface. We establish the requirements for such a system, and present the results from two case studies. One is a four month mhealth deployment with cardiac patients; the other is a usability evaluation of a study to help parents keep track of their child's development.

    Social Media Civic Engagement

    We are developing technological platforms, human and social processes, and the underlying creative abilities to leverage social media and digital networks to enhance democracies especially in Africa. Our current work has focused on participation in and monitoring of elections while future work will expand that frame seeking to foster broader and deeper civic engagement especially among youth, the promotion of government accountability and efficacy, and the constructive creative ability to hack democracy. The Social Media Tracking Center (SMTC) encompasses a process to monitor and respond in real-time to reports from systems such as Twitter, Facebook, Google+ and Ushahidi. The technological platform facilitating the SMTC is Aggie, our social media tracking software, which allows the aggregation and integration of social media data streams along with real-time trend analysis and visualizations.

    Social Media in Cuba: Who is speaking, and who is listening?

    Social Media is changing our world. Traditional narratives paint a picture of many-to-many democratization of information--making ideas, opinions, education, and knowledge available to many people in many places at any time. This research examines whether this is&nbsp;actually the case, especially in countries with a totalitarian government. Through an interview-based study of Cuban participants on social media, this project explores both the limitations and opportunities of cyber networks as tools for individuals to create collaborative narratives and affect social change. By shedding light on the situation in Cuba, this research seeks to&nbsp;illuminate the role of online social networks as tools of persuasion and influence in a global setting.

    Social TV

    Social TV is a mobile application that works in conjunction with watching TV. It is targeted to users who may be living in a new location and watch TV alone, or to users who want to increase the social aspect of watching TV. Through integration with Facebook, the mobile app presents a TV Feed where users can share information, such as screenshots, quotes and recommendations with other individuals in their network. The app has a TV chat feature, where users can engage in both synchronous and asynchronous chatting about what is occurring in the show of their choosing. Users have the option to add friends to their TV watching experience, expanding their social network. Other aspects of the app include the ability to keep track of your own and friends’ progress on TV shows, scheduling time to watch TV with others, and discovering new TV shows they may be interested in.

    Software Defined Exchange (SDX)

    The SDX project is applying the principles of Software Defined Networking and Infrastructure to the problems of peering between network operators and service providers. This work seeks to overcome the traditional limitations of peering protocols such as BGP to enable operators and their customers to express rich, application specific policies that facilitate the integration of cloud computing and virtualization into numerous applications.

    Solar System Sonification

    Students in the Sonification Lab and  Center for Music Technology designed Solar System Sonification, an auditory experience of the planets. Using non-speech audio to convey information, they built a musical model of the solar system.

    Planetariums typically rely on visuals with various levels of speech description, but have not explored using auditory cues to present information about space. Auditory displays, like the ones developed for Solar System Sonification, enable more immersive experiences and make information accessible to people with visual impairments. 

    The project offers two views: the solar system view and the planetary view. The solar system view looks at the whole system through how the planets and other objects interact. The planetary view focuses on properties of the planets themselves. 

    For the solar system view, attributes such as mass, length of year (orbital period), length of day, and distance from the sun are displayed. For the planetary view, attributes such as planet type, number of moons, number of rings, gravitational force, and the mean temperature range are represented. 

    See 2017 publication for details on the design of auditory displays for this project.

    Solid Sketch

    SolidSketch is a solid modeling program that enables users to rapidly construct 3D models through sketch and multi-touch input. The interaction design principles of SolidSketch are based on the cognitive science theory of enaction. This paper uses the enaction theory as a lens to describe why the interaction designs of conventional CAD tools often fail to support early stages of the design process.  We argue enactive interactions would support design creativity by enabling rapid iteration and continuous feedback throughout a flexible design exploration. SolidSketch is a unique sketch-modeling program that analyzes and interprets freehand sketch gestures to execute commands. The system infers the intention of the user by continuously analyzing the surrounding context and user’s behavior.  The program only provides a minimum set of GUI components to encourage the user to focus more on the canvas area rather than navigating intricate interface elements.

    Sonified Fantasy Sports

    The Sonified Fantasy Sports project has been exploring various ways to add sounds to online (web or mobile apps) fantasy sports in an attempt to make a more immersive user experience while also adding to the accessibility of fantasy sports for visually impaired or print disabled users. After identifying information needs and various strategies employed by users (who ranged from beginners to power users) we were able to identify a hierarchy in which to present information about &#39;my team&#39; and &#39;players&#39; using sound. Ongoing investigation is exploring additional ways to employ optimal soundscapes that will result in the most seamlessly integrated audio-visual experience while offering as much accessibility as possible

    Sound Happening

    Sound Happening is a collaborative music-making installation that allows several people to explore and create music in a space by playing with colorful bouncy balls. Using a webcam and Max/MSP, Sound Happening tracks each ball’s location relative to the space to manipulate and trigger various samples, resulting in intriguing sound combinations that are constantly changing as the balls move.

    Space Table: The life of a star

    The Space Table is an interactive informal experience to teach children about the formation of solar systems and how gravity, and mass play a role in their creation. Additionally, the project will explore the concepts of different celestial bodies such as stars, planets, asteroids, neutron stars, black holes and others.

    The installation will make use of three tangibles, one for a different star size. Interactors will use the tangibles to “stamp” a star on the table, which will spawn a digital star. After the star is created, interactors will be able to create asteroids and other space debris by sliding their fingers on the screen, which will in turn create bigger objects (such as moons or planets) when they collide with other small celestial bodies. Once the system is created, the expectation is for interactors to experiment with the celestial bodies, leading them to the discovery of cosmological concepts.

    SpaceSketch - Multitouch Exploration of Urban Public Safety Data

    Visualization tools for spatio-temporal data utilize map-based representations to help a user understand trends and outliers within a given region over time. Multitouch visualization tools allow us to recreate many of the capabilities of sketching directly on maps while still taking advantage of computational models of public safety. We will be demonstrating SpaceSketch, a multitouch approach to spatio-temporal visualization. Visitors will be allowed to explore crime and transit data in the city of Atlanta using our high-resolution Surface Hub Interface.

    SPAN: Increasing Social Participation of Teens w/ Brain Injury via App-based Peer Coaching and Training

    Social Participation and Navigation (SPAN) is an intervention that integrates mobile app-based training and “peer coach” support around key aspects of social participation, including social communication and problem-solving skills, identification and remediation of barriers to participation, and establishment of social connections with others.

    SPAN will focus on social participation as our core outcome, working with teens with TBI to identify and prioritize their own social participation goals and supports and barriers to participation, and teach strategies that teens can use to achieve their participation goals. Training of peer coaches and use of smartphone and web-based technology ensures that teens with TBI are directly supported in everyday life (school, community, and work), according to their own preferences and schedules, rather than in clinical settings where opportunities for social participation are limited.

    This project is a collaboration with Cincinatti Children's Hospital Medical Center, Tufts University, University of Wisconsin-Madison and Children's Health Care of Atlanta.

    Sparse Tangibles

    Sparse Tangibles investigates the use of novel tangible and gestural interactions for making sense of large biological datasets. Our current prototype employs active tangibles in combination with a large multi-touch tabletop displays to navigate and visualize gene regulatory network data from the BioGrid database.

    Stand Up: Device and App Solution to Track Standing & Sitting

    Today the public has become increasingly aware of the health impacts of a sedentary lifestyle.  Sedentary behavior research continues to uncover the detrimental effects of sitting and has shown that the average American now sits during the day more than the time they spend sleeping.  Activity trackers have reflected this shift in public awareness by beginning to track standing as well as the typical health metrics, like steps.  The most prominent example of this is the Apple Watch with its Activity rings - the last ring is for stand hours.  This project is to explore the best way to visually represent how much a user has stood and sat during the day.  The design will be informed by both the intended user and by expert opinion.

    Previous work on this project led to the creation of a simple activity tracker device that accurately determines when a user stands and sits.  This project will take that device, connect it to an iPhone app, and explore the best visual representation of the data.  Once a design has been created, it will be put in front of users in a usability test.  The test will focus on exploring whether the design effectively communicates to the user if their current patterns of standing and sitting are healthy. 


    Compete in an immersive steam punk environment hot-air-balloon race against your arch nemesis using a custom VR controller. The evil nemesis has sabotaged your balloon and stolen your robot, now you must work against the odds to keep your balloon afloat, save your robot, and prove you are the best inventor in Mountaintown! This game makes use of a specialized controller that combines a familiar motion with pressure sensitive pump to provide realtime feedback and a deeper immersive experience.

    Steering Multidimensional Scaling for Exploring Ego-network Evolution

    Dimensionality reduction (DR) is often used for exploring the evolution patterns of a collection of dynamic ego-networks. However, DR often lacks flexibility: as analysts’ questions evolve during data exploration, the low-dimensional projection remains static, limiting the depth of exploration. To address the inflexibility of DR, we designed a data transformation pipeline which enables analysts to transform dynamic ego-networks into event sequences for steering MDS to create different scatterplots of dynamic ego-networks. Based on the data transformation pipeline, we developed Segue, a visual analytics system for exploring different evolution patterns of ego-networks.

    Story Map

    A second-screen companion app to support viewers in following a densely populated storyworld, with prototype based on the FX Series Justified.

    StrangVR Things

    StrangVR Things is a VR prototype that crafts an immersive and engaging experience within the narrative world of Netflix’s original series, Stranger Things. The user takes on the role of Eleven and must escape a hostile environment by leveraging her telekinetic powers. Our design aims to allow show viewers to explore the dangers of the Stranger Things world through Eleven’s eyes and endow viewers with the same mysterious powers.

    Stream Cleaning: Simulation Revealing Impact of Urban Creek Contamination on Nearby Residents

    Conversations with employees and volunteers at the Proctor Creek Stewardship Council in Atlanta, GA revealed that contamination in urban creeks is a major problem. This is especially an issue when neighborhoods sit along sections of the creek that get contaminated and dumped in.

    Most often, this happens in lower-income neighborhoods where many residents are unaware of trash/recycling guidelines, the local government doesn’t feel accountable for the residents, and local businesses view the area as grounds for dumping. This creates a dangerous cycle where the water gets contaminated and sewer drains get clogged, leading to flooding, the flooding leads to mold and asbestos in homes, and the next time homes flood, the mold and asbestos goes back into the creek water. The creek therefore become a health and safety hazard.

    With the simulation, interactors learn what they can do in the real world to help solve this problem by taking on the role of a resident in a low-income urban neighborhood along a contaminated creek. Through the replayable simulation, they realize what types of events (with varying levels of control) occur in the neighborhood and how their choices impact the creek’s cleanliness and the community.

    Streetcar Sensors

    Starting this summer, the Atlanta Streetcar began using a real-time dispatching method developed at Georgia Tech that eliminates the need for schedules and cuts down on passenger wait times. School of Civil and Environmental Engineering Assistant Professor Kari Watkins and Ph.D student Simon Berrebi developed an algorithm that ensures each vehicle is spaced evenly along the 2.7 mile route in downtown Atlanta, maximizing the frequency of service. Unlike the previous method, the Georgia Tech algorithm uses real-time information.

    One problem the researchers faced was the “urban canyon” effect where the GPS reception reverberates on buildings, or is blocked entirely, creating an error in the signal and causing the apparent GPS location of the vehicles to wander. Watkins and Berrebi worked with Research Scientist Bill Eason of IPaT and the Georgia Tech Research Network Operations Center (GT-RNOC), and GT-RNOC Co-director Russ Clark to use a barometric pressure sensor. They found that the newest sensors, designed to be built into next-generation cell phones, are sensitive enough to detect changes in elevation of under a meter (3.3 feet). The sensor measures changes in elevation and allows for more accuracy in pinpointing the location of a streetcar in real-time.

    Student Well-being Reflection Tool

    The StudentLife project has been developing ways to combine multiple streams of data about student habits into meaningful holistic analyses of individual’s well-being.  Communicating those results to a student population poses a challenge to provide information in a legible form and provide meaningful and helpful, and importantly not harmful, feedback to enable students to improve their well-being.

    The objective of creating such a tool is to understand what elements are most useful and relevant to communicate to student users, and how to do so in a way that empowers them to benefit from that information, and not enter into ruminative states in which a negative affective response to awareness of a negative affective state becomes a downward spiral.  Therefore, a significant element of this project is understanding how people, especially students, respond to self-reflection, what elements of self-reflection can be harmful and helpful, and how to use that understanding to inform the development of a useful and engaging application. 

    A well designed reflection tool can provide a means for students to become more conscious of their mental state and mental patterns, reflect on their emotions, build capacity for stressful situations, and utilize resources during times of distress.  An effective means of building emotional intelligence, which would allow for better understanding and regulation of emotional well-being, provides utility for students who experience mood disorders as well as students who experience any kind of stress. A tool to utilize the data produced by studying student social media posts and provide beneficial reflection on emotions and well-being might take the form of an application that could be used in conjunction with social media to record and reflect on students’ well-being over time.  Such a tool would need to take into consideration a variety of factors, and will ideally be designed in response to the interests of students who wish to become more aware of their thought patterns and affect.  It could also provide motivation to participate in research regarding the kind of information such a tool would collect in order to provide feedback.

    This project consists of understanding the literature about how to provide helpful feedback.  Based on that information,  I have conducted an internet survey to gauge interest in a feedback tool, as well as different elements that could be incorporated into a feedback application.  I have also been collecting information on how providing study participants with feedback may incentivize participation in well-being related research, and the concerns participants express about using applications that track well-being.  I am building a prototype to test with student users to gauge responses to such an application.

    Supporting in-home collection and sharing of behavior evidence for diagnostic assessment of children with autism

    We have designed and evaluated a Clinician-directed Capture and Access system that can enable (1) parents to easily collect clinician-prescribed in-home behavior specimens(video evidence of behaviors) that have clinical utility, and (2) clinical experts to conduct diagnostic assessment for autism based on parent-collected in-home behavior specimens.
    Currently, in collaboration with an autism center, a clinical trial is being conducted to validate this remote autism diagnostic model on a larger scale.

    Sweet Auburn Digital Media Initiative

    Can locative media (Augmented and Mixed Reality, web applications, and social networking) serve as a platform for preservation of cultural heritage, informal education, and civic engagement?

    This is the question at the heart of the Auburn Avenue Research Project, a project that brings together researchers from variety of disciplines – including media theory, design studies, and human-computer interaction – to engage the above question in theory and practice. Through the creation of a tiered media strategy, the Auburn Avenue Research Project takes advantage of real world development project (e.g., new physical signage, street car) and potentials of digital technology to raise awareness of Auburn Avenue’s history an future trajectory, to increase the number of visitors to the neighborhood, and to support community preservation and revitalization efforts. Project objectives include:

    To explore the usage of locative media forms for their potential to increase civic engagement among visitors and residents.
    To make the rich cultural heritage and history of Auburn accessible to people by integrating new and old representational media.

    System for Wearable Audio Navigation (SWAN)

    The System for Wearable Audio Navigation (SWAN) serves as a navigation and orientation aid for persons temporarily or permanently visually impaired. SWAN is in the early stages of a software rewrite and technology upgrade. Interaction techniques are being prototyped in Virtual Reality (VR) to support preliminary user studies of new features.

    Tactile Teacher

    In a piano lesson, a student often imitates the teacher’s playing in terms of speed, dynamics, and fingering. While this learning model leverages one’s visual and even audial perception for emulation, it still lacks an important component of piano playing – the tactile sensation. We seek to convey the tactile sensations of the teacher's keystrokes and then signal the student's corresponding fingers. We implemented an instrumented fingerless glove called Tactile Teacher to detect finger taps on hard surfaces. Since finger taps generate acoustic signals and cause vibrations, we embedded three vibration sensors on the glove and use machine learning algorithms to analyze the data from the sensors. After a brief training procedure, this prototype can accurately identify single finger tap in a very good performance at above 89% accuracy, and two finger taps resulted in accuracy around 85%.



    TASC: Tangibles for Augmenting Spatial Cognition

    Spatial ability has been shown to be significantly correlated with interest and success in STEM fields. It also has been linked to embodiment in different ways. Tangible and embodied interfaces have been shown to support embodiment, including linking embodiment to changes in spatial ability. However, little research has linked the interaction design elements of tangible and embodied interfaces to specific effects on spatial cognition. Our research aims to gain a deeper understanding of the effects of tangible and embodied interfaces on spatial cognition and to develop interface protocols that enhance spatial ability training. Our current prototype employs tangible interaction with physical/digital blocks in a virtual reality environment to support perspective-taking spatial abilities.

    Tech Hustle: Understanding and Supporting Technology Learning and Employment

    Tech Hustle is a project to build upon indications from earlier project, Glitch. Glitch found that many low-income African American male high school students were more willing and able to engage in computer education when it was situated in an authentic work environments. The high school students in this previous work exhibited pride and excitement in being responsible workers, making their own money, and used work as a way to justify time spent on learning computer science and other more academic pursuits. We are researching programs to help young people set up their own side businesses (side hustles) providing tech support, web development, and computational and physical prototyping.

    Technology in museums: Avenues for personalized parent-child conversations

    Parents and children work together in museums and other informal learning settings to make meaning of the world around them. The parent-child conversations, the child's interests (islands of expertise) and the surrounding technology influence the overall informal learning experience. We conducted two studies at the Children's museum of Atlanta to better understand the interplay amongst those conversations, child's interest and technology. Drawing on the insights from the field studies and participatory design activities conducted with parent-child dyads, we created a prototype of a companion mobile app which may create avenues for having in-depth, contextualized and personalized parent-child conversations, which may, in turn, improve the informal learning experience.


    The Bag of Communities Approach: Identifying Abusive Behavior Online with Preexisting Internet Data

    Since its earliest days, harassment and abuse have plagued the Internet. Recent research has focused on in-domain methods to detect abusive content and faces several challenges, most notably the need to obtain large training corpora. In this paper, we introduce a novel computational approach to ad- dress this problem called Bag of Communities (BoC)—a technique that leverages large-scale, preexisting data from other Internet communities. We then apply BoC toward identifying abusive behavior within a major Internet community. Specifically, we compute a post’s similarity to 9 other communities from 4chan, Reddit, Voat and MetaFilter. We show that a BoC model can be used on communities “off the shelf” with roughly 75% accuracy—no training examples are needed from the target community. A dynamic BoC model achieves 91.18% accuracy after seeing 100,000 human-moderated posts, and uniformly outperforms in-domain methods. Using this conceptual and empirical work, we argue that the BoC approach may allow communities to deal with a range of common problems, like abusive behavior, faster and with fewer engineering resources.

    The Benefits of Robot Deception

    Deception is a common behavior not only in humans but also in animals. We focus on deceptive behavior in robotics because the appropriate use of deception is beneficial in several domains ranging from the military to a more everyday context. In this project, we proposed a taxonomy of robot deception and developed novel algorithms for robots' deceptive behaviors inspired by biological findings. In more recent work, we are developing computational models and conducting human-subject studies for a robot's other-oriented deception in the context of Human-Robot Interactions (HRI).

    The BoARd Game

    The BoARd Game is a room-scale interactive narrative designed specifically for the Hololens. In this narrative, the player is helping a couple of kids (non-playable characters) to escape a life-threatening board game. The game begins with the kids casting the dice and the game shrinking them to a small size after which they become actual characters in the game. The player will guide the kids through 5 separate puzzles that explore different interactions and designs to the end of the game which resets the world and returns everything back to the normal. The narrative is inspired from “Honey I shrunk the kids” and “Jumanji”.

    The Internet in Cuba

    With nearly four billion people still lacking access to the internet, efforts to expand internet access are growing rapidly across the world. Cuba remains one of few emerging nations where this access is still affected by historical trade embargoes and restrictions. Since the 2014 announcement of the normalization of relations between Cuba and the U.S., however, internet access in Cuba is increasing. This work is situated during this time of transition to explore the impacts of increasing internet access on individuals and communities living in Havana.  “The internet” in Cuba is currently made up of four separate components: slow access at places of work, content sold on USB thumb drives (“El Paquete”), an intranet custom designed by citizens (“StreetNet”), and public wifi hotspots available at exorbitant prices (opened in March 2015).  In this work, we seek to understand the particularity of each of these parts, and the bigger whole that emerges during this time of transition. The Cuban context provides for a compelling study on how on how a highly literate population with a history of state-controlled information, and thriving offline practices transitions to deriving meaning out of an online, global, networked infrastructure. Through ethnographic, qualitative research methods, this project aims to develop a ground-up, holistic understanding of the information infrastructures that have evolved in Cuba as a response to individual values and ongoing constraints. These findings will inform the design of technologies that are context-appropriate but also flexible enough for users to modify and appropriate them in meaningful ways.



    The Life and Death of Metadata

    In collections of scientific and cultural history that are too big to see, metadata act as virtual handles for rare and delicate artifacts from the past. At the Arnold Arboretum, a collection of long-lived trees, vines and shrubs managed by Harvard University, landscapes from around the world and across time are stitched together by metadata. However, metadata are worthy of study themselves. Created in varied social and technological eras, they register the organizational structures and values of their time. Through a combination of data visualization and interviews with Arboretum staff, this essay illuminates what metadata can teach us about their own social and material histories, as well as how to study collections digitally.

    The Light Orchard

    The Light Orchard is an interactive installation that invites people to walk into a grove of futuristic trees, lit with color. The trees are aware of the presence of people in their space, and can respond in many different ways. User can play different games, watch animations, and work together with different simulations, that allow them to easily collaborate, learn, and play together.

    The Lights of St. Etienne: An AR/MR experience in the cathedral in Metz, France

    The Augmented Reality experience, The Lights of St. Etienne, uses the AR-browser Argon as a platform for an embodied, location-based experience in St Etiènne Cathédral in Metz. The experience takes into account the cultural heritage and “hidden” stories embedded in its architecture, windows, and ornamentations. Lights focuses on five separate sites (geo-spots) in and around the cathedral to explore various dimensions of the cathedral, its history, and its place in Metz.

    The five spots are inside the cathedral, working with historical narratives relating the history of the cathedral and citizens’ experience of Metz at various points in time. 3D panoramas will be used to give the visitor a sense of the changing architecture of the vast church over the centuries. Photographic images, presented on the screen, will permit the visitor to examine more closely some of the stained glass located high overhead. Sound and music is prominent in the experience, changing as the visitor moves to the various sites. Ranging from the sounds of fire (cf. Bill Viola’s Fire Woman video work 2005 serves as example), to spoken narratives in 1st person point-of-view narratives of people telling personal stories relating to a particular moment. Favoring a situated personalized narrative, inspired by historical events and drawn from contemporaneous sources, the narratives present one view on the complexity of history that is centered on the cathedral. Music is also used in the application, underscoring contemplation or the sacral experience of being in the cathedral.

    Argon permits us to design using a variety of Augmented and Mixed reality techniques, as well as multimedia presented on the mobile device’s screen. This prototype was designed and produced by AEL researchers, students at the Metz campus of Georgia Tech (GT Lorraine), and students at Georgia Tech in Atlanta (US).

    The Object Ecology: designing edge cases for the Internet of Things

    Usually, objects are considered to be discrete instantiations of something unique and unitary, comprising one individual thing and not another. Some technological developments are making that assumption murkier. Ubiquitous computing and mobile devices have begun to shift how objects operate. As examples, clothing is being instrumented with computational capabilities, creating the fields of “wearables.” Networked appliances in the home become members of “the Internet of Things.” From bit players in the world, electronics and computation have augmented everyday objects into viable actors in their own right, active participants in the world that sense, report, and scheme to their own ends.

    By revealing these multiple obligations in more explicit ways, computational communication underscores the idea that objects in the world have always been members of multiple networks, involving themselves in social arrangements both subtly and dramatically. This simultaneous involvement in various networks—information, electronic, legal, cultural, material, and more—is what I call Object Ecology. What this ecological understanding of objects reveals is that objects cannot and should not be treated discretely. Instead, they must be considered as component members of an assemblage of actants.

    One ecosystem of objects that resonates deeply for most people is the home. ‘Domesticity’ is comprised of all sorts of objects: plates, furniture, heating vents, entertainment systems, family members, rugs and much more. By providing computational capabilities to materials in the home, the Internet of Things has entered this domain—brashly, but also intriguingly. It proffers a greater control of their environment to residents of “smart homes,” but access to this kind of technology is asymmetrical. Many communities and styles of living are excluded from the usual residential understanding of the Internet of Things. These outliers—cohousing communities, tiny homes, combination live/work spaces, homes with multiple adult roommates, and so on—offer a vantage to both critique contemporary IoT practices and provide a provocative set of sites to do design work from an ecological perspective. 

    The Shape of Story

    The Shape of Story is an interactive story circle experience in which participants collectively create a story line-by-line. Artificial intelligence in narrative understanding is used in conjunction with a symbolic visual language in order to visualize this story in real-time. The result is a communally created narrative art piece.

    Thinking Outside the Brick: Lifelong Learning Through Digital Play

    This MS project applies approaches from DIY and maker culture to lifelong learning principles. It aims to introduce practical workshop settings and guidelines for the use of rapid prototyping technology to continuous education of senior citizens.

    Tiamat Media ARG

    Tiamat Media is an ARG that was specifically designed for Dragon Con 2013, the largest multimedia fan conventions in the country. It was designed to fit within the socio-cultural context of the convention, appropriating established convention actions into the design.

    Our narrative revolved around the fictional company we created called Tiamat Media. Tiamat Media is a publishing company that specializes in publishing fan artwork. It operates under the guise of helping fans spread their artwork and encouraging fan empowerment. Unbeknownst to the outside, however, Tiamat’s true intentions are to steal these works and publish them as their own.

    The goal of the players is to peel back the facade of Tiamat Media and discover the sordid truth about the company. Players must then team up with those fighting against the company in order to turn the company from its evil ways.

    The game had three main game mechanic modules for players to solve: a casual picture tag game, a hide and seek game, and a artifact puzzle game. Each of these connected to each other through rabbit holes and once players unlocked the mysteries, they would receive information from characters that would help them take down Tiamat Media.

    Tiny Tinkering Platforms

    We are creating platforms that help people solve small-scale problems in their everyday lives. Here, gratification comes not from making something new and remarkable to show off to the world, but in making something unremarkable that nevertheless feels important to an individual or small community. When to water a particular plant, when hot coffee has reached a preferred temperature, when the mail has been delivered, and so on.

    We aim for an ecosystem that reduces hardware engineering complexity for small-scale, ubiquitous problems like these. It allows novices to prototype solutions quickly and straightforwardly. Today's "maker" platforms are priced for enthusiasts rather than ordinary people. Our platform needs to be very inexpensive to encourage applications that are never meant to be cherished. A focus on getting something done rather than on doing it.

    Transmedia Storyscape: An ecology of transmedia storytelling

    The story of The Ghost Club is bigger than just a feature film – it inhabits a complex world with a deep history and mythology that engages its audience members, inviting them into the Ghost Club storyscape.
    The concept of the Club, its team members, and the reality and rules of this world are introduced through a variety of non-traditional media channels – web series, social networks, online games, augmented reality mobile applications, and more.

    The Ghost Club transmedia storyscape generates a cohesive alternate reality that engages fans, encouraging them to discover, explore, and even participate in the world of ghosts and hunters.
    Storyscape components include:

    THE GHOST CLUB WEBISODES introduce viewers to the team and the show, and highlight story elements only hinted at in the feature film – including Noreen being a reporter who is secretly investigating the Club.
    TEAM FACEBOOK PROFILES & TWITTER FEEDS are where The Ghost Club team members post and tweet about the other investigations taking place during their final season. This establishes the rules of the show, the personalities of the investigators, and expands on team member relationships only hinted at during the feature film – such as Austin and Caitlin’s flirtations.
    THE GHOST CLUB WEBSITE serves as the “official” site for The Ghost Club, including the club genealogy, current team member bios, findings from past investigations, ghost tech diagrams, investigative techniques, and how-to tips.
    GHOST-PEDIA is a wiki allowing fans and amateur investigators to enter information about hauntings, ghosts, and investigation techniques.
    THE GHOST CLUB AUGMENTED REALITY APP: GHOST vs. CLUB is a mobile game that allows our viewers to either become ghost hunters and search for geo-tagged spirits or to become spiritualists who summon ghosts for the hunters to find.
    THE GHOST CLUB FLASH GAMES are a variety of online games that let fans try their hand at investigating ghosts. The flash games introduce the different techniques and equipment of ghost hunting.


    Under the cover of night, the player inflitrates the palace of a despotic ruler. Your mission: steal a prized sword without being caught by a guard. Trespass invites players to step into the shoes of an archetypal ninja and play through a thrilling interactive vignette of Japanese history. The story is based in part on historical accounts of the life of Ishikawa Goemon, a well known ninja of the 16th century. Our intention in building Trespass is to explore how players respond to spatial barriers in the virtual world. By utilizing HTC Vive and Unity, Trespass will immerse the player in grand digital rooms and subtle performances of stealth. Additionally, the game takes advantage of the affordances of VR by allowing players to physically walk around digital spaces that are "folded" to fit within the boundaries of a much smaller physical room.


    TuneTable is a responsive tabletop application with a tangible user interface. The intention is to teach basic computer programming concepts to middle school-aged and high school-aged students (9-16 years old) using physical blocks that work as snippets of code. TuneTable applies computational elements like: functions, parameters, and nested loops. Users compose short songs by building chains of blocks that represent code. Each block has a unique design on the bottom that, when placed on the acrylic surface of the table, is identified by the software using cameras mounted underneath the surface of the table. When the arrangement of blocks is recognized, the application outputs musical and visual feedback.



    Twitch Plays Improv

    Twitch Plays Improv is an experiment in participatory narrative creation that utilizes the live streaming video platform Twitch.tv as an arena for broadcasting improvisational theatre games. Drawing on the structures and implications of the “Twitch Plays Pokemon” phenomenon, Twitch Plays Improv digitizes the performative and participatory conventions of improv, allowing actors and audience members to collaboratively develop dynamic scenes and stories.

    Typeface Discovery for Designers

    This project will focus on the impact of typeface identification to facilitate the visual design process. It will help identify the issues that hinder designer's creativity, provide support for their acquisition of inspirations and relieve them from organizing collectibles, design artifacts. The expected prototype will reflect the mental model for typeface categorization, and make a designer's workflow smooth and enjoyable. 

    Understanding Approaches to Adolescent Health Education

    Lab: TanDEm

    Cultural taboos and inflexible social norms make it challenging to teach and communicate about menstrual health education in India. We present an investigation of current approaches used to educate adolescents about menstruation, examining the perspectives of parents, teachers, social workers, and health professionals for identifying design opportunities and potential for impact. There is also a palpable difference in attitudes regarding who must be taught, how, where, and when. We articulate factors that could shape access and receptivity to this knowledge. This project is in collaboration with scholars in India.

    Universal Design for Wayfinding

    Wayfinding in unfamiliar place has been challenging for everybody, including older adults and people with disabilities. This project investigates the barriers and difficulties people have with wayfinding systems, including environments, technology and their interaction with human. With understanding of the existing barriers and difficulties, this project also aims to provide design solutions from a universal design perspective. 

    Universal Threshold Object

    This project explores the possibilities, challenges, and benefits of using a tangible object as a controller and feedback device in an interactive television show. The Universal Threshold Object (UTO) enables interactors to realize emotional choices in a narrative world through physical action with objects in a way that emphasizes dramatic immersion rather than gamelike skill mastery. Research methods include prototyping based on real television content and iterative design and testing in a demonstration environment. Design strategies for interaction include mapping gesture to dramatic expectations and reinforcing physical presence in the virtual world. The project is in collaboration with the SynLab and was supported by Intel Corporation.

    Universe United

    Universe United is a second screen experience designed to bring transparency to transmedia storytelling, focused on connections between storytelling conventions such as items, characters, events, and locations. With this approach, we hope to enlighten both newcomers and veterans of a particular cinematic and/or television universe.

    Unpacking Informal Volunteerism: WhatsApp Use for Crisis Relief in Ecuador

    Lab: TanDEm

    When Ecuador was hit by a 7.8 intensity earthquake on April 16, 2016, the coordination efforts that unfolded relied heavily on the use and appropriation of social media such as WhatsApp, Facebook, and Twitter. While studies on informal volunteers in crisis situations have largely examined digital volunteerism and visible online activities, behind-the-scenes interactions among informal volunteers on the ground remain understudied. We present a qualitative interview study of how Ecuadorian informal volunteers self-organized to provide relief efforts in response to the earthquake. We found that informal groups of volunteers appropriated WhatsApp to articulate relief efforts within their groups and beyond. Drawing on our findings, we emphasize that the design of technologies for crisis response must consider how informal volunteers on the ground harness existing technology practices and situated expertise to address the ever-changing demands of the crisis relief environment.

    User Centered Design of a Patient Monitoring Dashboard

    Patient care happens in homes as well as away from them. Care providers spend a significant amount of time in trying to piece these different pieces together and come upto speed with the patient's current status. This user centered design of dashboard will provide them with a means for patient monitoring and help them get all the information that they need in that moment at a glance.

    At this time, this project focuses on achieving its goals in the context of Breast Cancer, specifically for Cancer Navigators.

    Using AR in Comic Art: Attention as a Commodity

    How can we use AR in combination with Comic Artwork to shift attention from the page to the screen and back? The project combines HCI with the nineth art: comics. It applies design criteria from comic scholars and pracitioners such as Will Eisner and Scott McCloud to develop effective AR designs for hybrid comic pieces. We will present a prototype sample of this project at work.

    Using Facebook Ad Audience Estimates to Study Psychosis Awareness in the US

    The objective of the study was to explore how the awareness of psychotic disorders varies with different demographics, like gender, age, education and ethnic affinity on social media across each of the states of United States. The facebook ad audience api, which returns an estimate of reach, when queried with demographic parameters like gender, age, education level, region, ethnic affinity and target interest was used in this study. The target interest in our study was Psychosis and psychotic disorders like Schizoaffective Disorder, Hallucination, Schizophrenia Awareness and Paranoid Schizophrenia. The api-returned statistic was roughly validated against the Pew tally of adult facebook users in United States. In addition, we used Government’s Mental Health Services Administration data for the count of state-wise mental health institutions, Census data for population and Human Development Index data. We did descriptive statistical analysis on the data, computed correlation metrics and fit regression models for the psychosis-related interest counts with the above parameters. The variation and quantitative association of awareness of psychotic disorders on facebook, with different demographic parameters was an interesting outcome of the study.  

    Using Social Media to Model Stress in College Campuses Around Violent Incidents

    Stress is a mental state involving emotional strain and tension resulting from adverse or demanding circumstances. Individuals in academic campuses, particularly students often go through periods of stress throughout the academic calendar. Additionally, extreme events involving violence, such as shooting, mass stabbing, or terrorist attack in college campuses, induces fear, resulting in increased stress among the campus population, which gets reflected on their social media activities. In this study, we propose to analyze how different campus population collectively react at such adverse events, using a causal inference based technique. We choose 8 violent incidents that occurred in US college campuses in the last four years, including the 2015 Chapel Hill Shooting, 2016 UCLA Shooting and 2016 OSU Attack.  Building a machine learning classifier of stress, which uses language models and expert validation, we also aim to identify the language cues for the expression of stress on social media.

    Using Visualization to Explore Social and Communicative Behaviors

    Psychology researchers use basic statistical visualizations such as bar charts, line charts, and box plots to explore their datasets. These charts are useful for visualizing one- or two-dimensional data but too simple to capture more complex data such as social and communicative behaviors. To more deeply explore temporal behavioral patterns, especially among a large group of subjects, researchers could use better visualization tools.

    We developed visualization tools to help developmental psychology researchers explore social and communicative behaviors. Based on our conversations with researchers, we learned that they need tools to find groups of children that exhibit commonalities in their behaviors.

    Our sample dataset consists of dyadic social interactions between a child and an examiner. Many behaviors from four modes of communication: gaze, speech, gesture and vocal affect from the children were coded by human annotators and visualized. We designed two visualization tools. One explicitly group children by their behaviors and the other implicitly suggests groups of children with commonalities in their behaviors.

    Validating a Conceptual Model of Spatial Presence to Enhance Virtual Reality Applications for Older and Younger Adults

    Virtual reality (VR) have a variety of applications with the potential to enhance life across physical, cognitive, and socio-emotional domains.  The defining characteristic of virtual reality is its ability to induce within the user a 'sense of being' physically located in the technology-mediated environment.  This experience is referred to as spatial presence.  Research has shown that virtual reality applications such as entertainment, physical and therapeutic interventions, social interaction, and training tend to be more effective and enjoyable when users experience spatial presence in the virtual environment.  However, the research in this area focuses primarily on the qualities of the VR system that influence spatial presence.  The user characteristics that influence the likelihood of experiencing spatial presence and the nature of those influences are not well-understood.  One such characteristic is the age of the user, as some sensory, physical, and cognitive abilities tend to undergo changes in older age.  Our current study aims to validate a conceptual model of spatial presence in virtual reality with an emphasis on potential age-related differences in if/how the 'sense of being' is formed and maintained.  Findings will provide insights into designing virtual reality experiences that incease their effectiveness for users of various ages.

    Virtual Medical Assistant

    Sensiotec, an ATDC startup, has developed Virtual Medical Assistant®, the world's first FDA-cleared, truly remote, totally non-contact, real-time cardiorespiratory and patient motion monitor – an invisible, non-wearable, contact-free system that provides scalable, high value, low cost, location-agnostic patient care. VMA measures heart and lung function as well as body movement, streaming critical spot and trend data to a nurse’s station, tablet device, or smartphone – all without electrodes, sensors or pads ever touching the patient directly or indirectly, thereby providing convenience, comfort and compliance. A key aspect of the technology platform is proprietary sensor technology that resides within a thin sensor panel. The panel simply slides under the mattress, bed or chair, providing convenience, comfort, and compliance with no electrodes or leads attached to, or anything required of, the patient. The Virtual Medical Assistant Workstation application was developed at the Interactive Media Technology Center.

    Virtual Research Assistant: MILA

    Protecting the environment is among the biggest challenges facing our society. Big data is an essential element of addressing this big challenge. Encyclopedia of Life  (EOL) is the world’s largest database of biological species and other biodiversity information. EOL also works closely with scores of other biodiversity datasets such as BISON, GBIF, and OBIS. We seek to make EOL and related biodiversity data sources accessible, usable, and useful, by integrating extant AI tools for information extraction, modeling and simulation, and question answering; we call the resulting system EOL+. The focus of this project is on the data engineering required for constructing EOL+ and on building a user community around EOL+. Professional and citizen scientists, and teachers and students alike, will be able to access EOL+ through the South BD Hub’s webportal, and use it for modeling and analysis, explanation and prediction, as well as education and workforce development in biological diversity, ecological modeling and environmental sustainability.
    MILA-S is an interactive tool developed at Georgia Tech that enables interactive construction of conceptual models of ecological phenomena and automatically spawns simulation models from the conceptual models. We will integrate MILA-S with EOL, enabling professional and citizen scientists to generate and test explanatory hypotheses as well as make predictions about ecosystems.  Watson is IBM’s cognitive system for question answering. At Georgia Tech, we have added semantic processing to Watson so that the resulting Watson+ system acts as a virtual research assistant. We will train Watson+ for answering questions about biological species using EOL as the datasource. 

    Virtual Teaching Assistant: Jill Watson

    It has been said that Jill Watson is the most famous teaching assistant in the world. Jill’s origin actually is quite humble. She was conceived in summer 2015 with the purpose of helping Georgia Tech’s Online MS in CS Program (OMSCS) and specifically with my online course on knowledge-based artificial intelligence (KBAI) as a part of OMSCS program (http://www.omscs.gatech.edu/cs-7637-knowledge-based-artificial-intellige...). Jill had a very difficult birth in fall 2015.

    Jill was quite precocious almost from the beginning. While still a baby, she joined the teaching staff of the Spring 2016 KBAI class. As a virtual teaching assistant, she answered questions in the online discussion forum for online KBAI class. Jill answered only routine, frequently answered questions on the forum, but she did so with close to 100% accuracy and precision, and with an authenticity that the students in the class did not figure out that Jill was actually an AI agent. When we shared the identity of Jill with the students, the students were thrilled by the experiment! One student even wanted to nominate her for the College of Computing Outstanding TA Award. 

    By now, we know that Jill is a veritable prodigy. In Fall 2016, we introduced two avatars of Jill Watson (named Ian Braun and Stacy Sisko) into both the online and residential sections of the KBAI class. Once again many students had difficulty discovering the identities of the virtual TAs among the teaching staff. Preliminary analysis from Fall 2016 suggests that student interaction with the virtual TAs helps improve motivation and engagement. In the current Spring 2017 semester, we have introduced three avatars of Jill into the online KBAI class (names withheld to protect the identities of the virtual TAs). With each generation, the performance of Jill has improved. By now ~25 human TAs have worked in league with Jill and ~1000 students have interacted with her.

    Virtually spatialized audio over bone conduction to support auditory situation awareness and increase pedestrian safety

    Most sound comes through our ears. However, it is also possible to pass vibrations through the bones of the head, and bypass much of the normal hearing pathway. This is called bone conduction audio, and can be used in situations where the ears need to be plugged, or where you need to leave the ears open to hear ambient sounds. We are studying the psychoacoustics as well as the applied aspects of bone conduction audio. A current application being investigated is the scenario of the distracted cyclist or pedestrian. This research is aimed at determining additional ways of further improving awareness of ambient sounds, in addition to using bone conduction devices, through techniques such as virtual spatialization or audio filters. This research will improve the safety of the many pedestrians and cyclists who currently wear headphones while going about their daily lives.

    Visual Policy Initiative

    The Visual Policy Initiative aims to transform complex policy issues into easy to understand data visualizations using empirically-derived evidence. The Visual Policy team is comprised of a group of researchers from both public policy and digital media. Through this collaborative effort, we aim to transform complex policy issues into easy to understand data visualizations using empirically-derived evidence. In our current endeavor, our team is focused on showing the economic and societal costs of autism spectrum disorder (ASD), and how those costs vary depending on age of diagnosis and age of intervention.

    Autism prevalence rates in the United States have more than doubled since 2000 (from 1 in 150 to 1 in 68 children being identified). Despite this trend as the nation’s fastest growing developmental disability, many insurance providers, including Medicaid, do not cover autism services or early intervention services for Autism Spectrum Disorders (ASD). Our research project draws on policy research and human-centered design research to build communication tools and strategies (“digital boundary objects”) that aid the public and legislators in understanding the negative economic impact of late intervention and present the existing evidence that justifies the passage and implementation of early intervention services in ASD. The first set of these communication tools is aimed at policymakers to improve the continuum of care and interagency system of supports for children with autism. We foresee variations of the developed communication strategies to be used by the public for raising awareness and enabling collective action.


    Visual Story Cubes

    Goal of this study is to understand how people interact with tangible user interfaces through play with the intent to support exploration, discovery, and learning of visual narratives. We designed the interactive system with the use of tangible blocks as primary interaction mode in mind, which contain various design elements of sequential art (graphic novels), for creating, editing, modifying visual narratives through different configuration and combination of blocks viewed under a device, such as such as a mobile phone in the form of augmented reality. 

    Visualization Journalism

    Visualization Journalism is focused on developing an interface and graphical metalanguage for massive multimodal news datasets. Such datasets are increasingly available, but for copyright reasons they cannot be made entirely open to the public. The project seeks to offer an abstracted and legal representation of news data, to enable comparative, cooperative and computer-supported analysis of trends across news events and networks. Combining quantitative methods from computational linguistics with opportunities for qualitative analysis, the project will help pundits and publics deliberate the structural characteristics that shape emergent news “narratives” and “points of view” on topics of broad social import. Existing platforms (i.e. InArticle, NewsMap, Archive.org) have only begun to demonstrate the potential for alternative forms of criticism that can handle the increased scale—and constraints—of news access. Our project makes use of UCLA's NewsScape, a growing collection of video for more than 300,000 broadcast news programs, extending back to Watergate.

    Visualization of Innovation in Global Supply Chain Networks

    This study describes a data-driven visualization approach to the systemic study of innovations in global supply chain networks. We demonstrate its applicability with illustrative examples of real-world supply chain in the electronics industry. Our visualization approach enhances the hypothesis-generating process as it can reveal important clusters, patterns, trends, and outliers in the networks.

    Visualization of the History of Worldwide Plane Crashes

    The project's goal is to help people to identify some of the variables involved in air traffic safety, and understand that air traffic safety relies on both technology and the people who control it.  Hopefully, it will be illuminating to anyone concerned with air traffic safety.

    Visualization Support for Early-Phase Complex Engineered System Design

    The design and production of complex engineered systems (CES) requires analysis of massive amounts of detailed information, including data on products and materials, engineering designs, manufacturing specifications, supply chain and delivery data, and changing customer needs. Visual analytics promises to offer tools and methods that will help stakeholders interactively explore, discover, and make sense of the underlying data. Our work focuses on the early design phase during which a large design space is explored, inconsistencies are identified, poor alternatives are pruned, and valuable alternatives are considered further. We demonstrate our ideas through an example of a two-degree-of-freedom robot and look at opportunities for future work for visualization in manufacturing design.

    Visualizing Figure Skating Jumps

    A visualization system for portraying the jumps in men's and ladies' single figure skating programs. Data is from the International Skating Union's score tables for world championships in the last 6 years. The objective is to better interpret the score tables by visualizing the program composition of top skaters, as well as showing a trend of the sport in general.

    Visualizing PGA Tour golf shot data

    The PGA Tour provides an extensive data collection of information about players' performance and individual shots over the past few years.  This collection is called ShotLink data. In this project we are building an interactive visualization system that will allow the viewer to easily browse and explore the golf shot data to learn more about the performance of all the players on tour.  The system presents a variety of different statistics including scoring, driving accuracy, greens in regulation, putting, and so on. We are now working to integrate visual representations of shot patterns on specific holes and players' summary performances when hitting shots from different distances.

    Visualizing the Top Golf Courses in the US

    We have created a visual interface to explore the history of the top 100 U.S. golf course rankings from Golf Digest and Golf Magazines. A viewer can explore the courses geographically via a map or through the individual ordered lists from the magazines. The system shows how each course's ranking has changes over the years, and it allows the viewer to explore courses by particular architects.


    Visually Exploring NFL Football Draft and Performance Data

    In this project, we have designed visualizations that show the recent history of a team's draft selections as well as each team's regular season and playoff history. Our goal is to provide an easy-to-browse and -understand interface for exploring the data and learning about teams' pasts.


    Face-to-face video communication technologies have grown tremendously in recent years, however they are not designed to provide a persistent sense of remote presence. More recently, telepresence robots give single users the ability to have a remote and mobile physical presence in another space. Combining telepresence robotics with persistent large-scale displays and multiple viewports, VPorter creates a telepresence ecology to support team collaboration across remote but connected lab spaces.

    Warning, Bias May Occur: Detecting Cognitive Bias in Visual Analytics

    Visual analytic tools provide interactive interfaces to help people gain insights and understanding about data. Such tools show visualizations depicting the output of analytic models. People can explore different views, change model parameters, and control aspects of the system through user interaction that allow them to ask questions of the data and see responses visually. This discourse between people and systems is often referred to as "human-in-the-loop" visual analytics, and is preferred to automated data mining alternatives in situations where human reasoning and domain expertise are required. However, little consideration has yet been given to the ways inherent human biases might shape the visual analytic process. We describe five preliminary metrics for detecting cognitive bias based on users' interactions and show how these metrics might be used in a visual analytic system. 

    Westside Atlanta Land Trust: A Community Mapping and Design Collaboration

    A few fridays ago the Public Design Workshop (PDW) led a Design and Policy Jam with the Westside Atlanta Land Trust (WALT) Program. WALT’s mission is “to organize the community’s power for self-determination; to serve and preserve in-place residents, small business owners, and their successive generations in redeveloping areas.” The jam session supported this mission by tasking participants with researching and producing an argument for a city-wide community land trust (CLT) policy. This argument would take the shape of maps, infographics, draft policy texts, and a powerpoint presentation to be presented to the City of Atlanta’s Community Development and Human Resources Committee.

    PDW lab members Andy Nelson (Master's HCI) and Amanda Meng (PhD INTA) work on a weekly basis with WALT to continue supporting data collection and mapping efforts. The team will also implement an online document editing platform to crowdsource policy texts that support a city wide Community Land Trust model.

    Westside Soul

    Westside Soul is an interactive installation that was displayed at the Historic Westside Cultural Arts Council's Black History Month Celebration. The installation displays videos of westside residents discussing issues in their community and allows viewers to add new responses through text and video. The installation is part of on ongoing partnership with the westside to explore the use of mobile and social computing and digital media to connect community members and instigate alternate forms of civic engagement.

    You can explore the installation at http://westsidesoul.net

    When the Internet Goes Down in Bangladesh

    Lab: TanDEm

    We present a study of Internet use and its forced non-use in Bangladesh. In light of current initiatives on state and industry actors to improve Internet access and bridge the 'digital divide' for underserved, under-resourced, and under-represented communities across the world, we offer a situated, qualitative perspective on what the current state of Internet use looks like for select social groups in Bangladesh. We analyze how a state-imposed ban attempted to effect the non-use of particular web-based services and how the affected populations found or did not find workarounds in response. We also discuss takeaways for researchers as well as industry and state actors studying and working towards more equitable access to the Internet in the 'developing' world.


    Who Told it How

    The news is the source citizens turn to in order to gain accurate information about the current events of the world. Unfortunately, a large number of trusted news sources are the worst perpetrators of bias; effectively skewing the public’s perception of important material. “Who Told it How” is a web-based interactive visualization that displays various elements of the Wendy Davis abortion filibuster, as written in articles by four major news syndicates, in order to expose bias and provide perspective.

    Women's Safety in Public Spaces: The Efficacy of Panic Buttons in New Delhi

    Lab: TanDEm

    We present a qualitative inquiry through the lens of feminist human computer interaction (HCI) into women's perceptions of personal safety in New Delhi, India. Since a brutal gang-rape incident in December 2012 that received global attention, the Indian government has issued a mandate to implement a panic button on every new phone by 2017. We draw on interview and survey data to examine women's reactions to the mandate as well as what factors influence their perceptions of safety, both positively and negatively. Our findings indicate that women's sense of safety may be deconstructed into a multitude of factors--personal, public, social, technological--that must be aligned for this sense of safety to be preserved. We then discuss the implications these factors have for the success and design of the panic button.

    Worked Examples and Subgoal Labeling: Impacts on Learning

    Procedural instructions and worked examples have been shown to be effective learning aids in science, technology, engineering, and mathematic (STEM) learning materials. Procedural instructions are texts that describe a general method to reach a goal, while worked examples demonstrate how to apply this method to a specific instance. Research supporting the use of advanced organizers predicts that if learners see the worked example first, they can develop a basis for the problem solving procedure. Learners can then use the procedural text to abstract their understanding, increasing both the initial and transfer performance.
    Subgoals have been shown to increase novice performance when included in procedural text and worked examples. A subgoal groups a set of solution steps by their purpose, which allows novice learners to create a framework for problem solving. The proposed research investigates the potential interactions of instructional order and subgoal labeling on performance.