Explain & Visualize to help AI generate better rationales

Info about the Project

Building on our prior work on AI agents to think out loud in plain English, we are taking the next step. You will not only see Frogger think out loud using plan English, you will also be able to visually connect which parts of his language correspond to which parts of the game. For e.g, the statement "I am trying to avoid the red truck to the left" will be accompanied by a visual indication of the red truck. While language is instrumental in making black-boxed AI systems explainable to lay users, having a layer of visual correlation makes our approach even more powerful.
Faculty: 
Mark Riedl
Students: 
Upol Ehsan, Shukan Shah, Pradyumna Tambwekar,