A picture may be worth 1000 words, but can we also teach computers to create stories from the stories that lie inside our images? In this episode, Devi Parikh of Georgia Tech’s school of interactive computing discusses her work training computers to determine the semantic meaning within images. Devi talks about the stories behind her open-access article “Bringing Semantics Into Focus Using Visual Abstraction,” which she co-authored with Larry Zitnick in 2013.

This slideshow requires JavaScript.

Websites

• Devi on Twitter
• Devi presenting on this research (video)
• Article in IEEE’s PAMI journal
• Dhruv Batra’s website (AI researcher, and Devi’s husband)

Open Science Resources

• Link to research code, datasets, clipart, and additional publications
• Example descriptions and scenes from the dataset (including the unattended hot dog!)
• The team’s more recent clipart dataset (larger, more realistic, more intricate)
• Example scenes from Devi’s visual humor project

Bonus clips

Patrons of Parsing Science gain exclusive access to bonus clips from all our episodes and can also download mp3s of every individual episode.

Support us for as little as $1 per month at Patreon. Cancel anytime.