A picture may be worth 1000 words, but can we also teach computers to create stories from the stories that lie inside our images? In this episode, Devi Parikh of Georgia Tech’s school of interactive computing discusses her work training computers to determine the semantic meaning within images. Devi talks about the stories behind her article “Bringing Semantics Into Focus Using Visual Abstraction,” which she co-authored with Larry Zitnick in 2013.

Semantic Meaning in Images - Devi Parikh
Semantic Meaning in Images - Devi Parikh
Click bottom of waveform to add your comments

Subscribe: iTunes | Google Play | Android | RSS

This slideshow requires JavaScript.

Devi on Facebook / Twitter

Devi presenting on this research (video)

Article in IEEE’s PAMI journal

Dhruv Batra’s website (AI researcher, and Devi’s husband)

Open Science Resources

Link to research code, datasets, clipart, and additional publications

Example descriptions and scenes from the dataset (including the unattended hot dog!)

The team’s more recent clipart dataset (larger, more realistic, more intricate)

Example scenes from Devi’s visual humor project

Bonus clips


▲ the utility of the gaze in deciphering meaning


▲ visual consistency


▲ features and classes within scenes


▲ creating an app for studying the importance of sequencing of creating scenes


▲ moving to Natural Language Processing (NLP)


▲ IEEE scholarship


▲ IEEE reviewer feedback


▲ collaborating with Larry Zitnick

Hosts / Producers

Doug Leigh & Ryan Watkins

How to Cite

Leigh, D., & Watkins, R.. (2017, September 5). Parsing Science – Semantic Meaning in Images with Devi Parikh (Version 1). figshare. https://doi.org/10.6084/m9.figshare.5907418.v1


What’s The Angle? by Shane Ivers