Mandy B. Korpusik profile photo

Mandy B. Korpusik

Assistant Professor of Computer Science | Seaver College of Science and Engineering

Los Angeles, CA, UNITED STATES

Seaver College of Science and Engineering

Media

Publications

Documents

Photos

Audio

Video

Image for vimeo videos on These Defense 2019! :)

Biography

Professor Mandy Korpusik received her B.S. in Electrical and Computer Engineering from Franklin W. Olin College of Engineering in May, 2013. She completed her S.M. in Computer Science at MIT in June, 2015 and received her Ph.D. from MIT in June, 2019. Her primary research interests include natural language processing and spoken language understanding for dialogue systems. Professor Korpusik used deep learning models to build the Coco Nutritionist application for iOS that allows obesity patients to more easily track the food they eat by speaking naturally. Her long-term research goal is to deploy a collection of AI-based conversational agents that improve the health, well-being, and productivity of people.

Education

Franklin W. Olin College of Engineering

B.S., Electrical and Computer Engineering

2013

Massachusetts Institute of Technology

S.M., Electrical Engineering and Computer Science

2015

Massachusetts Institute of Technology

Ph.D., Electrical Engineering and Computer Science

2019

Social

Areas of Expertise

Spoken Dialogue SystemsNatural Language ProcessingElectrical Engineering and Computer ScienceDeep Learning

Industry Expertise

  • Computer Software
  • Health and Wellness

Media Appearances

Voice-controlled calorie counter

MIT News  online

2016-05-24

Spoken-language app makes meal logging easier, could aid weight loss.

view more

MIT researchers launched a new AI system for counting calories.

Inside AI Newsletter  online

2019-01-24

The app is called Coco, and users simply say what they have eaten to log their meals. Data is held anonymously and the app is part of an ongoing MIT research project on speech understanding. — COCO NUTRITION

view more

Exploring the nature of intelligence

MIT News  online

2019-02-21

Undergraduate research projects show how students are advancing research in human and artificial intelligence, and applying intelligence tools to other disciplines.

view more

Sample Talks

Conference: A Comparison of Deep Learning Methods for Language Understanding

M.Korpusik, Z. Liu, J. Glass Interspeech 2019, Graz, Austria

Workshop: A Food Logging System for iOS with Natural Spoken Language Meal Descriptions

M. Korpusik, S. Taylor, S.Das, C. Gilhooly, S. Roberts, J. Glass Nutrition 2019, Baltimore

Conference: Dialogue State Tracking with Convolutional Semantic Taggers

M. Korpusik, J. Glass ICASSP 2019, Brighton, UK

Workshop: Convolutional Neural Endoder for the 7th Dialogue System Technology Challenge

M. Korpusik, J. Glass DSTC7 Workshop, Honolulu (2019)

Conference: Convolutional Neural Networks for Dialogue State Tracking without Pre-trained Word Vectors or Semantic Dictionaries

M. Korpusik, J. Glass SLT 2018, Athens

Patents

Behavior prediction on social media using neural networks

14/966438

2015-12-11

view more

A System and Method for Semantic Mapping ofNatural Language Input to Database Entries via Convolutional NeuralNetworks.

15/92239

2018-03-16

Articles

Deep Learning for Database Mapping and Asking Clarification Questions in Dialogue Systems | IEEE Transactions on Audio, Speech, and Language Processing (Volume: 27 , Issue: 8 , Aug. 2019)

2019-05-23

M. Korpusik, J. Glass

Abstract:
Food logging is recommended by dieticians for prevention and treatment of obesity, but currently available mobile applications for diet tracking are often too difficult and time-consuming for patients to use regularly. For this reason, we propose a novel approach to food journaling that uses speech and language understanding technology in order to enable efficient self-assessment of energy and nutrient consumption. This paper presents ongoing language understanding experiments conducted as part of a larger effort to create a nutrition dialogue system that automatically extracts food concepts from a user's spoken meal description. We first summarize the data collection and annotation of food descriptions performed via Amazon Mechanical Turk (AMT), for both a written corpus and spoken data from an in-domain speech recognizer. We show that the addition of word vector features improves conditional random field (CRF) performance for semantic tagging of food concepts, achieving an average F1 test score of 92.4 on written data; we also demonstrate that a convolutional neural network (CNN) with no hand-crafted features outperforms the best CRF on spoken data, achieving an F1 test score of 91.3. We illustrate two methods for associating foods with properties: segmenting meal descriptions with a CRF, and a complementary method that directly predicts associations with a feed-forward neural network. Finally, we conduct an end-to-end system evaluation through an AMT user study with worker ratings of 83% semantic tagging accuracy.

view more
Spoken Language Understanding for a Nutrition Dialogue System | IEEE Transactions on Audio, Speech, and Language Processing (Volume 25 , Issue: 7 , July 2017)

2017-04-17

M. Korpusik, J. Glass

Abstract:
Food logging is recommended by dietitians for prevention and treatment of obesity, but currently available mobile applications for diet tracking are often too difficult and time-consuming for patients to use regularly. For this reason, we propose a novel approach to food journaling that uses speech and language understanding technology in order to enable efficient self-assessment of energy and nutrient consumption. This paper presents ongoing language understanding experiments conducted as part of a larger effort to create a nutrition dialogue system that automatically extracts food concepts from a user's spoken meal description. We first summarize the data collection and annotation of food descriptions performed via Amazon Mechanical Turk (AMT), for both a written corpus and spoken data from an in-domain speech recognizer. We show that the addition of word vector features improves conditional random field (CRF) performance for semantic tagging of food concepts, achieving an average F1 test score of 92.4 on written data; we also demonstrate that a convolutional neural network (CNN) with no hand-crafted features outperforms the best CRF on spoken data, achieving an F1 test score of 91.3. We illustrate two methods for associating foods with properties: segmenting meal descriptions with a CRF, and a complementary method that directly predicts associations with a feed-forward neural network. Finally, we conduct an end-to-end system evaluation through an AMT user study with worker ratings of 83% semantic tagging accuracy.

view more