Title: Pattern Mining and Concept Discovery for Multimodal Content Analysis
Speaker: Hongzhi Li
Venue: 202, 1-A（行政楼办公楼202）
Time: 12thSeptember 2018, 10:00-11:00（9月12日上午10:00）
Host: Xin-Shun Xu（许信顺）
Abstract:
Visual patterns are the discriminative and representative image content found in objects or local image regions seen in an image collection. Visual patterns can also be used to summarize the major visual concepts in an image collection. In this talk, I’ll discuss the following question: given a new target domain and associated data corpora, how do we rapidly discover nameable content patterns that are semantically coherent, visually consistent, and can be automatically named with semantic concepts related to the events of interest in the target domains? We develop pattern discovery methods that focus on visual content as well as multimodal data including text and visual. Traditional visual pattern mining methods only focus on analysis of the visual content, and do not have the ability to automatically name the patterns. To address this, we propose a new multimodal visual pattern mining and naming method that specifically addresses this shortcoming. The named visual patterns can be used as discovered semantic concepts relevant to the target data corpora. By combining information from multiple modalities, we can ensure that the discovered patterns are not only visually similar, but also have consistent meaning, as well. To discover better visual patterns, we further improve the visual model in the multimodal visual pattern mining pipeline, by developing a convolutional neural network (CNN) architecture that allows for the discovery of scale-invariant patterns.
Bio:
Dr. Hongzhi Li is a Senior Research SDE in Microsoft Research. His research interests are machine learning, multimodal content analysis and cloud based computing. His current research is focused on deep learning for visual intelligence and its applications on cloud computing platform.Dr. Li received his PhD degree in Computer Science from Columbia University in 2016. Before that, he received his Bachelor and Master degree in Computer Science from Zhejiang University, China and Columbia University, US, in 2010 and 2012, respectively.Dr. Li has published in ACM Multimedia, TMM, ICMR, EMNLP, NAACL, SPIE and other venues. He received best poster award in ACM ICMR 2018. He is the winner of grand challenge (first place) in ACM Multimedia 2012. Dr. Li severed in program committee of major international conferences, including ACM MM, ICME, IJCAI, etc. He also severed as a reviewer in journals including IEEE TMM, IEEE TCSVT, TPAMI, JVCI, JVIS, etc.
