ConceptLearner: Discovering Visual Concepts from Weakly Labeled Image Collections.

Bolei Zhou* Vignesh Jagadeesh+ Robinson Piramuthu+
Massachusetts Institute of Technology*,
eBay Research Labs+.

Download Paper

Discovering visual knowledge from weakly labeled data are crucial to scale up computer vision recognition system, since it is expensive to obtain fully labeled data for a large number of concept categories while the weakly labeled data could be collected from the Internet cheaply and massively. In this paper we proposes a scalable approach to discover visual concepts from weakly labeled image collections, with thousands of visual concept detectors learned. Then we show that the learned detectors could be applied to recognize concepts at image-level and to detect concepts at image region-level accurately. Under domain-selected supervision, we further evaluate the learned concepts for scene recognition on SUN database and for object detection on Pascal VOC 2007. It shows promising performance compared to the fully supervised and weakly supervised methods.

Read our CVPR'15 paper for details.

Introduction of ConceptLearner

As shown below, images uploaded to social network usually have some short description written by the user. There are a lot of interesting knowledge and concepts in the description associated with image contents. After collecting thousands of images with tags/sentence description, ConceptLearner first extracts the semantic phrases from these sentences, as weak labels for these images. ConceptLearner applies concept discovery algorithm to learn concept detectors from thousands of these images with weak labels.

After the concept detectors are learned from weakly labeled image collections using the algorithm of max margin visual concept discovery, conceptLearner will apply them for concept recognition and concept detection. Here are some examplar results of concept recognition by ConceptLearner: Given an image, our system will predict the most probable visual concepts relevant to the image.

Reference & Acknowledgement

B. Zhou, V. Jagadeesh, and R. Piramuthu
ConceptLearner: Discovering Visual Concepts from Weakly Labeled Image Collections.
Computer Vision and Pattern Recognition (CVPR), 2015.
  title={{ConceptLearner: Discovering Visual Concepts from Weakly Labeled Image Collections.}},
  author={Zhou, Bolei and Jagadeesh, Vignesh and Piramuthu, Robinson},
  journal={Computer Vision and Pattern Recognition (CVPR)},