Deep Learning

Deep learning is a new area of machine learning research, which have demonstrated state-of-the-art performance on many artificial intelligence tasks, e.g., computer vision, speech recognition and natural language processing. We are primarily interested on developing new deep architectures and training algorithms.

Confident Deep Neural Networks

The predictive uncertainty (e.g., entropy of softmax distribution of a deep classifier) is indispensable as it is useful in many machine learning applications (e.g., active learning and ensemble learning) as well as when deploying the trained model in real-world systems. In order to improve the quality of the predictive uncertainty, we proposed a novel loss function for training deep models. Our key idea is to force the predictive distribution become being uniform for out-of-distribution samples drawn from a generative adversarial network.

Training Confidence-Calibrated Classifiers for Detecting Out-of-Distribution Samples (code)

Kimin Lee, Honglak Lee, Kibok Lee and Jinwoo Shin
ICLR 2018

We study more informative novelty detection methods based on the hierarchical classification framework. For an objective of a novel class, we aim for finding its closest super class in the hierarchical taxonomy among all known classes. To this end, we propose two different architectures termed top-down and flatten. The essential ingredients of our methods are confident deep neural networks, relabeling and the leave-one-out strategy.

Hierarchical Novelty Detection for Visual Object Recognition

Kibok Lee, Kimin Lee, Kyle Min, Yuting Zhang, Jinwoo Shin and Honglak Lee
CVPR 2018

We propose new ensemble methods specialized for deep neural networks, called confident multiple choice learning (CMCL) utilizing the known concept of multiple choice learning (MCL). For resolving the overconfidence issue of the original MCL, we utilize the confident deep neural networks. We show that CMCL can outperform all baseline ensemble methods such as independent ensemble and MCL for image classification and segmentation tasks.

Confident Multiple Choice Learning (code)

Kimin Lee, Changho Hwang, Kyoungsoo Park and Jinwoo Shin
ICML 2017

(edited by Kimin Lee and Jinwoo Shin, Feb 2018)