A lecture by-
Chris Wiggins
Columbia University and
Matthew L. Jones
Princeton University
The co-authors of the book How Data Happened will trace the dynamic relationships among data, truth, and power, exploring how data-empowered algorithms have come to shape our personal, professional, and political realities.
Location: 1008 Humanities
Over the past decade, larger datasets, hardware accelerations, and network architecture improvements have contributed to phenomenal achievements in many tasks of computer
vision. However, in the absence of large datasets, computer vision models struggle to learn
general representations which results in poor performance. Few-shot learning tries to address
this problem by proposing models which learn from a few examples.
I first give an overall review of few-shot learning methods. I particularly focus on generative Few Shot Learning(FSL) methods, which augment the scarce categories in a dataset by generating samples for those rare categories. As the actual class distribution can be complex and lie very close to each other, the sample generated for one class can be noisy or lie close to another class. However, none of the current FS generative methods perform any form of quality control of the generated samples.
In this work, I propose to identify and remove the generated samples that are less likely to be in the distribution of the few-shot class. Here I particularly deal with few-shot scenarios where the
prior information of the relationship between the classes based on visual similarity is available. The main idea is to exploit these priors to better identify the unreliable generated samples.
Particularly, I have proposed two methods based on class relationship to detect noisy generated samples. In the first method, we assume that the embedding space of each class follows a Gaussian distribution. From this assumption, I propose Gaussian Neighborhood (GN), a method to estimate how likely a generated sample is drawn from the estimated distribution of a few-shot class. We evaluate this method on the Hematopoiesis dataset. By simply eliminating samples based on thresholding our proposed GN scores, the few-shot classification performance is improved by 5% and 2% in five shot and one shot respectively, compared to the model trained on all generated images.
The GN scores represent the similarity distances from the generated samples to their classes, based on the assumption that each class is a Gaussian distribution. However, this assumption might be strict in many scenarios since the real distributions of data can be arbitrarily complex. Thus in my second proposed method, I aim to learn such similarity distances directly from data via metric learning. I propose to train a deep-network to regress the similarity distance between a pair of samples. This network is trained using both the class-level visual similarity information and the class labels. This method improves the 1-shot and 5-shot classification performances by 0.5% and 1% respectively, compared to GN.
