Abstract:
Gaze-following is an important ability that allows us to understand what people are doing, and what are their intentions. Current research on gaze following is focused on using a single image and restricts the task based on whether the target is in frame or out of frame. A survey of gaze-following datasets revealed defects in the way they collect gaze point labels. Previous datasets rely on annotator votes for determining the actual gaze target location leading to ambiguous ground truths. Also, no existing method investigates cases of occlusion, a common phenomenon in real-world scenarios. Occlusion implies that the gaze target is in frame, but is obstructed by another object making the target unidentifiable. However, current approaches fail to differentiate between visible and occlusion cases when the target is in frame which could lead to incorrect predictions. In this work, we have collected a new dataset, MultiViewGaze, with precise point target annotations in six views for four scenes, that simulate an office, a restaurant kitchen, a shop and a lounge area. We also extend the gaze labels beyond the traditional in-frame and out-of-frame by handling cases of occlusion. We further introduce fine-grained labels for three categories of occlusions - Self-Occlusion, Occlusion due to another object and Occlusion due to subject. While occlusion is common in everyday situations, prior approaches have not delved into this task. We modify two single-view approaches primarily used for gaze following - the VideoAttentionTarget model and the ChildPlay model and train them for the task of Occlusion Detection, achieving good baseline results. Overall, we believe that classifying cases of occlusion and detecting obstructed targets to improve Gaze Following methods is a challenging and important problem and is instrumental in effective scene understanding.

Zoom meeting: https://stonybrook.zoom.us/j/95416846966?pwd=YjNHVkFsT1UrNkN1eEZHcFRJeU5...
Meeting ID: 95416846966
Passcode: 749434