Action localization with tubelets from motion.pdf

Action localization with tubelets from motion Mihir Jain? Jan van Gemert Herve? Je?gou? Patrick Bouthemy? Cees G.M. Snoek ?Inria University of Amsterdam Abstract This paper considers the problem of action localization, where the objective is to determine when and where certain actions appear. We introduce a sampling strategy to pro- duce 2D+t sequences of bounding boxes, called tubelets. Compared to state-of-the-art alternatives, this drastically reduces the number of hypotheses that are likely to include the action of interest. Our method is inspired by a re- cent technique introduced in the context of image localiza- tion. Beyond considering this technique for the first time for videos, we revisit this strategy for 2D+t sequences obtained from super-voxels. Our sampling strategy advantageously exploits a criterion that reflects how action related motion deviates from background motion. We demonstrate the interest of our approach by extensive experiments on two public datasets: UCF Sports and MSR- II. Our approach significantly outperforms the state-of-the- art on both datasets, while restricting the search of actions to a fraction of possible bounding box sequences. 1. Introduction Recognizing actions in videos is an active area of re- search in computer vision. Because of the many fine- grained spatio-temporal variations in action appearance, the current performance is far from that achieved in other recog- nition tasks such as image search. The goal of action clas- sification is to determine which action appears in the video. Temporal action detection estimates, additionally, when it occurs. This paper specifically considers the problem of ac- tion localization: the objective is to detect when and where an action of interest occurs. The expected output of such an action localization system is typically a subvolume encom- passing the action of interest. Since a localized action only covers a fraction of the spatio-temporal volume in a video, the task is consi