A Novel Approach for Abstractive Video Visualization

We present a novel approach for abstractive video visualization, which can help users understand the semantic information from the video in a fast and effective manner. We use the scale-invariant feature transform(SIFT) algorithm to detect features of each frame, together with a modified bag of words algorithm to construct a feature vocabulary in order to compute the feature frequencies. By mapping the video sequence onto a 3D curve in a high dimensional vocabulary space with the use of the multi-dimensional scaling (MDS) algorithm, the video is abstracted and embedded into a visually recognizable curve in 3D space. This generated visualization result can vividly illustrate the evolvement of the video contents, while well protecting and preserving the semantic meaning that are encoded within the video. Experimental results indicate that this curve-based visualization technique can uncover the semantic relationship between the frames, characterize the transition of video contents, and help the users understand the semantic structure of the underlying video sequence with a quick glance.