Sub-Optimal Low-Rank Decomposition
- Chenglong Li, Liang Lin, Wangmeng Zuo, Shuicheng Yan, and Jin Tang. SOLD: Sub-Optimal Low-Rank Decomposition for Efficient Video Segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),2015. PDF Supplementary Code
- Chenglong Li, Liang Lin, Wangmeng Zuo, Wenzhong Wang, and Jin Tang. An Approach to Streaming Video Segmentation with Sub-Optimal Low-Rank Decomposition. IEEE Transactions on Image Procssing(T-IP), 2016.
This project investigates how to perform robust and efficient video segmentation while suppressing the effects of data noises and/or corruptions.We propose a general algorithm, called Sub-Optimal Low-rank Decomposition (SOLD), which pursues the low-rank representation for video segmentation. Given the data matrix formed by supervoxel representation of an observed video sequence, SOLD seeks a sub-optimal solution by making the matrix rank explicitly determined. In particular, the affinity matrix with the fixed rank can be decomposed into two submatrices of low rank, and then we iteratively optimize them with closed-form solutions. Moreover, we incorporate a discriminative replication prior into SOLD based on the observation that smallsize video patterns tend to recur frequently within the same object. The Normalized-Cut (NCut) algorithm is adopted with the low-rank representation to segment the video into several spatio-temporal regions. The video is processed in a streaming fashion, i.e. sequentially segmenting a batch of frames, where we further design several temporal consistent constraints to improve the robustness. Extensive experiments on two public challenging datasets VSB100 and SegTrack suggest that our framework outperforms other video segmentation approaches in both accuracy and efficiency.
In this project, we have proposed a general algorithm for lowrank representation pursuit by decomposing the matrix with the fixed rank and proved that a sub-optimal solution can be achieved by alternating closed-form optimization. Based on this algorithm, we have developed an effective and efficient framework that automatically segments streaming videos in both unsupervised and interactive way. Extensive experiments on the standard benchmarks have demonstrated the superior performances of our approach over other video segmentation methods. In future work, we will improve our video segmentation framework by introducing more robust video features and over-segmentation methods. Our low-rank decomposition algorithm can be also extended to other vision tasks such as multi-object tracking and saliency detection.
1. F. Galasso, N. Nagaraja, T. Cardenas, T. Brox, and B. Schiele. A unified video segmentation benchmark: annotation, metrics and analysis. In Proc. IEEE Int. Conf. Comput. Vis., 2013.
2. J. Corso, E. Sharon, S. Dube, S. El-Saden, U. Sinha, and A. Yuille. Efficient multilevel brain tumor segmentation with integrated bayesian model classification. IEEE Trans. Med. Imag., 27(5): 629–640, May 2008.
3. F. Galasso, R. Cipolla, and B. Schiele. Video segmentation with superpixels. In Proc. Asian Conf. Comput. Vis., 2012.
4. M. Grundmann, V. Kwatra, M. Han, and I. Essa. Efficient hierarchical graph-based video segmentation. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2010.
5. C. Xu, C. Xiong, and J. J. Corso. Streaming hierarchical video segmentation. In Proc. Eur. Conf. Comput. Vis., 2012.