Semantic Video Segmentation

Feature Space Optimization for Semantic Video Segmentation

Paper Slides Poster Code Talk

Abstract

We present an approach to long-range spatio-temporal regularization in semantic video segmentation. Temporal regularization in video is challenging because both the camera and the scene may be in motion. Thus Euclidean distance in the space-time volume is not a good proxy for correspondence. We optimize the mapping of pixels to a Euclidean feature space so as to minimize distances between corresponding points. Structured prediction is performed by a dense CRF that operates on the optimized features. Experimental results demonstrate that the presented approach increases the accuracy and temporal consistency of semantic video segmentation.

Videos

Results on Cityscapes dataset

Comparison on Camvid dataset

Citation

For attribution in academic contexts, please cite this work as

  1. @inproceedings{KunduCVPR2016VideoFSO,
  title = {Feature Space Optimization for Semantic Video Segmentation},
  author = {Kundu, Abhijit and Vineet, Vibhav and Koltun, Vladlen},
  booktitle = {CVPR},
  year = {2016},
  doi = {10.1109/CVPR.2016.345},
  url = {https://doi.org/10.1109/CVPR.2016.345},
}