Parametric Image Segmentation of Humans with Structural Shape Priors

Abstract: The figure-ground segmentation of humans in images captured in natural environments is an outstanding open problem due to the presence of complex backgrounds, articulation, varying body proportions, partial views and viewpoint changes. In this work we propose class-specific segmentation models that leverage parametric max-flow image segmentation and a large dataset of human shapes. Our contributions are as follows: (1) formulation of a sub-modular energy model that combines class-specific structural constraints and data-driven shape priors, within a parametric max-flow optimization methodology that systematically computes all breakpoints of the model in polynomial time; (2) design of a data-driven class-specific fusion methodology, based on matching against a large training set of exemplar human shapes (100,000 in our experiments), that allows the shape prior to be constructed on-the-fly, for arbitrary viewpoints and partial views. (3) demonstration of state of the art results, in two challenging datasets, H3D and MPII (where figure-ground segmentation annotations have been added by us), where we substantially improve on the first ranked hypothesis estimates of mid-level segmentation methods, by 20%, with hypothesis set sizes that are up to one order of magnitude smaller.