In real-world applications, e.g. law enforcement and video retrieval, one often needs to search a certain person in long videos with just one portrait. This is much more challenging than the conventional settings for person re-identification, as the search may need to be carried out in the environments different from where the portrait was taken. In this paper, we aim to tackle this challenge and propose a novel framework, which takes into account the identity invariance along a tracklet, thus allowing person identities to be propagated via both the visual and the temporal links. We also develop a novel scheme called Progressive Propagation via Competitive Consensus, which significantly improves the reliability of the propagation process. To promote the study of person search, we construct a large-scale benchmark, which contains 127K manually annotated tracklets from 192 movies. Experiments show that our approach remarkably outperforms mainstream person re-id methods, raising the mAP from 42.16% to 62.27%.

Competitive Consensus

Competitive Consensus is a novel scheme for label propagation. Compared to the conventional linear diffusion, it improves the reliability by propagating the most confident information.

Progressive Propagation

Progressive Propagation is a simple but effective scheme to accelerate the propagation process and reduce the effects of noise, which freezes the label of a certain fraction of nodes at each iteration according to their confidence.

Accuracy on CSM

Examples of Search Results

Cast Search in Movies (CSM) contains more than 127K tracklets (with more than 11M person instances) of 1,218 cast from 192 movies. Bounding box and indentity of key frames are manually annotated. Bounding box of other instances of tracklets are gotten by a person detector based on Faster-RCNN.

Comparison between CSM and Related Datasets

Examples of CSM

@inproceedings{huang2018person,
    title={Person Search in Videos with One Portrait Through Visual and Temporal Links},
    author={Huang, Qingqiu and Liu, Wentao and Lin, Dahua},
    booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
    pages={425--441},
    year={2018}
}

hq016@ie.cuhk.edu.hk