Which Are You In A Photo?

Abstract: Automatic image tagging has been a long standing problem, it mainly relies on image recognition techniques of which the accuracy is still not satisfying. This paper attempts to explore out-of-band sensing base on the mobile phone to sense the people in a picture while the picture is being taken and create name tags on-the-fly. The major challenges pertain to two aspects - "Who" and "Which". (1) "Who": discriminating people who are in the picture from those that are not; (2) "Which": correlating each name tag with its corresponding people in the picture. We propose an accurate acoustic scheme applying on the mobile phones, which leverages the Doppler effect of sound wave to address these two challenges. As a proof of concept, we implement the scheme on 7 android phones and take pictures in various real-life scenarios with people positioning in different ways. Extensive experiments show that the accuracy of tag correlation is above 85% within 3m for picturing.