Papers
Topics
Authors
Recent
Search
2000 character limit reached

Facial Kinship Verification from remote photoplethysmography

Published 14 Sep 2023 in cs.CV | (2309.08006v2)

Abstract: Facial Kinship Verification (FKV) aims at automatically determining whether two subjects have a kinship relation based on human faces. It has potential applications in finding missing children and social media analysis. Traditional FKV faces challenges as it is vulnerable to spoof attacks and raises privacy issues. In this paper, we explore for the first time the FKV with vital bio-signals, focusing on remote Photoplethysmography (rPPG). rPPG signals are extracted from facial videos, resulting in a one-dimensional signal that measures the changes in visible light reflection emitted to and detected from the skin caused by the heartbeat. Specifically, in this paper, we employed a straightforward one-dimensional Convolutional Neural Network (1DCNN) with a 1DCNN-Attention module and kinship contrastive loss to learn the kin similarity from rPPGs. The network takes multiple rPPG signals extracted from various facial Regions of Interest (ROIs) as inputs. Additionally, the 1DCNN attention module is designed to learn and capture the discriminative kin features from feature embeddings. Finally, we demonstrate the feasibility of rPPG to detect kinship with the experiment evaluation on the UvANEMO Smile Database from different kin relations.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (29)
  1. Hamdi Dibeklioglu, “Visual transformation aided contrastive learning for video-based kinship verification,” in ICCV. IEEE, 2017, pp. 2459–2468.
  2. “Audio-visual kinship verification: a new dataset and a unified adaptive adversarial multimodal learning approach,” IEEE Transactions on Cybernetics, 2022.
  3. “Kinship verification from gait,” in ICIP. IEEE, 2020, pp. 2950–2954.
  4. “Facial kinship verification: A comprehensive review and outlook,” International Journal of Computer Vision, vol. 130, no. 6, pp. 1494–1525, 2022.
  5. “Generative adversarial networks: A survey toward private and secure applications,” ACM Computing Surveys, vol. 54, no. 6, pp. 1–38, 2021.
  6. “Stylegene: Crossover and mutation of region-level facial genes for kinship face synthesis,” in CVPR. IEEE/CVF, 2023, pp. 20960–20969.
  7. “A non-contact ppg biometric system based on deep neural network,” in BTAS. IEEE, 2018, pp. 1–7.
  8. “A robust and reusable ecg-based authentication and data encryption scheme for ehealth systems,” in GLOBECOM. IEEE, 2016, pp. 1–6.
  9. “Person identification based on parametric processing of the eeg,” in ICECS’99. IEEE, 1999, vol. 1, pp. 283–286.
  10. “pyvhr: a python framework for remote photoplethysmography,” PeerJ Computer Science, vol. 8, pp. e929, 2022.
  11. Weitz W, “Studien an eineiigen zwillingen,” Z Klin Med, pp. 489–501, 1925.
  12. “Heritability of ecg measurements in adult male twins,” Journal of electrocardiology, vol. 30, pp. 64–68, 1998.
  13. “Bioelectrical signals as emerging biometrics: Issues and challenges,” International Scholarly Research Notices, vol. 2012, 2012.
  14. “Heartid: Cardiac biometric recognition,” in BTAS. IEEE, 2010, pp. 1–5.
  15. “A survey on heart biometrics,” ACM Computing Surveys, vol. 53, no. 6, pp. 1–38, 2020.
  16. “Face2ppg: An unsupervised pipeline for blood volume pulse extraction from faces,” IEEE Journal of Biomedical and Health Informatics, pp. 1–12, 2023.
  17. “Are you really smiling at me? spontaneous versus posed enjoyment smiles,” in ECCV. Springer, 2012, pp. 525–538.
  18. “Time series classification from scratch with deep neural networks: A strong baseline,” in IJCNN. IEEE, 2017, pp. 1578–1585.
  19. “Towards computational models of kinship verification,” in ICIP. IEEE, 2010, pp. 1577–1580.
  20. “Neighborhood repulsed metric learning for kinship verification,” IEEE transactions on pattern analysis and machine intelligence, vol. 36, no. 2, pp. 331–345, 2014.
  21. “Kinship verification from faces via similarity metric based convolutional neural network,” in ICIAR. Springer, 2016, pp. 539–548.
  22. Andrei Shadrikov, “Achieving better kinship recognition through better baseline,” in FG. IEEE, 2020, pp. 872–876.
  23. “Can we generate real faces from rppg signals? probably not,” in PerCom Workshops. IEEE, 2024.
  24. “Non-contact heart rate measurement from deteriorated videos,” arXiv preprint arXiv:2304.14789, 2023.
  25. “Remote plethysmographic imaging using ambient light.,” Optics express, vol. 16, no. 26, pp. 21434–21445, 2008.
  26. “Robust pulse rate from chrominance-based rppg,” IEEE Transactions on Biomedical Engineering, vol. 60, no. 10, pp. 2878–2886, 2013.
  27. “Local group invariance for heart rate estimation from face videos in the wild,” in CVPR Workshop. IEEE, 2018, pp. 1254–1262.
  28. “Algorithmic principles of remote ppg,” IEEE Transactions on Biomedical Engineering, vol. 64, no. 7, pp. 1479–1491, 2016.
  29. “Squeeze-and-excitation networks,” in CVPR. IEEE, 2018, pp. 7132–7141.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up