Measure heart rate from webcam videos (yarppg)
The human eye is an amazing and powerful organ. However, humans are not good at spotting small changes in color or intensity. By contrast, cameras obtain a numeric representation of a scene with incredibly high color resolution. A camera can therefore be used to detect tiny differences in successive video frames in order to gain meaningful information. Using some basic tools from computer vision and signal processing, it is even possible to measure the heart rate of a person sitting in front of a regular webcam.
Remote photoplethysmography (rPPG)
Remote photoplethysmography (rPPG) refers to the contact-free measurement of a blood volume pulse signal. It works by detecting small changes in skin color that originate from the pulsation of blood1. When light hits the skin, a portion enters the tissue, where it is absorbed and scattered. Some of the scattered rays exit the skin again and are picked up by the camera (diffuse reflection2). The intensity of the diffusely reflected light depends on the blood volume and consequently changes over time with each heartbeat. In particular, the camera’s green channel varies most significantly as green light is absorbed more strongly by hemoglobin (blood pigment).
yarppg
In yarppg (yet another rPPG), I have implemented a small GUI application that continuously reads a webcam input and detects color changes in a face. The user interface looks something like this:
Besides I/O and user interface, there are three main parts to this rPPG application (and basically most others):
- Region of interest (ROI) identification
- blood volume pulse signal extraction
- heart rate calculation
Several solutions have been introduced for each of these tasks. Rouast et al.3 give a thorough overview of “traditional” techniques. Recently, also deep learning has been applied to the challenge, a literature review and general rPPG discussion is provided by Cheng et al.4
In the following posts, I will illustrate how this is done in yarppg and discuss some of the challenges.
(References)
-
W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote plethysmographic imaging using ambient light,” Optics express, vol. 16, no. 26, pp. 21434–45, 2008, doi:10.1364/oe.16.021434. ↩︎
-
W. Wang, A. C. den Brinker, S. Stuijk, and G. de Haan, “Algorithmic Principles of Remote PPG,” IEEE Transactions on Biomedical Engineering, vol. 64, no. 7, pp. 1479–1491, 2017, doi:10.1109/TBME.2016.2609282. ↩︎
-
P. V Rouast, M. T. P. Adam, R. Chiong, D. Cornforth, and E. Lux, “Remote heart rate measurement using low-cost RGB face video: a technical literature review,” Frontiers of Computer Science, vol. 12, no. 5, pp. 858–872, 2018, doi:10.1007/s11704-016-6243-6. ↩︎
-
C. H. Cheng, K. L. Wong, J. W. Chin, T. T. Chan, and R. H. Y. So, “Deep Learning Methods for Remote Heart Rate Measurement: A Review and Future Research Agenda,” Sensors, vol. 21, no. 18, p. 6296, 2021, doi:10.3390/S21186296. ↩︎