Robotic Detection and Estimation of Single Scuba Diver Respiration Rate from Underwater Video

Abstract: Human respiration rate (HRR) is an important physiological metric for diagnosing a variety of health conditions from stress levels to heart conditions. Estimation of HRR is well-studied in controlled terrestrial environments, yet robotic estimation of HRR as an indicator of diver stress in underwater for underwater human robot interaction (UHRI) scenarios is to our knowledge unexplored. We introduce a novel system for robotic estimation of HRR from underwater visual data by utilizing bubbles from exhalation cycles in scuba diving to time respiration rate. We introduce a fuzzy labeling system that utilizes audio information to label a diverse dataset of diver breathing data on which we compare four different methods for characterizing the presence of bubbles in images. Ultimately we show that our method is effective at estimating HRR by comparing the respiration rate output with human analysts.

• The paper introduces a robotic system that estimates a scuba diver’s respiration rate by detecting exhalation bubbles from underwater video data.
• It employs methods like SVM, CNN, and CNN-LSTM, with CNN achieving around 97% detection accuracy in various conditions.
• Results indicate that CNN-LSTM performs best in complex visual environments, paving the way for reliable, non-contact health monitoring in underwater operations.

Robotic Detection and Estimation of Single Scuba Diver Respiration Rate from Underwater Video

The paper, authored by Demetrious T. Kutzke and Junaed Sattar, presents a novel approach for robotic estimation of a scuba diver’s human respiration rate (HRR) using underwater visual data. The focus of the study is to utilize exhalation bubbles captured in video footage to determine the respiration cycle, given the lack of existing research on underwater HRR estimation for underwater human-robot interaction (UHRI) scenarios.

Context and Importance

HRR is a critical physiological metric linked to various health conditions such as stress, exposure to extreme environments, lung, and heart issues. Traditional HRR measurement methods rely heavily on contact-based technologies that are impractical in underwater environments due to issues like signal attenuation and requirement for direct skin contact obstructed by diving gear. Consequently, there’s a significant gap in real-time health monitoring for divers using automated systems.

Proposed Solution

The authors’ solution leverages visual data captured by an autonomous underwater vehicle (AUV) to identify and count exhalation bubbles, thereby estimating the respiration rate. This estimation method could potentially assist in mitigating risks associated with underwater operations, such as excessive gas consumption and catastrophic health failures.

Methodology

The system integrates two primary components:

1. Bubble Detection: Identification of bubbles in video frames using binary classification methods.
2. Respiration Rate Tracking: Calculation of time intervals between detected exhalation bubbles to estimate HRR.

To facilitate bubble detection, the authors introduce multiple methodologies including an audio signal classification system for initial labeling, and three image-based methods: a Support Vector Machine (SVM), Convolutional Neural Network (CNN), and a Convolutional Long Short-term Memory Network (CNN-LSTM).

Data Collection

Diver respiration data was collected across varied underwater environments—closed-water pools, freshwater lakes, and seawater—ensuring diverse conditions in terms of light, visibility, and diver gear. Approximately 40,000 images and several million audio samples constitute the dataset, providing a robust basis for training and evaluating the detection algorithms.

Results

The study evaluates the detection accuracy of SVM, CNN, and CNN-LSTM methods. Overall, CNN performed best in terms of classification accuracy (~97%), slightly outperforming CNN-LSTM. Despite CNN's superior detection accuracy, CNN-LSTM demonstrated better HRR estimation in more complex visual conditions, such as seawater environments.

The HRR estimation results compared against human analysts showed mixed performance. The CNN-LSTM model exhibited the lowest error in seawater conditions, highlighting its potential for capturing temporal breathing patterns. Nevertheless, errors were significant in certain freshwater videos, indicating limitations in generalization across varied underwater scenarios.

Implications

The practical implications of this research are substantial: it provides a foundation for developing AUV systems capable of enhancing diver safety through real-time monitoring and potentially improving communication efficiency by timing signal exchanges to diver inhalations. On a theoretical level, this work paves a path for further investigations into non-contact based physiological monitoring solutions in aquatic environments.

Future Directions

There are several promising directions for future research:

1. Advanced Training Datasets: Expanding training datasets to include more realistic and diverse diving scenarios to improve algorithmic robustness.
2. Refined Detection Algorithms: Integrating more sophisticated models and techniques to reduce false positives and enhance detection accuracy under challenging conditions.
3. Hybrid Models: Combining visual and acoustic data more effectively to exploit their complementary strengths in reducing ambiguities.
4. Extended Analysis: Exploring the applicability of this approach in different UHRI contexts beyond diving, such as submarine crew monitoring or autonomous underwater exploration.

In conclusion, the presented work demonstrates a significant stride towards addressing a hitherto unexplored problem in underwater robotics, offering innovative solutions with broad implications for diver safety and underwater communications.