SCIENCE@home

IoT-V2E: An Uncertainty-Aware Cross-Modal Hashing Retrieval Between Infrared-Videos and EEGs for Automated Sleep State Analysis

Jianan Han,Aidong Men,Yang Liu,Ziming Yao,Shenmin Zhang,Yan-Tao Yan,Qingchao Chen

Published 2024 in IEEE Internet of Things Journal

ABSTRACT

Estimating and monitoring the sleep states at home using ubiquitous infrared (IR) visual camera sensors is an essential healthcare problem. Currently, the common challenge of using IoT sensors to predict sleep stages is the “semantic gap” between the IoT sensory signals and the medical signals, where fewer correlations between IoT sensory signals and the sleep stage labels are observed. To bridge this gap, we propose a novel systematic and methodological IoT design (IoT-V2E) to retrieve the most similar electroencephalogram signal representations in a database given an IR visual query for sleep-related analysis. Specifically, we make the following specific contributions: 1) we collect a cross-modal retrieval data set, including the IR sensory signals and the synchronized Polysomnography signals with sleep stage ground-truth annotations; 2) we propose a novel uncertainty-aware hashing retrieval method, presenting superior performances, sufficient interpretability, and high memory efficiency; 3) our method achieves the state-of-the-art sleep stage retrieval results and provides the uncertainty for each query in the inference; and 4) most importantly, our system is evaluated to be able to assist the physicians not only in diagnosing sleep-related diseases but also finding the subjects with the most similar sleep patterns. Our project is available at https://github.com/SPIresearch/IoT-V2E.

PUBLICATION RECORD

CITATION MAP

EXTRACTION MAP

CLAIMS

  • No claims are published for this paper.

CONCEPTS

  • No concepts are published for this paper.

REFERENCES

Showing 1-77 of 77 references · Page 1 of 1

CITED BY

Showing 1-11 of 11 citing papers · Page 1 of 1