Acoustic-based sensing and applications: A survey

Abstract With advancements of wireless and sensing technologies, recent studies have demonstrated technical feasibility and effectiveness of using acoustic signals for sensing. In the past decades, low-cost audio infrastructures are widely-deployed and integrated into mobile and Internet of Things (IoT) devices to facilitate a broad array of applications including human activity recognition, tracking, localization, and security monitoring. The technology underpinning these applications lies in the analysis of propagation properties of acoustic signals (e.g., reflection, diffraction, and scattering) when they encounter human bodies. As a result, these applications serve as the foundation to support various daily functionalities such as safety protection, smart healthcare, and smart appliance interaction. The already-existing acoustic infrastructure could also complement RF-based localization and other approaches based on short-range communications such as Near-Field Communication (NFC) and Quick Response (QR) code. In this paper, we provide a comprehensive review on acoustic-based sensing in terms of hardware infrastructure, technical approaches, and its broad applications. First we describe different methodologies and techniques of using acoustic signals for sensing including Time-of-Arrival (ToA), Frequency Modulated Continuous Wave (FMCW), Time-Difference-of-Arrival (TDoA), and Channel Impulse Response (CIR). Then we classify various applications and compare different acoustic-based sensing approaches: in recognition and tracking, we review daily activity recognition, human health and behavioral monitoring hand gesture recognition, hand movement tracking, and speech recognition; in localization and navigation, we discuss ranging and direction finding, indoor and outdoor localization, and floor map construction; in security and privacy, we survey user authentication, keystroke snooping attacks, audio adversarial attacks, acoustic vibration attacks, and privacy protection schemes. Lastly we discuss future research directions and limitations of the acoustic-based sensing.

• Publication year

  2020

• Venue

  Comput. Networks

• Publication date

  2020-11-01

• Fields of study

  Physics, Computer Science, Engineering

• Identifiers
  DOI 10.1016/j.comnet.2020.107447
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• An Indirect Eavesdropping Attack of Keystrokes on Touch Screen through Acoustic Sensing

  2021cited by this paper
• User authentication on mobile devices: Approaches, threats and trends

  2020cited by this paper
• Metamorph: Injecting Inaudible Commands into Over-the-air Voice Controlled Systems

  2020cited by this paper
• Wireless Sensing for Human Activity: A Survey

  2020cited by this paper
• Practical Adversarial Attacks Against Speaker Recognition Systems

  2020cited by this paper
• Alexa

  2020cited by this paper
• Enable Traditional Laptops with Virtual Writing Capability Leveraging Acoustic Signals

  2020cited by this paper
• Adversarial Music: Real World Audio Adversary Against Wake-word Detection System

  2019cited by this paper
• Lip Reading-Based User Authentication Through Acoustic Sensing on Smartphones

  2019cited by this paper
• Defeating hidden audio channel attacks on voice assistants via audio-induced surface vibrations

  2019cited by this paper
• BreathListener: Fine-grained Breathing Monitoring in Driving Environments Utilizing Acoustic Signals

  2019cited by this paper
• You Talk Too Much: Limiting Privacy Exposure Via Voice Input

  2019cited by this paper
• Poster: Inaudible High-throughput Communication Through Acoustic Signals

  2019cited by this paper
• KeyLiSterber: Inferring Keystrokes on QWERTY Keyboard of Touch Screen through Acoustic Signals

  2019cited by this paper
• Spearphone: A Speech Privacy Exploit via Accelerometer-Sensed Reverberations from Smartphone Loudspeakers

  2019cited by this paper
• Prεεch: A System for Privacy-Preserving Speech Transcription

  2019cited by this paper
• TDOA-Based Multiple Acoustic Source Localization Without Association Ambiguity

  2018cited by this paper
• Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces

  2018cited by this paper
• Using Acoustic Signal and Image to Achieve Accurate Indoor Localization

  2018influential reference
• Acousticcardiogram: Monitoring Heartbeats using Acoustic Signals on Smart Devices

  2018influential reference
• Next-generation of virtual personal assistants (Microsoft Cortana, Apple Siri, Amazon Alexa and Google Home)

  2018cited by this paper
• SilentKey

  2018cited by this paper
• Audio Adversarial Examples: Targeted Attacks on Speech-to-Text

  2018cited by this paper
• Home Alone: The Insider Threat of Unattended Wearables and A Defense using Audio Proximity

  2018cited by this paper
• Potential-Field-Based Active Exploration for Acoustic Simultaneous Localization and Mapping

  2018cited by this paper
• 2MA: Verifying Voice Commands via Two Microphone Authentication

  2018cited by this paper
• PatternListener: Cracking Android Pattern Lock Using Acoustic Signals

  2018cited by this paper
• Vernier: Accurate and Fast Acoustic Motion Tracking Using Mobile Devices

  2018influential reference
• CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition

  2018cited by this paper
• LipPass: Lip Reading-based User Authentication on Smartphones Leveraging Acoustic Signals

  2018influential reference
• WordRecorder: Accurate Acoustic-based Handwriting Recognition Using Deep Learning

  2018influential reference
• Ubicoustics: Plug-and-Play Acoustic Activity Recognition

  2018cited by this paper
• VPad: Virtual Writing Tablet for Laptops Leveraging Acoustic Signals

  2018influential reference
• Smartphone Localization Using Active-Passive Acoustic Sensing

  2018influential reference
• Alexa, Are You Listening?

  2018cited by this paper
• EchoPrint: Two-factor Authentication using Acoustics and Vision on Smartphones

  2018influential reference
• Proximity-Proof: Secure and Usable Mobile Two-Factor Authentication

  2018cited by this paper
• Listening Watch: Wearable Two-Factor Authentication using Speech Signals Resilient to Near-Far Attacks

  2018cited by this paper
• Smartphone-based Acoustic Indoor Space Mapping

  2018cited by this paper
• Leveraging Audio Signals for Early Recognition of Inattentive Driving with Smartphones

  2018cited by this paper
• Towards Environment Independent Device Free Human Activity Recognition

  2018influential reference
• SteerTrack: Acoustic-Based Device-Free Steering Tracking Leveraging Smartphones

  2018cited by this paper
• X-Vectors: Robust DNN Embeddings for Speaker Recognition

  2018cited by this paper
• UltraGesture: Fine-Grained Gesture Sensing and Recognition

  2018influential reference
• Shooter localization in wireless acoustic sensor networks

  2017cited by this paper
• Time-of-Arrival-Based Indoor Smartphone Localization Using Light-Synchronized Acoustic Waves

  2017cited by this paper
• An investigation of recurrent neural network for daily activity recognition using multi-modal signals

  2017cited by this paper
• SoundWrite II: Ambient Acoustic Sensing for Noise Tolerant Device-Free Gesture Recognition

  2017cited by this paper
• CondioSense: high-quality context-aware service for audio sensing system via active sonar

  2017cited by this paper
• SmarPer: Context-Aware and Automatic Runtime-Permissions for Mobile Devices

  2017cited by this paper
• DolphinAttack: Inaudible Voice Commands

  2017influential reference
• BackDoor: Making Microphones Hear Inaudible Sounds

  2017influential reference
• Deep Learning for Sensor-based Activity Recognition: A Survey

  2017cited by this paper
• Strata: Fine-Grained Acoustic-based Device-Free Tracking

  2017influential reference
• Physical activity recognition by smartphones, a survey

  2017cited by this paper
• Camera Based Two Factor Authentication Through Mobile and Wearable Devices

  2017cited by this paper
• Continuous Authentication for Voice Assistants

  2017cited by this paper
• S2M: A Lightweight Acoustic Fingerprints-Based Wireless Device Authentication Protocol

  2017cited by this paper
• BatMapper: Acoustic Sensing Based Indoor Floor Plan Construction Using Smartphones

  2017influential reference
• EchoTrack: Acoustic device-free hand tracking on smart phones

  2017cited by this paper
• PitchIn: Eavesdropping via Intelligible Speech Reconstruction Using Non-acoustic Sensor Fusion

  2017cited by this paper
• Indoor Self-Localization and Orientation Estimation of Smartphones Using Acoustic Signals

  2017cited by this paper
• Acoustic Sensor Self-Localization: Models and Recent Results

  2017cited by this paper
• A Survey of Sound Source Localization Methods in Wireless Acoustic Sensor Networks

  2017cited by this paper
• Hearing Your Voice is Not Enough: An Articulatory Gesture Based Liveness Detection for Voice Authentication

  2017cited by this paper
• Gait-based Human identification using acoustic sensor and deep neural network

  2017cited by this paper
• Eavesdrop with PoKeMon: Position free keystroke monitoring using acoustic data

  2017cited by this paper
• Exploiting Spatial Degrees of Freedom for High Data Rate Ultrasound Communication with Implantable Devices

  2017cited by this paper
• PACMAN: Personal Agent for Access Control in Social Media

  2017cited by this paper
• A Software-Defined Ultrasonic Networking Framework for Wearable Devices

  2017cited by this paper
• Smart User Authentication through Actuation of Daily Activities Leveraging WiFi-enabled IoT

  2017cited by this paper
• Stride-in-the-Loop Relative Positioning Between Users and Dummy Acoustic Speakers

  2017cited by this paper
• A feasibility analysis on the use of ultrasonic multihop communications for E-health applications

  2017cited by this paper
• BreathPrint: Breathing Acoustics-based User Authentication

  2017influential reference
• Generalized End-to-End Loss for Speaker Verification

  2017cited by this paper
• ER: Early recognition of inattentive driving leveraging audio devices on smartphones

  2017influential reference
• EchoLoc: Accurate Device-Free Hand Localization Using COTS Devices

  2016cited by this paper
• A floor plan creation tool utilizing a smartphone with an ultrasonic sensor gadget

  2016cited by this paper
• FingerIO: Using Active Sonar for Fine-Grained Finger Tracking

  2016cited by this paper
• Securing acoustics-based short-range communication systems: an overview

  2016cited by this paper
• VoiceLive: A Phoneme Localization based Liveness Detection for Voice Authentication on Smartphones

  2016cited by this paper
• AMIL: Localizing neighboring mobile devices through a simple gesture

  2016cited by this paper
• Messages behind the sound: real-time hidden acoustic signal capture with smartphones

  2016influential reference
• Near-ultrasound communication for TV's 2nd screen services

  2016cited by this paper
• CAT: high-precision acoustic motion tracking

  2016influential reference
• The Sounds of the Phones: Dangers of Zero-Effort Second Factor Login based on Ambient Audio

  2016cited by this paper
• User Identification and Authentication Using Keystroke Dynamics with Acoustic Signal

  2016influential reference
• WalkieLokie: sensing relative positions of surrounding presenters by acoustic signals

  2016influential reference
• Device-free gesture tracking using acoustic signals

  2016influential reference
• AudioGest: enabling fine-grained hand gesture detection by decoding echo signal

  2016influential reference
• Accurate Combined Keystrokes Detection Using Acoustic Signals

  2016cited by this paper
• Underwater Acoustic Communication

  2015cited by this paper
• Contactless Sleep Apnea Detection on Smartphones

  2015influential reference
• Spartacus: Spatially-Aware Interaction for Mobile Devices Through Energy-Efficient Audio Sensing

  2015cited by this paper
• Fine-grained sleep monitoring: Hearing your breathing with smartphones

  2015influential reference
• Sound-Proof: Usable Two-Factor Authentication Based on Ambient Sound

  2015influential reference
• Turning a Mobile Device into a Mouse in the Air

  2015cited by this paper
• Chirp signal-based aerial acoustic communication for smart devices

  2015influential reference
• AccelWord: Energy Efficient Hotword Detection through Accelerometer

  2015cited by this paper
• SoundWrite: Text Input on Surfaces through Mobile Acoustic Sensing

  2015influential reference

• Safe Search and Rescue Operations Based on Autonomous Robots: A Systematic Review of the General System Architecture

  2026cites this paper
• Device-Free Visible Light Sensing: A Survey

  2026cites this paper
• A Survey of Wireless Sensing Security From a Role-Based View

  2026cites this paper
• TouchAI: Toward Waterproof Mobile Touchscreen Interface via Acoustic–Inertial Sensing Modalities

  2026cites this paper
• Automated and Scalable Footstep Vibration-Based Pedestrian Localization in Built Environments Using Deep Learning

  2026cites this paper
• Acoustic sensing mechanisms, technologies, and applications: a survey

  2025influential citation
• A new deep sparse unfolding network with variational Bayesian enhancement for high-resolution localization of acoustic sources

  2025cites this paper
• Wi-GR: Wi-Fi-Based Gait Recognition Using Multi-Part Velocity Profile

  2025cites this paper
• Teaching Physical Awareness to LLMs through Sounds

  2025influential citation
• Deep learning-enhanced anti-noise triboelectric acoustic sensor for human-machine collaboration in noisy environments

  2025cites this paper
• Dual Interferometric Interrogation for DFB Laser-Based Acoustic Sensing

  2025cites this paper
• Toward Open-World-Aware User Authentication Based on Human Bodies Using mmWave Signals

  2025cites this paper
• Biomimetic Robotics and Intelligence: A Survey

  2025cites this paper
• Pilot agent implied efficient data communication in pervasive acoustic wireless sensor network

  2025cites this paper
• Human Behavior Recognition in Retail Environments With Graph-Driven RFID Signal Embedding

  2025cites this paper
• The Role, Trends, and Applications of Machine Learning in Undersea Communication: A Bangladesh Perspective

  2025cites this paper
• A Study on the Lightweight and Fast Response GRU Techniques for Indoor Continuous Motion Recognition Based on Wi-Fi CSI

  2025cites this paper
• Environmental Acoustic Features Robustness Analysis: A Multi-Aspecs Study

  2025cites this paper
• A Voxel-Based 3D reconstruction and action recognition method for construction workers

  2025cites this paper
• LaserKey: Eavesdropping Keyboard Typing Leveraging Vibrational Emanations via Laser Sensing

  2025cites this paper
• MoGL: A robust multi-view learning framework for unsupervised acoustic sensing

  2025cites this paper
• AcouListener: An Inaudible Acoustic Side-Channel Attack on AR/VR Systems

  2025cites this paper
• Stretchable electrospun nanocomposite for energy storage and harvesting from both vibrational and acoustic excitations

  2025cites this paper
• Evaluating Egyptian citizens’ perception toward introducing voice assistant technology as a means of improving public service delivery: utilizing machine learning as an additional perception’s predictor

  2025cites this paper
• A Novel Acoustic Source Localization Technique for Edge AI Applications: A Lightweight Framework and Implementation for IoT and Smart Sensing Devices

  2025cites this paper
• High-Performance Acoustic Transducers with Exfoliated NbSe2 Nanosheets and Hybrid Force Mechanisms

  2025cites this paper
• A review of sound-based pig monitoring for enhanced precision production

  2024cites this paper
• Current Applications of Machine Learning in Additive Manufacturing: A Review on Challenges and Future Trends

  2024cites this paper
• System Aspects of Direct Localization of Multiple Acoustic Sources Using Distributed Sensor Networks

  2024cites this paper
• Enhancing smart home device identification in WiFi environments for futuristic smart networks-based IoT

  2024cites this paper
• AFace

  2024cites this paper
• Kirigami

  2024cites this paper
• Plug-and-play multi-dimensional attention module for accurate Human Activity Recognition

  2024cites this paper
• Analysis of Machine Learning Classifiers for Speaker Identification: A Study on SVM, Random Forest, KNN, and Decision Tree

  2024cites this paper
• Design, fabrication and experimental analysis of piezoresistive bidirectional acoustic sensor

  2024cites this paper
• The EarSAVAS Dataset

  2024cites this paper
• A survey of acoustic eavesdropping attacks: Principle, methods, and progress

  2024cites this paper
• Passive Monitoring of Dangerous Driving Behaviors Using mmWave Radar

  2024cites this paper
• EchoGest: A Highly Scalable Unseen Gesture Recognition System Based on Feature-Wise Transformation

  2024cites this paper
• Sok: Comprehensive Security Overview, Challenges, and Future Directions of Voice-Controlled Systems

  2024cites this paper
• RFSpy: Eavesdropping on Online Conversations with Out-of-Vocabulary Words by Sensing Metal Coil Vibration of Headsets Leveraging RFID

  2024cites this paper
• A Systematic Review of Contemporary Indoor Positioning Systems: Taxonomy, Techniques, and Algorithms

  2024cites this paper
• PreGesNet: Few-Shot Acoustic Gesture Recognition Based on Task-Adaptive Pretrained Networks

  2024influential citation
• Underwater small and occlusion object detection with feature fusion and global context decoupling head-based YOLO

  2024cites this paper
• FDME-RATE: Frequency-Domain-Based Multipath Extraction for Robust Acoustic TOA Estimation

  2024cites this paper
• Heart of the future home: a multidimensional model of inclusive kitchen for older people in the UK

  2024cites this paper
• Secure Audio Classification for Voice Assistants: A Multi-Party Homomorphic Approach

  2024cites this paper
• AGR: Acoustic Gait Recognition Using Interpretable Micro-Range Profile

  2024cites this paper
• OpenAuth: Human Body-Based User Authentication Using mmWave Signals in Open-World Scenarios

  2024cites this paper
• An efficient multi-modal sensors feature fusion approach for handwritten characters recognition using Shapley values and deep autoencoder

  2024cites this paper
• Acoustic data acquisition and integration for semantic organization of sentimental data and analysis in a PWSN

  2024cites this paper
• Stretchable electrospun PVDF/TPU nanofibers membranes: Acoustic signals detectors

  2024cites this paper
• USpeech: Ultrasound-Enhanced Speech with Minimal Human Effort via Cross-Modal Synthesis

  2024cites this paper
• Enhanced cell phone security: An ultrasonic and sensor fusion-based persistent cell phone protection method integrating anti-theft & identity authentication

  2024cites this paper
• DASIV: Directional Acoustic Sensing based Intelligent Vehicle Interaction System

  2024cites this paper
• GOTCHA: Physical Intrusion Detection with Active Acoustic Sensing Using a Smart Speaker

  2024cites this paper
• Exploring the Landscape of Acoustic Authentication: Techniques, Performance and Security

  2024cites this paper
• EchoSense: Acoustic Sensing for Enclosed Structure Monitoring and Recognition

  2023cites this paper
• MAA: Modulation-adaptive Acoustic Gesture Recognition

  2023cites this paper
• Molybdenum disulfide as a propitious electrochemical sensing material: a mini review

  2023cites this paper
• Self‐Powered e‐Skin Based on Integrated Flexible Organic Photovoltaics and Transparent Touch Sensors

  2023cites this paper
• Research Progress of Human–Computer Interaction Technology Based on Gesture Recognition

  2023cites this paper
• Prospects and Applications of Incoherent Light in Non-contact Wireless Sensing Systems

  2023cites this paper
• Real-time TDOA-based stationary sound source direction finding

  2023cites this paper
• Self-aware artificial auditory neuron with a triboelectric sensor for spike-based neuromorphic hardware

  2023cites this paper
• mmAssist : Passive Monitoring of Driver's Attentiveness Using mmWave Sensors

  2023cites this paper
• A Review of Wireless Positioning Techniques and Technologies: From Smart Sensors to 6G

  2023cites this paper
• mmDrive: mmWave Sensing for Live Monitoring and On-Device Inference of Dangerous Driving

  2023cites this paper
• Sub-Nanometer Acoustic Vibration Sensing Using a Tapered-Tip Optical Fiber Microcantilever

  2023cites this paper
• Scribe

  2023cites this paper
• Continuous Multi-user Activity Tracking via Room-Scale mmWave Sensing

  2023cites this paper
• Non-contact sensing technology enables precision livestock farming in smart farms

  2023cites this paper
• A Survey on Acoustic Positioning Systems for Location-Based Services

  2022cites this paper
• Experience: practical problems for acoustic sensing

  2022cites this paper
• BreathMentor

  2022cites this paper
• Evaluation of Low-cost Air Quality Sensor Calibration Models

  2022cites this paper
• Scalable real-time sound source localization method based on TDOA

  2022cites this paper
• SPiDR: ultra-low-power acoustic spatial sensing for micro-robot navigation

  2022cites this paper
• Information Disclosure in the Era of Voice Technology

  2022cites this paper
• ITrackU: tracking a pen-like instrument via UWB-IMU fusion

  2021cites this paper
• Owlet: enabling spatial information in ubiquitous acoustic devices

  2021cites this paper
• Sound Source Localization Using a Convolutional Neural Network and Regression Model

  2021cites this paper
• EXPERIENCE

  2020cites this paper