YORU: Animal behavior detection with object-based approach for real-time closed-loop feedback

The advent of deep learning methodologies for animal behavior analysis has revolutionized neuroethology studies. However, the analysis of social behaviors, characterized by dynamic interactions among multiple individuals, continues to represent a major challenge. In this study, we present “YORU” (your optimal recognition utility), a behavior detection approach leveraging an object detection deep learning algorithm. Unlike conventional approaches, YORU directly identifies behaviors as “behavior objects” based on the animal’s shape, enabling robust and accurate detection. YORU successfully classified several types of social behaviors in species ranging from vertebrates to insects. Furthermore, YORU enables real-time behavior analysis and closed-loop feedback. In addition, we achieved real-time delivery of photostimulation feedback to specific individuals during social behaviors, even when multiple individuals are close together. This system overcomes the challenges posed by conventional pose estimation methods and presents an alternative approach for behavioral analysis.

• Publication year

  2026

• Venue

  Science Advances

• Publication date

  2026-02-11

• Fields of study

  Biology, Medicine, Computer Science

• Identifiers
  DOI 10.1126/sciadv.adw2109PMID 41671367PMCID 12893285
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Simple Behavioral Analysis (SimBA) as a platform for explainable machine learning in behavioral neuroscience

  2024cited by this paper
• YOLO-Behaviour: A simple, flexible framework to automatically quantify animal behaviours from videos

  2024cited by this paper
• What is YOLOv5: A deep look into the internal features of the popular object detector

  2024cited by this paper
• The Zebrafish Cerebellar Neural Circuits Are Involved in Orienting Behavior

  2024cited by this paper
• Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics

  2024cited by this paper
• Two distinct functions of Lim1 in the Drosophila antenna

  2024cited by this paper
• Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling and cloud-native open-source tools

  2024cited by this paper
• Advancing social behavioral neuroscience by integrating ethology and comparative psychology methods through machine learning.

  2023cited by this paper
• A-SOiD, an active learning platform for expert-guided, data efficient discovery of behavior

  2023cited by this paper
• LabGym: Quantification of user-defined animal behaviors using learning-based holistic assessment.

  2023cited by this paper
• Nested neural circuits generate distinct acoustic signals during Drosophila courtship

  2023cited by this paper
• Metabolic division of labor in social insects.

  2023cited by this paper
• Event-triggered feedback system using YOLO for optogenetic manipulation of neural activity

  2023cited by this paper
• Piezo-mediated mechanosensation contributes to stabilizing copulation posture and reproductive success in Drosophila males

  2023cited by this paper
• Multi-animal pose estimation, identification and tracking with DeepLabCut

  2022influential reference
• Assessing the impact of image quality on object-detection algorithms

  2022cited by this paper
• Posteromedial cortical networks encode visuomotor prediction errors

  2022influential reference
• Performance Study of YOLOv5 and Faster R-CNN for Autonomous Navigation around Non-Cooperative Targets

  2022cited by this paper
• Analysis of social interactions in group-housed animals using dyadic linear modelsf

  2022cited by this paper
• Editorial: The Neuroethology of Social Behavior

  2022cited by this paper
• SLEAP: A deep learning system for multi-animal pose tracking

  2022influential reference
• The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions

  2021cited by this paper
• Communication in Ant Societies

  2021cited by this paper
• Python Parallel Processing and Multiprocessing: A Rivew

  2021cited by this paper
• A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit

  2021influential reference
• Chemogenetics: Beyond Lesions and Electrodes

  2021cited by this paper
• Molecular and neural mechanisms regulating sexual motivation of virgin female Drosophila

  2021cited by this paper
• New Open-Source Tools: Using Bonsai for Behavioral Tracking and Closed-Loop Experiments

  2021cited by this paper
• Decision letter: TRex, a fast multi-animal tracking system with markerless identification, and 2D estimation of posture and visual fields

  2020cited by this paper
• Interactive Stickies: Low-latency projection mapping for dynamic interaction with projected images on a movable surface

  2020cited by this paper
• LabelStoma: A tool for stomata detection based on the YOLO algorithm

  2020cited by this paper
• Real-time, low-latency closed-loop feedback using markerless posture tracking

  2020cited by this paper
• Neural Evolution of Context-Dependent Fly Song.

  2019cited by this paper
• Sensorimotor processing in the rodent barrel cortex

  2019cited by this paper
• The HIV-1 latent reservoir is largely sensitive to circulating T cells

  2019cited by this paper
• Deep learning tools for the measurement of animal behavior in neuroscience

  2019cited by this paper
• Understanding of Object Detection Based on CNN Family and YOLO

  2018cited by this paper
• GABAergic Local Interneurons Shape Female Fruit Fly Response to Mating Songs

  2018cited by this paper
• Open source modules for tracking animal behavior and closed-loop stimulation based on Open Ephys and Bonsai

  2018cited by this paper
• Forebrain Control of Behaviorally Driven Social Orienting in Zebrafish.

  2018influential reference
• DeepLabCut: markerless pose estimation of user-defined body parts with deep learning

  2018influential reference
• An unbiased template of the Drosophila brain and ventral nerve cord

  2018cited by this paper
• Can Multilayer Networks Advance Animal Behavior Research?

  2018cited by this paper
• Global Representations of Goal-Directed Behavior in Distinct Cell Types of Mouse Neocortex

  2017cited by this paper
• Optogenetic inhibition of behavior with anion channelrhodopsins

  2017cited by this paper
• Neuroscience Needs Behavior: Correcting a Reductionist Bias.

  2017cited by this paper
• BORIS: a free, versatile open‐source event‐logging software for video/audio coding and live observations

  2016cited by this paper
• You Only Look Once: Unified, Real-Time Object Detection

  2015influential reference
• Social Eavesdropping in Zebrafish: Tuning of Attention to Social Interactions

  2015influential reference
• The Pascal Visual Object Classes Challenge: A Retrospective

  2014cited by this paper
• Natural Whisker-Guided Behavior by Head-Fixed Mice in Tactile Virtual Reality

  2014cited by this paper
• Optogenetic control of selective neural activity in multiple freely moving Drosophila adults

  2014cited by this paper
• Bonsai: an event-based framework for processing and controlling data streams

  2014cited by this paper
• Optogenetics and thermogenetics: technologies for controlling the activity of targeted cells within intact neural circuits.

  2012cited by this paper
• The development and application of optogenetics.

  2011cited by this paper
• The aligned rank transform for nonparametric factorial analyses using only anova procedures

  2011cited by this paper
• Neuronal control of Drosophila courtship song.

  2011cited by this paper
• A New Chamber for Studying the Behavior of Drosophila

  2010cited by this paper
• Neurogenetics of courtship and mating in Drosophila.

  2008cited by this paper
• Of mice and missing data: what we know (and need to learn) about male sexual behavior.

  2004cited by this paper
• Geodesic estimation for large deformation anatomical shape averaging and interpolation.

  2004cited by this paper
• S C I E N C E

  year unknowncited by this paper

• No citing papers are available for this paper.