Relief-Based Feature Selection: Introduction and Review

Feature selection plays a critical role in biomedical data mining, driven by increasing feature dimensionality in target problems and growing interest in advanced but computationally expensive methodologies able to model complex associations. Specifically, there is a need for feature selection methods that are computationally efficient, yet sensitive to complex patterns of association, e.g. interactions, so that informative features are not mistakenly eliminated prior to downstream modeling. This paper focuses on Relief-based algorithms (RBAs), a unique family of filter-style feature selection algorithms that have gained appeal by striking an effective balance between these objectives while flexibly adapting to various data characteristics, e.g. classification vs. regression. First, this work broadly examines types of feature selection and defines RBAs within that context. Next, we introduce the original Relief algorithm and associated concepts, emphasizing the intuition behind how it works, how feature weights generated by the algorithm can be interpreted, and why it is sensitive to feature interactions without evaluating combinations of features. Lastly, we include an expansive review of RBA methodological research beyond Relief and its popular descendant, ReliefF. In particular, we characterize branches of RBA research, and provide comparative summaries of RBA algorithms including contributions, strategies, functionality, time complexity, adaptation to key data characteristics, and software availability.

• Publication year

  2017

• Venue

  Journal of Biomedical Informatics

• Publication date

  2017-11-22

• Fields of study

  Medicine, Computer Science, Mathematics

• Identifiers
  DOI 10.1016/j.jbi.2018.07.014arXiv 1711.08421PMID 30031057PMCID PMC6299836
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Imbalanced data classification using MapReduce and relief

  2018cited by this paper
• Collective feature selection to identify crucial epistatic variants

  2018cited by this paper
• Feature Selection for Detecting Gene-Gene Interactions in Genome-Wide Association Studies

  2018cited by this paper
• PMLB: a large benchmark suite for machine learning evaluation and comparison

  2017cited by this paper
• Benchmarking Relief-Based Feature Selection Methods

  2017cited by this paper
• Differential privacy‐based evaporative cooling feature selection and classification with relief‐F and random forests

  2017cited by this paper
• A Survey and Comparative Study of Filter and Wrapper Feature Selection Techniques

  2016cited by this paper
• A Weighted Feature Selection Method for Instance-Based Classification

  2016cited by this paper
• Multithreaded and Spark parallelization of feature selection filters

  2016cited by this paper
• XGBoost: A Scalable Tree Boosting System

  2016cited by this paper
• ExSTraCS 2.0: description and evaluation of a scalable learning classifier system

  2015cited by this paper
• A review of feature selection methods with applications

  2015cited by this paper
• Very Large Scale ReliefF Algorithm on GPU for Genome-Wide Association Study

  2015influential reference
• New technique for feature selection: Combination between elastic net and relief

  2015cited by this paper
• Feature selection of medical data sets based on RS-RELIEFF

  2015cited by this paper
• Scalable extensions of the ReliefF algorithm for weighting and selecting features on the multi-label learning context

  2015cited by this paper
• Epistasis analysis using ReliefF.

  2015influential reference
• A survey on feature selection methods

  2014cited by this paper
• Feature Selection for Classification: A Review

  2014cited by this paper
• Stable feature selection with ensembles of multi-reliefF

  2014cited by this paper
• An Extended Michigan-Style Learning Classifier System for Flexible Supervised Learning, Classification, and Data Mining

  2014cited by this paper
• Relief for regression with missing data in variable selection

  2014cited by this paper
• A review of feature selection methods on synthetic data

  2013influential reference
• Feature selection based on ReliefF and PCA for underwater sound classification

  2013cited by this paper
• Multiple Threshold Spatially Uniform ReliefF for the Genetic Analysis of Complex Human Diseases

  2013influential reference
• ReliefF for Multi-label Feature Selection

  2013cited by this paper
• Orange: data mining toolbox in python

  2013cited by this paper
• Extending ReliefF for Hierarchical Multi-label Classification ?

  2013cited by this paper
• Feature Selection Using Cooperative Game Theory and Relief Algorithm

  2013cited by this paper
• ReliefF-ML: An Extension of ReliefF Algorithm to Multi-label Learning

  2013cited by this paper
• A Fast Clustering-Based Feature Subset Selection Algorithm for High-Dimensional Data

  2013cited by this paper
• A new algorithm relief hybrid (HRelief) for biological motifs selection

  2013cited by this paper
• ReliefSeq: A Gene-Wise Adaptive-K Nearest-Neighbor Feature Selection Tool for Finding Gene-Gene Interactions and Main Effects in mRNA-Seq Gene Expression Data

  2013cited by this paper
• A Comparative Analysis of Biomarker Selection Techniques

  2013cited by this paper
• Review of feature selection techniques in bioinformatics

  2012cited by this paper
• Using Expert Knowledge to Guide Covering and Mutation in a Michigan Style Learning Classifier System to Detect Epistasis and Heterogeneity

  2012cited by this paper
• Application of a spatially-weighted Relief algorithm for ranking genetic predictors of disease

  2012cited by this paper
• ReliefF-MI: An extension of ReliefF to multiple instance learning

  2012cited by this paper
• A Review and Comparative Study on Univariate Feature Selection Techniques

  2012cited by this paper
• Classification and regression trees

  2012cited by this paper
• Filter Approach Feature Selection Methods to Support Multi-label Learning Based on ReliefF and Information Gain

  2012cited by this paper
• Multi-label ReliefF and F-statistic feature selections for image annotation

  2012cited by this paper
• Feature Weighting by RELIEF Based on Local Hyperplane Approximation

  2012cited by this paper
• RELIEF-DISC: An Extended RELIEF Algorithm Using Discretization Approach for Continuous Features

  2011cited by this paper
• RELIEF-C: Efficient Feature Selection for Clustering over Noisy Data

  2011cited by this paper
• RELIEF Algorithm and Similarity Learning for k-NN

  2011cited by this paper
• Scikit-learn: Machine Learning in Python

  2011cited by this paper
• Review and Evaluation of Feature Selection Algorithms in Synthetic Problems

  2011influential reference
• Implementing ReliefF filters to extract meaningful features from genetic lifetime datasets

  2011cited by this paper
• Feature selection is the ReliefF for multiple instance learning

  2010cited by this paper
• Algorithms for subsetting attribute values with Relief

  2010cited by this paper
• Generalized iterative RELIEF for supervised distance metric learning

  2010cited by this paper
• Ensemble-based classifiers

  2010cited by this paper
• Local-Learning-Based Feature Selection for High-Dimensional Data Analysis

  2010influential reference
• The Informative Extremes: Using Both Nearest and Farthest Individuals Can Improve Relief Algorithms in the Domain of Human Genetics

  2010influential reference
• A comparison of random forest and its Gini importance with standard chemometric methods for the feature selection and classification of spectral data

  2009cited by this paper
• Spatially Uniform ReliefF (SURF) for computationally-efficient filtering of gene-gene interactions

  2009influential reference
• ReliefMSS: a variation on a feature ranking ReliefF algorithm

  2009cited by this paper
• Searching for interacting features in subset selection

  2009cited by this paper
• The WEKA data mining software: an update

  2009cited by this paper
• Capturing the Spectrum of Interaction Effects in Genetic Association Studies by Simulated Evaporative Cooling Network Analysis

  2009cited by this paper
• Simulated Annealing

  2008cited by this paper
• Robust Feature Selection Using Ensemble Feature Selection Techniques

  2008cited by this paper
• Multi-RELIEF: a method to recognize specificity determining residues from multiple sequence alignments using a Machine-Learning approach for feature weighting

  2008cited by this paper
• Very large scale ReliefF for genome-wide association analysis

  2008influential reference
• A RELIEF Based Feature Extraction Algorithm

  2008cited by this paper
• Iterative RELIEF for Feature Weighting: Algorithms, Theories, and Applications

  2007influential reference
• A review of feature selection techniques in bioinformatics

  2007cited by this paper
• Evaporative cooling feature selection for genotypic data involving interactions

  2007cited by this paper
• extraRelief: Improving Relief by Efficient Selection of Instances

  2007cited by this paper
• Extended Relief Algorithms in Instance-Based Feature Filtering

  2007cited by this paper
• Non-Myopic Feature Quality Evaluation with (R)ReliefF

  2007influential reference
• Tuning ReliefF for Genome-Wide Genetic Analysis

  2007influential reference
• Genomic mining for complex disease traits with “random chemistry”

  2007cited by this paper
• Machine Learning Techniques and Chi-Square Feature Selection for Cancer Classification Using SAGE Gene Expression Profiles

  2006cited by this paper
• Multi Criteria Wrapper Improvements to Naive Bayes Learning

  2006cited by this paper
• Orthogonal Relief Algorithm for Feature Selection

  2006cited by this paper
• Iterative RELIEF for feature weighting

  2006influential reference
• Multivariate Non-Linear Feature Selection with Kernel Methods

  2005cited by this paper
• Addendum: Regularization and variable selection via the elastic net

  2005cited by this paper
• IEEE Transactions on Pattern Analysis and Machine Intelligence

  2004cited by this paper
• A feature set measure based on Relief

  2004cited by this paper
• Overcoming the Myopia of Inductive Learning Algorithms with RELIEFF

  2004cited by this paper
• Efficient Feature Selection via Analysis of Relevance and Redundancy

  2004cited by this paper
• Instance-based learning algorithms

  2004cited by this paper
• Iterative Relief

  2003influential reference
• Experiments with Cost-Sensitive Feature Evaluation

  2003cited by this paper
• Theoretical and Empirical Analysis of ReliefF and RReliefF

  2003influential reference
• An Introduction to Variable and Feature Selection

  2003influential reference
• Multivariate Non-Linear Feature Selection with Kernel Multiplicative Updates and Gram-Schmidt Relief

  2003cited by this paper
• Evaluating Feature Relevance: Reducing Bias in Relief

  2002influential reference
• Reviewing RELIEF and its extensions: a new approach for estimating attributes considering high-correlated features

  2002influential reference
• Feature selection from huge feature sets

  2001cited by this paper
• Comprehensible Interpretation of Relief's Estimates

  2001cited by this paper
• Multifactor-dimensionality reduction reveals high-order interactions among estrogen-metabolism genes in sporadic breast cancer.

  2001cited by this paper
• Feature Subset Selection by Bayesian network-based optimization

  2000cited by this paper
• Feature selection from huge feature sets in the context of computer vision

  2000cited by this paper
• Feature Selection via Concave Minimization and Support Vector Machines

  1998cited by this paper
• Selection of Relevant Features and Examples in Machine Learning

  1997cited by this paper
• An adaptation of Relief for attribute estimation in regression

  1997influential reference

• Synergistic machine learning and DFT driven high-throughput screening strategy for A2BX6-Type inorganic perovskite photovoltaic materials

  2026cites this paper
• CLCNet: a contrastive learning and chromosome-aware network for genomic prediction in plants

  2026cites this paper
• Improved artemisinin optimization for high-dimensional medical feature selection

  2026cites this paper
• Utilizing interpretable machine learning algorithms and multiple features from multi-temporal Sentinel-2 imagery for predicting wheat fusarium head blight

  2026cites this paper
• A Novel Hybrid Adaptive Multi-Resolution Feature Extraction Method for Power Quality Disturbance Detection

  2026cites this paper
• A three-stage feature selection approach for human activity recognition

  2026cites this paper
• Behavioral Cloning of Physiotherapists in Adapting Robot Control Parameter

  2026cites this paper
• Pipeline-optimized machine learning for chronic fatigue syndrome diagnosis: A lightweight, interpretable model using blood biochemical and metabolomic data.

  2026cites this paper
• Exploration of prognostic prediction models for renal cell carcinoma using diffusion relaxation correlation spectroscopic imaging

  2026cites this paper
• Enhanced classification of unsound sorghum kernels based on decision-level fusion of hyperspectral and imaging textural features

  2026cites this paper
• Sensor-Derived Parameters from Standardized Walking Tasks Can Support the Identification of Patients with Parkinson's Disease at Risk of Gait Deterioration.

  2026cites this paper
• Leveraging Machine Learning to Predict Food Waste Quantity: Focusing on Military Dining Facilities as Large‐Scale Food Service Operations

  2026cites this paper
• SA-AVAE: A sex-Aware adversarial variational autoencoder network for biological brain age estimation in IoMT-Enabled multimodal neuroimages

  2026cites this paper
• Turning heterogeneity of statistical epistasis networks to an advantage

  2026cites this paper
• Dynamic Prediction of Reducing Sugar Content in Daqu Based on a Time Series–Microenvironment Coupled Stacking Model

  2026cites this paper
• 基于高光谱与 LiDAR 融合的尾矿库 Pb 元素浓度反演

  2026cites this paper
• Development of a Radiomics-Based Machine Learning Model Utilizing CT-Guided Biopsy Images: Virtual Biopsy of Thoracic Cancers

  2026cites this paper
• Data-driven modelling of the fume emission rate for gas metal arc welding

  2026cites this paper
• Machine Learning-Driven Prediction of Chloride Resistance and Service Life Estimation in Blended Cement Concrete

  2026cites this paper
• Multidisciplinary prediction of running-related injuries using machine learning.

  2026cites this paper
• Predicting grain growth kinetic in steels using machine learning and XAI for mechanical properties

  2026cites this paper
• EEG-based Alzheimer’s disease detection: Past, present, and future

  2026cites this paper
• Automatic diagnosis of age-related macular degeneration using machine learning and image processing techniques

  2026cites this paper
• Cross-Subject EEG Mental State Recognition via Correlation-Based Feature Selection

  2026cites this paper
• Multimodal fusion of stable isotope and infrared spectral data based on a GMM–ReliefF–BP framework: a novel approach for intelligent source attribution of plastic evidence in forensic science

  2026cites this paper
• A cloud-enabled predictive analytics model for assessing health risks under climate variation

  2026cites this paper
• Dynamic feature selection with distributional Markov chain error correction for financial time series forecasting

  2026cites this paper
• Fault diagnosis for railway point machines based on improved multi-scale derivative wavelet packet energy entropy and two-stage feature selection

  2026cites this paper
• Enhancing ReliefF for Multi-Label Text Classification via High-Order Label Correlation

  2026cites this paper
• Synthetic Data Guided Feature Selection for Robust Activity Recognition in Older Adults

  2026cites this paper
• Optimization of Dye and Plasticizer Concentrations in Halochromic Sensor Films for Rapid pH Response Using Bird-Inspired Metaheuristic Algorithms

  2025cites this paper
• A Single-Channel EEG Approach for Sleep Stage-Independent Automatic Detection of REM Sleep Behavior Disorder

  2025cites this paper
• Integrating relieff-based feature selection and ensemble machine learning for robust biomarker identification in colorectal cancer

  2025cites this paper
• A deep learning model for gender recognition using voice data

  2025cites this paper
• A Least Absolute Shrinkage and Selection Operator Based Artificial Neural Network for Breast Cancer Identification

  2025cites this paper
• Multi-Level Feature Selection and Transfer Learning Framework for Scalable and Explainable Machine Learning Systems in Real-Time Applications

  2025cites this paper
• Frugal Machine Learning for Energy-efficient, and Resource-aware Artificial Intelligence

  2025cites this paper
• A cooperative intrusion detection system for internet of things using fuzzy logic and ensemble of convolutional neural networks

  2025cites this paper
• Enhancing machinery reliability in lunar bases: Optimized machine learning for bearing fault classification in DC power distribution networks

  2025cites this paper
• Cross study transcriptomic investigation of Alzheimer’s brain tissue discoveries and limitations

  2025cites this paper
• Enhancing DDoS Attack Classification through SDN and Machine Learning: A Feature Ranking Analysis

  2025cites this paper
• A feature selection method utilizing path accumulation cost, redundancy minimization, and interaction maximization for the diagnosis of coronary heart disease

  2025cites this paper
• RBON: A Robust Bi-objective Feature Selection Framework Using Hybrid Ensemble Technique and NSGA-II

  2025cites this paper
• Optimized feature representation and selection for malware detection using portable executable headers and machine learning

  2025influential citation
• Leveraging the Machine Learning Techniques for Diagnosing Liver Diseases

  2025influential citation
• Computer-based quantitative image texture analysis using multi-collinearity diagnosis in chest X-ray images

  2025cites this paper
• Classification of Parotid Tumors with Robust Radiomic Features from DCE- and DW-MRI

  2025cites this paper
• Comprehensive Review of Hybrid Feature Selection Methods for Microarray-Based Cancer Detection

  2025cites this paper
• Data-Driven Customer Retention Strategies in E-Commerce: A Fuzzy Z-Number Approach

  2025cites this paper
• Automated Lightweight Model for Asthma Detection Using Respiratory and Cough Sound Signals

  2025cites this paper
• Feature Selection to Address High Dimensionality in Industry 4.0 Multi-Emitter Laser Modules Assembly Lines

  2025cites this paper
• Feature Selection Method with Feature Subcategorization in Regression

  2025cites this paper
• Landslide risk assessment on the China–Pakistan Economic Corridor (CPEC): a comparative study of quantitative and machine learning approaches

  2025cites this paper
• The Role of Advanced Machine Learning in COVID-19 Medical Imaging: A Technical Review

  2025cites this paper
• Multihead Takagi–Sugeno–Kang Fuzzy System

  2025cites this paper
• MF$^{2}$-Net: Exploring a Meta-Fuzzy Multimodal Fusion Network for Depression Recognition

  2025cites this paper
• Accuracy Study Based on Different Preprocessing Machine Learning Approaches

  2025cites this paper
• Feature Selection for Stock Movement Direction Prediction Using Sparse Support Vector Machine

  2025cites this paper
• A Sample Size–Driven Approach to Heart Disease Risk Prediction

  2025cites this paper
• Practical semi-supervised learning framework for real-time warning of aerodynamic instabilities: Applications from compressors to gas turbine engines

  2025cites this paper
• An experimental study on the delamination fracture of a unidirectional thermoplastic composite under different in-plane loading modes using acoustic emission

  2025cites this paper
• Efficient hybrid approach for feature extraction using metaheuristic algorithm and majority voting in social media networks

  2025cites this paper
• Spatial-Spectral feature extraction of hyperspectral images for tongue diagnosis digitalization and classification

  2025cites this paper
• Identification of fungal diseases in strawberry by analysis of hyperspectral images using machine learning methods

  2025cites this paper
• A Comparative Analysis of Various Soft Computing Techniques for Indian Stock Market Prediction

  2025cites this paper
• Improve Black Ice Image Detection for Road Safety

  2025cites this paper
• Vibration Based Honeycomb Core Milling Diagnostics Using Machine Learning Approaches

  2025cites this paper
• Integrating Feature Selection and Deep Learning: A Hybrid Approach for Smart Agriculture Applications

  2025cites this paper
• Enhancing precision agriculture through cloud based transformative crop recommendation model

  2025cites this paper
• Insights into landslide susceptibility: a comparative evaluation of multi-criteria analysis and machine learning techniques

  2025influential citation
• Bearing Prognostics Using the PRONOSTIA Data: A Comparative Study

  2025cites this paper
• A new representation in genetic programming with hybrid feature ranking criterion for high-dimensional feature selection

  2025cites this paper
• A systematic literature review on sentiment analysis techniques, challenges, and future trends

  2025cites this paper
• GA-SFS: A Two-Stage Hybrid Algorithm for Feature Selection in Biomedical Data

  2025cites this paper
• Multi-label feature selection based on binary hashing learning and dynamic graph constraints

  2025cites this paper
• Separating Chickens’ Heads and Legs in Thermal Images via Object Detection and Machine Learning Models to Predict Avian Influenza and Newcastle Disease

  2025cites this paper
• Two-Stage Hybrid Feature Selection: Integrating ACO Algorithms With a Statistical Ensemble Technique for EV Demand Prediction

  2025cites this paper
• An integrated feature selection and machine learning framework for PM10 concentration prediction

  2025cites this paper
• Evolutionary LightGBM‐Based Intrusion Detection System for IoT Networks

  2025cites this paper
• Advancing Cardiac Disease Detection Using Feature Extraction, Feature Selection, and Ensemble Learning Approaches

  2025cites this paper
• An Improved Fault Diagnosis Method for Rolling Bearing Based on Relief-F and Optimized Random Forests Algorithm

  2025cites this paper
• Automatic Classification of Parkinson's Disease Through the Fusion of Sustained Vowel Descriptors

  2025cites this paper
• Application of Radiomics Analysis on Mammography for Differentiating Benign and Malignant Masses

  2025cites this paper
• Multifeature-Driven Multistep Wind Speed Forecasting Using NARXR and Modified VMD Approaches

  2025cites this paper
• Predicting Insomnia Response to Acupuncture With the Development of Innovative Machine Learning

  2025cites this paper
• Prediction of IPL Player Selection from T20 formats using a Novel Data Balancing Approach

  2025cites this paper
• Comparative Study of Hardware Footprints of Neural Network Activation Functions for EMG Based Diabetic Sensorimotor Polyneuropathy Severity Classifier

  2025cites this paper
• Automatic detection of Alzheimer's disease from EEG signals using hybrid PSO-GWO algorithm

  2025cites this paper
• A multi-feature selection fused with investor sentiment for stock price prediction

  2025cites this paper
• Rank based RELIEFG Method for Spam Mail Detection and Classification

  2025cites this paper
• Quantum-inspired feature extraction model from EEG frequency waves for enhanced schizophrenia detection

  2025cites this paper
• Detection of Hepatocellular Carcinoma Using Optimized miRNA Combinations and Interpretable Machine Learning Models

  2025cites this paper
• Using machine learning to investigate the influence of the prenatal chemical exposome on neurodevelopment of young children.

  2025cites this paper
• Optimal feature selection for supervised bridge damage classification using responses collected from a passing vehicle

  2025cites this paper
• Predicting Depression in Older Adults: A Novel Feature Selection and Neural Network Framework

  2025cites this paper
• Biomarker-driven drug repurposing for NAFLD-associated hepatocellular carcinoma using machine learning integrated ensemble feature selection

  2025influential citation
• Internet Usage Prediction in Cellular Networks by Ensemble of Deep Belief Networks (DBNs) and Particle Swarm Optimization (PSO)

  2025cites this paper
• Non-destructive prediction of the moisture content of individual wheat kernels combining hyperspectral imaging and WGAN data augmentation algorithm.

  2025cites this paper
• Identification of relevant features using SEQENS to improve supervised machine learning models predicting AML treatment outcome

  2025influential citation
• FusionNet: Dual Input Feature Fusion Network with Ensemble Based Filter Feature Selection for Enhanced Brain Tumor Classification.

  2025cites this paper