REPAIR: Removing Representation Bias by Dataset Resampling

Modern machine learning datasets can have biases for certain representations that are leveraged by algorithms to achieve high performance without learning to solve the underlying task. This problem is referred to as “representation bias”. The question of how to reduce the representation biases of a dataset is investigated and a new dataset REPresentAtion bIas Removal (REPAIR) procedure is proposed. This formulates bias minimization as an optimization problem, seeking a weight distribution that penalizes examples easy for a classifier built on a given feature representation. Bias reduction is then equated to maximizing the ratio between the classification loss on the reweighted dataset and the uncertainty of the ground-truth class labels. This is a minimax problem that REPAIR solves by alternatingly updating classifier parameters and dataset resampling weights, using stochastic gradient descent. An experimental set-up is also introduced to measure the bias of any dataset for a given representation, and the impact of this bias on the performance of recognition models. Experiments with synthetic and action recognition data show that dataset REPAIR can significantly reduce representation bias, and lead to improved generalization of models trained on REPAIRed datasets. The tools used for characterizing representation bias, and the proposed dataset REPAIR algorithm, are available at https://github.com/JerryYLi/Dataset-REPAIR/.

• Publication year

  2019

• Venue

  Computer Vision and Pattern Recognition

• Publication date

  2019-04-16

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1109/CVPR.2019.00980arXiv 1904.07911
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Women also Snowboard: Overcoming Bias in Captioning Models

  2018cited by this paper
• Fine-grained Video Classification and Captioning

  2018cited by this paper
• ON THE EFFECTIVENESS OF TASK GRANULARITY FOR TRANSFER LEARNING

  2018cited by this paper
• Recognition in Terra Incognita

  2018cited by this paper
• GENERATIVE ADVERSARIAL NETS

  2018cited by this paper
• What have We Learned from Deep Representations for Action Recognition?

  2018cited by this paper
• RESOUND: Towards Action Recognition Without Representation Bias

  2018influential reference
• What Makes a Video a Video: Analyzing Temporal Information in Video Understanding Models and Datasets

  2018cited by this paper
• ActionVLAD: Learning Spatio-Temporal Aggregation for Action Classification

  2017cited by this paper
• Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study

  2017cited by this paper
• Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints

  2017cited by this paper
• ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases

  2017cited by this paper
• Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset

  2017cited by this paper
• The Kinetics Human Action Video Dataset

  2017cited by this paper
• Temporal Relational Reasoning in Videos

  2017cited by this paper
• Equality of Opportunity in Supervised Learning

  2016cited by this paper
• Temporal Segment Networks: Towards Good Practices for Deep Action Recognition

  2016cited by this paper
• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings

  2016cited by this paper
• A Deeper Look at Dataset Bias

  2015cited by this paper
• Deep Residual Learning for Image Recognition

  2015cited by this paper
• Contextual Action Recognition with R*CNN

  2015cited by this paper
• Beyond short snippets: Deep networks for video classification

  2015cited by this paper
• Two-Stream Convolutional Networks for Action Recognition in Videos

  2014cited by this paper
• ImageNet Large Scale Visual Recognition Challenge

  2014cited by this paper
• Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks

  2014cited by this paper
• Certifying and Removing Disparate Impact

  2014cited by this paper
• Learning Spatiotemporal Features with 3D Convolutional Networks

  2014cited by this paper
• Unsupervised Visual Domain Adaptation Using Subspace Alignment

  2013cited by this paper
• Learning Fair Representations

  2013cited by this paper
• Towards Understanding Action Recognition

  2013cited by this paper
• Undoing the Damage of Dataset Bias

  2012cited by this paper
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild

  2012cited by this paper
• HMDB: A large video database for human motion recognition

  2011cited by this paper
• Action recognition by dense trajectories

  2011cited by this paper
• Unbiased look at dataset bias

  2011influential reference
• On Space-Time Interest Points

  2005cited by this paper
• Contextual Priming for Object Detection

  2003cited by this paper
• SMOTE: Synthetic Minority Over-sampling Technique

  2002cited by this paper
• Gradient-based learning applied to document recognition

  1998cited by this paper
• Author manuscript, published in "International Conference on Computer Vision (2013)" Action Recognition with Improved Trajectories

  year unknowncited by this paper

• Team, Then Trim: An Assembly-Line LLM Framework for High-Quality Tabular Data Generation

  2026cites this paper
• Mitigating Shortcut Learning via Feature Disentanglement in Medical Imaging: A Benchmark Study

  2026cites this paper
• Detecting Dataset Bias in Medical AI: A Generalized and Modality-Agnostic Auditing Framework

  2025cites this paper
• Preventing Shortcut Learning in Medical Image Analysis through Intermediate Layer Knowledge Distillation from Specialist Teachers

  2025cites this paper
• Automatic Pruning and Quality Assurance of Object Detection Datasets for Autonomous Driving

  2025cites this paper
• AI Image Generation Technology in Ophthalmology: Use, Misuse and Future Applications.

  2025cites this paper
• The PanAf-FGBG Dataset: Understanding the Impact of Backgrounds in Wildlife Behaviour Recognition

  2025cites this paper
• Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Video

  2025cites this paper
• Debiasing Global Workspace: A Cognitive Neural Framework for Learning Debiased and Interpretable Representations

  2025cites this paper
• MLHOps: Machine Learning Health Operations

  2025cites this paper
• A Critical Review of Predominant Bias in Neural Networks

  2025cites this paper
• Distillation-Guided Representation Learning for Unconstrained Video Human Authentication

  2025cites this paper
• Active Learning Methods for Efficient Data Utilization and Model Performance Enhancement

  2025cites this paper
• Rethinking Temporal Context in Video-QA: A Comprehensive Study of Single-Frame Static Bias

  2025cites this paper
• Ethical Considerations in Artificial Intelligence (AI) Applications in Smart Grid

  2025cites this paper
• Project-Probe-Aggregate: Efficient Fine-Tuning for Group Robustness

  2025cites this paper
• Misogynous Memes Recognition: Training vs Inference Bias Mitigation Strategies

  2025influential citation
• Diffusing DeBias: Synthetic Bias Amplification for Model Debiasing

  2025cites this paper
• Diffusing DeBias: a Recipe for Turning a Bug into a Feature

  2025cites this paper
• Prediction-Powered Causal Inferences

  2025cites this paper
• Separating Shared and Domain-Specific LoRAs for Multi-Domain Learning

  2025cites this paper
• Learning Better Representations for Crowded Pedestrians in Offboard LiDAR-Camera 3D Tracking-by-detection

  2025cites this paper
• Impact on bias mitigation algorithms to variations in inferred sensitive attribute uncertainty

  2025cites this paper
• Harnessing Diffusion-Generated Synthetic Images for Fair Image Classification

  2025cites this paper
• Leveraging Text Guidance for Enhancing Demographic Fairness in Gender Classification

  2025cites this paper
• Learning to Look: Cognitive Attention Alignment with Vision-Language Models

  2025cites this paper
• Adaptive Group Robust Ensemble Knowledge Distillation

  2024cites this paper
• The Pitfalls of Memorization: When Memorization Hurts Generalization

  2024cites this paper
• CosFairNet:A Parameter-Space based Approach for Bias Free Learning

  2024cites this paper
• Towards Improved Perception System’s Generalization Through Generative Artificial Intelligence

  2024cites this paper
• Effective Guidance for Model Attention with Simple Yes-no Annotations

  2024cites this paper
• Does SpatioTemporal information benefit Two video summarization benchmarks?

  2024cites this paper
• Common-Sense Bias Discovery and Mitigation for Classification Tasks

  2024cites this paper
• Fairness and Bias Mitigation in Computer Vision: A Survey

  2024cites this paper
• Mitigating Bias in Dataset Distillation

  2024cites this paper
• Revisiting the Dataset Bias Problem from a Statistical Perspective

  2024cites this paper
• Model Debiasing by Learnable Data Augmentation

  2024cites this paper
• OxonFair: A Flexible Toolkit for Algorithmic Fairness

  2024cites this paper
• Video Interaction Recognition using an Attention Augmented Relational Network and Skeleton Data

  2024cites this paper
• Understanding Video Transformers via Universal Concept Discovery

  2024cites this paper
• Marginal Debiased Network for Fair Visual Recognition

  2024cites this paper
• Spatio-temporal Filter Analysis Improves 3D-CNN For Action Classification

  2024cites this paper
• Fairness without Harm: An Influence-Guided Active Sampling Approach

  2024influential citation
• HydraGAN: A Cooperative Agent Model for Multi-Objective Data Generation

  2024cites this paper
• Enhancing Skeleton-Based Action Recognition in Real-World Scenarios Through Realistic Data Augmentation

  2024cites this paper
• Comprehensive study of driver behavior monitoring systems using computer vision and machine learning techniques

  2024cites this paper
• Discover and Mitigate Multiple Biased Subgroups in Image Classifiers

  2024cites this paper
• Artificial Intelligent Agent Architecture and Clinical Decision-Making in the Healthcare Sector

  2024cites this paper
• PreciseDebias: An Automatic Prompt Engineering Approach for Generative AI to Mitigate Image Demographic Biases

  2024cites this paper
• Feature Selection-driven Bias Deduction in Histopathology Images: Tackling Site-Specific Influences

  2024cites this paper
• Bias Assessment and Data Drift Detection in Medical Image Analysis: A Survey

  2024cites this paper
• Medical Image Debiasing by Learning Adaptive Agreement from a Biased Council

  2024cites this paper
• Data Collection-free Masked Video Modeling

  2024cites this paper
• Improving Robustness to Multiple Spurious Correlations by Multi-Objective Optimization

  2024cites this paper
• Learning Decomposable and Debiased Representations via Attribute-Centric Information Bottlenecks

  2024cites this paper
• DCRP: Class-Aware Feature Diffusion Constraint and Reliable Pseudo-Labeling for Imbalanced Semi-Supervised Learning

  2024cites this paper
• Personalized Federated Learning with Spurious Features: An Adversarial Approach

  2024cites this paper
• LG-CAV: Train Any Concept Activation Vector with Language Guidance

  2024cites this paper
• Language-guided Detection and Mitigation of Unknown Dataset Bias

  2024cites this paper
• A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective

  2024cites this paper
• Runtime Monitoring and Enforcement of Conditional Fairness in Generative AIs

  2024cites this paper
• Fair Classifiers Without Fair Training: An Influence-Guided Data Sampling Approach

  2024influential citation
• Ameliorate Spurious Correlations in Dataset Condensation

  2024cites this paper
• Configurable Fairness: Direct Optimization of Parity Metrics via Vision-Language Models

  2024cites this paper
• A causal perspective on dataset bias in machine learning for medical imaging

  2024cites this paper
• Task-Free Fairness-Aware Bias Mitigation for Black-Box Deployed Models

  2024cites this paper
• Leveraging CLIP for Inferring Sensitive Information and Improving Model Fairness

  2024cites this paper
• Mapping the Potential of Explainable Artificial Intelligence (XAI) for Fairness Along the AI Lifecycle

  2024cites this paper
• Mitigating Biases in Blackbox Feature Extractors for Image Classification Tasks

  2024cites this paper
• Mapping the Potential of Explainable AI for Fairness Along the AI Lifecycle

  2024cites this paper
• Push Quantization-Aware Training Toward Full Precision Performances via Consistency Regularization

  2024cites this paper
• Disentangling the intrinsic feature from the related feature in image classification using knowledge distillation and object replacement

  2024cites this paper
• Fairness in Visual Clustering: A Novel Transformer Clustering Approach

  2023cites this paper
• Default Prediction of Internet Finance Users Based on Imbalance-XGBoost

  2023cites this paper
• LINGO : Visually Debiasing Natural Language Instructions to Support Task Diversity

  2023cites this paper
• DataComp: In search of the next generation of multimodal datasets

  2023cites this paper
• Actor-Aware Self-Supervised Learning for Semi-Supervised Video Representation Learning

  2023cites this paper
• PASS: Peer-Agreement based Sample Selection for training with Noisy Labels

  2023cites this paper
• Feature Importance Disparities for Data Bias Investigations

  2023cites this paper
• Domain Generalization in Machine Learning Models for Wireless Communications: Concepts, State-of-the-Art, and Open Issues

  2023cites this paper
• Decomposed Cross-Modal Distillation for RGB-based Temporal Action Detection

  2023cites this paper
• UnbiasedNets: a dataset diversification framework for robustness bias alleviation in neural networks

  2023influential citation
• HaLP: Hallucinating Latent Positives for Skeleton-based Self-Supervised Learning of Actions

  2023cites this paper
• Intersectional Fairness: A Fractal Approach

  2023cites this paper
• ANetQA: A Large-scale Benchmark for Fine-grained Compositional Reasoning over Untrimmed Videos

  2023cites this paper
• Linking convolutional kernel size to generalization bias in face analysis CNNs

  2023cites this paper
• Pushing the Accuracy-Group Robustness Frontier with Introspective Self-play

  2023cites this paper
• Improving Long-Term Guided Wave Damage Detection With Measurement Resampling

  2023cites this paper
• Interpreting Disparate Privacy-Utility Tradeoff in Adversarial Learning via Attribute Correlation

  2023cites this paper
• I P REFER NOT TO S AY : A RE U SERS P ENALIZED FOR P ROTECTING P ERSONAL D ATA ?

  2023cites this paper
• Measuring Bias

  2023cites this paper
• Sifer: Overcoming simplicity bias in deep networks using a feature sieve

  2023cites this paper
• GELDA: A generative language annotation framework to reveal visual biases in datasets

  2023cites this paper
• De-Biasing Methods in Neural Networks: A Survey

  2023cites this paper
• Bias Mitigation in Misogynous Meme Recognition: A Preliminary Study

  2023cites this paper
• Self-Supervised Scene-Debiasing for Video Representation Learning via Background Patching

  2023cites this paper
• A N ADVERSARIAL FEATURE LEARNING STRATEGY FOR DEBIASING NEURAL NETWORKS

  2023cites this paper
• Beyond Distribution Shift: Spurious Features Through the Lens of Training Dynamics

  2023cites this paper
• GLAD: Global-Local View Alignment and Background Debiasing for Unsupervised Video Domain Adaptation with Large Domain Gap

  2023cites this paper
• AI-based association analysis for medical imaging using latent-space geometric confounder correction

  2023cites this paper