Online stratified sampling: evaluating classifiers at web-scale

No abstract is available for this paper.

• Publication year

  2010

• Venue

  International Conference on Information and Knowledge Management

• Publication date

  2010-10-26

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1145/1871437.1871677
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Threshold selection for web-page classification with highly skewed class distribution

  2009influential reference
• A simple and efficient sampling method for estimating AP and NDCG

  2008influential reference
• A stratified traffic sampling methodology for seeing the big picture

  2008influential reference
• Optimized stratified sampling for approximate query processing

  2007influential reference
• Million Query Track 2007 Overview

  2007influential reference
• Online supervised spam filter evaluation

  2007cited by this paper
• IMPROVING THE PRECISION OF ESTIMATES OF THE FREQUENCY OF RARE EVENTS

  2005influential reference
• Learning and evaluating classifiers under sample selection bias

  2004cited by this paper
• Stratification Strategies for Sampling-based Non-intrusive Measurements of One-way Delay

  2003cited by this paper
• Open Directory Project

  2003cited by this paper
• Using asymmetric distributions to improve text classifier probability estimates

  2003cited by this paper
• Reducing multiclass to binary by coupling probability estimates

  2001cited by this paper
• Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods

  1999cited by this paper
• Assessing Agreement on Classification Tasks: The Kappa Statistic

  1996cited by this paper
• Evaluating and optimizing autonomous text classification systems

  1995cited by this paper

• Label-Efficient Monitoring of Classification Models via Stratified Importance Sampling

  2026cites this paper
• Scaling Up Active Testing to Large Language Models

  2025cites this paper
• Efficient sampling of polycyclic aromatic compounds for free energy predictions through active learning

  2025cites this paper
• Neuron Coverage Maximization for Effective Test Set Construction with Respect to the Model

  2024cites this paper
• Operationalizing content moderation "accuracy" in the Digital Services Act

  2023cites this paper
• Actively Testing Your Model While It Learns: Realizing Label-Efficient Learning in Practice

  2023cites this paper
• Precision/Recall on Imbalanced Test Data

  2023cites this paper
• A Large Scale Study and Classification of VirusTotal Reports on Phishing and Malware URLs

  2022cites this paper
• SIRAJ: A Unified Framework for Aggregation of Malicious Entity Detectors

  2022cites this paper
• FILA: Online Auditing of Machine Learning Model Accuracy under Finite Labelling Budget

  2022influential citation
• Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation

  2022cites this paper
• Active Assessment of Prediction Services as Accuracy Surface Over Attribute Combinations

  2021cites this paper
• Low-Shot Validation: Active Importance Sampling for Estimating Classifier Performance on Rare Categories

  2021cites this paper
• Active Testing: Sample-Efficient Model Evaluation

  2021cites this paper
• Chemical disease relation extraction task using genetic algorithm with two novelvoting methods for classifier subset selection

  2020cites this paper
• A general framework for label-efficient online evaluation with asymptotic guarantees

  2020cites this paper
• Estimating Precisions for Multiple Binary Classifiers Under Limited Samples

  2020cites this paper
• Needle in a Haystack: Label-Efficient Evaluation under Extreme Class Imbalance

  2020influential citation
• Near Optimal Stratified Sampling

  2019influential citation
• Interactive Evaluation of Classifiers Under Limited Resources

  2018influential citation
• Fe b 20 18 Classifier Risk Estimation under Limited Labeling Resources

  2018cites this paper
• Algorithms for Active Classifier Selection: Maximizing Recall with Precision Constraints

  2017cites this paper
• In Search of an Entity Resolution OASIS: Optimal Asymptotic Sequential Importance Sampling

  2017influential citation
• Classifier Risk Estimation under Limited Labeling Resources

  2016cites this paper
• Optimizing Speech Recognition Evaluation Using Stratified Sampling

  2016cites this paper
• DRAFT On Machine Perception of Sound Ph

  2016cites this paper
• Recall estimation for rare topic retrieval from large corpuses

  2014cites this paper
• Random Sampling in an Age of Automation: Minimizing Expenditures through Balanced Collection and Annotation

  2014cites this paper
• A Lazy Man's Approach to Benchmarking: Semisupervised Classifier Evaluation and Recalibration

  2013cites this paper
• Active Evaluation of Classifiers on Large Datasets

  2012influential citation
• Active evaluation of predictive models

  2012cites this paper
• Semisupervised Classifier Evaluation and Recalibration

  2012cites this paper
• Generalized expectation criteria for lightly supervised learning

  2011influential citation
• Beat the Machine: Challenging Workers to Find the Unknown Unknowns

  2011cites this paper
• Toward interactive training and evaluation

  2011influential citation