Scalability of Continuous Active Learning for Reliable High-Recall Text Classification

No abstract is available for this paper.

• Publication year

  2016

• Venue

  International Conference on Information and Knowledge Management

• Publication date

  2016-10-24

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1145/2983323.2983776
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• TREC 2016 Total Recall Track Overview

  2016cited by this paper
• Engineering Quality and Reliability in Technology-Assisted Review

  2016cited by this paper
• Autonomy and Reliability of Continuous Active Learning for Technology-Assisted Review

  2015influential reference
• Waterloo (Cormack) Participation in the TREC 2015 Total Recall Track

  2015cited by this paper
• LSHTC: A Benchmark for Large-Scale Text Classification

  2015cited by this paper
• Multi-Faceted Recall of Continuous Active Learning for Technology-Assisted Review

  2015cited by this paper
• Comments on “ The Implications of Rule 26 ( g ) on the Use of Technology-Assisted Review ”

  2014cited by this paper
• Evaluation of machine-learning protocols for technology-assisted review in electronic discovery

  2014cited by this paper
• Towards minimizing the annotation cost of certified text classification

  2013cited by this paper
• Sequential testing in classifier evaluation yields biased estimates of effectiveness

  2013cited by this paper
• Approximate Recall Confidence Intervals

  2012cited by this paper
• Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review

  2011cited by this paper
• Results of the Active Learning Challenge

  2011cited by this paper
• Active learning for biomedical citation screening

  2010cited by this paper
• Document categorization in legal electronic discovery: computer classification vs. manual review

  2010cited by this paper
• Cochrane Handbook for Systematic Reviews of Interventions

  2010cited by this paper
• Active Learning Strategies for Multi-Label Text Classification

  2009cited by this paper
• Effective multi-label active learning for text classification

  2009cited by this paper
• Greedy is not Enough: An Efficient Batch Mode Active Learning Algorithm

  2009cited by this paper
• Active Learning Literature Survey

  2009cited by this paper
• A stopping criterion for active learning

  2008cited by this paper
• Searching for Studies

  2008cited by this paper
• Learning using Large Datasets

  2007influential reference
• LEONARDO - The computational intelligence (CI) model selection wizard

  2007cited by this paper
• Large-scale text categorization by batch mode active learning

  2006cited by this paper
• The TREC 2005 robust track

  2006influential reference
• Large Scale Transductive SVMs

  2006cited by this paper
• Reducing workload in systematic review preparation using automated citation classification.

  2006cited by this paper
• Forming test collections with no system pooling

  2004cited by this paper
• RCV1: A New Benchmark Collection for Text Categorization Research

  2004influential reference
• Building a filtering test collection for TREC 2002

  2003cited by this paper
• Machine learning in automated text categorization

  2001cited by this paper
• The Philosophy of Information Retrieval Evaluation

  2001cited by this paper
• How reliable are the results of large-scale information retrieval experiments?

  1998cited by this paper
• Efficient construction of large test collections

  1998cited by this paper
• Variations in relevance judgments and the measurement of retrieval effectiveness

  1998cited by this paper
• A sequential algorithm for training text classifiers

  1994cited by this paper

• Don't Stop Me Now, `Cause I'm Having a Good Time Screening: Evaluation of Stopping Methods for Safe Use of Priority Screening in Systematic Reviews

  2026cites this paper
• Weakly Supervised Active Learning for Abstract Screening Leveraging LLM-Based Pseudo-Labeling

  2025cites this paper
• Teaching postsecondary students about the ethics of artificial intelligence: A scoping review protocol

  2025influential citation
• Principles and Guidelines for the Use of LLM Judges

  2025cites this paper
• A Generalised and Adaptable Reinforcement Learning Stopping Method

  2025cites this paper
• Simulation-based active learning for systematic reviews: A scoping review of literature

  2025cites this paper
• LLM-Assisted Relevance Assessments: When Should We Ask LLMs for Help?

  2024cites this paper
• RLStop: A Reinforcement Learning Stopping Method for TAR

  2024influential citation
• Empirical legal analysis simplified: reducing complexity through automatic identification and evaluation of legally relevant factors

  2024cites this paper
• Learning from Litigation: Graphs and LLMs for Retrieval and Reasoning in eDiscovery

  2024cites this paper
• Unbiased Validation of Technology-Assisted Review for eDiscovery

  2024cites this paper
• SciMine: An Efficient Systematic Prioritization Model Based on Richer Semantic Information

  2023influential citation
• Can Generative Pre-trained Transformers (GPT) Pass Assessments in Higher Education Programming Courses?

  2023cites this paper
• Reducing the user labeling effort in effective high recall tasks by fine-tuning active learning

  2023influential citation
• Stopping Methods for Technology-assisted Reviews Based on Point Processes

  2023influential citation
• Confidence Sequences for Evaluating One-Phase Technology-Assisted Review

  2023cites this paper
• SAL_τ: efficiently stopping TAR by improving priors estimates

  2023influential citation
• Technology-Assisted Review for Spreadsheets and Noisy Text

  2023cites this paper
• QuOTeS: Query-Oriented Technical Summarization

  2023cites this paper
• TARexp: A Python Framework for Technology-Assisted Review Experiments

  2022cites this paper
• From Limited Annotated Raw Material Data to Quality Production Data: A Case Study in the Milk Industry (Technical Report)

  2022cites this paper
• From Limited Annotated Raw Material Data to Quality Production Data: A Case Study in the Milk Industry (Technical Report)

  2022cites this paper
• Fuzzy Law: Towards Creating a Novel Explainable Technology-Assisted Review System for e-Discovery

  2022cites this paper
• Towards semantic-driven boolean query formalization for biomedical systematic literature reviews

  2022cites this paper
• Extreme Systematic Reviews: A Large Literature Screening Dataset to Support Environmental Policymaking

  2022cites this paper
• Evaluation of MTARSI2 Dataset for Aircraft Type Recognition in Remote Sensing Images

  2022cites this paper
• Extracting Salient Facts from Company Reviews with Scarce Labels

  2022cites this paper
• From Little Things Big Things Grow: A Collection with Seed Studies for Medical Systematic Review Literature Search

  2022cites this paper
• Automatic identification of cassava leaf diseases utilizing morphological hidden patterns and multi-feature textures with a distributed structure-based classification approach

  2022cites this paper
• Search Strategy Formulation for Systematic Reviews: issues, challenges and opportunities

  2021cites this paper
• Cross-Domain Generalization and Knowledge Transfer in Transformers Trained on Legal Data

  2021cites this paper
• On the cost-effectiveness of neural and non-neural approaches and representations for text classification: A comprehensive comparative study

  2021cites this paper
• Goldilocks: Just-Right Tuning of BERT for Technology-Assisted Review

  2021cites this paper
• Heuristic stopping rules for technology-assisted review

  2021influential citation
• On minimizing cost in legal document review workflows

  2021cites this paper
• Certifying One-Phase Technology-Assisted Reviews

  2021cites this paper
• Artificial Intelligence in Evidence-Based Medicine

  2021cites this paper
• Utilizing Active Machine Learning for Quality Assurance: A Case Study of Virtual Car Renderings in the Automotive Industry

  2021cites this paper
• From Limited Annotated Raw Material Data to Quality Production Data: A Case Study in the Milk Industry

  2021cites this paper
• APS: An Active PubMed Search System for Technology Assisted Reviews

  2020cites this paper
• Sentence Embeddings and High-Speed Similarity Search for Fast Computer Assisted Annotation of Legal Documents

  2020cites this paper
• Automatic Boolean Query Formulation for Systematic Review Literature Search

  2020cites this paper
• UvA-DARE (Digital Academic Repository) When to Stop Reviewing in Technology-Assisted Reviews Sampling from an Adaptive Distribution to Estimate Residual Relevant Documents

  2020influential citation
• Participation in TREC 2020 COVID Track Using Continuous Active Learning

  2020influential citation
• Effective collection construction for information retrieval evaluation and optimization

  2020cites this paper
• UvA-DARE (Digital Academic Repository) When to Stop Reviewing in Technology-Assisted Reviews Sampling from an Adaptive Distribution to Estimate Residual Relevant Documents

  2020influential citation
• When to Stop Reviewing in Technology-Assisted Reviews

  2020influential citation
• WeSAL: Applying Active Supervision to Find High-quality Labels at Industrial Scale

  2020cites this paper
• Active Learning with Query Generation for Cost-Effective Text Classification

  2020cites this paper
• Fair Active Learning

  2020cites this paper
• Towards Question-based High-recall Information Retrieval

  2020cites this paper
• Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

  2019cites this paper
• On the improvement of reinforcement active learning with the involvement of cross entropy to address one-shot learning problem

  2019cites this paper
• TERMINATOR: better automated UI test case prioritization

  2019cites this paper
• Searching for Test Case Prioritization Schemes: An Expert System-Assisted Literature Review.

  2019cites this paper
• Poetry: Identification, Entity Recognition, and Retrieval

  2019cites this paper
• Increasing the Efficiency of High-Recall Information Retrieval

  2019cites this paper
• A Reinforcement One-Shot Active Learning Approach for Aircraft Type Recognition

  2019cites this paper
• A Study on a Mixed Stopping Strategy for Total Recall Tasks

  2019influential citation
• A Content-Based Schema Matching Tool

  2019cites this paper
• Assessing expert system-assisted literature reviews with a case study

  2019cites this paper
• Searching for Better Test Case Prioritization Schemes: a Case Study of AI-assisted Systematic Literature Review

  2019cites this paper
• The Quest for Total Recall

  2018cites this paper
• Efficient Test Collection Construction via Active Learning

  2018cites this paper
• Evaluating sentence-level relevance feedback for high-recall information retrieval

  2018influential citation
• Interactive Machine Learning with Applications in Health Informatics

  2018cites this paper
• Beyond Pooling

  2018influential citation
• Cost-aware Vulnerability Prediction: the HARMLESS Approach

  2018cites this paper
• Hybridization of Active Learning and Data Programming for Labeling Large Industrial Datasets

  2018cites this paper
• Discovering Play Store Reviews Related to Specific Android App Issues

  2018influential citation
• Improving Vulnerability Inspection Efficiency Using Active Learning

  2018cites this paper
• Total recall, language processing, and software engineering

  2018influential citation
• Using R Markdown for Replicable Experiments in Evidence Based Medicine

  2018cites this paper
• A Study on Manual Query Reformulation for Systematic Medical Reviews

  2018cites this paper
• Refresh Strategies in Continuous Active Learning

  2018cites this paper
• Geometric Active Learning via Enclosing Ball Boundary

  2018cites this paper
• Application of dynamic logistic regression with unscented Kalman filter in predictive coding

  2017cites this paper
• Empirical evaluations of active learning strategies in legal document review

  2017influential citation
• Testing Reading Tactics for Automated Reading Assistance: Is it Useful to Apply Old Knowledge?

  2017cites this paper
• Navigating Imprecision in Relevance Assessments on the Road to Total Recall: Roger and Me

  2017cites this paper
• FAST2: An intelligent assistant for finding relevant papers

  2017cites this paper
• Active Search for High Recall: A Non-stationary Extension of Thompson Sampling

  2017cites this paper
• On Design and Evaluation of High-Recall Retrieval Systems for Electronic Discovery

  2017influential citation
• Technology-Assisted Review in Empirical Medicine: Waterloo Participation in CLEF eHealth 2017

  2017cites this paper
• Data Balancing for Technologically Assisted Reviews: Undersampling or Reweighting

  2017cites this paper
• An Empirical Study of Active Learning for Text Classification

  2017cites this paper
• UvA-DARE (Digital Academic Repository) When to Stop Reviewing in Technology-Assisted Reviews Sampling from an Adaptive Distribution to Estimate Residual Relevant Documents

  year unknowninfluential citation