Do user preferences and evaluation measures line up?

No abstract is available for this paper.

• Publication year

  2010

• Venue

  Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

• Publication date

  2010-07-19

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1145/1835449.1835542
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Overview of the TREC 2009 Web Track

  2009cited by this paper
• Expected reciprocal rank for graded relevance

  2009cited by this paper
• A dynamic bayesian network click model for web search ranking

  2009cited by this paper
• An Effectiveness Measure for Ambiguous and Underspecified Queries

  2009cited by this paper
• Diversifying search results

  2009cited by this paper
• Crowdsourcing for relevance evaluation

  2008cited by this paper
• The good and the bad system: does the test collection predict users' effectiveness?

  2008cited by this paper
• How does clickthrough data reflect retrieval quality?

  2008cited by this paper
• A new rank correlation coefficient for information retrieval

  2008influential reference
• Novelty and diversity in information retrieval evaluation

  2008cited by this paper
• Creating a test collection to evaluate diversity in image retrieval

  2008cited by this paper
• User adaptation: good results from poor systems

  2008influential reference
• Students' search process and outcome in Medline in writing an essay for a class on evidence-based medicine

  2008cited by this paper
• The relationship between IR effectiveness measures and user satisfaction

  2007cited by this paper
• On GMAP: and other transformations

  2006cited by this paper
• Evaluation by comparing result sets in context

  2006cited by this paper
• User performance versus precision measures for simple search tasks

  2006cited by this paper
• A General Inductive Approach for Analyzing Qualitative Evaluation Data

  2006cited by this paper
• When will information retrieval be "good enough"?

  2005cited by this paper
• Evaluating Retrieval Performance Using Clickthrough Data

  2003cited by this paper
• Beyond independent relevance: methods and evaluation metrics for subtopic retrieval

  2003cited by this paper
• A taxonomy of web search

  2002cited by this paper
• Factors associated with success in searching MEDLINE and applying evidence to answer clinical questions.

  2002cited by this paper
• Why batch and user evaluations do not give the same results

  2001cited by this paper
• Do batch and user evaluations give the same results?

  2000cited by this paper
• Factors associated with successful answering of clinical questions using an information retrieval system.

  2000cited by this paper
• User-Defined Relevance Criteria: An Exploratory Study

  1994cited by this paper
• Evaluation Measures for Interactive Information Retrieval

  1992cited by this paper
• Estimating the satisfaction of information users.

  1977cited by this paper

• CLUE: Using Large Language Models for Judging Document Usefulness in Web Search Evaluation

  2025cites this paper
• On the Reliability of Sampling Strategies in Offline Recommender Evaluation

  2025cites this paper
• Leveraging LLMs to Evaluate Usefulness of Document

  2025cites this paper
• Preaching to the ChoIR: Lessons IR Should Share with AI

  2025cites this paper
• The EDIT UX Framework: A User-Centered Approach to Effective Product Redesign

  2024cites this paper
• Individual Persistence Adaptation for User-Centric Evaluation of User Satisfaction in Recommender Systems

  2024cites this paper
• Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs

  2024cites this paper
• Unbiased Learning to Rank: On Recent Advances and Practical Applications

  2024cites this paper
• A Meta-Evaluation of C/W/L/A Metrics: System Ranking Similarity, System Ranking Consistency and Discriminative Power

  2023cites this paper
• Report on the Dagstuhl Seminar on Frontiers of Information Access Experimentation for Research and Education

  2023cites this paper
• Recent Advances in the Foundations and Applications of Unbiased Learning to Rank

  2023cites this paper
• Safe Deployment for Counterfactual Learning to Rank with Exposure-Based Risk Minimization

  2023cites this paper
• Report from Dagstuhl Seminar 23031: Frontiers of Information Access Experimentation for Research and Education

  2023cites this paper
• Recall, Robustness, and Lexicographic Evaluation

  2023cites this paper
• Recent Advancements in Unbiased Learning to Rank

  2023cites this paper
• Validating Synthetic Usage Data in Living Lab Environments

  2023cites this paper
• How Well do Offline Metrics Predict Online Performance of Product Ranking Models?

  2023cites this paper
• What the Actual...Examining User Behaviour in Information Retrieval

  2022cites this paper
• Search Interfaces for Biomedical Searching: How do Gaze, User Perception, Search Behaviour and Search Performance Relate?

  2022cites this paper
• Designing Formulae for Ranking Search Results: Mixed Methods Evaluation Study

  2022cites this paper
• AxIoU: An Axiomatically Justified Measure for Video Moment Retrieval

  2022cites this paper
• Offline Retrieval Evaluation Without Evaluation Metrics

  2022cites this paper
• A Flexible Framework for Offline Effectiveness Metrics

  2022cites this paper
• Reaching the End of Unbiasedness: Uncovering Implicit Limitations of Click-Based Learning to Rank

  2022cites this paper
• Batch Evaluation Metrics in Information Retrieval: Measures, Scales, and Meaning

  2022cites this paper
• Matching Search Result Diversity with User Diversity Acceptance in Web Search Sessions

  2022cites this paper
• DiffIR: Exploring Differences in Ranking Models' Behavior

  2021cites this paper
• Understanding Bias and Variance of Learning-to-Rank Algorithms: An Empirical Framework

  2021cites this paper
• A user-centred approach to information retrieval

  2021cites this paper
• Incorporating Query Reformulating Behavior into Web Search Evaluation

  2021cites this paper
• POSSCORE: A Simple Yet Effective Evaluation of Conversational Search with Part of Speech Labelling

  2021cites this paper
• Meta-evaluation of Conversational Search Evaluation Metrics

  2021influential citation
• Robust Generalization and Safe Query-Specializationin Counterfactual Learning to Rank

  2021influential citation
• Perceptions of Diversity in Electronic Music: the Impact of Listener, Artist, and Track Characteristics

  2021cites this paper
• User behavior modeling for Web search evaluation

  2020cites this paper
• Good Evaluation Measures based on Document Preferences

  2020cites this paper
• How searching under time pressure impacts clinical decision making

  2020cites this paper
• Offline Evaluation without Gain

  2020cites this paper
• Hierarchical Knowledge Graphs: A Novel Information Representation for Exploratory Search Tasks

  2020influential citation
• Models Versus Satisfaction: Towards a Better Understanding of Evaluation Metrics

  2020cites this paper
• Retrieval Evaluation Measures that Agree with Users’ SERP Preferences

  2020influential citation
• Learning from user interactions with rankings

  2020cites this paper
• Which Diversity Evaluation Measures Are "Good"?

  2019influential citation
• Towards Context-Aware Evaluation for Image Search

  2019cites this paper
• Revisiting Online Personal Search Metrics with the User in Mind

  2019cites this paper
• Grid-based Evaluation Metrics for Web Image Search

  2019cites this paper
• On Annotation Methodologies for Image Search Evaluation

  2019cites this paper
• Pseudo Descriptions for Meta-Data Retrieval

  2018cites this paper
• How Well do Offline and Online Evaluation Metrics Measure User Satisfaction in Web Image Search?

  2018cites this paper
• Using Semantic Context to Rank the Results of Keyword Search

  2018influential citation
• Answer-biased summarization

  2018cites this paper
• Online Evaluation of Rankers Using Multileaving

  2018cites this paper
• ANALYSING EFFECTIVE METHODOLOGIES USED FOR TEXT CLUSTERING USING WEIGHTED ALGORITHMS

  2018cites this paper
• Dealing with Ties in Rank Correlation

  2018cites this paper
• Embracing imperfection in learning analytics

  2018cites this paper
• Some Common Mistakes In IR Evaluation, And How They Can Be Avoided

  2018cites this paper
• Hybrid crawling for time-based personalized web search ranking

  2017cites this paper
• Evaluating Evaluation Measures with Worst-Case Confidence Interval Widths

  2017cites this paper
• Evaluating Mobile Search with Height-Biased Gain

  2017cites this paper
• The machine in multimedia analytics

  2017cites this paper
• Prediction and Ranking of Highly Popular Web Content

  2017cites this paper
• User Preference Prediction in Mobile Search

  2017cites this paper
• Going beyond relevance: role of effort in information retrieval

  2017influential citation
• Efficient Hybrid Usage-Based Ranking Model Kanchanmala

  2017cites this paper
• Online In-Situ Interleaved Evaluation of Real-Time Push Notification Systems

  2017cites this paper
• Scalability and Performance of Random Forest based Learning-to-Rank for Information Retrieval

  2017cites this paper
• The Treatment of Ties in AP Correlation

  2017cites this paper
• Ephemeral relevance and user activities in a search session

  2017cites this paper
• Adaptive Persistence for Search Effectiveness Measures

  2017cites this paper
• Exploring the Effectiveness of Convolutional Neural Networks for Answer Selection in End-to-End Question Answering

  2017cites this paper
• Going Beyond Relevance: Incorporating Effort in Information Retrieval

  2016cites this paper
• Two-layered Summaries for Mobile Search: Does the Evaluation Measure Reflect User Preferences?

  2016cites this paper
• Predicting Pre-click Quality for Native Advertisements

  2016cites this paper
• When does Relevance Mean Usefulness and User Satisfaction in Web Search?

  2016cites this paper
• Data-Driven Relevance Judgments for Ranking Evaluation

  2016cites this paper
• Simulating Ideal and Average Users

  2016cites this paper
• Search Success Evaluation with Translation Model

  2016cites this paper
• Implications of Computational Cognitive Models for Information Retrieval

  2016cites this paper
• Tweet‐biased summarization

  2016cites this paper
• IR Evaluation: Modeling User Behavior for Measuring Effectiveness

  2015cites this paper
• A Short Survey on Online and Offline Methods for Search Quality Evaluation

  2015cites this paper
• Incorporating Non-sequential Behavior into Click Models

  2015cites this paper
• Improving single document summarization in a multi-document environment

  2015cites this paper
• An overview of semantic search evaluation initiatives

  2015influential citation
• Analytic Quality: Evaluation of Performance and Insight in Multimedia Collection Analysis

  2015cites this paper
• User Perspectives on Query Difficulty

  2015cites this paper
• Incorporating Non-sequential interactions into Click Models

  2015cites this paper
• How Do Gain and Discount Functions Affect the Correlation between DCG and User Satisfaction?

  2015cites this paper
• IR Evaluation: Designing an End-to-End Offline Evaluation Pipeline

  2015cites this paper
• Development and evaluation of a geographic information retrieval system using fine grained toponyms

  2015cites this paper
• Are Data Sets Like Documents?: Evaluating Similarity-Based Ranked Search over Scientific Data

  2015cites this paper
• Search Result Diversification

  2015cites this paper
• Assessor Differences and User Preferences in Tweet Timeline Generation

  2015cites this paper
• An Efficient Hybrid Usage-based Ranking model for information retrieval systems & web search Engine

  2015cites this paper
• Overview of the NTCIR-11 IMine Task

  2014cites this paper
• Bridging Between Information Retrieval and Databases

  2014cites this paper
• Dissimilarity Based Query Selection for Efficient Preference Based IR Evaluation

  2014cites this paper
• Ranked Similarity Search of Scientific Datasets: An Information Retrieval Approach

  2014cites this paper
• Optimally selecting and combining assessment and assessor types for information retrieval evaluation

  2014cites this paper
• Revisiting the Evaluation of Diversified Search Evaluation Metrics with User Preferences

  2014cites this paper