Retrieval of Research-level Mathematical Information Needs: A Test Collection and Technical Terminology Experiment

In this paper, we present a test collection for mathematical information retrieval composed of real-life, researchlevel mathematical information needs. Topics and relevance judgements have been procured from the on-line collaboration website MathOverflow by delegating domain-specific decisions to experts on-line. With our test collection, we construct a baseline using Lucene’s vectorspace model implementation and conduct an experiment to investigate how prior extraction of technical terms from mathematical text can affect retrieval efficiency. We show that by boosting the importance of technical terms, statistically significant improvements in retrieval performance can be obtained over the baseline.

• Publication year

  2015

• Venue

  Annual Meeting of the Association for Computational Linguistics

• Publication date

  2015-07-01

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.3115/v1/P15-2055
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Collaborative problem solving: a study of MathOverflow

  2014cited by this paper
• NTCIR-10 Math Pilot Task Overview

  2013cited by this paper
• What does mathoverflow tell us about the production of mathematics?

  2013cited by this paper
• Participation in an online mathematics community: differentiating motivations to add

  2012cited by this paper
• Predicting the perceived quality of online mathematics contributions from users' reputations

  2011cited by this paper
• Information Retrieval Evaluation

  2011cited by this paper
• Using graded-relevance metrics for evaluating community QA answer selection

  2011cited by this paper
• Web Interface and Collection for Mathematical Retrieval :WebMIaS and MREC

  2011influential reference
• Information-Retrieval: Evaluation

  2010cited by this paper
• Transforming Large Collections of Scientific Publications to XML

  2010cited by this paper
• Overview of the NTCIR-8 Community QA Pilot Task (Part II): System Evaluation

  2010cited by this paper
• Overview of the NTCIR-8 Community QA Pilot Task (Part I): The Test Collection and the Task

  2010cited by this paper
• Synthesis Lectures on Information Concepts, Retrieval, and Services

  2009cited by this paper
• A Methodology for Building a Patent Test Collection for Prior Art Search

  2008cited by this paper
• Questioning Yahoo! Answers

  2007cited by this paper
• TREC: Experiment and Evaluation in Information Retrieval

  2007cited by this paper
• Creating a Test Collection for Citation-based IR Experiments

  2006cited by this paper
• Test Collections for Patent Retrieval and Patent Classification in the Fifth NTCIR Workshop

  2006cited by this paper
• Overview of Patent Retrieval Task at NTCIR-3

  2003cited by this paper
• Overview of the TREC 2003 Web Track

  2003cited by this paper
• Overview of TREC 2001

  2001cited by this paper
• Ranking retrieval systems without relevance judgments

  2001cited by this paper
• Overview of the TREC 2001 Question Answering Track

  2001cited by this paper
• The C-value/NC-value Method of Automatic Recognition for Multi-Word Terms

  1998cited by this paper
• Variations in relevance judgments and the measurement of retrieval effectiveness

  1998cited by this paper
• Extracting terminological expressions

  1996cited by this paper
• Overview of the first TREC conference

  1993cited by this paper
• The significance of the Cranfield tests on index languages

  1991cited by this paper
• INFORMATION RETRIEVAL TEST COLLECTIONS

  1976cited by this paper
• Report on the need for and provision of an 'ideal' information retrieval test collection

  1975cited by this paper
• Information Retrieval Experiment

  1971cited by this paper
• Aslib Cranfield research project - Factors determining the performance of indexing systems; Volume 1, Design; Part 2, Appendices

  1966cited by this paper
• Factors determining the performance of indexing systems

  1966cited by this paper
• Aslib Cranfield research project: report on the testing and analysis of an investigation into the comparative efficiency of indexing systems

  1962influential reference
• Report on the first stage of an investigation into the comparative efficiency of indexing systems

  1960influential reference

• Exploiting Query Reformulation and Reciprocal Rank Fusion in Math-Aware Search Engines

  2025cites this paper
• Mathematical Information Retrieval: A Review

  2024cites this paper
• BERT Is Not The Count: Learning to Match Mathematical Statements with Proofs

  2023influential citation
• Introduction to Mathematical Language Processing: Informal Proofs, Word Problems, and Supporting Tasks

  2023cites this paper
• A Survey in Mathematical Language Processing

  2022cites this paper
• MathUSE: Mathematical information retrieval system using universal sentence encoder model

  2022cites this paper
• Overview of ARQMath-3 (2022): Third CLEF Lab on Answer Retrieval for Questions on Math (Working Notes Version)

  2022cites this paper
• Learning to Match Mathematical Statements with Proofs

  2021influential citation
• Mathematical Information Retrieval

  2020cites this paper
• Developing a Concept-Oriented Search Engine for Isabelle Based on Natural Language: Technical Challenges

  2020cites this paper
• Variable Typing: Assigning Meaning to Variables in Mathematical Text

  2018influential citation
• Mathematical Document Categorization with Structure of Mathematical Expressions

  2017influential citation
• Utilizing dependency relationships between math expressions in math IR

  2017cites this paper
• Mathematical Information Retrieval based on Type Embeddings and Query Expansion

  2016influential citation