ReAttn: Improving Attention-based Re-ranking via Attention Re-weighting

The strong capabilities of recent Large Language Models (LLMs) have made them highly effective for zero-shot re-ranking task. Attention-based re-ranking methods, which derive relevance scores directly from attention weights, offer an efficient and interpretable alternative to generation-based re-ranking methods. However, they still face two major limitations. First, attention signals are highly concentrated a small subset of tokens within a few documents, making others indistinguishable. Second, attention often overemphasizes phrases lexically similar to the query, yielding biased rankings that irrelevant documents with mere lexical resemblance are regarded as relevant. In this paper, we propose \textbf{ReAttn}, a post-hoc re-weighting strategy for attention-based re-ranking methods. It first compute the cross-document IDF weighting to down-weight attention on query-overlapping tokens that frequently appear across the candidate documents, reducing lexical bias and emphasizing distinctive terms. It then employs entropy-based regularization to mitigate over-concentrated attention, encouraging a more balanced distribution across informative tokens. Both adjustments operate directly on existing attention weights without additional training or supervision. Extensive experiments demonstrate the effectiveness of our method.

• Publication year

  2026

• Venue

  Unknown venue

• Publication date

  2026-02-23

• Fields of study

  Computer Science

• Identifiers
  arXiv 2602.19969
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAG

  2026cited by this paper
• Which Attention Heads Matter for In-Context Learning?

  2025cited by this paper
• UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations

  2025cited by this paper
• Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-ranking

  2025cited by this paper
• Focus Directions Make Your Language Models Pay More Attention to Relevant Contexts

  2025cited by this paper
• Entropy-based Exploration Conduction for Multi-step Reasoning

  2025cited by this paper
• RATT: A Thought Structure for Coherent and Correct LLM Reasoning

  2024cited by this paper
• LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory

  2024cited by this paper
• Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankers

  2024influential reference
• IRCAN: Mitigating Knowledge Conflicts in LLM Generation via Identifying and Reweighting Context-Aware Neurons

  2024cited by this paper
• Retrieval Head Mechanistically Explains Long-Context Factuality

  2024cited by this paper
• Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models

  2024cited by this paper
• Identifying Semantic Induction Heads to Understand In-Context Learning

  2024cited by this paper
• Data Engineering for Scaling Language Models to 128K Context

  2024cited by this paper
• LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

  2024cited by this paper
• Attention Sorting Combats Recency Bias In Long Context Language Models

  2023cited by this paper
• Extending Context Window of Large Language Models via Positional Interpolation

  2023cited by this paper
• ConvGQR: Generative Query Reformulation for Conversational Search

  2023cited by this paper
• In-context Learning and Induction Heads

  2022cited by this paper
• Improving Passage Retrieval with Zero-Shot Question Generation

  2022cited by this paper
• BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models

  2021cited by this paper
• Calibrate Before Use: Improving Few-Shot Performance of Language Models

  2021cited by this paper
• Document Ranking with a Pretrained Sequence-to-Sequence Model

  2020cited by this paper
• Distilling Knowledge from Reader to Retriever for Question Answering

  2020cited by this paper
• CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims

  2020cited by this paper
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

  2020cited by this paper
• Fact or Fiction: Verifying Scientific Claims

  2020cited by this paper
• SPECTER: Document-level Representation Learning using Citation-informed Transformers

  2020cited by this paper
• Dense Passage Retrieval for Open-Domain Question Answering

  2020cited by this paper
• Natural Questions: A Benchmark for Question Answering Research

  2019cited by this paper
• Multi-Stage Document Ranking with BERT

  2019cited by this paper
• Are Sixteen Heads Really Better than One?

  2019cited by this paper
• Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned

  2019cited by this paper
• Attention is not not Explanation

  2019cited by this paper
• Is Attention Interpretable?

  2019cited by this paper
• FEVER: a Large-scale Dataset for Fact Extraction and VERification

  2018influential reference
• WWW'18 Open Challenge: Financial Opinion Mining and Question Answering

  2018influential reference
• DBpedia-Entity v2: A Test Collection for Entity Search

  2017influential reference
• A Full-Text Learning to Rank Dataset for Medical Information Retrieval

  2016cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014cited by this paper
• The Probabilistic Relevance Framework: BM25 and Beyond

  2009cited by this paper

• No citing papers are available for this paper.