AGORA: Incentivizing Group Emergence Capability in LLMs via Group Distillation

Progress in complex reasoning is constrained by the static nature of the current training datasets. We propose structured interaction as a new scaling axis, moving beyond the prevailing paradigm of increasing model parameters. Our self-evolving framework, AGORA, enables a collaborative ensemble to achieve reasoning performance exceeding state-of-the-art monolithic systems by up to 4.45 percentage points on challenging mathematical benchmarks. This gain stems from group emergent ability-the synthesis of collective capabilities unattainable by isolated models, validating interaction as a scalable driver of intelligence. Our results position the engineering of collaborative ecosystems as a vital frontier for capability emergence.

• Publication year

  2025

• Venue

  arXiv.org

• Publication date

  2025-07-25

• Fields of study

  Computer Science

• Identifiers
  DOI 10.48550/arXiv.2507.21166arXiv 2507.21166
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• LIMO: Less is More for Reasoning

  2025cited by this paper
• MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention

  2025cited by this paper
• MiniCPM4: Ultra-Efficient LLMs on End Devices

  2025cited by this paper
• Qwen3 Technical Report

  2025cited by this paper
• Accelerating Chain-of-Thought Reasoning: When Goal-Gradient Importance Meets Dynamic Skipping

  2025cited by this paper
• Ultra-FineWeb: Efficient Data Filtering and Verification for High-Quality LLM Training Data

  2025cited by this paper
• A Strategic Coordination Framework of Small LLMs Matches Large LLMs in Data Synthesis

  2025cited by this paper
• Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing

  2024cited by this paper
• The Llama 3 Herd of Models

  2024cited by this paper
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

  2024cited by this paper
• WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct

  2023cited by this paper
• GPQA: A Graduate-Level Google-Proof Q&A Benchmark

  2023cited by this paper
• MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning

  2023cited by this paper
• From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning

  2023cited by this paper
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model

  2023cited by this paper
• Training language models to follow instructions with human feedback

  2022cited by this paper
• Distilling Knowledge via Knowledge Review

  2021cited by this paper
• Training Verifiers to Solve Math Word Problems

  2021cited by this paper
• Self-Distillation: Towards Efficient and Compact Neural Networks

  2021cited by this paper
• Measuring Mathematical Problem Solving With the MATH Dataset

  2021cited by this paper
• Knowledge Distillation: A Survey

  2020cited by this paper
• Scaling Laws for Neural Language Models

  2020cited by this paper
• Language Models are Few-Shot Learners

  2020influential reference
• Large scale distributed neural network training through online distillation

  2018cited by this paper
• Proximal Policy Optimization Algorithms

  2017cited by this paper
• Distilling a Neural Network Into a Soft Decision Tree

  2017cited by this paper
• Deep Learning

  2016cited by this paper
• Distilling the Knowledge in a Neural Network

  2015cited by this paper
• Complexity: A Guided Tour

  2009cited by this paper
• A multi-scale, multi-model approach for analyzing the future dynamics of European land use

  2008cited by this paper
• TrueSkill™: A Bayesian Skill Rating System

  2007cited by this paper
• Multi agent systems

  2005cited by this paper
• Complex adaptive systems

  1992cited by this paper
• in the hidden-order

  year unknowncited by this paper

• No citing papers are available for this paper.