Batch-wise Convergent Pre-training: Step-by-Step Learning Inspired by Child Language Development

,

• Publication year

  2025

• Venue

  Proceedings of the First BabyLM Workshop

• Publication date

  Unknown publication date

• Fields of study

  Not labeled

• Identifiers
  DOI 10.18653/v1/2025.babylm-main.36
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora

  2025influential reference
• Towards Data-Efficient Language Models: A Child-Inspired Approach to Language Learning

  2025cited by this paper
• BabyLM Challenge: Exploring the Effect of Variation Sets on Language Model Training Efficiency

  2024cited by this paper
• Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora

  2024cited by this paper
• [Call for Papers] The 2nd BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus

  2024influential reference
• Elements of World Knowledge (EWOK): A cognition-inspired framework for evaluating basic world knowledge in language models

  2024cited by this paper
• Is Child-Directed Speech Effective Training Data for Language Models?

  2024cited by this paper
• KidLM: Advancing Language Models for Children – Early Insights and Future Directions

  2024cited by this paper
• GPT or BERT: why not both?

  2024cited by this paper
• Cloze probability, predictability ratings, and computational estimates for 205 English sentences, aligned with existing EEG and reading time data

  2023cited by this paper
• TinyStories: How Small Can Language Models Be and Still Speak Coherent English?

  2023cited by this paper
• Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting

  2020cited by this paper
• When Do You Need Billions of Words of Pretraining Data?

  2020cited by this paper
• Scaling Laws for Neural Language Models

  2020cited by this paper
• BLiMP: The Benchmark of Linguistic Minimal Pairs for English

  2019cited by this paper
• Predict and redirect: Prediction errors support children's word learning.

  2019cited by this paper
• Language Models are Unsupervised Multitask Learners

  2019cited by this paper
• LAMAL: LAnguage Modeling Is All You Need for Lifelong Language Learning

  2019cited by this paper
• SuperGlue: Learning Feature Matching With Graph Neural Networks

  2019cited by this paper
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

  2018cited by this paper
• SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing

  2018cited by this paper
• Continual Lifelong Learning with Neural Networks: A Review

  2018cited by this paper
• Continual Learning Through Synaptic Intelligence

  2017cited by this paper
• Mapping the Early Language Environment Using All-Day Recordings and Automated Analysis.

  2017cited by this paper
• Gradient Episodic Memory for Continual Learning

  2017cited by this paper
• Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learner

  2016cited by this paper
• Overcoming catastrophic forgetting in neural networks

  2016cited by this paper
• Language Development and Assessment in the Preschool Period

  2012cited by this paper
• Knowing a lot for one's age: Vocabulary skill and not age is associated with anticipatory incremental sentence interpretation in children and adults.

  2012cited by this paper
• Age-of-acquisition ratings for 30,000 English words

  2012cited by this paper
• Curriculum learning

  2009cited by this paper
• Catastrophic forgetting in connectionist networks.

  1999cited by this paper
• The CHILDES project: tools for analyzing talk

  1992cited by this paper
• ConcreteGPT: A Baby GPT-2 Based on Lexical Concreteness and Curriculum Learning

  year unknowncited by this paper
• ELC-ParserBERT: Low-Resource Language Modeling Utilizing a Parser Network With ELC-BERT

  year unknowncited by this paper

• No citing papers are available for this paper.