ICDAR 2019 CROHME + TFD: Competition on Recognition of Handwritten Mathematical Expressions and Typeset Formula Detection

We summarize the tasks, protocol, and outcome for the 6th Competition on Recognition of Handwritten Mathematical Expressions (CROHME), which includes a new formula detection in document images task (+ TFD). For CROHME + TFD 2019, participants chose between two tasks for recognizing handwritten formulas from 1) online stroke data, or 2) images generated from the handwritten strokes. To compare LATEX strings and the labeled directed trees over strokes (label graphs) used in previous CROHMEs, we convert LATEX and stroke-based label graphs to label graphs defined over symbols (symbol-level label graphs, or symLG). More than thirty (33) participants registered for the competition, with nineteen (19) teams submitting results. The strongest formula recognition results were produced by the USTC-iFLYTEK research team, for both stroke-based (81%) and image-based (77%) input. For the new typeset formula detection task, the Samsung R&D Institute Ukraine (Team 2) obtained a very strong F-score (93%). System performance has improved since the last CROHME - still, the competition results suggest that recognition of handwritten formulae remains a difficult structural pattern recognition task.

• Publication year

  2019

• Venue

  IEEE International Conference on Document Analysis and Recognition

• Publication date

  2019-09-01

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1109/ICDAR.2019.00247
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Acceleration of Online Recognition of 2D Sequences Using Deep Bidirectional LSTM and Dynamic Programming

  2019cited by this paper
• An Attention-Based End-to-End Model for Multiple Text Lines Recognition in Japanese Historical Documents

  2019cited by this paper
• Stroke Extraction for Offline Handwritten Mathematical Expression Recognition

  2019cited by this paper
• Pattern Generation Strategies for Improving Recognition of Handwritten Mathematical Expressions

  2019cited by this paper
• Track, Attend, and Parse (TAP): An End-to-End Framework for Online Handwritten Mathematical Expression Recognition

  2019cited by this paper
• YOLOv3: An Incremental Improvement

  2018cited by this paper
• Image-to-Markup Generation via Paired Adversarial Learning

  2018cited by this paper
• Watch, attend and parse: An end-to-end neural network based approach to handwritten mathematical expression recognition

  2017cited by this paper
• NTCIR-12 MathIR Task Overview

  2016cited by this paper
• Text/shape classifier for mobile applications with handwriting input

  2016cited by this paper
• A system for recognizing online handwritten mathematical expressions by using improved structural analysis

  2016cited by this paper
• Advancing the state of the art for handwritten math recognition: the CROHME competitions, 2011–2014

  2016influential reference
• Recognition and retrieval of mathematical expressions

  2011influential reference
• INFTY: an integrated OCR system for mathematical documents

  2003cited by this paper
• Overview of the 2003 KDD Cup

  2003cited by this paper

• Vision–Tactile Sensor Fusion System for Fabric Sorting and Robotic Grasping in Textile Recycling

  2026cites this paper
• Local and Global Graph Modeling with Edge-weighted Graph Attention Network for Handwritten Mathematical Expression Recognition

  2026cites this paper
• PositionOCR: Augmenting Positional Awareness in Multi-Modal Models via Hybrid Specialist Integration

  2026cites this paper
• Automatic Scoring System for Digitized Handwritten Answers

  2026cites this paper
• Symbol-Aware Reasoning with Masked Discrete Diffusion for Handwritten Mathematical Expression Recognition

  2026influential citation
• The Return of Structural Handwritten Mathematical Expression Recognition

  2025influential citation
• Complex Mathematical Expression Recognition: Benchmark, Large-Scale Dataset and Strong Baseline

  2025cites this paper
• Enhancing Handwritten Mathematical Expression Recognition with Structure and Counting Aware Network

  2025cites this paper
• Skeletonization-Based Adversarial Perturbations on Large Vision Language Model’s Mathematical Text Recognition

  2025cites this paper
• Towards Scalable Training for Handwritten Mathematical Expression Recognition

  2025cites this paper
• Mask & Match: Learning to Recognize Handwritten Math with Self-Supervised Attention

  2025cites this paper
• WHATSNet: a wavelet-guided hybrid attention token selector for handwritten mathematical expression recognition

  2025cites this paper
• MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigm

  2025cites this paper
• AutoScaler: Self scale alignment for handwritten mathematical expression recognition

  2025cites this paper
• AI-Powered LaTeX Generation from Handwritten Mathematical Equations

  2025cites this paper
• Uni-MuMER: Unified Multi-Task Fine-Tuning of Vision-Language Model for Handwritten Mathematical Expression Recognition

  2025cites this paper
• HiE-VL: A Large Vision-Language Model with Hierarchical Adapter for Handwritten Mathematical Expression Recognition

  2025cites this paper
• Innovative approaches in image processing: enhancing feature extraction and recognition capabilities

  2025cites this paper
• Fuzzy-based DCKN: Fuzzy-based deep convolutional kronecker network for semantic analysis of summarized video

  2025cites this paper
• MMHMER:Multi-viewer and Multi-task for Handwritten Mathematical Expression Recognition

  2025cites this paper
• Valley2: Exploring Multimodal Models with Scalable Vision-Language Design

  2025cites this paper
• MathAssist: A Handwritten Mathematical Expression Autocomplete Technique

  2024cites this paper
• GAP: A Grammar and Position-Aware Framework for Efficient Recognition of Multi-Line Mathematical Formulas

  2024cites this paper
• Viewing Writing as Video: Optical Flow based Multi-Modal Handwritten Mathematical Expression Recognition

  2024cites this paper
• MoCoMER: A Self Supervised Representation Learning for Handwritten Mathematical Expression Recognition✱

  2024cites this paper
• CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacy

  2024cites this paper
• Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction

  2024influential citation
• MEMix: Improving HMER with Diverse Formula Structure Augmentation

  2024cites this paper
• Generating Handwritten Mathematical Expressions From Symbol Graphs: An End-to-End Pipeline

  2024cites this paper
• Enhancing Complex Formula Recognition with Hierarchical Detail-Focused Network

  2024cites this paper
• Character Relationship Refinement Network for Handwritten Mathematical Expression Recognition

  2024cites this paper
• CDM: A Reliable Metric for Fair and Accurate Formula Recognition Evaluation

  2024cites this paper
• Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matching

  2024cites this paper
• Attention Guidance Mechanism for Handwritten Mathematical Expression Recognition

  2024influential citation
• Contrastive representation enhancement and learning for handwritten mathematical expression recognition

  2024cites this paper
• PosFormer: Recognizing Complex Handwritten Mathematical Expression with Position Forest Transformer

  2024influential citation
• Multimodal Dependence Attention and Large-Scale Data Based Offline Handwritten Formula Recognition

  2024cites this paper
• Improving Handwritten Mathematical Expression Recognition via Integrating Convolutional Neural Network With Transformer and Diffusion-Based Data Augmentation

  2024cites this paper
• ICAL: Implicit Character-Aided Learning for Enhanced Handwritten Mathematical Expression Recognition

  2024cites this paper
• A Real-Time License Plate Detection and Recognition Model in Unconstrained Scenarios

  2024cites this paper
• A survey on handwritten mathematical expression recognition: The rise of encoder-decoder and GNN models

  2024cites this paper
• Research on Automatic Grading of Handwritten Homework in Intelligent Education Based on Model Fusion Approach

  2024cites this paper
• UniMERNet: A Universal Network for Real-World Mathematical Expression Recognition

  2024influential citation
• MathNet: A Data-Centric Approach for Printed Mathematical Expression Recognition

  2024cites this paper
• Bidirectional Trained Tree-Structured Decoder for Handwritten Mathematical Expression Recognition

  2023cites this paper
• Spatial Attention and Syntax Rule Enhanced Tree Decoder for Offline Handwritten Mathematical Expression Recognition

  2023cites this paper
• Discriminative estimation of probabilistic context-free grammars for mathematical expression recognition and retrieval

  2023cites this paper
• Offline handwritten mathematical expression recognition based on YOLOv5s

  2023cites this paper
• General Category Network: Handwritten Mathematical Expression Recognition with Coarse-Grained Recognition Task

  2023cites this paper
• The IBEM dataset: A large printed scientific image dataset for indexing and searching mathematical expressions

  2023cites this paper
• Equation Detection in the Camera Captured Handwritten Document

  2023cites this paper
• DenseBAM-GI: Attention Augmented DeneseNet with momentum aided GRU for HMER

  2023cites this paper
• Line-of-Sight with Graph Attention Parser (LGAP) for Math Formulas

  2023cites this paper
• Recognizing Handwritten Mathematical Expressions of Vertical Addition and Subtraction

  2023cites this paper
• Nougat: Neural Optical Understanding for Academic Documents

  2023cites this paper
• Survey of Structural Analysis in Mathematical Expression Recognition

  2023cites this paper
• Read Ten Lines at One Glance: Line-Aware Semi-Autoregressive Transformer for Multi-Line Handwritten Mathematical Expression Recognition

  2023influential citation
• An optimized neural Network-based character recognition and relation finding for mathematical expression images

  2023cites this paper
• Mathematical formula detection in document images: A new dataset and a new approach

  2023cites this paper
• A tree-based model with branch parallel decoding for handwritten mathematical expression recognition

  2023cites this paper
• Online Q&A for Machine Learning Courses Based on Formula Recognition

  2023cites this paper
• A Review of Emerging Research Directions in Abstract Visual Reasoning

  2022cites this paper
• Syntax-Aware Network for Handwritten Mathematical Expression Recognition

  2022cites this paper
• Interdisciplinary Discovery of Nanomaterials Based on Convolutional Neural Networks

  2022cites this paper
• Structural String Decoder for Handwritten Mathematical Expression Recognition

  2022cites this paper
• Transformer-based method for mathematical expression recognition in document images

  2022cites this paper
• Handwritten Mathematical Equation Recognition and Solver

  2022cites this paper
• Accessible PDFs: Applying Artificial Intelligence for Automated Remediation of STEM PDFs

  2022cites this paper
• A Tree-Based Structure-Aware Transformer Decoder for Image-To-Markup Generation

  2022cites this paper
• Self-Supervised Pretraining of Graph Neural Network for the Retrieval of Related Mathematical Expressions in Scientific Articles

  2022cites this paper
• Tree-based data augmentation and mutual learning for offline handwritten mathematical expression recognition

  2022cites this paper
• CoMER: Modeling Coverage for Transformer-based Handwritten Mathematical Expression Recognition

  2022cites this paper
• TDv2: A Novel Tree-Structured Decoder for Offline Mathematical Expression Recognition

  2022cites this paper
• Multi-Scale Features Integration for Handwritten Mathematical Expression Recognition

  2022influential citation
• Dataset of solution-based inorganic materials synthesis procedures extracted from the scientific literature

  2022cites this paper
• Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformer

  2021influential citation
• Dataset of Solution-based Inorganic Materials Synthesis Recipes Extracted from the Scientific Literature

  2021cites this paper
• Handwritten Mathematical Expression Recognition with Self-Attention

  2021cites this paper
• Syntactic data generation for handwritten mathematical expression recognition

  2021cites this paper
• Scallop: From Probabilistic Deductive Databases to Scalable Differentiable Reasoning

  2021cites this paper
• An end‐to‐end framework for the detection of mathematical expressions in scientific document images

  2021cites this paper
• ScanSSD-XYc: Faster Detection for Math Formulas

  2021influential citation
• A Math Formula Extraction and Evaluation Framework for PDF Documents

  2021influential citation
• Segmentation and Graph Matching for Online Analysis of Student Arithmetic Operations

  2021influential citation
• MRD: A Memory Relation Decoder for Online Handwritten Mathematical Expression Recognition

  2021cites this paper
• Cascade Network with Deformable Composite Backbone for Formula Detection in Scanned Document Images

  2021cites this paper
• A Transformer-based Math Language Model for Handwritten Math Expression Recognition

  2021influential citation
• 1st Place Solution for ICDAR 2021 Competition on Mathematical Formula Detection

  2021cites this paper
• Global Context for improving recognition of Online Handwritten Mathematical Expressions

  2021cites this paper
• GSSF: A Generative Sequence Similarity Function based on a Seq2Seq model for clustering online handwritten mathematical answers

  2021influential citation
• Clustering of Handwritten Mathematical Expressions for Computer-Assisted Marking

  2021cites this paper
• Recursive Recognition of Offline Handwritten Mathematical Expressions

  2021cites this paper
• Generation of Hypergraphs from the N-Best Parsing of 2D-Probabilistic Context-Free Grammars for Mathematical Expression Recognition

  2021cites this paper
• Clustering online handwritten mathematical expressions

  2021cites this paper
• Online Handwritten Mathematical Expression Recognition and Applications: A Survey

  2021cites this paper
• Opportunities and challenges of text mining in aterials research

  2021cites this paper
• A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics

  2021cites this paper
• Generating Synthetic Online Handwritten Mathematical Expressions from Markup Languages

  2021cites this paper
• SRD: A Tree Structure Based Decoder for Online Handwritten Mathematical Expression Recognition

  2021cites this paper
• A New Approach to Data Annotation Automation for Online Handwritten Mathematical Expression Recognition based on Recurrent Neural Networks

  2021cites this paper