SCIENCE@home

Legal Docket Classification: Where Machine Learning Stumbles

Ramesh Nallapati,Christopher D. Manning

Published 2008 in Conference on Empirical Methods in Natural Language Processing

ABSTRACT

We investigate the problem of binary text classification in the domain of legal docket entries. This work presents an illustrative instance of a domain-specific problem where the state-of-the-art Machine Learning (ML) classifiers such as SVMs are inadequate. Our investigation into the reasons for the failure of these classifiers revealed two types of prominent errors which we call conjunctive and disjunctive errors. We developed simple heuristics to address one of these error types and improve the performance of the SVMs. Based on the intuition gained from our experiments, we also developed a simple propositional logic based classifier using hand-labeled features, that addresses both types of errors simultaneously. We show that this new, but simple, approach outperforms all existing state-of-the-art ML models, with statistically significant gains. We hope this work serves as a motivating example of the need to build more expressive classifiers beyond the standard model classes, and to address text classification problems in such non-traditional domains.

PUBLICATION RECORD

  • Publication year

    2008

  • Venue

    Conference on Empirical Methods in Natural Language Processing

  • Publication date

    2008-10-25

  • Fields of study

    Law, Computer Science

  • Identifiers
    DOI 10.3115/1613715.1613771
  • External record

    Open on Semantic Scholar

  • Source metadata

    Semantic Scholar

CITATION MAP

EXTRACTION MAP

CLAIMS

  • No claims are published for this paper.

CONCEPTS

  • No concepts are published for this paper.

REFERENCES

Showing 1-16 of 16 references · Page 1 of 1

CITED BY

Showing 1-40 of 40 citing papers · Page 1 of 1