SCIENCE@home

Using unsupervised topic modeling to uncover document hierarchy and latent topics in prostate cancer clinical texts

T. Denti,W. Sun,S. Ji,H. Moen,O. Kerro,A. Rannikko,P. Marttinen,M. Koskinen

Published 2024 in medRxiv

ABSTRACT

Objective: Medical records reflect patients' health status and journey through healthcare services and medical specialties. As a result, document collections contain various text types, even about one patient, which makes medical texts unique. Our aim was to take the advantage of the implicit document hierarchy of medical records to evaluate contextualized neural topic modeling as a tool to gain insight into a mixed set of notes related to prostate cancer treatment, i.e., to uncover document types, their contents, and relations. Materials and Methods: We collected clinical text documents from 21,872 prostate cancer patients and organized the documents into a hierarchy using the document metadata. We trained neural topic models without the metadata to index the document collections, performed rigorous numerical evaluations of topic and clustering quality to optimize the topic count, visualized the latent representation of the models, and evaluated the topic clusters with respect to document metadata. Results: Topic clusters reflected the structure of the document hierarchy and provided information about the contents of different text types. The determination of the optimal number of topics required complementary information by topic and clustering quality metrics. Discussion: The topic modeling was found useful in visualizing and indexing large document collections, in providing understanding of document contents, and in revealing document organization comparable to metadata-based hierarchy. Conclusion: Hospital databases contain masses of text documents and topic modeling can provide means for analysts and researchers to group documents into discernable and explainable classes.

PUBLICATION RECORD

CITATION MAP

EXTRACTION MAP

CLAIMS

  • No claims are published for this paper.

CONCEPTS

  • No concepts are published for this paper.

REFERENCES

Showing 1-34 of 34 references · Page 1 of 1

CITED BY