Prognostic prediction of sepsis patient using transformer with skip connected token for tabular data

Sepsis is known as a common syndrome in intensive care units (ICU), and severe sepsis and septic shock are among the leading causes of death worldwide. The purpose of this study is to develop a deep learning model that supports clinicians in efficiently managing sepsis patients in the ICU by predicting mortality, ICU length of stay (>14 days), and hospital length of stay (>30 days). The proposed model was developed using 591 retrospective data with 16 tabular data related to a sequential organ failure assessment (SOFA) score. To analyze tabular data, we designed the modified architecture of the transformer that has achieved extraordinary success in the field of languages and computer vision tasks in recent years. The main idea of the proposed model is to use a skip-connected token, which combines both local (feature-wise token) and global (classification token) information as the output of a transformer encoder. The proposed model was compared with four machine learning models (ElasticNet, Extreme Gradient Boosting [XGBoost]), and Random Forest) and three deep learning models (Multi-Layer Perceptron [MLP], transformer, and Feature-Tokenizer transformer [FT-Transformer]) and achieved the best performance (mortality, area under the receiver operating characteristic (AUROC) 0.8047; ICU length of stay, AUROC 0.8314; hospital length of stay, AUROC 0.7342). We anticipate that the proposed model architecture will provide a promising approach to predict the various clinical endpoints using tabular data such as electronic health and medical records.

• Publication year

  2024

• Venue

  Artif. Intell. Medicine

• Publication date

  2024-03-01

• Fields of study

  Medicine, Computer Science

• Identifiers
  DOI 10.1016/j.artmed.2024.102804PMID 38462275
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• TransFG: A Transformer Architecture for Fine-grained Recognition

  2021cited by this paper
• Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet

  2021cited by this paper
• Machine Learning-Based Heart Disease Diagnosis: A Systematic Literature Review

  2021cited by this paper
• Trends in the incidence and outcome of sepsis using data from a Japanese nationwide medical claims database-the Japan Sepsis Alliance (JaSA) study group-

  2021cited by this paper
• Machine Learning–Based Short-Term Mortality Prediction Models for Patients With Cancer Using Electronic Health Record Data: Systematic Review and Critical Appraisal

  2021cited by this paper
• Feature Fusion Vision Transformer for Fine-Grained Visual Categorization

  2021cited by this paper
• Revisiting Deep Learning Models for Tabular Data

  2021cited by this paper
• CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification

  2021cited by this paper
• Sepsis trends: increasing incidence and decreasing mortality, or changing denominator?

  2020cited by this paper
• Machine learning-based in-hospital mortality prediction models for patients with acute coronary syndrome.

  2020cited by this paper
• A comparison of machine learning models versus clinical evaluation for mortality prediction in patients with sepsis

  2020cited by this paper
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

  2020cited by this paper
• Using machine learning methods to predict in-hospital mortality of sepsis patients in the ICU

  2020cited by this paper
• Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the Global Burden of Disease Study

  2020cited by this paper
• Comparison of the accuracy of three early warning scores with SOFA score for predicting mortality in adult sepsis and septic shock patients admitted to intensive care unit

  2019cited by this paper
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

  2019cited by this paper
• Variable Selection

  2019cited by this paper
• TabNet: Attentive Interpretable Tabular Learning

  2019cited by this paper
• The early change of SOFA score as a prognostic marker of 28-day sepsis mortality: analysis through a derivation and a validation cohort

  2019cited by this paper
• The global burden of sepsis: barriers and potential solutions

  2018cited by this paper
• Sepsis mortality score for the prediction of mortality in septic patients

  2018cited by this paper
• Utility of SOFA score, management and outcomes of sepsis in Southeast Asia: a multinational multicenter prospective observational study

  2018cited by this paper
• Adaptive Input Representations for Neural Language Modeling

  2018cited by this paper
• Attention is All you Need

  2017cited by this paper
• Reducing the global burden of sepsis

  2017cited by this paper
• Development of a novel score for the prediction of hospital mortality in patients with severe sepsis: the use of electronic healthcare records with LASSO regression

  2017cited by this paper
• Prediction of In-hospital Mortality in Emergency Department Patients With Sepsis: A Local Big Data-Driven, Machine Learning Approach.

  2016cited by this paper
• Predictive modelling of survival and length of stay in critically ill patients using sequential organ failure scores

  2015cited by this paper
• Comparison of PIRO, SOFA, and MEDS scores for predicting mortality in emergency department patients with severe sepsis and septic shock.

  2014cited by this paper
• Model for predicting short-term mortality of severe sepsis

  2009cited by this paper
• Acute renal failure in the ICU: risk factors and outcome evaluated by the SOFA score

  2000cited by this paper
• Use of the SOFA score to assess the incidence of organ dysfunction/failure in intensive care units: results of a multicenter, prospective study. Working group on "sepsis-related problems" of the European Society of Intensive Care Medicine.

  1998cited by this paper
• Support Vector Method for Function Approximation, Regression Estimation and Signal Processing

  1996cited by this paper
• Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients.

  1993cited by this paper
• APACHE II: A severity of disease classification system

  1985cited by this paper
• APACHE II: a severity of disease classification system.

  1985cited by this paper
• A simplified acute physiology score for ICU patients

  1984cited by this paper

• Transformer-based model for predicting length of stay in intensive care unit in sepsis patients

  2025cites this paper
• Training and Evaluation of Guideline-Based Medical Reasoning in LLMs

  2025cites this paper
• Text understanding and generation using transformer models for intelligent e-commerce recommendations

  2024cites this paper
• Airport Visibility Classification Based on Multimodal Fusion of Image-Tabular Data

  2024cites this paper
• EpiBrCan-Lite: A lightweight deep learning model for breast cancer subtype classification using epigenomic data

  2024cites this paper
• ICU-TGNN: A Hybrid Multitask Transformer and Graph Neural Network Model for Predicting Clinical Outcomes of Patients in the ICU

  2024cites this paper