Defining and measuring completeness of electronic health records for secondary use

We demonstrate the importance of explicit definitions of electronic health record (EHR) data completeness and how different conceptualizations of completeness may impact findings from EHR-derived datasets. This study has important repercussions for researchers and clinicians engaged in the secondary use of EHR data. We describe four prototypical definitions of EHR completeness: documentation, breadth, density, and predictive completeness. Each definition dictates a different approach to the measurement of completeness. These measures were applied to representative data from NewYork-Presbyterian Hospital's clinical data warehouse. We found that according to any definition, the number of complete records in our clinical database is far lower than the nominal total. The proportion that meets criteria for completeness is heavily dependent on the definition of completeness used, and the different definitions generate different subsets of records. We conclude that the concept of completeness in EHR is contextual. We urge data consumers to be explicit in how they define a complete record and transparent about the limitations of their data.

• Publication year

  2013

• Venue

  Journal of Biomedical Informatics

• Publication date

  2013-10-01

• Fields of study

  Medicine, Computer Science

• Identifiers
  DOI 10.1016/j.jbi.2013.06.010PMID 23820016PMCID PMC3810243
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• A collaborative approach to developing an electronic health record phenotyping algorithm for drug-induced liver injury.

  2013cited by this paper
• Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research

  2013cited by this paper
• Improved Cardiovascular Risk Prediction Using Nonparametric Regression and Electronic Health Record Data

  2013cited by this paper
• Extracting and Normalizing Temporal Expressions in Clinical Data Requests from Researchers

  2013cited by this paper
• Poster: Drawing Area-Proportional Venn-3 Diagrams Using Ellipses

  2012cited by this paper
• Leveraging Derived Data Elements in Data Analytic Models for Understanding and Predicting Hospital Readmissions

  2012cited by this paper
• Use of diverse electronic medical record systems to identify genetic risk for type 2 diabetes within a genome-wide association study

  2012cited by this paper
• An Electronic Medical Record-Based Model to Predict 30-Day Risk of Readmission and Death Among HIV-Infected Inpatients

  2012cited by this paper
• EliXR-TIME: A Temporal Knowledge Representation for Clinical Research Eligibility Criteria

  2012cited by this paper
• A predictive model for progression of chronic kidney disease to kidney failure.

  2011cited by this paper
• Variants near FOXE1 are associated with hypothyroidism and other thyroid conditions: using electronic medical records for genome- and phenome-wide studies.

  2011cited by this paper
• A real-time screening alert improves patient recruitment efficiency.

  2011cited by this paper
• Quantifying the longitudinal value of healthcare record collections for pharmacoepidemiology.

  2011cited by this paper
• Combining PubMed knowledge and EHR data to develop a weighted bayesian network for pancreatic cancer prediction

  2011cited by this paper
• Prediction Modeling Using EHR Data: Challenges, Strategies, and a Comparison of Machine Learning Approaches

  2010cited by this paper
• Robust replication of genotype-phenotype associations across multiple diseases in an electronic medical record.

  2010cited by this paper
• Review: Electronic Health Records and the Reliability and Validity of Quality Measures: A Review of the Literature

  2010cited by this paper
• Secondary Use of EHR: Data Quality Issues and Informatics Opportunities

  2010cited by this paper
• Implementing Electronic Health Record-Based Quality Measures for Developmental Screening

  2009cited by this paper
• Using Electronic Health Records to Measure Physician Performance for Acute Conditions in Primary Care: Empirical Evaluation of the Community-Acquired Pneumonia Clinical Quality Measure Set

  2009cited by this paper
• Case Report: Electronic Screening Improves Efficiency in Clinical Trial Recruitment

  2009cited by this paper
• Assessment of COPD-related outcomes via a national electronic medical record database

  2008cited by this paper
• Predicting 6-Year Mortality Risk in Patients With Type 2 Diabetes

  2008cited by this paper
• Measuring quality of care for rheumatic diseases using an electronic medical record

  2008cited by this paper
• Toward a national framework for the secondary use of health data: an American Medical Informatics Association White Paper.

  2007cited by this paper
• Measuring Performance Directly Using the Veterans Health Administration Electronic Medical Record: A Comparison With External Peer Review

  2007cited by this paper
• Adding value to the electronic health record through secondary use of data for quality assurance, research, and surveillance.

  2007cited by this paper
• Paper Versus Computer: Feasibility of an Electronic Medical Record in General Pediatrics

  2006cited by this paper
• Advanced and secure architectural EHR approaches

  2006cited by this paper
• Agreement between GPRD smoking data: a survey of general practitioners and a population‐based survey

  2004cited by this paper
• Systematic review of scope and quality of electronic patient record data in primary care

  2003cited by this paper
• Model Formulation: Defining and Improving Data Quality in Medical Registries: A Literature Review, Case Study, and Generic Framework

  2002cited by this paper
• Missing data: our view of the state of the art.

  2002cited by this paper
• Measuring the quality of medical records: a method for comparing completeness and correctness of clinical encounter data

  2001cited by this paper
• Accuracy of data in computer-based patient records.

  1997cited by this paper
• Beyond Accuracy: What Data Quality Means to Data Consumers

  1996cited by this paper
• Outcomes research using the electronic patient record: Beth Israel Hospital's experience with anticoagulation.

  1995cited by this paper
• Data quality in computerized patient records

  1994cited by this paper
• Markers of data quality in computer audit: the Manchester Orthopaedic Database.

  1993cited by this paper
• Quality of data in the Manchester orthopaedic database.

  1992cited by this paper
• Juran's Quality Control Handbook

  1988cited by this paper
• INFERENCE AND MISSING DATA

  1975cited by this paper

• A Study on the Behavioural Intentions of Hong Kong Pet Owners using Chatbots for Veterinary Consultation

  2026cites this paper
• Evolution of Computerized Provider Order Entry Documentation at a Leading Tertiary Care Referral Center in Riyadh

  2026influential citation
• Robust Multimodal Deep Learning for Lymphoma Subtype Classification Using 18F-FDG PET Maximum Intensity Projection Images and Clinical Data: A Multi-Center Study

  2026cites this paper
• The digital milestone: exploring the relationship between smartphone use and developmental domains in children aged 1–5 years

  2026cites this paper
• Secondary use of health records for prediction, detection, and treatment planning in the clinical decision support system: a systematic review

  2025cites this paper
• Strategies for Creating Robust Patient Groups to Study Diverse Conditions with Electronic Health Records.

  2025cites this paper
• Quantification of Information Gained by Linking Claims Data to an Electronic Health Record Cohort of Patients With Metastatic Breast Cancer

  2025cites this paper
• Analisis Faktor Ketidaklengkapan Pada Resume Medis Di RSUD Siti Fatimah Provinsi Sumatera Selatan

  2025cites this paper
• Rapid chronic kidney disease progression in younger, First Nations patients in the Northern Territory

  2025cites this paper
• Estimating the Observability of an Outcome from an Electronic Health Records Dataset Using External Data.

  2025cites this paper
• Artificial intelligence algorithm for real-time diagnostic assist in orofacial pain.

  2025cites this paper
• Assessing the Impact of the Quality of Textual Data on Feature Representation and Machine Learning Models: Quantitative Study Using Large Language Models

  2025cites this paper
• Preoperative Colchicine and Conduction Disturbances After Transcatheter Aortic Valve Implantation: A US Retrospective Cohort Analysis

  2025cites this paper
• The future of electronic health record‐based research: Leveraging informatics in genetic counseling research

  2025cites this paper
• Ontology-driven identification of inconsistencies in clinical data: A case study in lung cancer phenotyping

  2025cites this paper
• Analisis Efektivitas Rekam Medis Elektronik Berdasarkan Mutu Rekam Medis di Rawat Inap Rumah Sakit Patria IKKT

  2025cites this paper
• Completeness of Data on Race and Ethnicity and Timeliness of Electronic Case Reports for COVID-19 at 4 Health Care Organizations in Florida, December 2020.

  2025cites this paper
• Innovación tecnológica en la atención sociosanitaria

  2025cites this paper
• Incompleteness of Electronic Health Records: An Impending Process Problem Within Healthcare

  2025influential citation
• From Klebsiella and Candida to artificial intelligence: a perspective from infectious diseases doctors and researchers

  2025cites this paper
• Assessing the Impact of the Quality of Textual Data on Feature Representation and Machine Learning Models

  2025cites this paper
• Prognostic models for predicting patient arrivals in emergency departments: an updated systematic review and research agenda

  2025cites this paper
• Modeling diagnostic code dropout of schizophrenia in electronic health records improves phenotypic data quality and cross-ancestry transferability of polygenic scores

  2025cites this paper
• Exploring the implementation of a data trust committee: a qualitative evaluation of processes and practices

  2025cites this paper
• The Role of Artificial Intelligence in Optimizing Electronic Health Records for Early Detection of Disease

  2025cites this paper
• The AI and Automation in Anaesthesia (TRIPLE‐A) Perioperative Database in Eastern Denmark 2017–: A Platform for Epidemiology, AI‐Driven Prediction, Quality Control and Automated Data Retrieval

  2025cites this paper
• An assessment of data quality and sociodemographic variation in health service utilisation of general practice, emergency department and admitted services in a New South Wales linked health data asset: a retrospective cohort study of Lumos

  2025cites this paper
• Evaluation of the impact of defining observable time in real-world data on outcome incidence

  2025cites this paper
• Newborn screening for primary congenital hypothyroidism: past, present and future

  2025cites this paper
• A Hybrid AI Framework to Address the Issue of Frequent Missing Values with Application in EHR Systems: the Case of Parkinson's Disease

  2024cites this paper
• Navigating electronic health record accuracy by examination of sex incongruent conditions

  2024cites this paper
• MEDS-Tab: Automated tabularization and baseline methods for MEDS datasets

  2024cites this paper
• Natural language processing of multi-hospital electronic health records for public health surveillance of suicidality

  2024cites this paper
• Estimating the Prevalence of Child Abuse and Neglect Among Adolescents in Primary Care Through Diagnoses Codes and Free-Text EHR Clinical Notes.

  2024cites this paper
• In silico and in vitro evaluation of the immunogenic potential of Leishmania donovani ascorbate peroxidase and its derived peptides.

  2024cites this paper
• The Significance of Information Quality for the Secondary Use of the Information in the National Health Care Quality Registers in Finland

  2024cites this paper
• Biases in Race and Ethnicity Introduced by Filtering Electronic Health Records for “Complete Data”: Observational Clinical Data Analysis

  2024cites this paper
• Personalized approach to treatment choice using a digital profile of patient with cardiovascular diseases: the features of clinical decision-making support service

  2024cites this paper
• Clinical prediction with localized modeling using similarity-based cohorts: A scoping review

  2024cites this paper
• Assessing racial bias in healthcare predictive models: Practical lessons from an empirical evaluation of 30-day hospital readmission models

  2024cites this paper
• Differences in changes of data completeness after the implementation of an electronic medical record in three surgical departments of a German hospital–a longitudinal comparative document analysis

  2024cites this paper
• Performance of machine learning versus the national early warning score for predicting patient deterioration risk: a single-site study of emergency admissions

  2024cites this paper
• Application of a Data Quality Framework to Ductal Carcinoma In Situ Using Electronic Health Record Data From the All of Us Research Program

  2024cites this paper
• Assessing Real-World Data From Electronic Health Records for Health Technology Assessment: The SUITABILITY Checklist: A Good Practices Report of an ISPOR Task Force.

  2024cites this paper
• Ensuring patient safety: a case report on vigilance and noncompliance with healthcare regulations

  2024cites this paper
• Converting OMOP CDM to phenopackets: A model alignment and patient data representation evaluation

  2024cites this paper
• Linking clinical trial participants to their U.S. real-world data through tokenization: A practical guide

  2024cites this paper
• Toward Fairness, Accountability, Transparency, and Ethics in AI for Social Media and Health Care: Scoping Review

  2024cites this paper
• Graph Artificial Intelligence in Medicine.

  2024cites this paper
• Towards the automatic calculation of the EQUAL Candida Score: Extraction of CVC-related information from EMRs of critically ill patients with candidemia in Intensive Care Units

  2024cites this paper
• The role and limitations of electronic medical records versus patient interviews for determining symptoms, underlying comorbidities, and medication usage for patients with COVID-19.

  2024cites this paper
• Validation of automated data abstraction for SCCM discovery VIRUS COVID-19 registry: practical EHR export pathways (VIRUS-PEEP)

  2023cites this paper
• Graph AI in Medicine

  2023cites this paper
• Outliers in diagnosis ratios: A clue toward possibly absent data.

  2023cites this paper
• Improving Real-World Mortality Data Quality in Oncology Research: Augmenting Electronic Medical Records With Obituary, Social Security Death Index, and Commercial Claims Data

  2023cites this paper
• Automating Electronic Health Record Data Quality Assessment

  2023cites this paper
• Diagnostic rate estimation from Medicare records: Dependence on claim numbers and latent clinical features

  2023cites this paper
• Extracting and structuring information from the electronic medical text: state of the art and trendy directions

  2023cites this paper
• Adjusting for the progressive digitization of health records: working examples on a multi-hospital clinical data warehouse

  2023cites this paper
• Data Quality– and Utility-Compliant Anonymization of Common Data Model–Harmonized Electronic Health Record Data: Protocol for a Scoping Review

  2023cites this paper
• A broadly applicable approach to enrich electronic-health-record cohorts by identifying patients with complete data: a multisite evaluation

  2023cites this paper
• Approach to a preparation of dataset combining digital mammographic images and patient clinical data from electronic medical records

  2023cites this paper
• An EHR data quality evaluation approach based on medical knowledge and text matching

  2023cites this paper
• Detecting automatically the layout of clinical documents to enhance the performances of downstream natural language processing

  2023cites this paper
• Assessment study of the importance of computer technology in nursing health care

  2023cites this paper
• Digital Health Data Quality Issues: Systematic Review

  2023cites this paper
• A New Strategy for Evaluating the Quality of Laboratory Results for Big Data Research: Using External Quality Assessment Survey Data (2010–2020)

  2023cites this paper
• Association between Estimated Cardiorespiratory Fitness and Abnormal Glucose Risk: A Cohort Study

  2023cites this paper
• Leveraging electronic health record data for endometriosis research

  2023influential citation
• Electronic health record data quality assessment and tools: a systematic review

  2023cites this paper
• Five-dimensional evaluation system and perceptron intelligent computing performance measurement methods based on medical heterogeneous equipment health data

  2023cites this paper
• Personalized hypertension treatment recommendations by a data-driven model

  2023cites this paper
• Evaluating Algorithmic Bias in 30-Day Hospital Readmission Models: Retrospective Analysis

  2023cites this paper
• Assessing the effect of electronic health information exchange on the completeness and validity of data for measuring viral load testing turnaround time in Nigeria

  2023cites this paper
• Towards FATE in AI for Social Media and Healthcare: A Systematic Review

  2023cites this paper
• Legal and technological aspects for the creation of a European Health Data Space

  2022cites this paper
• Assessment of Data Quality Variability across Two EHR Systems through a Case Study of Post-Surgical Complications.

  2022cites this paper
• An Interoperable Electronic Health Record System for Clinical Cardiology

  2022cites this paper
• The Implication of Latent Information Quality to the Reproducibility of Secondary Use of Electronic Health Records

  2022cites this paper
• Information extraction from clinical records: an example for breast cancer*

  2022cites this paper
• Establishing a National Cardiovascular Disease Surveillance System in the United States Using Electronic Health Record Data: Key Strengths and Limitations

  2022cites this paper
• Using Electronic Dental Records to Assess Osteoradionecrosis Risk in Irradiated Head and Neck Cancer

  2022cites this paper
• Comparing medical history data derived from electronic health records and survey answers in the All of Us Research Program

  2022cites this paper
• Evaluating the Risk of Paroxysmal Atrial Fibrillation in Noncardioembolic Ischemic Stroke Using Artificial Intelligence-Enabled ECG Algorithm

  2022cites this paper
• Development and usage of an anesthesia data warehouse: lessons learnt from a 10-year project

  2022cites this paper
• An architectural design study of electronic healthcare record systems with associated context parameters on MIMIC III

  2022cites this paper
• Adaptive Semi-Supervised Inference for Optimal Treatment Decisions with Electronic Medical Record Data

  2022cites this paper
• Discrimination, trust, and withholding information from providers: Implications for missing data and inequity

  2022cites this paper
• Harnessing the Data Universe to Understand and Reduce Clinical Deterioration in Children.

  2022cites this paper
• The Secondary Use of Electronic Health Records for Data Mining: Data Characteristics and Challenges

  2022cites this paper
• Predicting pharmacotherapeutic outcomes for type 2 diabetes: An evaluation of three approaches to leveraging electronic health record data from multiple sources

  2022cites this paper
• Glomerular Diseases in the Colombian Caribbean: Data from the Colombian Nephropathy Registry (NEFRORED®)

  2022cites this paper
• Using Electronic Health Records to understand the population of local children captured in a large health system in Durham County, NC, USA, and implications for population health research.

  2022cites this paper
• Tailoring Rule-Based Data Quality Assessment to the Patient-Centered Outcomes Research Network (PCORnet) Common Data Model (CDM)

  2022cites this paper
• An Explanatory Study of User Satisfaction: Evidence From Brunei Health Information and Management System (Bru-HIMS)

  2022cites this paper
• Quantifying Electronic Health Record Data Quality in Telehealth and Office-Based Diabetes Care

  2022cites this paper
• Evaluating the Impact of Health Care Data Completeness for Deep Generative Models

  2022cites this paper
• Information extraction from electronic medical documents: state of the art and future research directions

  2022cites this paper
• Clinical encounter heterogeneity and methods for resolving in networked EHR data: A study from N3C and RECOVER programs

  2022cites this paper
• Determining the requirements of a medical records electronic deficiency management system: a mixed-method study

  2022cites this paper