Enhancing disease surveillance with novel data streams: challenges and opportunities

Novel data streams (NDS), such as web search data or social media updates, hold promise for enhancing the capabilities of public health surveillance. In this paper, we outline a conceptual framework for integrating NDS into current public health surveillance. Our approach focuses on two key questions: What are the opportunities for using NDS and what are the minimal tests of validity and utility that must be applied when using NDS? Identifying these opportunities will necessitate the involvement of public health authorities and an appreciation of the diversity of objectives and scales across agencies at different levels (local, state, national, international). We present the case that clearly articulating surveillance objectives and systematically evaluating NDS and comparing the performance of NDS to existing surveillance data and alternative NDS data is critical and has not sufficiently been addressed in many applications of NDS currently in the literature.

• Publication year

  2015

• Venue

  EPJ Data Science

• Publication date

  2015-10-16

• Fields of study

  Medicine, Computer Science, Environmental Science

• Identifiers
  DOI 10.1140/epjds/s13688-015-0054-0PMID 27990325PMCID 5156315
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Twitter Improves Seasonal Influenza Prediction

  2018cited by this paper
• Wikipedia Usage Estimates Prevalence of Influenza-Like Illness in the United States in Near Real-Time

  2014cited by this paper
• Global Disease Monitoring and Forecasting with Wikipedia

  2014cited by this paper
• Guess Who’s Not Coming to Dinner? Evaluating Online Restaurant Reservations for Disease Surveillance

  2014cited by this paper
• Using Clinicians’ Search Query Data to Monitor Influenza Epidemics

  2014cited by this paper
• Could behavioral medicine lead the web data revolution?

  2014cited by this paper
• What's the healthiest day?: Circaseptan (weekly) rhythms in healthy considerations.

  2014cited by this paper
• What can digital disease detection learn from (an external revision to) Google Flu Trends?

  2014cited by this paper
• Population health concerns during the United States' Great Recession.

  2014cited by this paper
• Do celebrity cancer diagnoses promote primary cancer prevention?

  2014cited by this paper
• The Parable of Google Flu: Traps in Big Data Analysis

  2014influential reference
• Using Online Reviews by Restaurant Patrons to Identify Unreported Cases of Foodborne Illness — New York City, 2012–2013

  2014cited by this paper
• Use of Hangeul Twitter to Track and Predict Human Influenza Infection

  2013cited by this paper
• Seasonality in seeking mental health information on Google.

  2013cited by this paper
• Digital detection for tobacco control: online reactions to the 2009 U.S. cigarette excise tax increase.

  2013cited by this paper
• Seasonal trends in restless legs symptomatology: evidence from Internet search query data.

  2013cited by this paper
• Internet-based surveillance systems for monitoring emerging infectious diseases

  2013cited by this paper
• Predicting National Suicide Numbers with Social Media Data

  2013cited by this paper
• Web‐based participatory surveillance of infectious diseases: the Influenzanet participatory surveillance experience

  2013cited by this paper
• BioSense 2.0

  2013cited by this paper
• Influenza Forecasting with Google Flu Trends

  2013cited by this paper
• Reassessing Google Flu Trends Data for Detection of Seasonal and Pandemic Influenza: A Comparative Epidemiological Study at Three Geographic Scales

  2013cited by this paper
• Separating Fact from Fear: Tracking Flu Infections on Twitter

  2013cited by this paper
• Gonorrhea incidence forecasting research based on Baidu search data

  2013cited by this paper
• Using Google Trends for Influenza Surveillance in South China

  2013cited by this paper
• Using search queries for malaria surveillance, Thailand

  2013cited by this paper
• National and Local Influenza Surveillance through Twitter: An Analysis of the 2012-2013 Influenza Epidemic

  2013cited by this paper
• Monitoring Influenza Epidemics in China with Search Query from Baidu

  2013cited by this paper
• The Complex Relationship of Realspace Events and Messages in Cyberspace: Case Study of Influenza and Pertussis Using Tweets

  2013cited by this paper
• Ubiquity symposium: Evolutionary computation and the processes of life: what the no free lunch theorems really mean: how to improve search algorithms

  2013cited by this paper
• Predicting Flu Incidence from Portuguese Tweets

  2013cited by this paper
• Real-Time Influenza Forecasts during the 2012–2013 Season

  2013cited by this paper
• Flu Near You: An Online Self-reported Influenza Surveillance System in the USA

  2013cited by this paper
• Internet Search Patterns of Human Immunodeficiency Virus and the Digital Divide in the Russian Federation: Infoveillance Study

  2013cited by this paper
• Correlation between National Influenza Surveillance Data and Google Trends in South Korea

  2013cited by this paper
• Digital Detection for tobacco Control: Online reactions to the United states’ 2009 Cigarette excise tax increase

  2013cited by this paper
• Using digital surveillance to examine the impact of public figure pancreatic cancer announcements on media and search query outcomes.

  2013cited by this paper
• Optimizing Provider Recruitment for Influenza Surveillance Networks

  2012cited by this paper
• Head Lice Surveillance on a Deregulated OTC-Sales Market: A Study Using Web Query Data

  2012cited by this paper
• Norovirus disease surveillance using Google Internet query share data.

  2012cited by this paper
• Social and news media enable estimation of epidemiological patterns early in the 2010 Haitian cholera outbreak.

  2012cited by this paper
• Machine learning - a probabilistic perspective

  2012cited by this paper
• Prediction of Infectious Disease Spread Using Twitter: A Case of Influenza

  2012cited by this paper
• Nowcasting Events from the Social Web with Statistical Learning

  2012cited by this paper
• Novel surveillance of psychological distress during the great recession.

  2012cited by this paper
• Enhancing Twitter Data Analysis with Simple Semantic Filtering: Example in Tracking Influenza-Like Illnesses

  2012cited by this paper
• Comparison: Flu Prescription Sales Data from a Retail Pharmacy in the US with Google Flu Trends and US ILINet (CDC) Data as Flu Activity Indicator

  2012cited by this paper
• Lightweight methods to estimate influenza rates and alcohol sales volume from Twitter messages

  2012cited by this paper
• Influenza Patients Are Invisible in the Web: Traditional Model Still Improves the State of the Art Web Based Influenza Surveillance

  2012cited by this paper
• Internet suicide searches and the incidence of suicide in young people in Japan

  2011cited by this paper
• Internet Queries and Methicillin-Resistant Staphylococcus aureus Surveillance

  2011cited by this paper
• The Use of Twitter to Track Levels of Disease Activity and Public Concern in the U.S. during the Influenza A H1N1 Pandemic

  2011cited by this paper
• Using Web Search Query Data to Monitor Dengue Epidemics: A New Model for Neglected Tropical Disease Surveillance

  2011cited by this paper
• You Are What You Tweet: Analyzing Twitter for Public Health

  2011cited by this paper
• Web query-based surveillance in Sweden during the influenza A(H1N1)2009 pandemic, April 2009 to February 2010.

  2011cited by this paper
• Twitter Catches The Flu: Detecting Influenza Epidemics using Twitter

  2011cited by this paper
• Dengue surveillance based on a computational model of spatio-temporal locality of Twitter

  2011cited by this paper
• Predicting Epidemic Tendency through Search Behavior Analysis

  2011cited by this paper
• Association of Internet search trends with suicide death in Taipei City, Taiwan, 2004-2009.

  2011cited by this paper
• Using Search Query Surveillance to Monitor Tax Avoidance and Smoking Cessation following the United States' 2009 “SCHIP” Cigarette Tax Increase

  2011cited by this paper
• Predicting Flu Trends using Twitter data

  2011cited by this paper
• Use of Google Insights for Search to track seasonal and geographic kidney stone incidence in the United States.

  2011cited by this paper
• Tuberculosis Surveillance by Analyzing Google Trends

  2011cited by this paper
• Applying a New Model for Sharing Population Health Data to National Syndromic Influenza Surveillance: DiSTRIBuTE Project Proof of Concept, 2006 to 2009

  2011cited by this paper
• Determination of geographic variance in stroke prevalence using Internet search engine analytics.

  2011cited by this paper
• Prediction of Dengue Incidence Using Search Query Surveillance

  2011cited by this paper
• Notifiable infectious disease surveillance with data collected by search engine

  2010cited by this paper
• Eye-Opening Approach to Norovirus Surveillance

  2010cited by this paper
• Tracking the flu pandemic by monitoring the social web

  2010cited by this paper
• Do Seasons Have an Influence on the Incidence of Depression? The Use of an Internet Search Engine Query Data as a Proxy of Human Affect

  2010cited by this paper
• Pandemics in the Age of Twitter: Content Analysis of Tweets during the 2009 H1N1 Outbreak

  2010cited by this paper
• Text and Structural Data Mining of Influenza Mentions in Web and Social Media

  2010cited by this paper
• A neural netwok based approach to detect influenza epidemics using search engine query data

  2010cited by this paper
• The utility of "Google Trends" for epidemiological research: Lyme disease as an example.

  2010cited by this paper
• Detecting influenza epidemics using search engine query data

  2009cited by this paper
• Early detection of disease outbreaks using the Internet

  2009cited by this paper
• Web Queries as a Source for Syndromic Surveillance

  2009cited by this paper
• Google trends: a web-based tool for real-time surveillance of disease outbreaks.

  2009cited by this paper
• More Diseases Tracked by Using Google Trends

  2009cited by this paper
• Model Formulation: HealthMap: Global Infectious Disease Monitoring through Automated Classification and Visualization of Internet Media Reports

  2008cited by this paper
• Using internet searches for influenza surveillance.

  2008cited by this paper
• Healthcare Epidemiology: Use of Prediction Markets to Forecast Infectious Disease Activity

  2007cited by this paper
• Infodemiology: Tracking Flu-Related Searches on the Web for Syndromic Surveillance

  2006cited by this paper
• Monitoring over-the-counter medication sales for early detection of disease outbreaks--New York City.

  2005cited by this paper
• Cancer Internet Search Activity on a Major Search Engine, United States 2001-2003

  2005cited by this paper
• Analysis of Web Access Logs for Surveillance of Influenza

  2004cited by this paper
• The Elements of Statistical Learning: Data Mining, Inference, and Prediction

  2001cited by this paper
• Stacked Density Estimation

  1997cited by this paper
• In Advances in Neural Information Processing Systems

  1996cited by this paper
• A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection

  1995cited by this paper
• Guidelines for evaluating surveillance systems.

  1988cited by this paper

• The Waxing and Waning of Fear Influence the Control of Vector-Borne Diseases

  2025cites this paper
• Invisibility and Exposure: Challenges of Using AI in Healthcare

  2025cites this paper
• Mathematical Analysis and Modeling of Ebola Virus Dynamics via Optimal Control and Neural Network Paradigms

  2025cites this paper
• Modern Information and Molecular Technologies in the Practice of Epidemiological Surveillance of Natural Focal Infections

  2025cites this paper
• One pathogen does not an epidemic make: A review of interacting contagions, diseases, beliefs, and stories.

  2025cites this paper
• Comprehensive age-period-cohort analysis of global pancreatitis burden and socioeconomic disparities (1990–2021)

  2025cites this paper
• DIGITAL EPIDEMIOLOGY IN CENTRAL ASIA: USING SEARCH DATA TO MONITOR INFLUENZA-LIKE ILLNESS TRENDS

  2025cites this paper
• Predicting Disease Outbreaks with AI: An In-depth Analysis of Infectious Diseases Surveillance

  2025cites this paper
• One pathogen does not an epidemic make: a review of interacting contagions, diseases, beliefs, and stories

  2025cites this paper
• Integrating emotion dynamics in mental health: A trimodal framework combining ecological momentary assessment, physiological measurements, and speech emotion recognition

  2025cites this paper
• Modeling Intraday Aedes-human exposure dynamics enhances dengue risk prediction

  2025cites this paper
• PREPARE: PREdicting PAndemic's REcurring Waves Amidst Mutations, Vaccination, and Lockdowns

  2024cites this paper
• Comparative AnalysisOfComputational ModelsForInfectiousDiseases

  2024cites this paper
• Telephone calls to emergency medical service as a tool to predict influenza-like illness: A 10-year study.

  2024cites this paper
• Estimating behavioural relaxation induced by COVID-19 vaccines in the first months of their rollout

  2024cites this paper
• Pharmacovigilance in the Age of Legalized Cannabis: Using Social Media to Monitor Drug–Drug Interactions Between Immunosuppressants and Cannabis-Derived Products

  2024cites this paper
• Human behavior-driven epidemic surveillance in urban landscapes

  2024cites this paper
• Harnessing Telehealth: Improving Epidemic Prediction and Response.

  2024cites this paper
• Secondary data for global health digitalisation.

  2023cites this paper
• Ethical Framework for Multi-Source Big Data Analytics Disease Surveillance in Public Healthcare

  2023cites this paper
• Real-time Hybrid Dashboard and App for Mpox Outbreak Surveillance

  2023cites this paper
• Modern technologies and solutions to enhance surveillance and response systems for emerging zoonotic diseases

  2023cites this paper
• A Novel Approach for the Early Detection of Medical Resource Demand Surges During Health Care Emergencies: Infodemiology Study of Tweets

  2023cites this paper
• Mining Google Trends data for nowcasting and forecasting colorectal cancer (CRC) prevalence

  2023cites this paper
• Operationalizing cultural adaptation to climate change: contemporary examples from United States agriculture

  2023cites this paper
• Investigating How Liverpool City Council Use Big Data to Control Covid-19 Transmission

  2023cites this paper
• Challenges and Opportunities of Secondary Use of health Data in Developing Countries

  2023cites this paper
• Crime, inequality and public health: a survey of emerging trends in urban data science

  2023cites this paper
• The Birth of Digital Epidemiology in South Korea

  2023cites this paper
• Predicting the dengue cluster outbreak dynamics in Yogyakarta, Indonesia: a modelling study

  2023cites this paper
• Mpox Panic, Infodemic, and Stigmatization of the Two-Spirit, Lesbian, Gay, Bisexual, Transgender, Queer or Questioning, Intersex, Asexual Community: Geospatial Analysis, Topic Modeling, and Sentiment Analysis of a Large, Multilingual Social Media Database

  2023cites this paper
• Addressing the socioeconomic divide in computational modeling for infectious diseases

  2022cites this paper
• Mpox panic, infodemic, and stigmatization of the 2SLGBTQIAP+ community: geospatial analysis, topic modeling, and sentiment analysis of a large, multilingual social media database (Preprint)

  2022cites this paper
• Inequality, Crime and Public Health: A Survey of Emerging Trends in Urban Data Science

  2022cites this paper
• State-Level COVID-19 Symptom Searches and Case Data: Quantitative Analysis of Political Affiliation as a Predictor for Lag Time Using Google Trends and Centers for Disease Control and Prevention Data

  2022cites this paper
• Dynamic downscaling and daily nowcasting from influenza surveillance data

  2022cites this paper
• Exploring the Relationship among Human Activities, COVID-19 Morbidity, and At-Risk Areas Using Location-Based Social Media Data: Knowledge about the Early Pandemic Stage in Wuhan

  2022cites this paper
• Challenges in the development of digital public health interventions and mapped solutions: Findings from a scoping review

  2022cites this paper
• The Volume and Tone of Twitter Posts About Cannabis Use During Pregnancy: Protocol for a Scoping Review

  2022cites this paper
• Identifying Health-Related Discussions of Cannabis Use on Twitter by Using a Medical Dictionary: Content Analysis of Tweets

  2022cites this paper
• Estimating the potential for global dissemination of pandemic pathogens using the global airline network and healthcare development indices

  2022cites this paper
• Data-driven methods for present and future pandemics: Monitoring, modelling and managing

  2021cites this paper
• Identifying health-related discussions of cannabis use on Twitter: a content analysis informed by a medical dictionary (Preprint)

  2021cites this paper
• Rapid indicators of deprivation using grocery shopping data

  2021cites this paper
• Identifying health-related discussions of cannabis use on Twitter: a proof-of-concept study (Preprint)

  2021cites this paper
• NextGen Public Health Surveillance and the Internet of Things (IoT)

  2021cites this paper
• Cannabis as a medicine from the social media perspective: Narrative review of the literature (Preprint)

  2021cites this paper
• Emerging Applications of Machine Learning in Food Safety.

  2021cites this paper
• From “The Interpersonal Theory of Suicide” to “The Interpersonal Trust”: an unexpected and effective resource to mitigate economic crisis-related suicide risk in times of Covid-19?

  2021cites this paper
• Using smartphone-GPS data to understand pedestrian-scale behavior in urban settings: A review of themes and approaches

  2021cites this paper
• Wikipedia, Google Trends and Diet: Assessment of Temporal Trends in the Internet Users’ Searches in Italy before and during COVID-19 Pandemic

  2021influential citation
• Prominent Features in Sleep Disorder Disease in Bioinformatics Using Relevant Data Sets

  2021cites this paper
• What can internet users' behaviours reveal about the mental health impacts of the COVID-19 pandemic? A systematic review

  2021cites this paper
• Leveraging Individual and Collective Regularity to Profile and Segment User Locations from Mobile Phone Data

  2021cites this paper
• Spatial aggregation choice in the era of digital and administrative surveillance data

  2021cites this paper
• Harnessing Social Media in the Modelling of Pandemics—Challenges and Opportunities

  2021cites this paper
• Methodological approaches for the prediction of opioid use-related epidemics in the United States: a narrative review and cross-disciplinary call to action.

  2021cites this paper
• Cell-phone traces reveal infection-associated behavioral change

  2021cites this paper
• Forecasting influenza-like illness trends in Cameroon using Google Search Data

  2021cites this paper
• Data-Driven Surveillance: Effective Collection, Integration, and Interpretation of Data to Support Decision Making

  2021cites this paper
• In search of art: rapid estimates of gallery and museum visits using Google Trends

  2020cites this paper
• New Means of Data Collection and Accessibility

  2020cites this paper
• Social Media- and Internet-Based Disease Surveillance for Public Health.

  2020cites this paper
• Recommendations for Cardiovascular Health and Disease Surveillance for 2030 and Beyond: A Policy Statement From the American Heart Association

  2020cites this paper
• Asynchronous dual-pipeline deep learning framework for online data stream classification

  2020cites this paper
• Survey data and human computation for improved flu tracking

  2020cites this paper
• A re-organizing biosurveillance framework based on fog and mobile edge computing

  2020cites this paper
• Infectious Disease Forecasting for Public Health

  2020cites this paper
• Data-Driven Methods to Monitor, Model, Forecast and Control Covid-19 Pandemic: Leveraging Data Science, Epidemiology and Control Theory

  2020cites this paper
• Time Series Methods and Ensemble Models to Nowcast Dengue at the State Level in Brazil

  2020cites this paper
• A Quantitative Portrait of Wikipedia's High-Tempo Collaborations during the 2020 Coronavirus Pandemic

  2020cites this paper
• Crowdsourcing in Crisis: Rising to the Occasion.

  2020cites this paper
• Trending on Social Media: Integrating Social Media into Infectious Disease Dynamics

  2020cites this paper
• Nowcasting Influenza-like Illness Trends in Cameroon

  2020cites this paper
• Early warnings of COVID-19 outbreaks across Europe from social media

  2020cites this paper
• Big Data Analytics in the Fight against Major Public Health Incidents (Including COVID-19): A Conceptual Framework

  2020cites this paper
• GeoDenStream: An improved DenStream clustering method for managing entity data within geographical data streams

  2020cites this paper
• A Google–Wikipedia–Twitter Model as a Leading Indicator of the Numbers of Coronavirus Deaths

  2020cites this paper
• A framework for estimating crime location choice based on awareness space

  2020cites this paper
• Predicting seasonal influenza using supermarket retail records

  2020cites this paper
• Turn to the Internet First? Using Online Medical Behavioral Data to Forecast COVID-19 Epidemic Trend

  2020cites this paper
• A Social Media Time-Series Data Analytics Approach for Digital Epidemiology

  2020cites this paper
• Google searches accurately forecast RSV hospitalizations

  2019cites this paper
• SleepOMICS: How Big Data Can Revolutionize Sleep Science

  2019cites this paper
• Harnessing digital data and data science to achieve 90-90-90 goals to end the HIV epidemic.

  2019cites this paper
• Movement and Spatial Specificity Support Scaling in Ant Colonies and Immune Systems: Application to National Biosurveillance

  2019cites this paper
• The future of influenza forecasts

  2019cites this paper
• Modeling Relational Drug-Target-Disease Interactions via Tensor Factorization with Multiple Web Sources

  2019cites this paper
• Topic Clustering of E-Cigarette Submissions Among Reddit Communities: A Network Perspective

  2019cites this paper
• Social media surveillance for perceived therapeutic effects of cannabidiol (CBD) products.

  2019cites this paper
• Influenza Surveillance Using Wearable Mobile Health Devices

  2019cites this paper
• What can urban mobility data reveal about the spatial distribution of infection in a single city?

  2019cites this paper
• Épidémiologie de la grippe en population générale à partir d'une cohorte participative

  2019cites this paper
• Computational socioeconomics

  2019cites this paper
• Reappraising the utility of Google Flu Trends

  2019cites this paper
• The impact of news exposure on collective attention in the United States during the 2016 Zika epidemic

  2018cites this paper
• Vaccines Meet Big Data: State-of-the-Art and Future Prospects. From the Classical 3Is (“Isolate–Inactivate–Inject”) Vaccinology 1.0 to Vaccinology 3.0, Vaccinomics, and Beyond: A Historical Overview

  2018cites this paper
• Epidemiological surveillance through cellphone metadata

  2018cites this paper
• Global Research on Syndromic Surveillance from 1993 to 2017: Bibliometric Analysis and Visualization

  2018cites this paper
• Modeling Spatiotemporal Factors Associated With Sentiment on Twitter: Synthesis and Suggestions for Improving the Identification of Localized Deviations

  2018cites this paper