Assessing the reliability of ecotoxicological studies: An overview of current needs and approaches

In general, reliable studies are well designed and well performed, and enough details on study design and performance are reported to assess the study. For hazard and risk assessment in various legal frameworks, many different types of ecotoxicity studies need to be evaluated for reliability. These studies vary in study design, methodology, quality, and level of detail reported (e.g., reviews, peer‐reviewed research papers, or industry‐sponsored studies documented under Good Laboratory Practice [GLP] guidelines). Regulators have the responsibility to make sound and verifiable decisions and should evaluate each study for reliability in accordance with scientific principles regardless of whether they were conducted in accordance with GLP and/or standardized methods. Thus, a systematic and transparent approach is needed to evaluate studies for reliability. In this paper, 8 different methods for reliability assessment were compared using a number of attributes: categorical versus numerical scoring methods, use of exclusion and critical criteria, weighting of criteria, whether methods are tested with case studies, domain of applicability, bias toward GLP studies, incorporation of standard guidelines in the evaluation method, number of criteria used, type of criteria considered, and availability of guidance material. Finally, some considerations are given on how to choose a suitable method for assessing reliability of ecotoxicity studies. Integr Environ Assess Manag 2017;13:640–651. © 2016 The Authors. Integrated Environmental Assessment and Management published by Wiley Periodicals, Inc. on behalf of Society of Environmental Toxicology & Chemistry (SETAC)

• Publication year

  2017

• Venue

  Integrated Environmental Assessment and Management

• Publication date

  2017-07-01

• Fields of study

  Medicine, Engineering, Environmental Science

• Identifiers
  DOI 10.1002/ieam.1870PMID 27869364
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• How we can make ecotoxicology more valuable to environmental protection.

  2017cited by this paper
• Assessing the relevance of ecotoxicological studies for regulatory decision making

  2017cited by this paper
• Criteria for Reporting and Evaluating ecotoxicity Data (CRED): comparison and perception of the Klimisch and CRED methods for evaluating reliability and relevance of ecotoxicity studies

  2016cited by this paper
• CRED: Criteria for reporting and evaluating ecotoxicity data

  2016influential reference
• Stepwise Information‐Filtering Tool (SIFT): A method for using risk assessment metadata in a nontraditional way

  2015influential reference
• Facilitating the use of non‐standard in vivo studies in health risk assessment of chemicals: a proposal to improve evaluation criteria and reporting

  2014influential reference
• Effects of Atrazine in Fish, Amphibians, and Reptiles: An Analysis Based on Quantitative Weight of Evidence

  2014cited by this paper
• Bad Reporting or Bad Science? Systematic Data Evaluation as a Means to Improve the Use of Peer-Reviewed Studies in Risk Assessments of Chemicals

  2014influential reference
• Best Practices for Evaluating Method Validity, Data Quality and Study Reliability of Toxicity Studies for Chemical Hazard and Risk Assessments

  2013influential reference
• Creative in the kitchen

  2012cited by this paper
• Information Quality in Regulatory Decision Making: Peer Review versus Good Laboratory Practice

  2012cited by this paper
• Comparison of four different methods for reliability evaluation of ecotoxicity data: a case study of non-standard test data used in environmental risk assessments of pharmaceutical substances

  2011cited by this paper
• Reporting and evaluation criteria as means towards a transparent use of ecotoxicity data for environmental risk assessment of pharmaceuticals.

  2011influential reference
• Good Laboratory Practices Are Not Synonymous with Good Scientific Practices, Accurate Reporting, or Valid Data

  2010cited by this paper
• Comparison of Data Used for Setting Occupational Exposure Limits

  2010cited by this paper
• A New Quality Assurance System for the Evaluation of Ecotoxicity Studies Submitted Under the New Substances Notification Regulations in Canada

  2009influential reference
• "ToxRTool", a new tool to assess the reliability of toxicological data.

  2009influential reference
• Manual for summarising and evaluating environmental aspects of plant protection products

  2008cited by this paper
• Evaluation of Criteria Used to Assess the Quality of Aquatic Toxicity Data

  2005influential reference
• Good Laboratory Practice: the Why and the How

  2000cited by this paper
• Data Quality Evaluation of Toxicological Studies Used to Derive Ecotoxicological Benchmarks

  2000influential reference
• Guidelines for ecological risk assessment

  1998cited by this paper
• A systematic approach for evaluating the quality of experimental toxicological and ecotoxicological data.

  1997influential reference

• An integrated ecotoxicological study reliability (EcoSR) framework for use in toxicity value development.

  2025cites this paper
• Weight of evidence assessment of effects of sulfoxaflor on aquatic invertebrates: sulfoxaflor environmental science review part II

  2025cites this paper
• Occurrence, dispersal, and associated environmental risk assessment of pesticides and their transformation products in small water bodies of Northeastern France

  2024cites this paper
• How Effective is Score-Based Data Quality Assessment? An Illustration with Fish BCF Data.

  2024cites this paper
• Incorporating a weight‐of‐evidence approach into a tiered assessment for chemicals management, with emphasis on program development and applications in developing countries and emerging economies

  2024cites this paper
• Evidence on the effects of flame retardant substances at ecologically relevant endpoints: a systematic map protocol

  2024cites this paper
• THE EFFECTS OF COMMERCIAL 2,4-D HERBICIDE ON GAME FISH SPECIES: NATURAL LAKE WATER vs. LABORATORY SYSTEM WATER.

  2024cites this paper
• STYGOTOX: A Quality‐Assessed Database of (Eco)Toxicological Data on Stygofauna and Other Aquatic Subterranean Organisms

  2024cites this paper
• Strategy towards producing relevant and reliable data for the hazard assessment of micro- and nanoplastics in agricultural soils

  2024cites this paper
• Implementation of the CREED approach for environmental assessments

  2024cites this paper
• Incorporation of sediment‐ and soil‐specific aspects in the Criteria for Reporting and Evaluating Ecotoxicity Data (CRED)

  2024cites this paper
• Proposal of critical appraisal tools for the evaluation of ecotoxicology studies

  2023cites this paper
• Considerations for the inclusion of metabarcoding data in the plant protection product risk assessment of the soil microbiome

  2023cites this paper
• Using freshwater snail Biomphalaria glabrata (Say, 1818) as a biological model for ecotoxicology studies: a systematic review

  2023cites this paper
• Impact of Endocrine Disrupting Chemicals (EDCs) on Reproductive Health of Human

  2022cites this paper
• Characterisation and analysis of key studies used to restrict substances under REACH

  2022cites this paper
• The ECOTOXicology Knowledgebase: A Curated Database of Ecologically Relevant Toxicity Tests to Support Environmental Research and Risk Assessment

  2022cites this paper
• Screening and prioritization of nano- and microplastic particle toxicity studies for evaluating human health risks – development and application of a toxicity study assessment tool

  2022cites this paper
• Response to "A comprehensive review on environmental toxicity of azole compounds to fish".

  2021cites this paper
• A Critical Review of Organic Ultraviolet Filter Exposure, Hazard, and Risk to Corals

  2021cites this paper
• Comparisons of PNEC derivation logic flows under example regulatory schemes and implications for ecoTTC

  2021cites this paper
• Open Science in regulatory environmental risk assessment

  2021cites this paper
• Environmental fate and effects assessment of human pharmaceuticals: lessons learnt from regulatory data

  2021cites this paper
• Ecotoxicology of Glyphosate, Its Formulants, and Environmental Degradation Products.

  2021cites this paper
• Environmental Assessors Should Use State‐of‐Practice Weight‐of‐Evidence Processes

  2021cites this paper
• Does the scientific knowledge reflect the chemical diversity of environmental pollution? – A twenty-year perspective

  2021cites this paper
• Coral Ecotoxicological Data Evaluation for the Environmental Safety Assessment of Ultraviolet Filters

  2021cites this paper
• A Critical Review of the Availability, Reliability, and Ecological Relevance of Arctic Species Toxicity Tests for Use in Environmental Risk Assessment

  2021cites this paper
• The achievement of regulatory excellence in the oil and gas industry in Nigeria: the 2017 National Oil and Gas Policy

  2020cites this paper
• Conducting fit‐for‐purpose food safety risk assessments

  2019cites this paper
• Creation of a Curated Aquatic Toxicology Database: EnviroTox

  2019cites this paper
• Towards Sustainable Environmental Quality: Priority Research Questions for the Australasian Region of Oceania

  2019cites this paper
• Workshop Synthesis Towards Sustainable Environmental Quality: Priority Research Questions for the Australasian Region of Oceania

  2019cites this paper
• Effects of atrazine on fish, amphibians, and reptiles: update of the analysis based on quantitative weight of evidence

  2019cites this paper
• Ecotoxicological Risk Assessment of Endocrine Disruptors: Current and Future Approaches

  2019cites this paper
• Strength of methods assessment for aquatic primary producer toxicity data: A critical review of atrazine studies from the peer-reviewed literature.

  2019cites this paper
• Improving environmental risk assessments of chemicals: Steps towards evidence-based ecotoxicology.

  2019influential citation
• Guiding Principles and Key Elements for Establishing a Weight of Evidence for Chemical Assessment

  2019cites this paper
• Explanatory note on literature searching conducted in the context of GMO applications for (renewed) market authorisation and annual post‐market environmental monitoring reports on GMOs authorised in the EU market

  2019cites this paper
• TRANSPARENCY WITHIN REACH?

  2018cites this paper
• Evidence of citation bias in the pesticide ecotoxicology literature

  2018cites this paper
• General Guidance on Endocrine Assessment: Assays and Endpoints

  2018cites this paper
• Targeted selection of existing aquatic in vivo bioassay data in ecotoxicological hazard quantification

  2018cites this paper
• Data quality scoring system for microcosm and mesocosm studies used to derive a level of concern for atrazine

  2018influential citation
• Reliability and relevance evaluations of REACH data

  2018cites this paper
• GLPs, Human Error, and Deviations: When Are Quality and Integrity Compromised?

  2018cites this paper
• Genotoxicity induced by metal oxide nanoparticles: a weight of evidence study and effect of particle surface and electronic properties

  2018cites this paper
• What training and skills will the ecotoxicologists of the future require?

  2017cites this paper
• Weight of evidence approaches for the identification of endocrine disrupting properties of chemicals: Review and recommendations for EU regulatory application

  2017cites this paper
• Quantitative weight of evidence assessment of risk to honeybee colonies from use of imidacloprid, clothianidin, and thiamethoxam as seed treatments: a postscript

  2017cites this paper
• A weight of evidence framework for environmental assessments: Inferring quantities

  2017cites this paper
• Explanatory note on literature searching conducted in the context of GMO applications for (renewed) market authorisation and annual post‐market environmental monitoring reports on GMOs authorised in the EU market

  2017cites this paper
• Assessing the relevance of ecotoxicological studies for regulatory decision making

  2017cites this paper
• Guidance on the use of the weight of evidence approach in scientific assessments

  2017cites this paper
• New approach to weight‐of‐evidence assessment of ecotoxicological effects in regulatory decision‐making

  2017cites this paper
• Quantitative weight of evidence assessment of higher-tier studies on the toxicity and risks of neonicotinoid insecticides in honeybees 1: Methods

  2017cites this paper
• A weight of evidence framework for environmental assessments: Inferring qualities

  2017cites this paper