Generalized Additive Modeling of Ecological Data With mgcv: New Adequacy Assessment Tools

ABSTRACT Generalized additive models (GAMs) are a semi‐parametric extension of generalized linear models (GLMs) that allow incorporating different forms of nonlinearities commonly encountered in ecological relationships, thus frequently offering a better statistical description than GLMs in such cases. Due to the use of smooth functions, however, validating that the observed data represent a plausible realization of a fitted GAM according to the underlying distributional assumptions being used is less straightforward than with GLMs. Moreover, if the number of basis dimensions used in smooth terms to control the degree of flexibility is set too large, overfitting can arise despite in‐built penalization procedures aimed at preventing excessive wiggliness. Here, we present how GAMs fitted with the mgcv package in R can be assessed for their adequacy based on half‐normal plots with a simulated envelope using newly‐available helper functions for the hnp package. A proposed metric relying on the mgcViz package is also presented to help detect both under‐ and overfitting relative to a predictor of interest from a realized coverage perspective. Three fisheries‐related examples analyzing continuous data, counts, and discrete proportions are then presented to illustrate the usefulness of these approaches in providing more statistical context for the interpretation of nonlinear ecological relationships.

• Publication year

  2026

• Venue

  Ecology and Evolution

• Publication date

  2026-01-01

• Fields of study

  Medicine, Environmental Science

• Identifiers
  DOI 10.1002/ece3.72825PMID 41522220PMCID 12784473
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Limited Demographic Effects One Decade After Implementation of a Harvest‐Slot Length Limit for Walleye (Sander vitreus) in the St. Lawrence River, Québec, Canada

  2025influential reference
• Krill swarm biomass, energetic density, and species composition drive humpback whale distribution in the Northern California Current

  2025cited by this paper
• A comprehensive analysis of R's application in ecological research from 2008 to 2023

  2025cited by this paper
• Assessing abundance–suitability models to prioritize conservation areas for the dwarf caimans in South America

  2024cited by this paper
• Generalized nonlinearity in animal ecology: Research, review, and recommendations

  2024cited by this paper
• Veni, vidi, vici? Future spread and ecological impacts of a rapidly expanding invasive predator population

  2023cited by this paper
• An improved method for the estimation and comparison of mortality rates in fish from catch‐curve data

  2021influential reference
• Performance evaluation of cetacean species distribution models developed using generalized additive models and boosted regression trees

  2020influential reference
• Evaluating the popularity of R in ecology

  2019cited by this paper
• The consequences of checking for zero‐inflation and overdispersion in the analysis of count data

  2019cited by this paper
• Hierarchical generalized additive models in ecology: an introduction with mgcv

  2019cited by this paper
• Ant species richness and interactions in canopies of two distinct successional stages in a tropical dry forest

  2019cited by this paper
• Scalable Visualization Methods for Modern Generalized Additive Models

  2018cited by this paper
• Half-Normal Plots and Overdispersed Models in R: The hnp Package

  2017influential reference
• itsadug: Interpreting Time Series and Autocorrelated Data Using GAMMs

  2015influential reference
• Rising complexity and falling explanatory power in ecology

  2014cited by this paper
• On quantile quantile plots for generalized linear models

  2012cited by this paper
• Practical variable selection for generalized additive models

  2011influential reference
• Maximum Likelihood Estimation and Inference: With Examples in R, SAS and ADMB

  2011cited by this paper
• Finding the Smoothest Path to Success: Model Complexity and the Consideration of Nonlinear Patterns in Nest-Survival Data

  2010cited by this paper
• Consequences of recruitment decisions and heterogeneity on age-specific breeding success in a long-lived seabird.

  2009cited by this paper
• Mixed Effects Models and Extensions in Ecology with R

  2009influential reference
• Generalized linear mixed models: a practical guide for ecology and evolution.

  2009cited by this paper
• Dealing with overdispersed count data in applied ecology

  2007cited by this paper
• Generalized Additive Models

  2006influential reference
• Zero tolerance ecology: improving ecological inference by modelling the source of zero observations.

  2005cited by this paper
• Remedies for pseudoreplication

  2004cited by this paper
• Generalized linear and generalized additive models in studies of species distributions: setting the scene

  2002cited by this paper
• Worm plot: a simple diagnostic device for modelling growth reference curves

  2001cited by this paper
• A small-sample correction for the Schwarz SIC model selection criterion

  1999cited by this paper
• Generalized Additive Models

  1986cited by this paper
• Monitoring reproduction in fish: Assessing the adequacy of ogives and the predicted uncertainty of their L50 estimates for more reliable biological inferences

  year unknowninfluential reference

• No citing papers are available for this paper.