Bias approximations for likelihood‐based estimators

Bias approximation has played an important rôle in statistical inference, and numerous bias calculation techniques have been proposed under different contexts. We provide a unified approach to approximating the bias of the maximum likelihood estimator and the l2 penalized likelihood estimator for both linear and nonlinear models, where the design variables are allowed to be random and the sample size can be a stopping time. The proposed method is based on the Woodroofe–Stein identity and is justified by very weak approximations. The accuracy of the derived bias formulas is assessed by simulation for several examples. The bias of the ridge estimator in high‐dimensional settings is also discussed.

• Publication year

  2020

• Venue

  Scandinavian Journal of Statistics

• Publication date

  2020-11-03

• Fields of study

  Mathematics

• Identifiers
  DOI 10.1111/sjos.12499
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• A general result on the estimation bias of ARMA models

  2018cited by this paper
• A general method for third-order bias and variance corrections on a nonlinear estimator

  2015cited by this paper
• A significance test for graph‐constrained estimation

  2015cited by this paper
• Corrected confidence intervals based on the signed root transformation for multi-parameter sequentially designed experiments

  2014cited by this paper
• Confidence intervals and hypothesis testing for high-dimensional regression

  2013cited by this paper
• Statistical significance in high-dimensional linear models

  2012cited by this paper
• Folklore Theorems, Implicit Maps, and Indirect Inference

  2012cited by this paper
• Estimation in high-dimensional linear models with deterministic design matrices

  2012cited by this paper
• A Bayesian Approximation Method for Online Ranking

  2011cited by this paper
• The Likelihood Principle

  2011cited by this paper
• The Second-Order Bias and Mean Squared Error of Estimators in Time Series Models

  2007influential reference
• Corrigendum to “The second-order bias and mean squared error of nonlinear estimators”: [Journal of Econometrics 75(2) (1996) 369–395]

  2005cited by this paper
• Ridge Regression: Biased Estimation for Nonorthogonal Problems

  2000cited by this paper
• Approximate bias calculations for sequentially designed experiments

  1998influential reference
• Corrected confidence sets for sequentially designed experiments

  1997cited by this paper
• Corrected confidence intervals after sequential testing with applications to survival analysis

  1996cited by this paper
• Point and interval estimation following a sequential clinical trial

  1996cited by this paper
• The second-order bias and mean squared error of nonlinear estimators

  1996cited by this paper
• Bias correction in ARMA models

  1994cited by this paper
• Bias reduction of maximum likelihood estimates

  1993cited by this paper
• Estimation with Quadratic Loss

  1992cited by this paper
• Bias Correction in Generalized Linear Models

  1991cited by this paper
• Multiple Imputation of Industry and Occupation Codes in Census Public-use Samples Using Bayesian Logistic Regression

  1991cited by this paper
• On stopping times and stochastic monotonicity

  1990cited by this paper
• Very weak expansions for sequentially designed experiments: linear models

  1989cited by this paper
• On the bias of maximum likelihood estimation following a sequential test

  1986cited by this paper
• Very Weak Expansions for Sequential Confidence Levels

  1986cited by this paper
• On the coverage probability of confidence sets based on a prior distribution

  1985cited by this paper
• Bias correction in maximum likelihood logistic regression.

  1985cited by this paper
• Experimental Design for Binary Data

  1983influential reference
• Estimation of the Mean of a Multivariate Normal Distribution

  1981cited by this paper
• A Note On Bias Correction in Maximum Likelihood Estimation with Logistic Discrimination

  1980cited by this paper
• Logistic Discrimination and Bias Correction in Maximum Likelihood Estimation

  1979cited by this paper
• Mathematics for econometrics

  1978cited by this paper
• Maximum likelihood estimation in small samples

  1977cited by this paper
• A General Definition of Residuals

  1968influential reference
• APPROXIMATE CONFIDENCE INTERVALS

  1953influential reference

• The effects of adaptation on maximum likelihood inference for nonlinear models with normal errors

  2018cites this paper