Maximum Entropy Estimator for Hidden Markov Models: Reduction to Dimension 2

In the paper, we introduce the maximum entropy estimator based on 2-dimensional empirical distribution of the observation sequence of hidden Markov model , when the sample size is big: in that case computing the maximum likelihood estimator is too consuming in time by the classical Baum-Welch EM algorithm. We prove the consistency and the asymptotic normality of the maximum entropy estimator in a quite general framework, where the asymptotic covariance matrix is explicitly estimated in terms of the 2-dimensional Fisher information. To complement it, the 2-dimensional relative entropy is skillfully used to study the hypotheses testing problem. Furthermore, we propose 2-dimensional maximum entropy algorithm for finding the maximum entropy estimator, which works for very large observation dataset and large hidden states set. Some numerical examples are furnished and commented for illustrating our theoretical results.

• Publication year

  2023

• Venue

  Unknown venue

• Publication date

  2023-02-18

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 2303.08718
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Upper bounds on the relative entropy and Rényi divergence as a function of total variation distance for finite alphabets

  2015cited by this paper
• Nonparametric identification and maximum likelihood estimation for hidden Markov models.

  2014cited by this paper
• Statistical Inference in Hidden Markov Models Using k-Segment Constraints

  2013cited by this paper
• Distribution of Statistics of Hidden State Sequences Through the Sum-Product Algorithm

  2012cited by this paper
• Large deviations for random dynamical systems and applications to hidden Markov models

  2011cited by this paper
• Transportation inequalities for hidden Markov chains and applications

  2011cited by this paper
• CONSISTENCY OF THE MAXIMUM LIKELIHOOD ESTIMATOR FOR GENERAL HIDDEN MARKOV MODELS

  2009cited by this paper
• Hidden Topic Markov Models

  2007cited by this paper
• Recursive estimation of Hidden Markov Models

  2005cited by this paper
• Asymptotic properties of the maximum likelihood estimator in autoregressive models with Markov regime

  2004cited by this paper
• Transportation cost-information inequalities and applications to random dynamical systems and diffusions

  2004cited by this paper
• On contraction properties of Markov kernels

  2003cited by this paper
• Hidden Markov processes

  2002cited by this paper
• Asymptotics of the maximum likelihood estimator for general hidden Markov models

  2001cited by this paper
• Sequentially finding the N-Best List in Hidden Markov Models

  2001cited by this paper
• Asymptotic normality of the maximum-likelihood estimator for general hidden Markov models

  1998cited by this paper
• Consistent Estimation of Linear and Non‐linear Autoregressive Models with Markov Regime

  1998cited by this paper
• Exponential Forgetting and Geometric Ergodicity in Hidden Markov Models

  1997cited by this paper
• On identifiability and order of continuous-time aggregated Markov chains, Markov-modulated Poisson processes, and phase-type distributions

  1996cited by this paper
• Inference in hidden Markov models I: Local asymptotic normality in the stationary case

  1996cited by this paper
• An EM algorithm for estimation in Markov-modulated Poisson processes

  1996cited by this paper
• Consistent and Asymptotically Normal Parameter Estimates for Markov Modulated Poisson Processes

  1995cited by this paper
• Estimating the Order of Hidden Markov Models

  1995cited by this paper
• Consistent and Asymptotically Normal Parameter Estimates for Hidden Markov Models

  1994cited by this paper
• Markov Chains and Stochastic Stability

  1993cited by this paper
• Maximum likelihood estimation and identification directly from single-channel recordings

  1992cited by this paper
• Maximum mutual information estimation of hidden Markov model parameters for speech recognition

  1986cited by this paper
• Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper

  1977cited by this paper
• A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains

  1970cited by this paper
• On the Identifiability of Finite Mixtures

  1968cited by this paper
• Probabilistic functions of finite-state markov chains.

  1967influential reference
• Error bounds for convolutional codes and an asymptotically optimum decoding algorithm

  1967cited by this paper
• Statistical Inference for Probabilistic Functions of Finite State Markov Chains

  1966cited by this paper
• On the Identifiability Problem for Functions of Finite Markov Chains

  1957cited by this paper

• No citing papers are available for this paper.