Generalized Model Selection: An Information Theoretic Approach

S, Chaithanya K. (2015) Generalized Model Selection: An Information Theoretic Approach. Masters thesis, Indian Institute of Science Education and Research Kolkata.

[img] PDF (MS dissertation of Chaithanya K. S. (10MS82))
chaithanya_ks.pdf - Submitted Version
Restricted to Repository staff only

Download (454kB)
Official URL:


The problem of model selection for a data set is the task of selecting the `best' statistical model from a set of candidate models. After choosing the candidate models, we run a statistical analysis to identify the best model. But what we mean by `Best' is contro- versial. To add clarity, the best model should balance between goodness of fit and the simplicity of the model. For instance, the more complex the model is, the more it will be able to adapt its shape to the data. But this might lead to the over-fitting problem i.e. the additional parameters which were included for a better- fitted shape may not represent anything useful. Perhaps a few of them might be really random. The com- plexity of the model is usually represented by the number of parameters in the model. Likelihood ratio approach have been used to determine the goodness-of-fit statistic. But many of the classical goodness-of-fit tests do not penalize distributions for the number of parameters they use. Thus, a distribution with four parameters may fit the data better because it has a lot more exibility in shape than a two-parameter distribution, but the apparent improvement is spurious. This is where information criterion comes into play. The information criteria penalize distributions with a greater number of parameters, and thus helps avoid the over-fitting problem.

Item Type: Thesis (Masters)
Additional Information: Supervisor: Prof. Asok Kumar Nanda
Uncontrolled Keywords: AIC; Akaike's Entropy-based Information Criterion; Generalized Model Selection; Information Theoretic Approach; Information Theory of Model Selection; Model Selection; Problem of Model Selection
Subjects: Q Science > QA Mathematics
Divisions: Department of Mathematics and Statistics
Depositing User: IISER Kolkata Librarian
Date Deposited: 22 Aug 2016 05:53
Last Modified: 22 Aug 2016 05:54

Actions (login required)

View Item View Item