Bayesian and Classical Estimation Methods in Linear Regression Model: A Monte Carlo Simulation Analysis

Abstract

In the recent times the Bayesian approach has emerged as a strong competitor to the traditional Classical approach to econometric analysis. The main reason for the vicious controversy between the promoters of the two approaches is the notions of probability they employ. The framework of Bayesian approach allows for subjectivity in the choice of prior distribution. The main concern has been the relative performance of this approach especially in the areas of applications. This study focuses on the general linear regression model and investigates the differences between the Classical and the Bayesian approach in terms of model formulation and estimation procedures. The paper describes the procedure for executing Bayesian regression and the choice of the prior distribution to be employed. The Prior distribution is the degree of belief that the researcher has about the distribution of the parameter that he is trying to estimate. Non-Bayesians usually employ this information to lead them to add, drop, or modify variables in an ad hoc search for a “better” result. The Bayesians employ it ex ante in an explicit, upfront fashion. The performance of the Bayesian approach and the Classical was assessed through a series of Monte Carlo experiments. A large number of samples are drawn and the Classical and Bayesian estimators were computed for each sample. The approximate sampling distribution of the statistics were then determined. Using bias and mean square error criteria, the results showed that there was nothing to choose from when prior density is non-informative.