Marr's Levels of Analysis

Marr's 3 levels of analysis.
Marr's three levels of analysis.

Marr's three levels of analysis [cite key="marr1982"] promotes the idea that complex systems such as the brain, a computer or human behaviour should be understood at different levels. Marr's framework proved to be an elegant and popular way of reasoning about complex systems, and in the context of machine learning and statistics,  remains an intuitive framework that is often used when describing probabilistic models of cognition and perceptual systems.

Continue reading "Marr's Levels of Analysis"

On Marginal Likelihoods and Widely Applicable BIC

The marginal likelihood of a model is one the key quantities appearing throughout machine learning and statistics, since it provides an intuitive and natural objective function for model selection and parameter estimation. I recently read a new paper by Sumio Watanabe on A Widely applicable Bayesian information criterion (WBIC)[cite key="watanabe2012widely"] (and to appear in JMLR soon) that  provides a new, theoretically grounded and easy to implement method of approximating the marginal likelihood, which I will briefly describe in this post. I'll summarise some of the important aspects of the marginal likelihood and then briefly describe the WBIC and some thoughts and questions on its use.

Continue reading "On Marginal Likelihoods and Widely Applicable BIC"