Marr's three levels of analysis [cite key="marr1982"] promotes the idea that complex systems such as the brain, a computer or human behaviour should be understood at different levels. Marr's framework proved to be an elegant and popular way of reasoning about complex systems, and in the context of machine learning and statistics, remains an intuitive framework that is often used when describing probabilistic models of cognition and perceptual systems.
As Bayesian statisticians, I think this is a useful framework to be aware of, particularly as we move towards designing systems that are increasingly complex and designed to exhibit aspects of 'intelligence'. Marr's three levels framework (see figure) consists of a computational, algorithmic and implementational level [cite key="colombo2012bayes"]:
At this level we describe and specify the problems we are faced with in a generic manner, but do not say how these problems are to be solved. Do we aim to learn a function? Do we wish to estimate uncertainty?
This level forms a bridge between the computational and implementational levels, describing how the identified computational problems can be solved. It is here that Bayesian and machine learning methods find their place.
The physical substrate or mechanism, and its organisation, in which computation is performed. This could be biological in the case of neurons and synapses, or in silicon using transistors, etc.
Each level should be seen as a realisation of the level before it. Thus, the algorithmic level is a realisation of the computational level, describing how the general computational problems can be solved, and similarly, the implementational level is a realisation of the algorithmic level, specifying the mechanism that carries out our algorithms.
Marr introduced these levels as as a reaction to, and means of avoiding reductionism and vagueness, i.e. we should not focus solely on computational or implementational problems, but also look at what bridges these concerns. Modern thoughts, such as those by Poggio [cite key="poggio2012levels"], now seek to update Marr's levels to incorporate the integrated nature of problems at these levels.
Bayesian methods fit neatly within Marr's framework and can play a useful role in motivating and contextualising our research efforts. Indeed, their may be a more fundamental role for Bayesian analysis to play, as emphasised by the ongoing investigation into aspects of the Bayesian brain and the Bayesian coding hypothesis. Colombo and Series [cite key="colombo2012bayes"] provide a comprehensive discussion of the current philosophical discourse in this regard.