## Talk: Memory-based Bayesian Reasoning and Deep Learning

A talk that explores the convergence of deep learning and Bayesian inference.  We'll take a statistical tour of deep learning, think about approximate Bayesian inference, and explore the idea of doing inference-with-memory and the different ways that this manifests itself in contemporary machine learning. Slides See the slides using this link. Abstract Deep learning and Bayesian ...

## Machine Learning Trick of the Day (4): Reparameterisation Tricks12

Our ability to rewrite statistical problems in an equivalent but different form, to reparameterise them, is one of the most general-purpose tools we have in mathematical statistics. We used reparameterisation in all the tricks we explored in this series so far: trick 1 re-expressed a log-partition function in terms of copies (replicas) of the marginal probability, trick 2 re-expressed a ...

## Bayesian Reasoning and Deep Learning4

I gave a talk entitled 'Bayesian Reasoning and Deep Learning' recently. Here is the abstract and the slides for interest. Slides Bayesian Reasoning and Deep Learning Abstract Deep learning and Bayesian machine learning are currently two of the most active areas of machine learning research. Deep learning provides a powerful class of models and an ...

## Machine Learning Trick of the Day (3): Hutchinson's Trick6

Hutchinson's estimator  is a simple way to obtain a stochastic estimate of the trace of a matrix. This is a simple trick that uses randomisation to transform the algebraic problem of computing the trace into the statistical problem of computing an expectation of a quadratic function. The randomisation technique used in this trick is just one from a ...

## Machine Learning Trick of the Day (2): Gaussian Integral Trick

Today's trick, the Gaussian integral trick, is one that allows us to re-express a (potentially troublesome) function in an alternative form, in particular, as an integral of a Gaussian against another function — integrals against a Gaussian turn out not to be too troublesome and can provide many statistical and computational benefits. One popular setting where we can exploit such ...

## Machine Learning Trick of the Day (1): Replica Trick3

'Tricks' of all sorts are used throughout machine learning, in both research and in production settings. These tricks allow us to address many different types of data analysis problems, being roughly of either an analytical, statistical, algorithmic, or numerical flavour. Today's trick is in the analytical class and comes to us from statistical physics: the popular Replica trick. The replica ...

## A Statistical View of Deep Learning: Retrospective1

Over the past 6 months, I've taken to writing a series of posts (one each month) on a statistical view of deep learning with two principal motivations in mind. The first was as a personal exercise to make concrete and to test the limits of the way that I think about, and use deep learning in my every ...

## A Statistical View of Deep Learning (VI): What is Deep?6

Throughout this series, we have discussed deep networks by examining prototypical instances of these models, e.g., deep feed-forward networks, deep auto-encoders, deep generative models, but have not yet interrogated the key word we have been using. We have not posed the question what does 'deep' mean, and what makes a model deep. There is little in way of ...

## Chinese Edition: A Statistical View of Deep Learning (III)/ 从统计学角度来看深度学习

Colleagues from the Capital of Statistics, an online statistics community in China, have been kind enough to translate my third post in this series, A Statistical View of Deep Learning (III): Memory and Kernels,  in the hope that they might be of interest to machine learning and statistics researchers in China (and to Chinese readers). Find it here: ...

## Chinese Edition: A Statistical View of Deep Learning (II)/ 从统计学角度来看深度学习

Colleagues from the Capital of Statistics, an online statistics community in China, have been kind enough to translate my second post in this series, A statistical View of Deep Learning (II): Auto-encoders and Free Energy,  in the hope that they might be of interest to machine learning and statistics researchers in China (and to Chinese readers). Find ...