Machine Learning Trick of the Day (2): Gaussian Integral Trick

Today's trick, the Gaussian integral trick, is one that allows us to re-express a (potentially troublesome) function in an alternative form, in particular, as an integral of a Gaussian against another function — integrals against a Gaussian turn out not to be too troublesome … Continue reading Machine Learning Trick of the Day (2): Gaussian Integral Trick

Machine Learning Trick of the Day (1): Replica Trick

'Tricks' of all sorts are used throughout machine learning, in both research and in production settings. These tricks allow us to address many different types of data analysis problems, being roughly of either an analytical, statistical, algorithmic, or numerical flavour. Today's trick is in the analytical class and comes to us from statistical physics: the popular Replica trick. The replica trick [cite key="engel2001statistical"][cite key="sharp2011effective"][cite key="opper1995statistical"] is used for analytical computation of log-normalising constants (or log-partition functions). More formally, the replica trick provides one of the tools needed for a replica analysis of a probabilistic model — a theoretical analysis of the the properties and expected behaviour of a model. Replica … Continue reading Machine Learning Trick of the Day (1): Replica Trick

A Statistical View of Deep Learning: Retrospective

Over the past 6 months, I've taken to writing a series of posts (one each month) on a statistical view of deep learning with two principal motivations in mind. The first was as a personal exercise to make concrete and to test the limits of the way that I think about, and use deep learning in my every day work. The second, was to highlight important statistical connections and implications of deep learning that I do not see being made in the popular courses, reviews and books on deep learning, but which are extremely important to keep in mind. Post Links and Summary Links to each post with a short … Continue reading A Statistical View of Deep Learning: Retrospective

Chinese Edition: A Statistical View of Deep Learning (III)/ 从统计学角度来看深度学习

Colleagues from the Capital of Statistics, an online statistics community in China, have been kind enough to translate my third post in this series, A Statistical View of Deep Learning (III): Memory and Kernels,  in the hope that they might be of interest to machine learning and statistics researchers in China (and to Chinese readers). Find it here: 从统计学角度来看深度学习(3):记忆和核方法 Continue reading Chinese Edition: A Statistical View of Deep Learning (III)/ 从统计学角度来看深度学习

Chinese Edition: A Statistical View of Deep Learning (II)/ 从统计学角度来看深度学习

Colleagues from the Capital of Statistics, an online statistics community in China, have been kind enough to translate my second post in this series, A statistical View of Deep Learning (II): Auto-encoders and Free Energy,  in the hope that they might be of interest to machine learning and statistics researchers in China (and to Chinese readers). Find it here: 从统计学角度来看深度学习(2):自动编码器和自由能 Continue reading Chinese Edition: A Statistical View of Deep Learning (II)/ 从统计学角度来看深度学习

Chinese Edition: A statistical View of Deep Learning (I)/ 从统计学角度来看深度学习

Colleagues from the Capital of Statistics, an online statistics community in China, have been kind enough to translate my first post in this series, A statistical View of Deep Learning (I): Recursive GLMs,  in the hope that they might be of interest to machine learning and statistics researchers in China (and to Chinese readers). Find it here: 从统计学角度来看深度学习(1):递归广义线性模型   Continue reading Chinese Edition: A statistical View of Deep Learning (I)/ 从统计学角度来看深度学习

A Statistical View of Deep Learning (IV): Recurrent Nets and Dynamical Systems

Recurrent neural networks (RNNs) are now established as one of the key tools in the machine learning toolbox for handling large-scale sequence data. The ability to specify highly powerful models, advances in stochastic gradient descent, the availability of large volumes of … Continue reading A Statistical View of Deep Learning (IV): Recurrent Nets and Dynamical Systems