Learning in Brains and Machines (4): Episodic and Interactive Memory

· Read in · 2080 words · All posts in series  · [dropcap]My[/dropcap] memory, like yours, exerts a powerful influence over my interaction with the world. It is reconstructive and evocative; I can easily form an image of hot December days in the South African summer, and remember my first time—it was … Continue reading Learning in Brains and Machines (4): Episodic and Interactive Memory

A Year of Approximate Inference: Review of the NIPS 2015 Workshop

[dropcap]Probabilistic[/dropcap] inference lies no longer at the fringe. The importance of how we connect our observed data to the assumptions made by our statistical models—the task of inference—was a central part of this year's Neural Information Processing Systems (NIPS) conference: Zoubin Ghahramani's opening keynote … Continue reading A Year of Approximate Inference: Review of the NIPS 2015 Workshop

Machine Learning Trick of the Day (1): Replica Trick

'Tricks' of all sorts are used throughout machine learning, in both research and in production settings. These tricks allow us to address many different types of data analysis problems, being roughly of either an analytical, statistical, algorithmic, or numerical flavour. Today's trick is in the analytical class and comes to us from statistical physics: the popular Replica trick. The replica trick [cite key="engel2001statistical"][cite key="sharp2011effective"][cite key="opper1995statistical"] is used for analytical computation of log-normalising constants (or log-partition functions). More formally, the replica trick provides one of the tools needed for a replica analysis of a probabilistic model — a theoretical analysis of the the properties and expected behaviour of a model. Replica … Continue reading Machine Learning Trick of the Day (1): Replica Trick

A Statistical View of Deep Learning: Retrospective

Over the past 6 months, I've taken to writing a series of posts (one each month) on a statistical view of deep learning with two principal motivations in mind. The first was as a personal exercise to make concrete and to test the limits of the way that I think about, and use deep learning in my every day work. The second, was to highlight important statistical connections and implications of deep learning that I do not see being made in the popular courses, reviews and books on deep learning, but which are extremely important to keep in mind. Post Links and Summary Links to each post with a short … Continue reading A Statistical View of Deep Learning: Retrospective