Machine Learning Trick of the Day (5): Log Derivative Trick 5

Machine Learning Trick of the Day (5): Log Derivative Trick
Machine learning involves manipulating probabilities. These probabilities are most often represented as normalised-probabilities or as log-probabilities. An ability to shrewdly alternate between these two representations is a vital step towards strengthening the probabilistic dexterity we need to solve modern machine learning problems. Today's trick, the log derivative trick, helps us to do just this, using the property of the derivative of the logarithm. This ...

Machine Learning Trick of the Day (4): Reparameterisation Tricks 4

Machine Learning Trick of the Day (4): Reparameterisation Tricks
Our ability to rewrite statistical problems in an equivalent but different form, to reparameterise them, is one of the most general-purpose tools we have in mathematical statistics. We used reparameterisation in all the tricks we explored in this series so far: trick 1 re-expressed a log-partition function in terms of copies (replicas) of the marginal probability, trick 2 re-expressed a ...

Machine Learning Trick of the Day (3): Hutchinson's Trick 4

Machine Learning Trick of the Day (3): Hutchinson's Trick
Hutchinson's estimator  is a simple way to obtain a stochastic estimate of the trace of a matrix. This is a simple trick that uses randomisation to transform the algebraic problem of computing the trace into the statistical problem of computing an expectation of a quadratic function. The randomisation technique used in this trick is just one from a ...