What do Variational Autoencoders (VAEs), Diffusion Models behind generative AI and fuzzy clustering have in common? The answer lies in a single, elegant framework for fitting probability distributions to data.
We’ll show how to fit complex data with a mixture model. We’ll be using an energy-based formulation, drawing surprising analogies from thermodynamics. Discover the variational inequalities that form the mathematical backbone of these diverse methods, from the classic Expectation-Maximization algorithm to the cutting-edge models that generate stunning images from noise.

Plancks law for black body radiation explains the color of a glowing hot object, the light from our sun, and even the energy radiated by the Earth. This post offers a complete derivation of that famous equation for black body radiation, starting from the foundational principles of thermodynamics. We build the formula from the ground up, relying on just a few clearly stated postulates. You will see how a profound physical result emerges from a surprisingly simple and direct thermodynamical approach.

A natural derivation of the entropy from maximum likelihood.

The fundamental mathematical ideas behind thermodynamics

Some notes on the elementary properties of the exponential families