Softmax and Temperature
In this video, I explain the softmax function and the concept of temperature in the context of language models and neural networks. I start by revisiting the Markov chain coding challenge, then break down how softmax converts raw neural network outputs (logits) into probabilitie ...
Wikipedia article on the softmax function.
Interactive visual explanation of Markov chains by Victor Powell.
Visualization of Google's tri-gram data showing word frequency relationships.
Wikipedia article on Euler's number (e ≈ 2.718).
p5.js reference for the exp() function (Euler's number raised to a power).
Official documentation for the Transformers.js library.
The language model used in the demo.
Chapter of book that covers randomness, weighted selection, and evolutionary computing.
First video in this series covering how to load and use transformers.js.
The original Markov chain coding challenge revisited in this video.
Video explaining the weighted selection algorithm used for sampling.
Train an image classifier with Teachable Machine and use it in p5.js with ml5.js.
- Editing
by Mathieu Blanchette
- Animations
by Jason Heglund