AI resources

Here are a few AI-related resources I've produced. More may appear here eventually.

• Slides from a short expository talk called "A mathematical introduction to the transformer architecture" are here.

• I've been consuming Andrej Karpathy's beautiful Neural Networks: Zero to Hero lecture series, which starts essentially "from scratch" and arrives at (a baby version of) GPT-2 in lecture 7. It's really great: he goes through both the math and the code, and crucially he also explains the journey of how we (as a species) arrived at these various design decisions -- a sort of potted history of AI in the modern era. To solidify my own understanding, I've been writing notes on both the math and the journey (omitting the code), adding a bit of mathematical background and rigor along the way. You can see an incomplete draft here. If you'd like to be notified when the notes are complete, please fill out my subscription form.