The spelled-out intro to language modeling: building makemore

推荐理由

It's long (2h), but well worth the time if anyone is curious about the technical workings of LLMs.

强烈推荐islewis · 原文2024年2月

说明

这是2022年由Andrej Karpathy发布的一部教程，主题归在Machine learning & deep learning。它讲的是language modeling这一支里的具体问题，而不是把整个领域扫一遍。适合第一次接触、还没有先修课的人。内容上可以概括成：Andrej Karpathy builds a bigram character-level language model from scratch, introducing torch.Tensor and the framework of training, sampling and loss evaluation. It is the first step toward a GPT-style Transformer in his makemore series.这些读者给出的理由，核心是：HN recommends it as a long but worthwhile way to demystify how language models work technically.讲法不绑某一条新闻，过几年仍然可以当材料看。从2024年到2024年，有2位不同的读者在2个不同月份把它推荐出来，其中明确叫别人去看的有2次。适合打开电脑跟着做的人。如果只听不练，这集的价值会少一半。如果已经会了基本定义，就把注意力放在它怎么把language modeling变成能跟着复述的例子。

视频信息

频道

Andrej Karpathy

时长

1 小时 57 分

发布年份

2022

同主题更多

• The spelled-out intro to neural networks and backpropagation: building micrograd
• But what is a neural network? | Deep learning chapter 1
• AlphaGo - The Movie | Full award-winning documentary
• Transformers, the tech behind LLMs | Deep Learning Chapter 5
• USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?
• Lecture 1 | Machine Learning (Stanford)