LLM Lectures
- https://cmu-llms.org/schedule/
- https://www.phontron.com/class/lminference-fall2025/schedule
- https://llmsystem.github.io/llmsystem2026spring/docs/Syllabus
- https://llmsystem.github.io/llmsystem2025spring/docs/Syllabus/
The following content is generated by LLMs and may contain inaccuracies.
Context
This collection addresses the growing need for structured educational resources in large language model (LLM) development and deployment. As LLMs transition from research artifacts to production systems, practitioners require deep understanding across the full stack—from GPU programming and transformer architecture to distributed training and inference optimization. These courses from CMU and related institutions represent the maturation of LLM education, bridging theoretical foundations with systems engineering concerns that arise at scale.
Key Insights
-
Curriculum divergence reflects specialization paths: The CMU LLM Applications course emphasizes prompt engineering, RAG systems, and domain-specific applications (healthcare, code generation), while the LLM Systems courses dive into GPU kernel optimization, distributed training strategies (Megatron-LM, ZeRO), and serving infrastructure (vLLM, FlashAttention). This split mirrors industry roles—application engineers who orchestrate LLMs versus systems engineers who make them computationally feasible.
-
Hardware-algorithm co-design emerges as core competency: Multiple syllabi feature guest lectures from creators of foundational systems: Tri Dao on FlashAttention, Woosuk Kwon on vLLM’s PagedAttention, Hao Zhang on DistServe. This signals that modern LLM work requires understanding memory hierarchies and attention mechanisms simultaneously—algorithmic improvements are inseparable from hardware constraints.
-
From monolithic models to modular architectures: The progression from basic transformers to mixture-of-experts (DeepSeek-MoE), disaggregated serving (DistServe), and retrieval augmentation reflects the field’s shift toward composable systems. The LLM Inference course likely extends this toward inference-specific optimizations like speculative decoding and KV cache management.
Open Questions
- How should curricula balance depth in classical ML theory versus hands-on systems optimization as LLM architectures continue evolving? Will today’s FlashAttention become tomorrow’s deprecated technique?
- What pedagogical approaches best prepare students for the lag between academic research and production deployment, especially when industry systems (SGLang, vLLM) advance faster than publication cycles?
LLM讲座
- https://cmu-llms.org/schedule/
- https://www.phontron.com/class/lminference-fall2025/schedule
- https://llmsystem.github.io/llmsystem2026spring/docs/Syllabus
- https://llmsystem.github.io/llmsystem2025spring/docs/Syllabus/
以下内容由 LLM 生成，可能包含不准确之处。
背景
这个资源集合应对了大语言模型（LLM）开发和部署中日益增长的结构化教育资源需求。随着LLM从研究工件过渡到生产系统，从业者需要掌握整个技术栈的深入知识——从GPU编程和变换器架构到分布式训练和推理优化。来自CMU及相关机构的这些课程代表了LLM教育的成熟发展，在理论基础与大规模系统工程问题之间架起了桥梁。
关键洞察
-
课程分化反映了专业化路径：CMU LLM应用课程强调提示工程、RAG系统和特定领域应用（医疗保健、代码生成），而LLM系统课程深入探讨GPU内核优化、分布式训练策略（Megatron-LM、ZeRO）和服务基础设施（vLLM、FlashAttention）。这种分化反映了行业角色差异——应用工程师编排LLM，而系统工程师使其在计算上可行。
-
硬件-算法协同设计成为核心能力：多个课程大纲特别邀请了基础系统创始人进行讲座：Tri Dao讲FlashAttention、Woosuk Kwon讲vLLM的PagedAttention、Hao Zhang讲DistServe。这表明现代LLM工作需要同时理解内存层次结构和注意力机制——算法改进与硬件约束密不可分。
-
从单体模型到模块化架构：从基础变换器到专家混合模型（DeepSeek-MoE）、分解服务（DistServe）和检索增强的进展，反映了该领域向可组合系统的转变。LLM推理课程可能会进一步扩展到推理特定的优化，如推测解码和KV缓存管理。
待解问题
- 随着LLM架构不断演进，课程应如何平衡经典ML理论的深度与实践系统优化？今天的FlashAttention会成为明天的过时技术吗？
- 什么样的教学方法能最好地为学生准备应对学术研究与生产部署之间的滞后，特别是当行业系统（SGLang、vLLM）的进度快于发表周期时？