CS40008.01 LLMs · Fall 2026 · Fudan University

Course overview

Learn how modern LLMs work: tokenization, language modeling, embeddings, attention, and Transformers; training and evaluation; data quality, compute costs, and retrieval. Learn to design controlled experiments and investigate failures. Materials are mainly online, drawing on my previous lectures and Stanford CS336. Each student explores one meaningful question through an individual project. AI assistance is welcome; verify results, explain decisions, and support conclusions with evidence.

Prerequisites: Python and introductory ML, including probability and matrix operations. Each meeting spans three 45-minute periods with explanations, worked experiments, guided practice, and occasional quizzes. We will use GitHub.

Basic info

Class meetings

Time: Wednesdays 13:30–16:10 (periods 6–8)
Location: HGX103, Handan Campus
Dates: Sep 9 – Dec 23, 2026

People

Instructor: Baojian (bjzhou AT fudan.edu.cn)
TAs: Wang Yuxiang and Zhou Yu
Office hours: Mon. 14:00–15:30

Coursework

Quizzes

Five 15-minute quizzes; all count equally (2% each).

Quiz 1 · Sep 23 (Week 3)
Quiz 2 · Oct 10 (Week 5)
Quiz 3 · Oct 21 (Week 7)
Quiz 4 · Nov 11 (Week 10)
Quiz 5 · Dec 9 (Week 14)

Assignments

All count equally (15% each); submission dates are listed below.

A1 · Tokenization and language models
Due Sep 30, 23:59 (Asia/Shanghai)
A2 · Build and investigate a small language model
Due Oct 31, 23:59 (Asia/Shanghai)
A3 · Adapt and evaluate · Dec 2 (Week 13)

Assignment announcements

Individual project

Candidate projects released · Oct 10 (Week 5)
Proposal · Oct 21 (Week 7)
Progress update · Nov 25 (Week 12)
Final submission · Dec 30 (Week 17)

All deadlines are at 23:59 (Asia/Shanghai). The October 7 class meets on Saturday, October 10; time and room follow the university make-up notice. All project checkpoints are written submissions through eLearning. No project presentations, posters, or oral defenses are required.

Weekly schedule

Before opening local slides or notebooks, run uv sync, then uv run python scripts/slides.py serve. Weekly tasks · Progress board

Week	Date	Topic and question	Content and materials	Milestone
1	Sep 9	Introduction and tokenizationHow does the representation of text affect cost and what a model can learn?	Brief introduction to LLMs; bytes and Unicode; BPE; special tokens; vocabulary choices for Chinese and English. Practice: Trace BPE merges, check encode/decode round trips, and compare Chinese and English tokenization at different vocabulary sizes. Slides: lecture-01-slides Reading: Text preprocessing and tokenizer pipelines Exercise: lecture-01-exercise.ipynb Extended practice: lecture-01-exercise-tokenization.ipynb Papers: BPE (Sennrich et al., 2016) · SentencePiece (Kudo and Richardson, 2018) · Tokenizer reading list · Interactive version Survey results: Which LLM apps do you use most? · counts	—
2	Sep 16	N-gram language modelsHow do we build, evaluate, and use a language model?	N-gram LMs and smoothing; held-out loss, perplexity, and bits per byte; sampling; reference-model data filtering; neural probabilistic LMs. Practice: Estimate a bigram model, compare evaluation metrics, tune interpolation on development data, and sample from a smoothed model. Slides: lecture-02-slides Exercise: lecture-02-exercise.ipynb Reading: Language-model metrics: loss, perplexity, and bits per byte References: Jurafsky and Martin, Chapter 3: N-gram Language Models	A1 released Announcement and questions Due Sep 30, 23:59 (Asia/Shanghai)
3	Sep 23	Embeddings and PyTorch for language modelsHow do discrete tokens become trainable representations?	Embedding lookup; distributional hypothesis; one skip-gram example; tensor shapes, batching, autograd, and output projections. Practice: Trace tensors and gradients through a small language model; compare parameter counts with and without weight sharing. Slides: lecture-03-slides Exercise: lecture-03-exercise.ipynb References: Jurafsky and Martin, Chapter 5: Embeddings · CS336 Lecture 2: PyTorch and resource accounting · Weight tying (Press and Wolf, 2017)	Quiz 1
4	Sep 30	Neural LMs and Recurrent AttentionHow can a model retain and select useful context?	Feedforward LM training; recurrent states and gradients; LSTM memory; additive alignment in an RNN encoder–decoder. Practice: Fit a tiny batch, trace recurrent memory and gates, and calculate a weighted source context. Slides: lecture-04-slides Exercise: lecture-04-exercise.ipynb References: Bengio et al. (2003), Section 2 · Bahdanau et al. (2014/2015), Sections 2–3 · Pascanu et al. (2013), recurrent gradients	A1 due 23:59 (Asia/Shanghai) Submit on eLearning
5	Oct 10 Saturday	Attention and the TransformerHow does attention select useful context, and how does it lead to the Transformer?	Brief recurrent-alignment recap; queries, keys, values, and scaled dot products; causal self-attention; multi-head attention, positions, and Transformer blocks. Practice: Compute attention weights and outputs on a small example, verify causal masking, and trace shapes through a decoder block. Slides: lecture-05-slides Exercise: practice.ipynb Extended practice: lecture-05-exercise.ipynb References: Bahdanau et al. (2015), Section 3 · Luong et al. (2015), Section 3 · Vaswani et al. (2017), Sections 3–4	Quiz 2; A2 released Candidate projects released
6	Oct 14	Pretraining and decoding in practiceWhat evidence shows that a training run is working?	Data batches and shifted targets; AdamW; learning-rate schedules; gradient clipping; checkpoints; train/validation curves; greedy, temperature, and top-p decoding. Practice: Run and resume a small GPT training job; use training curves to diagnose instability. Slides: lecture-06-slides Exercise: practice.ipynb References: Loshchilov and Hutter, AdamW · Holtzman et al., nucleus sampling · Reading guide	—
7	Oct 21	Data preparation and qualityCan better data improve a fixed model under a fixed training budget?	Text extraction; source documentation; quality filtering; exact and near duplicates; language mixtures; contamination and split construction. Practice: Compare filtering, deduplication, or language mixtures under a fixed model, tokenizer, and training-token budget. References: To be updated.	Quiz 3 Project proposal due
8	Oct 28	Compute budgets and scalingHow should we spend a limited GPU budget?	Parameter, gradient, activation, and optimizer memory; approximate FLOPs; batch and sequence length; compute versus memory bottlenecks; model-size/data tradeoffs. Practice: Profile a small run's memory and compute; fit supplied scaling results and discuss uncertainty. References: To be updated.	A2 due Oct 31
9	Nov 4	Evaluation and experimental designDoes a measured improvement answer the question we care about?	Task and metric selection; loss versus downstream performance; prompt sensitivity; uncertainty; leakage; error analysis. Use a BERT classification example and one decoder evaluation. Practice: Audit an evaluation set, test a claim, and examine uncertainty or evaluator disagreement. References: To be updated.	—
10	Nov 11	Supervised fine-tuningWhen does task-specific training improve a pretrained model?	Instruction data; chat templates; response-token loss masking; optimization and forgetting; evaluate before and after adaptation. Practice: Adapt a supplied pretrained model using parameter-efficient tuning or a data-quality comparison; inspect improvements and regressions. References: To be updated.	Quiz 4; A3 released
11	Nov 18	Preferences, rewards, and alignmentWhat happens when the optimization target differs from what we actually want?	Preference pairs and reward models; DPO intuition and loss; RLHF/RLVR overview; verifiable rewards; reward hacking and safety evaluation. Practice: Compute a DPO loss and inspect a preference-learning or verifiable-reward example for reward hacking. References: To be updated.	—
12	Nov 25	Retrieval and retrieval-augmented generationWhen should a system retrieve evidence, and when should it abstain?	TF-IDF/BM25; dense retrieval; reranking; chunking; context construction; retrieval recall, answer support, and unanswerable questions. Practice: Compare retrieval or chunking choices in a course-notes RAG system; distinguish retrieval failures from generation failures. References: To be updated.	Project progress update due
13	Dec 2	Efficient inferenceWhich speed improvement matters for a particular workload?	Prefill and decoding; KV caching; batching; latency versus throughput; quantization tradeoffs; FlashAttention intuition. Practice: Profile a cache or batching change; compare speed and memory while checking output correctness. References: To be updated.	A3 due
14	Dec 9	Diffusion language modelsWhat changes when generation can update multiple token positions?	Review BERT masked prediction; corruption and iterative unmasking; compare sampling order, conditioning, and compute with autoregression. State assumptions behind the connections. Practice: Compare masked-generation samplers in a supplied demo, measuring quality and latency. References: To be updated.	Quiz 5
15	Dec 16	Agents and tool useWhen does a tool-use loop improve an outcome enough to justify its cost?	Bounded tool use; stopping conditions; trace inspection; prompt injection and tool permissions; compare with direct generation. Practice: Compare an agent with direct generation and inspect its tool-use trace, stopping behavior, and cost. References: To be updated.	—
16	Dec 23	Research synthesis and project revisionWhich claims are supported by our experiments, and what still needs to be checked?	Connect course methods to project findings; inspect baselines, controlled comparisons, uncertainty, and failure cases. Use written feedback to revise the final report and experiment record. Practice: Trace one project claim to its evidence, identify a limitation, and plan the final revision. References: To be updated.	—
17	Dec 30	Exam week	No lecture. Submit the individual project.	Final report, code, and experiment record due

There are 16 teaching sessions, including the Saturday make-up class on October 10. References will remain available on this course page. December 23 is reserved for research synthesis and project revision. Individual project feedback is provided through written comments and office hours.

Assessment

Quizzes 10%Assignments 45%Individual project 45%

Quizzes: Quizzes assess the common core already taught, using brief explanations, calculations, and interpretation of unfamiliar examples. They are closed-book, and AI tools are not allowed. All five quizzes count, each contributing 2% of the course grade. Advanced extensions are not tested.

Assignments: Assignments establish common skills and can seed the individual project. Reuse the code and data preparation, but identify the additional question and experiments credited to the project. Staff reference code/checkpoints are planned so an early implementation failure does not block later work. The tiny model in A2 teaches training mechanics; the supplied pretrained model supports meaningful adaptation experiments. The three practical assignments are:

A1 · Tokenization and language models (15%). Released September 16; due September 30, 2026, 23:59 (Asia/Shanghai). Build a byte-level BPE tokenizer, smoothed n-gram baselines, and a small neural language model; evaluate on held-out English and Chinese text. Handout, starter code, and data · Announcement and questions. Submit privately through eLearning.
A2 · Build and investigate a small language model (15%). Released October 10; due October 31, 2026, 23:59 (Asia/Shanghai). Build a causal Transformer, verify training and checkpoint restart, compare decoding settings, and investigate exact-document deduplication at a fixed training budget. Handout, starter code, and data. Submit privately through eLearning.
A3 · Adapt and evaluate (15%). Released November 11; due December 2, 2026, 23:59 (Asia/Shanghai). SFT on a supplied pretrained checkpoint with correct response-token masking; compare with the unadapted model on a held-out task. Inspect both improvements and regressions. DPO or GRPO is an extension.

Individual project: One project per student, centered on an important, interesting, and answerable question. Ambitious AI-assisted work is welcome. A strong project develops evidence through successive experiments and explains what changed in the student's understanding.

Propose your own project or explore 48 candidates · October 10, 2026. Start with a topic that interests you, or browse recent papers, concrete experiments, and open-source contribution opportunities. Student-proposed projects follow the same individual-project requirements and rubric.

Choosing a question

Importance: who needs the answer, and which decision could it change?
Uncertainty: what do prior results leave unclear, and what competing explanations are plausible?
Evidence: which baseline, dataset, metric, and controlled comparison can distinguish those explanations?
Feasibility: what can be learned with the available compute, data, and semester time?

Projects may investigate data, tokenization, training, evaluation, adaptation, retrieval, inference, diffusion, or agents. Building a system, reproducing a result, and analyzing a failure are all valid approaches. Novelty is not required for the common core. A useful negative result can receive full credit when the experiment and analysis are sound.

Survey-only projects are not allowed. Every project must include an implementation, experiment, or tested artifact with a meaningful comparison and analysis. Students may use their own hardware, laboratory resources, cloud credits, hosted APIs, or other resources they can obtain. Document their source and cost separately from any course-pool request; the same rubric applies to everyone.

Example questions: Does filtering improve a fixed model under a fixed token budget? When should a course-notes RAG system abstain? How does a decoding policy change accuracy and cost? These are starting points to narrow after reading related work, not claims of novelty.

Written checkpoints

Week 7 · Proposal (one page), due October 21. Motivation, relevant prior work, an uncertain question, baseline, evaluation plan, and approximate compute budget. Later-topic projects may use a supplied baseline before those lectures occur.
Week 12 · Progress update (up to two pages), due November 25. A working baseline, inspected evaluation examples, initial measurements, and one controlled comparison. Discuss a finding or failure, a competing explanation, and the next experiment. Staff provide written feedback; incorporate it in the final report.
Week 17 · Final submission, due December 30. The main report is limited to six pages; references and appendices have no page limit. Include code and a concise experiment record. Use the ACL template. Checkpoint text can be revised into the report.

No slides, posters, oral defenses, or group deliverables are required. Peer discussion is welcome; each student submits and is assessed on their own work.

Experiment record and proposed rubric

Record the configurations, data versions/splits, metrics, and artifact locations needed to trace the main results. Include a brief account of material AI assistance and key decisions: what was proposed, how it was checked, what was adopted or revised, and why. A complete prompt transcript is not required.

Criterion	Share of project grade	Evidence
Question and motivation	20%	Significance, prior work, and a clear uncertainty
Experimental design	25%	Baselines, controls, metrics, and appropriate comparisons
Evidence and reproducibility	25%	Correct implementation, traceable runs, and inspected results
Interpretation and iteration	20%	Failure analysis, alternative explanations, limitations, and follow-up experiments
Written communication and checkpoints	10%	Clear report, timely progress records, and transparent AI use

Policies

AI assistants: AI assistants are allowed for coding, debugging, and literature exploration in practical assignments and projects. Students must verify sources, inspect data and outputs, validate key computations, and take responsibility for submitted conclusions. Disclose material AI use and distinguish measured results from proposed or generated examples. Quizzes and explicitly designated concept checks are completed without AI. Other guided practical work may use AI.

Academic integrity and submission: Follow the course academic-integrity policy. Attribute reused code, data, and ideas. Fabricated results and unattributed copying are misconduct. Submit work through eLearning by 23:59 on the listed date.

References and computing resources

References:

Spring 2026 Fudan course: reusable lecture explanations and notebooks.
Stanford CS336, Spring 2026: our practicals will use local, reduced-scale adaptations.
Speech and Language Processing, Jurafsky and Martin: selected chapters matched to each foundational topic.
MIT Flow Matching and Diffusion Models: optional advanced study.

Compute and GitHub: Register for Fudan CFFF; GPU jobs use Qizhi. Per-student GPU and AI-tool budgets will be specified with the project handout. Core exercises are planned for a laptop or one GPU, with supplied checkpoints for costly experiments. Exact model sizes and run budgets will be set after staff pilots. Keep personal model weights, data, logs, and generated results in ignored storage, not in the course repository. Course-provided materials will identify their source and required environment.