Spring 2026 @ Fudan University
This course covers the foundations and modern frontiers of Natural Language Processing (NLP), with a heavy emphasis on Large Language Models (LLMs). You will learn the modern pipeline of building effective LLMs from basic tokenization to training, fine-tuning, and deploying modern LLM architectures.
All standard homework assignments are completed by Week 12. The final month (Weeks 13–16) is dedicated exclusively to the Course Project. Please submit your homework at https://elearning.fudan.edu.cn/
In our first lecture, we introduce text preprocessing, including tokenization (BPE/WordPiece), and vocab design.
- Thu., 1:30 pm – 4:10 pm, 03/05/2026
- Slides: Lecture 01 slides
- Readings:
- Excercise: lecture-01-exercise-tokenization.ipynb
Release Assignment 1
In this lecture, we introduce the concept of MLE, Smoothing, Perplexity, and Language Modeling basics.
In this lecture, we introduce text classification, Word2Vec, Distributional Hypothesis, and Intrinsic/Extrinsic evaluations.
- Slides: Lecture 03 slides
- Readings:
- Excercise: lecture-03-embeddings.ipynb
In this lecture, we introduce neural networks and how to build NN models for sequence learning problems. We will discuss some classic models like LSTM and how the encoder-decoder style models developed and why the attention is a effective component adding to encoder-decoder model.
- Slides: Lecture 04 slides
- Readings:
- Excercise: lecture-04-neural-lms.ipynb
In this lecture, we introduce the Transformer architecture.
Causal LM vs MLM, Chinchilla Scaling Laws, Data Mixtures
SFT, LoRA/QLoRA, Adapters, Instruction Tuning
Project Proposal Due
Benchmarks (MMLU/GSM8K), Contamination, LLM-as-a-judge
In-context learning, Chain-of-Thought, Prompt sensitivity
Dense Retrieval, Vector DBs, Reranking, Grounding
RLHF (PPO/DPO), Safety barriers, Red-teaming
KV Caching, Quantization (Int8/FP4), Latency/Throughput
Multimodal LLMs, Diffusion LMs, Future Directions