LLMs and Transformers

STATS 700, Fall 2024

The Attention mechanism and the Transformer architecture have completely changed the landscape of AI, deep learning, and NLP research in the past few years. This advanced graduate level course will consist of two parts. In the first part, we will review foundational material in information theory, statistical NLP, and deep learning theory. In the second part, student project teams will explore topics from the fast growing literature on Transformers and Large Language Models (LLMs) especially papers that provide theoretical and mathematical analyses. Topics include, but are not limited to:

• limitations and capabilities of the transformer architecture
• neural scaling laws and emergence of skills as the models are scaled up in size
• issues around copyright, trust, safety, fairness, and watermarking
• alignment to human values and techniques such as RLHF (reinforcement learning with human feedback)
• adaptation of LLMs to downstream tasks via few shot fine-tuning and in-context learning.

If there is time, we might look at the impact that LLMs are having in disciplines such as Cognitive Science, Linguistics, and Neuroscience. We might also discuss ongoing efforts to build LLMs and foundation models for science and mathematics.

Note: This course is primarily meant for Statistics PhD students. Others will need instructor’s permission to enroll. Graduate coursework in statistics, theoretical computer science, mathematics, or related disciplines required. Students will be expected to possess that hard-to-define quality usually referred to as “mathematical maturity”.

• Logistics & Schedule
  • Part 1
  • Part 2
• Courses / Blogs
• Classics
• Topics
  • Word Embeddings
  • Attention
  • Implicit Regularization
  • NTK Theory for Transformers
  • Reverse Engineering Transformers
  • Capabilities and Limitations of LLMs and Transformers
  • Beyond PAC Learning: Learning Distributions and Grammars, Learning to Generate
  • Emergence
  • In-Context Learning
  • Hallucinations
  • Assessing Model Uncertainty
  • RLHF
  • State Space Models
  • Transformers as Interacting Particle Systems
  • Language Modeling, Prediction, and Compression
  • LLMs, Online Learning, and Regret
  • Transformers in RL
  • LLMs and Causality
  • LLMs and Cognitive Science, Linguistics, Neuroscience
  • LLMs and Foundation Models for Science and Mathematics

Logistics & Schedule

Days and Times: Tuesdays and Thursdays, 11:30 am-1:00 pm
Location: USB2260

J&M = Speech and Language Processing (3rd ed. draft), Jurafsky and Martin
C&T = Elements of Information Theory (2nd ed.), Cover and Thomas

Part 1

• 8/27 Introduction slides
• 8/29 N-gram Language Models, J&M Chapter 3 annotated chapter
• 9/3, 9/5 Entropy, Relative Entropy, and Mutual Information, C&T Chapter 2, Sections 2.1-2.5 notes
• 9/10, 9/12 Entropy, Relative Entropy, and Mutual Information, C&T Chapter 2, Sections 2.6-2.10 notes
• 9/17, 9/19 Paper presentation notes
  • Representational Strengths and Limitations of Transformers
  • On Limitations of the Transformer Architecture
  • One-layer transformers fail to solve the induction heads task
• 9/19 (wrap up) Entropy, Relative Entropy, and Mutual Information, C&T Chapter 2, Sections 2.6-2.10 notes
• 9/24 Vector Semantics and Embeddings, J&M Chapter 6 annotated chapter
• 9/26, 10/1 Asymptotic Equipartition Property, C&T Chapter 3, Sections 3.1-3.3 notes
• 10/3 Entropy Rates of a Stochastic Process, C&T Chapter 4, Sections 4.1-4.3 notes
• 10/10 Project Pitches
• 10/15 FALL BREAK
• 10/17 Project Pitches
• 10/22 Entropy Rates of a Stochastic Process, C&T Chapter 4, Sections 4.4-4.5 notes
• 10/24 Guest Lecture by Vinod Raman: Generation through the lens of learning theory
• 10/28 Project Proposals Due

Supplementary Material

• Neural Networks, J&M Chapter 7 annotated chapter
• RNNs and LSTMs, J&M Chapter 8 annotated chapter
• Transformers, J&M Chapter 9 annotated chapter
• Large Language Models, J&M Chapter 10 annotated chapter

Part 2

• 10/29
  • T1
    • AI models collapse when trained on recursively generated data
    • The Curse of Recursion: Training on Generated Data Makes Models Forget
    • How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
  • T2
    • Prediction-Powered Ranking of Large Language Models
    • AutoEval Done Right: Using Synthetic Data for Model Evaluation
• 10/31
  • T4
    • The Last JITAI?
    • Thompson Sampling for Noncompliant Bandits
    • Neural Thompson Sampling
  • T5 On the Dimensionality of Word Embeddings
• 11/5
  • T9 SpectraTr: A novel deep learning model for qualitative analysis of drug spectroscopy based on transformer structure
  • T17 Distinguishing the Knowable from the Unknowable with Language Models
• 11/7
  • T12 Conformal Language Modeling
  • T13
    • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
    • Learning Transferable Visual Models From Natural Language Supervision
    • Indication as Prior Knowledge for Multimodal Disease Classification in Chest Radiographs with Transformers
• 11/12
  • T14
    • Large Language Models Are Zero-Shot Time Series Forecasters
    • Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
  • T15
    • AI agents that matter
    • Are more LLM calls all you need? Towards scaling laws of compound inference systems
• 11/14
  • T7 Improving Pretraining Data Using Perplexity Correlations
  • T16 Controllable Neural Symbolic Regression
• 11/19
  • T3
    • A statistical framework for weak-to-strong generalization
    • Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution
  • T6 Large Language Models as General Pattern Machines
• 11/21
  • T8
    • GNOT: A General Neural Operator Transformer for Operator Learning
    • Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context
  • T11 Repeat After Me: Transformers are Better than State Space Models at Copying
• 11/26
  • T10 Slaves to the Law of Large Numbers: An Asymptotic Equipartition Property for Perplexity in Generative Language Models
• 11/28 THANKSGIVING BREAK

Dec 3, Poster Session I: Understanding and Improving LLMs & Transformers

T1 Soham, Sunrit: Evolution of Iteratively Trained Generative AI Models
T2 Xuanyu, Yiling: Context-Aware Ranking of Large Language Models via Pairwise Comparison
T3 Elvin, Jason, Kellen, Mihir: Is Linear Probing better than Fine-tuning for Weak-to-Strong Generalization?
T5 Jake, Jaylin, Noah: Dimension Decisions: The Impact of Embedding Size on LLMs
T7 Eduardo, Felipe, Harry, Xinhe: shIRT: similarity heuristics for data selection using IRT
T10 Mojtaba, Tara: Beat LLMs in Their Own Game: Statistics-based Methods for AI Detection with Theoretical Guarantees
T11 Unique, Vinod: The Limitations of Self-Attention for Information Retrieval
T12 Paolo, Sahana: Budget-Constrained Learning to Defer for Autoregressive Models
T17 Yuezhou: Classifying Uncertainty with In-Context Learning Methods

Dec 5, Poster Session II: Applying LLMs & Transformers

T4 Gabe, Ki, Marc: Online RL Considerations for LLM-Assisted JITAIs
T6 Andrej: Modern autoregressive architectures for the collective variable problem
T8 Abhiti, Julian, Yash: Adaptive Spectral Neural Operators
T9 Joe: Applying Transformers to Spectral Data
T13 Qiyuan, Zhilin: A Unified Framework for Multimodal Learning: Integrating Image, Text, and Crowd-Sourced Annotations
T14 Victor: Forecasting Solar Flares Using Time Series Foundation Models and LLM-Based Models
T15 Jiwoo, Yue: AI Agent Evaluation
T16 Daniel: Incorporating Domain Knowledge in Transformer-based models for Symbolic Regression

Dec 13, Project Reports Due

Courses / Blogs

Stanford
Princeton
Berkeley
Michigan EECS
Borealis AI blog series:

• Intro to LLMs
• Transformers I
• Transformers II
• Transformers III
• Neural Natural Language Generation: Decoding Algorithms
• Neural Natural Language Generation: Sequence Level Training
• Training and fine-tuning LLMs
• Speeding up inference in LLMs

Basics

• Attention Is All You Need This is the paper that started the transformers revolution
• Formal Algorithms for Transformers
• The Illustrated Transformer
• The Annotated Transformer

Special Year on LLMs and Transformers at the Simons Institute, UC Berkeley:

• Part 1
• Part 2
• An earlier workshop

Classics

Markov (1913) An Example of Statistical Investigation of the Text Eugene Onegin Concerning the Connection of Samples in Chains
Shannon (1951) Prediction and Entropy of Printed English
Zipf (1935) The Psycho-Biology of Language Houghton, Mifflin. (Reprinted by MIT Press in 1965)
Good(-Turing) (1953) The Population Frequencies of Species and the Estimation of Population Parameters

Topics

Word Embeddings

Neural Word Embedding as Implicit Matrix Factorization
A Latent Variable Model Approach to PMI-based Word Embeddings
Skip-Gram – Zipf + Uniform = Vector Additivity
Analogies Explained: Towards Understanding Word Embeddings

Attention

What can a Single Attention Layer Learn? A Study Through the Random Features Lens
Inductive Biases and Variable Creation in Self-Attention Mechanisms

Implicit Regularization

Implicit Regularization of Gradient Flow on One-Layer Softmax Attention

NTK Theory for Transformers

Infinite attention: NNGP and NTK for deep attention networks
Tensor Programs II: Neural Tangent Kernel for Any Architecture
A Kernel-Based View of Language Model Fine-Tuning

Reverse Engineering Transformers

Transformer Circuits Thread Project
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small
Progress Measures for Grokking Via Mechanistic Interpretability

Capabilities and Limitations of LLMs and Transformers

On the Turing Completeness of Modern Neural Network Architectures
Are Transformers universal approximators of sequence-to-sequence functions?
Theoretical Limitations of Self-Attention in Neural Sequence Models
On the Ability and Limitations of Transformers to Recognize Formal Languages
Self-Attention Networks Can Process Bounded Hierarchical Languages
Transformers Learn Shortcuts to Automata
Representational Strengths and Limitations of Transformers
On Limitations of the Transformer Architecture
Transformers, parallel computation, and logarithmic depth
From Self-Attention to Markov Models: Unveiling the Dynamics of Generative Transformers
Slaves to the Law of Large Numbers: An Asymptotic Equipartition Property for Perplexity in Generative Language Models
One-layer transformers fail to solve the induction heads task
Thinking Like Transformers

Beyond PAC Learning: Learning Distributions and Grammars, Learning to Generate

On the Learnability of Discrete Distributions
Near-optimal Sample Complexity Bounds for Robust Learning of Gaussian Mixtures via Compression Schemes
Distribution Learnability and Robustness
Inherent limitations of dimensions for characterizing learnability of distribution classes
Grammatical Inference: Learning Automata and Grammars
Mathematical Linguistics especially Chapter 7 (Complexity) and Chapter 8 (Linguistic pattern recognition)
Language Generation in the Limit

Emergence

Are Emergent Abilities of Large Language Models a Mirage?
A Theory for Emergence of Complex Skills in Language Models

In-Context Learning

What Can Transformers Learn In-Context? A Case Study of Simple Function Classes
The Learnability of In-Context Learning
Supervised Pretraining Can Learn In-Context Reinforcement Learning
Large Language Models can Implement Policy Iteration
Trainable Transformer in Transformer
Trained Transformers Learn Linear Models In-Context
The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains

Hallucinations

Calibrated Language Models Must Hallucinate

Assessing Model Uncertainty

Distinguishing the Knowable from the Unknowable with Language Models
Conformal Language Modeling
Language Models with Conformal Factuality Guarantees

RLHF

Training language models to follow instructions with human feedback
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Efficient Exploration for LLMs

State Space Models

Mamba: Linear-time sequence modeling with selective state spaces
Repeat After Me: Transformers are Better than State Space Models at Copying

Transformers as Interacting Particle Systems

A mathematical perspective on Transformers

Language Modeling, Prediction, and Compression

On prediction by data compression
Language Modeling Is Compression
Prediction by Compression

LLMs, Online Learning, and Regret

Do LLM Agents Have Regret? A Case Study in Online Learning and Games

Transformers in RL

Transformers in Reinforcement Learning: A Survey

LLMs and Causality

Causal Reasoning and Large Language Models: Opening a New Frontier for Causality

LLMs and Cognitive Science, Linguistics, Neuroscience

Formal grammar and information theory: together again?
The debate over understanding in AI’s large language models
Language models and linguistic theories beyond words
Noam Chomsky: The False Promise of ChatGPT
Modern language models refute Chomsky’s approach to language
Dissociating language and thought in large language models
Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve
Shared computational principles for language processing in humans and deep language models
The neural architecture of language: Integrative modeling converges on predictive processing
Predictive Coding or Just Feature Discovery? An Alternative Account of Why Language Models Fit Brain Data

LLMs and Foundation Models for Science and Mathematics

On the Opportunities and Risks of Foundation Models
MIDAS Symposium 2024
MICDE Symposium 2024