STATS / DATA SCI 315, Winter 2024

This is an introductory deep learning course using the Python programming language and the TensorFlow deep learning library.

• Textbook: We will not follow any one textbook too closely. Here are a few references we will use:
  • Dive into Deep Learning by Zhang, Lipton, Li and Smola. An advanced text from research scientists at Amazon. It weaves together math, figures, and code in an interactive online resource available here. Code examples are provided in three frameworks: MXNet, PyTorch and TensorFlow.
  • Deep Learning with Python (2nd edition) by Chollet. A solid hands-on guide oriented towards programmers from the creator of the Keras deep learning library. Ebook and print versions are available from Manning Publications
  • Deep Learning by Goodfellow, Bengio and Courville. Written by three top deep learning researchers, this comprehensive book is required reading if you want to pursue your study of deep learning at a more advanced level. Print version is available from MIT Press and an online version is here.
  • Understanding Deep Learning by Simon J.D. Prince. It promises to be a more up-to-date version of Deep Learning by Goodfellow et al. and its spiritual successor. Print version is available from MIT Press and an online version is here.
  • Deep Learning: Foundations and Concepts by Christopher M. Bishop and Hugh Bishop. Bishop’s earlier machine learning textbooks (Neural Networks for Pattern Recognition and Pattern Recognition and Machine Learning) have been extremely popular. The “new Bishop” (as Yann LeCun puts it) will be no different! Online version available here and ebook and printed version can be obtained via Springer.
• Undergraduate Courses on Deep Learning: Many universities now offer an introductory deep learning course, e.g., Berkeley, CMU, MIT, Stanford
• Canvas: You should access the Canvas class page for this course frequently. It will let you access important announcements and track course deliverables. (requires UM login)
• Slack: The course slack workspace is at um-wn24-stats315.slack.com (requires UM login)
• Days and Times: Tuesdays and Thursdays, 1-2:30
• Location: 170 WEISER (links for virtual lectures, if any, will be saved in the syllabus page on canvas)

Instructor Information

Name: Ambuj Tewari
Office Hours: 45 minutes before and after every class (please send me an email or slack DM in advance)
Email: tewaria@umich.edu

GSI Information

Name: Sahana Rayan (Lab 002 Thu 8:30-10:00 in 335 WH)
Email: srayan@umich.edu

Name: Jacob (Jake) Trauger (Lab 003 Thu 2:30-4:00 in 2234 USB)
Email: jtrauger@umich.edu

Name: Abhiti Mishra (Lab 004 Thu 4:00-5:30 in 1084 EH)
Email: abhiti@umich.edu

Lab webpage (also has GSI office hours info): link

Grading

• Canvas quizzes (20%): Timed, multiple choice, open book. Will drop two lowest scores
• Homeworks (30%): Assigned roughly every other week. Will drop one lowest score
• Midterm Exam (20%): In class, timed, multiple choice, open book
• Final Exam (30%): In class, timed, multiple choice, open book

Academic Integrity

The University of Michigan community functions best when its members treat one another with honesty, fairness, respect, and trust. The college promotes the assumption of personal responsibility and integrity, and prohibits all forms of academic dishonesty and misconduct. All cases of academic misconduct will be referred to the LSA Office of the Assistant Dean for Undergraduate Education. Being found responsible for academic misconduct will usually result in a grade sanction, in addition to any sanction from the college. For more information, including examples of behaviors that are considered academic misconduct and potential sanctions, please see https://lsa.umich.edu/lsa/academics/academic-integrity.html

Accommodation for Students with Disabilities

If you think you need accommodation for a disability, please let me know at your earliest convenience. Some aspects of this course, the assignments, the in-class activities, and the way the course is usually taught may be modified to facilitate your participation and progress. As soon as you make me aware of your needs, we can work with the Office of Services for Students with Disabilities (SSD) to help us determine appropriate academic accommodations. SSD (734-763-3000; http://ssd.umich.edu/) typically recommends accommodations through a Verified Individualized Services and Accommodations (VISA) form. Any information you provide is private and confidential and will be treated as such.

Mental Health and Well-Being

Students may experience stressors that can impact both their academic experience and their personal well-being. These may include academic pressures and challenges associated with relationships, mental health, alcohol or other drugs, identities, finances, etc. If you are experiencing concerns, seeking help is a courageous thing to do for yourself and those who care about you. If the source of your stressors is academic, please contact me so that we can find solutions together. For personal concerns, U-M offers a variety of resources, many which are listed on the Resources for Student Well-being webpage. You can also search for additional well-being resources here.

Schedule

DLPy = Deep Learning with Python (2nd edition) by Chollet
DL = Deep Learning by Goodfellow, Bengio and Courville
D2L = Dive into Deep Learning by Zhang, Lipton, Li and Smola
UDL = Understanding Deep Learning by Prince

Note: A “V” in the date column denotes a virtual lecture.

Date	Topic	Reading Assignment
Jan 11	Course logistics Introduction slides	DLPy What is deep learning?, Chap. 1 DL Introduction, Chap. 1 D2L Introduction, Chap. 1
	Linear Algebra Boot Camp
Jan 16	Linear Algebra notebook	D2L Geometry and Linear Algebraic Operations, Sec. 22.1.1-2
Jan 18	Linear Algebra (continued) notebook	D2L Geometry and Linear Algebraic Operations, Sec. 22.1.3-5
Jan 23 V	Linear Algebra (continued) notebook	D2L Geometry and Linear Algebraic Operations, Sec. 22.1.6-7
	Basics
Jan 25 V	Basic Elements of Linear Regression slides	D2L Linear Regression, Sec. 3.1.1
Jan 30 Feb 01	Regression Loss functions and gradient descent slides	D2L Linear Regression, Sec. 3.1.1
Feb 06	Regression wrap-up slides	D2L Linear Regression, Sec. 3.1.3-4
Feb 08	Classification Softmax Operation Cross Entropy Loss Function slides	D2L Softmax Regression, Sec. 4.1.1 D2L Loss Function, Sec. 4.1.2
Feb 13	Softmax Derivatives Information Theory Basics slides	D2L Information Theory Basics, Sec. 4.1.3
	TensorFlow/Keras
Feb 15	TensorFlow, Keras, Google Colab notebook	DLPy, Sec. 3.1-4 DLPy, Sec. 3.5.1-2
Feb 20	First steps with TensorFlow notebook	DLPy, Sec. 2.4.4 DLPy, Sec. 3.5.3-4
Feb 22	STUDY DAY
Feb 27	SPRING BREAK
Feb 29	SPRING BREAK
Mar 05	MIDTERM EXAM
	Fully Connected aka Dense Neural Networks
Mar 07	Getting started with NNs: Classification MNIST notebook	DLPy, Sec. 2.1
Mar 12	Getting started with NNs: Classification IMDB notebook Getting started with NNs: Regression Boston Housing Price notebook	DLPy, Sec. 4.1 DLPy, Sec. 4.3
Mar 14	Generalization Evaluating ML models notebook	DLPy, Sec. 5.1-2
Mar 19	NO CLASS (MIDAS Symposium)
Mar 21	Improving model fit Regularizing your model notebook	DLPy, Sec. 5.3 DLPy, Sec. 5.4.4
	Convolutional Neural Networks
Mar 26	From Fully-Connected Layers to Convolutions notebook	D2L, Sec. 7.1
Mar 28	Convolutions for Images notebook	D2L, Sec. 7.2
Apr 02	Padding and Stride notebook Multiple Input and Multiple Output Channels notebook Pooling notebook LeNet Different ways to build Keras models notebook	D2L, Sec. 7.3-6 DLPy, Sec. 7.2
	Transformers / Machine Olfaction
Apr 04	Stats Smellathon An Introduction to Sensory Evaluation	Guest Lecture by Michelle Krell Kydd, trained “nose” in flavors and fragrances TEDx talk Blog
Apr 09	Transformers slides	UDL, Sec. 12.1-3
Apr 11	Transformers slides	UDL, Sec. 12.4-5 UDL, Sec. 12.7
Apr 16	Mapping Scent We’ll go through three sets of results showing that a) olfaction is a sense that can be mapped, albeit only in high-dimensions, b) this map can be used to predict human olfactory responses to molecules at superhuman accuracy, c) the same map works across nearly all species studied by olfactory neuroscience and d) a possible explanation for the broad evolutionary applicability of the map may be explained by an unappreciated link between biological metabolism and scent.	Guest Lecture by UM alum Alex Wiltschko, Founder and CEO of osmo.ai
Apr 18	STUDY DAY
Apr 23	FINAL EXAM