Notheastern CS7150 Deep Learning

Course Information

Deep learning is a subfield of machine learning that builds highly parameterized (deep) models from large volume of data, widely deployed in many industrial systems. Applications cover computer vision, language, speech, robotics, etc. Deep learning advances so rapidly that it has become one of the most popular field to study.

This course will introduce students to

1) basic building blocks of deep models, e.g., convolution, activation functions, attention

2) typical architectures, e.g., encoder-decoder for seq-to-seq tasks

3) training and evaluation of these models, e.g., variants of stochastic optimizers

Some cutting edge research topics will also be discussed, e.g., interpretability, model compression, self-supervised pretraining. There will be a course project that encourages student to explore more of these open-ended problems.

Class & Links

Lecture: Saturday 9:00 am - 12:20 pm, San Jose Campus 1010/1011

Office hours: Saturday 1:00 pm - 3:00 pm, San Jose Campus 1010/1011

TA: Pratyaksh Bhalla (bhalla.pr@northeastern.edu)

TA office hours: Friday morning (check with TA though)

Textbooks:

• Deep Learning by Goodfellow, Bengio and Courville

• Deep Learning, Foundations and Concepts by C. M. Bishop and H. Bishop

• Deep Reinforcement Learning by Plaat

Grade: 20% homework + 20% paper presentation + 20% midterm exam + 40% project

Homework: Due Friday 11:59pm

Paper presentation: Each student presents one paper in one of the classes. We reward good presenters and actively involved audience. The paper could be one of those suggested reading materials in the previous classes, or any recent paper identified as relevant by the presenter. There is no restriction/preference on form of the presentation. Slides may be used for clarity. Whiteboard can be used for derivations if necessary.

Khoury cloud: instruction1, instruction2

General Policies:

• Attend regularly: If you cannot make it, please email TA. Note: 3+ absence without extenuating reason (e.g., illness, attending conferences) can harm your grade

• Academic integrity (check here): In your homework or final project,

  • if you copied code somewhere from the web, please put a comment of the link

  • if you discussed with others, please acknowledge your collaborator

  • if you used chatgpt or any of its equivalence, please give details on how you prompted the language model.

Schedules

01/13

• Topics: Introduction, Prerequisites [slides] [recording 1, 2]

• Readings: textbook chapters linear algerbra, probabilities, PyTorch tutorial page

• Assignment: [homework1]

01/20

• Topics: Machine Learning Review, Classifiers, Multi-layer perceptron [slides] [recording] [Colab example]

• Readings: textbook chapter ML basics, hubless NN, deep vs shallow representations, ELU activation, GELU activation

• Assignment: [homework2]

01/27

• Topics: Convolutional network and computer vision [slides] [recording 1, 2]

• Readings: ResNet, DenseNet, visualization, transferability

• Assignment: [homework3]

02/03

• Topics: ngram, word2vec, RNN, LSTM, Transformer [slides] [recording 1, 2]

• Readings: Lipschitz RNN, cache LM, LM without positional encoding

• Assignment: [homework4]

02/10

• Topics: Encoders, Decoders and Enc-Dec Models [slides] [recording]

• Readings: seminal transformer paper, other papers cited in the slides

• Assignment: N/A

02/17

• Topics: More on Optimization, Overfitting, Generalization [slides]

• Readings: large batch training, Rethinking generalization

• Assignment: N/A

02/24 Midterm Exam

03/02

• Topics: Large Language Models, finetuning methods [slides] [recording]

• Readings: How does in-context learning work, Prompt Tuning

• Assignment: N/A

03/09

• Topics: Multi-task Learning, meta-learning, zero-shot Learning [slides] [recording 1, 2]

• Readings: N/A

• Assignment: N/A

03/16:

• Topics: Interpretability [slides] [recording 1, 2]

• Readings: CKA flawed, HiResCAM

• Assignment: N/A

03/23

• Topics: Generative Models [slides] [recording]. Guest lecture [slides] by Sibo Zhang

• Readings: N/A

• Assignment: N/A

03/30

• Topics: Project middle term review

04/06

• Topics: Sparsity, Distillation, Model Compression [slides] [recording 1, 2]

• Readings: survey on KD of LLM, Redundancy in multi-head attention

• Assignment: N/A

04/13

• Topics: Reinforcement Learning (Guest Lecture [slides] by Kaixiang Lin) [recording 1, 2]

• Readings: N/A

• Assignment: N/A

04/20

• Topics: Graph Neural Networks (Guest Lecture [slides] by Jian Zhang)

• Readings: N/A

• Assignment: N/A

04/27

• Final Project Presentation