BLACKBOARD PAGE | UNIT INFO | FORUM | SAFE

COMSM0018 - Applied Deep Learning

ADL Banner

Unit Information

Welcome to COMSM0018. The unit introduces the students to deep architectures for learning linear and non-linear transformations of big data towards tasks such as classification and regression. The unit paves the path from understanding the fundamentals of convolutional and recurrent neural networks through to training and optimisation as well as evaluation of learnt outcomes. The unit's approach is hands-on, focusing on the 'how-to' while covering the basic theoretical foundations. For further general information, see the syllabus for the unit and the SAFE website.


Staff

Dima Damen (DD)office 3.12 MVB. Unit Director
Tilo Burghardt (TB) office 3.42 MVB.

Teaching Assistants

Hazel Doughty (HD), Davide Moltisanti (DM), Miguel Fortiz (MF), Michael Wray (MW), Will Price (WP), Will Andrew (WA), Evangelos Kazakos (EK), Jonathan Munro (JM)


Unit Materials

Wks Monday Sessions Friday Sessions Labs
1 01/10/18, 11am, QB.F101 - Introduction to the Unit

01/10/18, 12am, QB.F101 - LECTURE 1
BASICS OF ARTIFICIAL NEURAL NETWORKS
(Introduction, Neural Networks, Perceptron, Cost Functions, Gradient Descent, Delta Rule, Deep Networks)
SLIDES

05/10/18, 11am, QB.1.68 - LECTURE 2
TOWARDS TRAINING DEEP FORWARD NETWORKS
(Optimization, Saddle Points, Stochastic Gradient Descent, AdaGrad)
SLIDES
(no scheduled lab for week 1)
GETTING STARTED:

Register Individually on BlueCrystal4
(details see below)

RECAP WORKSHEETS:
-Convolutions (Homework)
-Python (Homework)
2 08/10/18,11am,QB.F101 - PRACTICAL 1
Your first fully connected layer
gradient descent
stochastic gradient descent
12/10/18, 9am + 11am, QB.1.68 - LECTURE 3
BACKPROPAGATION ALGORITHM
(Automatic Reverse Differentiation, The Backpropagation Algorithm)
SLIDES
08/10/18, 12noon, QB.F101 - 1hr
-BC4 Stress Test
Lab 1 - Training your first Deep Neural Network
3 15/10/18, 11am, QB.F101 - LECTURE 4
CONVOLUTIONAL NEURAL NETWORKS
(sharing parameters, conv layers, pooling, CNN architectures)
slides
PRACTICAL 2
Your first convolutional connected layer
Introduction to TensorFlow
19/10/18, 9am, 11am, QB.1.68 - LECTURE 5
OPTIMISATION, COST FUNCTIONS, REGULARISATION
(SGD, RMSProp, Adam, Saddle Points, Key Cost Functions, L1 and L2 Regularisation, Dropout)
SLIDES
15/10/18, 12noon, QB.F101 - 2hr
Lab 2 - Your First Convolutional Connected Network
4 22/10/18, 11am, QB.LT1.18 - PRACTICAL 3
Error rate monitoring (training/validation/testing)
Batch-based training
Learning rate
Batch normalisation
Parameter intialisation
Slides
22/10/18, 9am, QB.1.68- LECTURE 6
SUMMARY OF THEORY INTRO
SLIDES
Talk Assignment (25%)
22/10/18, 12noon, QB.F101 - 2hrs

Lab 3 - Hyperparameters
5 29/10/18, 11am, QB.F101 - 29/10/18, 12am, QB.F101 - Practical 4 Intro
SLIDES
PRACTICAL 4
Debugging strategies
Hyperparameters (again)
Invited Talk 29/10/18, 12noon, QB.F101 - 2hrs

Lab 4
Data Augmentation
6 PRACTICAL 5 Intro
Baseline models
Adversal training
SLIDES
Invited Talk 05/11/18, 12noon, QB.F101 - 2hrs
Lab 5
Adversarial Training
7 12/11/18, 11am, QB.F101 - LECTURE 9 RECURRENT NEURAL NETWORKS
(temporal dependencies, RNN, bi-directional RNNs, encoder-decoder, LSTM, gated RNN)

PRACTICAL 5 (cont.)
Baseline models
Adversal training
Invited Talk 12/11/18, 12noon, QB.F101 - 2hrs
Lab 6
8 CS EXPLORE WEEK
9 26/11/18, 11am, QB.F101 - CW Q/A Invited Talk 26/11/18, 12noon, QB.F101 - 3hrs
CW Lab
10 03/12/18, 11am, QB.F101 - Assessment Talks (3hrs)
11 10/12/18, 11am, QB.F101 - Assessment Talks (3hrs)
12 17/12/18, 11am, QB.F101 - Assessment Talks (3hrs)

Assessment Details

The student will undertake a challenge of replicating a state-of-the-art performance on a publicly available dataset using one of the deep architectures discussed on the unit. The unit has three assessments:

  1. Lab Portfolio (individual, summative, 15%) [Wk10]
  2. Talk Assignment (individual, summative, 25%) - SCHEDULE [Wk10-12]
  3. Group (up to 3) Project - assessed by a final report (summative, 60%) [Wk13] - Register your teams on doodle

Github

All technical resources will be posted on the COMSM0018 ADL Github organisation. If you find any issues, please kindly raise an issue in the respective repository.


Textbook

Recommended Reading:
Goodfellow et al (2016). Deep Learning. MIT Press


Blue Crystal 4 Registration [only applicable for Bristol undergraduate students with corresponding email]

All students must apply online to register an account on BC4 for this unit. This also applies to students who already have accounts on BC4 for other units (e.g. HPC), in this case you must register again using the instructions below.

  1. Click on: https://www.acrc.bris.ac.uk/login-area/apply.cgi
  2. Enter your personal details
  3. Choose: "Join an existing project"
  4. Enter project code: COSC018263
  5. Keep Preferred log-in shell as bash
  6. Do not provide any additional information

Note that it takes up to 48 hours to enable your account on BC4.