MIT Deep Learning Lecture 1

Today I watched the first of 20 lectures on Deep Learning from MIT. The first 2 weeks are 6.S094: Deep Learning for Self-Driving Cars. Week 3 will be 6.S091: Deep Reinforcement Learning. Week 4 will be 6.S093: Human-Centered Artificial Intelligence.

I won’t create one of these for each video but I thought I would create one for the first one just to get started. I will do an overview of each course.

Lecture Slides

YouTube Video:

Since the slides are available I won’t just recreate them with my notes. Instead, I will just add my thoughts below.

  • Lex Fridman dresses WAY better than I would ever hope to dress.
  • The camera person was having trouble keeping Lex in the little window.
  • Data, data, data. Data is VERY important to anything in the field.
  • “AI began with an ancient wish to forge the gods.” – Pamela McCorduck
  • “Sky is the limit with the tooling”
  • “Why Deep Learning? Scalable Machine Learning”
  • The brain has a lot of neurons. Ha. In my talks, I alway mention how fast you can recognize your mother to show the scale and speed.
  • Activation functions:
    • Sigmoid -> Vanishing gradients. Not zero centered
    • Tanh -> Vanishing gradients.
    • ReLU -> Not zero centered.
  • Loss Functions:
    • Mean Squared Error (Regression): Real number
    • Cross Entrophy Loss (Classification): {0,1}
  • Lex did a pretty nice backpropagation slide
    • Task: Update the weights and biases to decrease the loss function
    • Subtask:
      • Forward pass to compute network output and “error”
      • Backward pass to compute gradients
      • A fraction of the weight’s gradient is subtracted from the weight
  • Semantic Segmentation
    • Encoding -> Decoding
    • I have never seen this before so I will be very interested when we expand on this later.
  • The text to speech slides were interesting but not sure I want to get into that area.
  • AutoML might take my job before I even start it.

For the most part, this was a review from grad school. But, it does make me regret that I didn’t get to take the self driving car class at GT.