Modern NLP using Deep Learning

Advance knowledge at modern NLP

Understand modern NLP techniques

Advance knowledge at modern DL

Understand modern DL techniques

Requirements

  • Motivation
  • Interset
  • Mathematical approach

Description

You will learn the newest state-of-the-art Natural language processing (NLP) Deep-learning approaches.

You will

  1. Get state-of-the-art knowledge regarding
    1. NMT
    2. Text summarization
    3. QA
    4. Chatbot
  2. Validate your knowledge by answering short and very easy 3-question queezes of each lecture
  3. Be able to complete the course by ~2 hours.

Syllabus

  1. Neural machine translation (NMT)
    1. Seq2seq
      A family of machine learning approaches used for natural language processing.
    2. Attention
      A technique that mimics cognitive attention.
    3. NMT
      An approach to machine translation that uses an artificial neural network to predict the likelihood of a sequence of words, typically modelling entire sentences in a single integrated model.
    4. Teacher-forcing
      An algorithm for training the weights of recurrent neural networks (RNNs).
    5. BLEU
      An algorithm for evaluating the quality of text which has been machine-translated from one natural language to another.
    6. Beam search
      A heuristic search algorithm that explores a graph by expanding the most promising node in a limited set.
  2. Text summarization
    1. Transformer
      A deep learning model that adopts the mechanism of self-attention, differentially weighting the significance of each part of the input data.
  3. Question Answering
    1. GPT-3
      An autoregressive language model that uses deep learning to produce human-like text.
    2. BERT
      A transformer-based machine learning technique for natural language processing (NLP) pre-training developed by Google.
  4. Chatbot
    1. LSH
      An algorithmic technique that hashes similar input items into the same “buckets” with high probability.
    2. RevNet
      A variant of ResNets where each layer’s activations can be reconstructed exactly from the next layer’s.
    3. Reformer
      Introduces two techniques to improve the efficiency of Transformers.

Resources

  • Wikipedia
  • Coursera

Who this course is for:

  • Anyone intersted in NLP
  • Anyone intersted in AI

Course content