Teaching

Representation Learning for NLP (Master Machine Learning & AI @ U Lyon 2)

Here is a list of suggested readings for this course:

Pre-Trained Word Representations

Main techniques to learn vector space representations of words

• SGNS: Distributed Representations of Words and Phrases and their Compositionality
• Glove: Global Vectors for Word Representation
• Link between SGNS, GloVe & PMI: Neural Word Embedding as Implicit Matrix Factorization

Supervised Learning with Pre-Trained Word Representations

Main neural architectures to solve document-level classification tasks

• CNN: Convolutional Neural Networks for Sentence Classification
• RNN & GRU: Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

Sequence to Sequence learning and attention mechanism

• Seq2Seq: Sequence to Sequence Learning with Neural Networks
• Dot-product Attention: Effective Approaches to Attention-based Neural Machine Translation
• Transformer: Attention is All You Need

Supervised Learning with Pre-Trained Language Models

• BERT: Bidirectional Encoder Representations from Transformers

Machine Learning with Graphs (Master Complex Systems @ ENS Lyon)

Here is a list of suggested readings for this course:

Pre-trained Node and Graph Representations

• DeepWalk: Online Learning of Social Representations
• Node2Vec: Scalable Feature Learning for Networks

Graph Neural Networks

• GCN: Semi Supervised Node Classification with Graph Convolutional Networks
• GAT: Graph Attention Networks