Nan Rosemary Ke
nan.rosemary.ke at gmail dot com
I am a second year PhD student at the Montreal Institute for Learning Algorithms
MILA, where I work on deep learning. I am advised by Professor Chris Pal. I recently spent time at Microsoft Research, Montreal.
Prior to joining MILA, I received a Bachelors in Computer Science at the University of Auckland. I also spent time at Carneigie Mellon University working on speech recognition and deep learning.
I am interested in new ways of training Recurrent Neural Networks, generative models and causal inference learning. I also spend time at Microsoft Research, Montreal, where I work on improved RNN training, generative models and language related research.
Google Scholar /
GitHub


News

Talks

Research
I am interested in new ways of training Recurrent Neural Networks, generative models and causal inference learning.


Focused Hierarchical RNNs for Conditional Sequence Processing
Nan Rosemary Ke,
Konrad Zolna, Alessandro Sordoni, Zhouhan Lin, Adam Trischler, Yoshua Bengio, Joelle Pineau, Laurent Charlin, Chris Pal
International Conference on Machine Learning (ICML), 2018
arXiv
/
blog post (coming soon)
/
We present a mechanism for focusing
RNN encoders for sequence modelling tasks
which allows them to attend to key parts of the
input as needed. We formulate this using a multilayer
conditional sequence encoder that reads in
one token at a time and makes a discrete decision
on whether the token is relevant to the context
or question being asked. The discrete gating
mechanism takes in the context embedding and
the current hidden state as inputs and controls information
flow into the layer above.

Publications

Sparse Attentive BackTracking: Long Range Credit Assignment in Recurrent Networks
Nan Rosemary Ke, Anirudh Goyal, Olexa Bilaniuk, Jonathan Binas, Laurent Charlin, Mike Mozer, Chris Pal, Yoshua Bengio
ICML Workshop on Principled Approaches to Deep Learning, 2018 , under review
A major drawback of backpropagation through time (BPTT) is the difficulty of learning longterm dependencies, coming from having to propagate credit information backwards through every single step of the forward computation. This makes BPTT both computationally impractical and biologically implausible. For this reason, full backpropagation through time is rarely used on long sequences, and truncated backpropagation through time is used as a heuristic. However, this usually leads to biased estimates of the gradient in which longer term dependencies are ignored. Addressing this issue, we propose an alternative algorithm, Sparse Attentive Backtracking, which might also be related to principles used by brains to learn longterm dependencies. Sparse Attentive Backtracking learns an attention mechanism over the hidden states of the past and selectively backpropagates through paths with high attention weights. This allows the model to learn long term dependencies while only backtracking for a small number of time steps, not just from the recent past but also from attended relevant past states.


Twin Networks: Using the future to generate sequences
Nan Rosemary Ke*, Dmitry Serdyuk*, Alessandro Sordoni, Adam Trischler, Chris Pal, Yoshua Bengio
International Conference on Learning Representations (ICLR), 2018
We propose a simple technique for encouraging generative RNNs to plan ahead.
We train a “backward” recurrent network to generate a given sequence in reverse
order, and we encourage states of the forward model to predict cotemporal states
of the backward model. The backward network is used only during training, and
plays no role during sampling or inference. We hypothesize that our approach
eases modeling of longterm dependencies by implicitly forcing the forward states
to hold information about the longerterm future (as contained in the backward
states).


Ethical Challenges in Dialogue Systems
Peter Henderson, Koustav Sinha, Nicolas Gontier,Nan Rosemary Ke, Geneive Fried, Ryan Lowe, Joelle Pinea
to appear AIES, 2018
The use of dialogue systems as a medium for humanmachine interaction is an increasingly prevalent paradigm. A growing number of dialogue systems use conversation strategies that are learned from large datasets. There are well documented instances where interactions with these system have resulted in biased or even offensive conversations due to the datadriven training process. Here, we highlight potential ethical issues that arise in dialogue systems research, including: implicit biases in datadriven systems, the rise of adversarial examples, potential sources of privacy violations, safety concerns, special considerations for reinforcement learning systems, and reproducibility concerns. We also suggest areas stemming from these issues that deserve further investigation. Through this initial survey, we hope to spur research leading to robust, safe, and ethically sound dialogue systems.


Z Forcing: Training Stochastic RNN's
Anirudh Goyal,
Alessandro Sordoni,
MarcAlexandre Côté,
Rosemary Nan Ke,
Yoshua Bengio,
Neural Information Processing System (NIPS), 2017
arXiv
/
code (coming soon)
We proposed a novel approach to incorporate stochastic latent variables in sequential neural networks. The method builds on recent architectures that use latent variables to condition the recurrent dynamics of the network. We augmented the inference network with an RNN that runs backward through the sequence and added a new auxiliary cost that forces the latent variables to reconstruct the state of that backward RNN, i.e. predict a summary of future observations.


Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net
Anirudh Goyal,
Nan Rosemary Ke,
Surya Ganguli,
Yoshua Bengio
Neural Information Processing System (NIPS), 2017
arXiv
/
blog post (coming soon)
/
code
We propose a novel method to directly learn a stochastic transition operator whose repeated application provides generated samples. Traditional undirected graphical models approach this problem indirectly by learning a Markov chain model whose stationary distribution obeys detailed balance with respect to a parameterized energy function.


A Deep Reinforcement Learning Chatbot
Iulian V. Serban, Chinnadhurai Sankar, Mathieu Germain, Saizheng Zhang, Zhouhan Lin,
Sandeep Subramanian, Taesup Kim, Michael Pieper, Sarath Chandar, Nan Rosemary Ke,
Sai Rajeshwar, Alexandre de Brebisson, Jose M. R. Sotelo, Dendi Suhubdy,
Vincent Michalski, Alexandre Nguyen, Joelle Pineau and Yoshua Bengio,
International Conference on Learning Representations (ICLR), 2017
arXiv
We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language generation and retrieval models, including templatebased models, bagofwords models, sequencetosequence neural network and latent variable neural network models. By applying reinforcement learning to crowdsourced data and realworld user interactions, the system has been trained to select an appropriate response from the models in its ensemble. The system has been evaluated through A/B testing with realworld users, where it performed significantly better than many competing systems. Due to its machine learning architecture, the system is likely to improve with additional data.


Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations
David Krueger, Tegan Maharaj, Janos Kramar, Mohammad Pezeshki, Nicolas Ballas,Nan Rosemary Ke, Anirudh Goyal
Yoshua Bengio,
Aaron Courville
Chris Pal
International Conference on Learning Representations (ICLR), 2017
arXiv
/
code
We propose zoneout, a novel method for regularizing RNNs. At each timestep, zoneout stochastically forces some hidden units to maintain their previous values. Like dropout, zoneout uses random noise to train a pseudoensemble, improving generalization. But by preserving instead of dropping hidden units, gradient information and state information are more readily propagated through time, as in feedforward stochastic depth networks.


Cascading Bandits for LargeScale Recommendation Problems
Shi Zong, Hao Ni, Kenny Sung, Nan Rosemary Ke, Zheng Wen, Branislav Kveton
Association for Uncertainty in Artificial Intelligence (UAI), 2016
arXiv
Most recommender systems recommend a list of items. The user examines the list, from the first item to the last, and often chooses the first attractive item and does not examine the rest. This type of user behavior can be modeled by the cascade model. In this work, we study cascading bandits, an online learning variant of the cascade model where the goal is to recommend K most attractive items from a large set of L candidate items. We propose two algorithms for solving this problem, which are based on the idea of linear generalization. The key idea in our solutions is that we learn a predictor of the attraction probabilities of items from their features, as opposing to learning the attraction probability of each item independently as in the existing work. This results in practical learning algorithms whose regret does not depend on the number of items L. We bound the regret of one algorithm and comprehensively evaluate the other on a range of recommendation problems. The algorithm performs well and outperforms all baselines

