Nan Rosemary Ke

Nan Rosemary Ke
rosemary.nan.ke at gmail dot com

I am a senior research scientist at Google Deepmind. I completed my PhD at Mila with Professors Yoshua Bengio and Chris Pal.

My research lies at the intersection of causal learning and deep learning. Causal models generalize probabilistic models by adding the capability to infer cause-and-effect relationships among variables and events. Deep learning systems are increasingly being asked to answer questions that are essentially causal in nature. What is the causal effect of an agent's action on the state of the world and its ability to reach a goal? Which genes cause a disease? While deep learning is a powerful tool, it can only answer these questions reliably if it is used in a way that has a causal grounding. My goal is to exploit synergies between causal learning and deep learning, obtaining the inferential strength of causal mechanisms with the scalability and representational capabilities of deep learning.

During my PhD, I have spent time at Google Deepmind, Facebook AI Research and Microsoft Research. I have been named a Rising Star in Machine Learning, a Rising Star in EECS. I have been awarded the Facebook fellowship from 2019 to 202 from 2019 to 20211.

Google Scholar / Twitter

News

At ICML 2022, I gave the tutorial Causality and Deep Learning: Synergies, Challenges and the Future.
At UAI 2022, I gave the tutorial Causality and Deep Learning: Synergies, Challenges and the Future.

Invited Talks

At IROS 2023, I gave an invited talk on Learning to Learn Causal Structure in Reinforcement Learning at the Causality for Robotics workshop.
At NeurIPS 2022 , I gave an invited talk on Learning Neural Causal Models at the NeurIPS 2022 Workshop on Causality for Real-world Impact .
At ICLR 2022 , I gave an invited talk on Modularity, Causality and Deep learning at the ICLR Workshop on the Elements of Reasoning: Objects, Structure, and Causality .
I gave an invited talk at CogX 2020 on "Causality in Deep Learning” to discuss how to incorrporate causality with deep learning to achieve better systematic generalization.

Research

My research interest lies in the intersection of deep learning learning and causality.

	Learning to induce causal structure. Nan Rosemary Ke, Silvia Chiappa, Jane Wang, Anirudh Goyal, Jorg Bornschein, Melanie Rey, Theophane Weber, Matthew Botvinick, Michael Mozer, Danilo Jimenez Rezende paper One of the fundamental challenges in causal induction is to infer the underlying graph structure given observational and/or interventional data. Most existing causal induction algorithms operate by generating candidate graphs and evaluating them using either score-based methods (including continuous optimization) or independence tests. In our work, we instead treat the inference process as a black box and design a neural network architecture that learns the mapping from both observational and interventional data to graph structures via supervised training on synthetic graphs. The learned model generalizes to new synthetic graphs, is robust to train-test distribution shifts, and achieves state-of-the-art performance on naturalistic graphs for low sample complexity.
	Sparse Attentive Backtracking: Temporal Credit Assignment Through Reminding Nan Rosemary Ke, Anirudh Goyal, Olexa Blanik, Jonathan Binas, Michael Mozer, Chris Pal, Yoshua Bengio The Thirty-second Annual Conference on Neural Information Processing Systems (NeuIPS) Spotlight presentation , 2018 Arxiv / blog post (coming soon) / Learning long-term dependencies in extended temporal sequences requires credit assignment to events far back in the past. The most common method for training recurrent neural networks, back-propagation through time (BPTT), requires credit information to be propagated backwards through every single step of the forward computation, potentially over thousands or millions of time steps. This becomes computationally expensive or even infeasible when used with long sequences. Importantly, biological brains are unlikely to perform such detailed reverse replay over very long sequences of internal states (consider days, months, or years.) However, humans are often reminded of past memories or mental states which are associated with the current mental state. We consider the hypothesis that such memory associations between past and present could be used for credit assignment through arbitrarily long sequences, propagating the credit assigned to the current state to the associated past state. Based on this principle, we study a novel algorithm which only back-propagates through a few of these temporal skip connections, realized by a learned attention mechanism that associates current states with relevant past states. We demonstrate in experiments that our method matches or outperforms regular BPTT and truncated BPTT in tasks involving particularly long-term dependencies, but without requiring the biologically implausible backward replay through the whole history of states. Additionally, we demonstrate that the proposed method transfers to longer sequences significantly better than LSTMs trained with BPTT and LSTMs trained with full self-attention.
Publications	Focused Hierarchical RNNs for Conditional Sequence Processing Nan Rosemary Ke, Konrad Zolna, Alessandro Sordoni, Zhouhan Lin, Adam Trischler, Yoshua Bengio, Joelle Pineau, Laurent Charlin, Chris Pal International Conference on Machine Learning (ICML), 2018 Arxiv / blog post (coming soon) / We present a mechanism for focusing RNN encoders for sequence modelling tasks which allows them to attend to key parts of the input as needed. We formulate this using a multilayer conditional sequence encoder that reads in one token at a time and makes a discrete decision on whether the token is relevant to the context or question being asked. The discrete gating mechanism takes in the context embedding and the current hidden state as inputs and controls information flow into the layer above.
	Twin Networks: Using the future to generate sequences Nan Rosemary Ke, Dmitry Serdyuk, Alessandro Sordoni, Adam Trischler, Chris Pal, Yoshua Bengio International Conference on Learning Representations (ICLR), 2018 We propose a simple technique for encouraging generative RNNs to plan ahead. We train a “backward” recurrent network to generate a given sequence in reverse order, and we encourage states of the forward model to predict cotemporal states of the backward model. The backward network is used only during training, and plays no role during sampling or inference. We hypothesize that our approach eases modeling of long-term dependencies by implicitly forcing the forward states to hold information about the longer-term future (as contained in the backward states).
	Z Forcing: Training Stochastic RNN's Anirudh Goyal, Alessandro Sordoni, Marc-Alexandre Côté, Rosemary Nan Ke, Yoshua Bengio, Neural Information Processing System (NIPS), 2017 arXiv / code (coming soon) We proposed a novel approach to incorporate stochastic latent variables in sequential neural networks. The method builds on recent architectures that use latent variables to condition the recurrent dynamics of the network. We augmented the inference network with an RNN that runs backward through the sequence and added a new auxiliary cost that forces the latent variables to reconstruct the state of that backward RNN, i.e. predict a summary of future observations.
	Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net Anirudh Goyal, Nan Rosemary Ke, Surya Ganguli, Yoshua Bengio Neural Information Processing System (NIPS), 2017 arXiv / blog post (coming soon) / code We propose a novel method to directly learn a stochastic transition operator whose repeated application provides generated samples. Traditional undirected graphical models approach this problem indirectly by learning a Markov chain model whose stationary distribution obeys detailed balance with respect to a parameterized energy function.
	A Deep Reinforcement Learning Chatbot Iulian V. Serban, Chinnadhurai Sankar, Mathieu Germain, Saizheng Zhang, Zhouhan Lin, Sandeep Subramanian, Taesup Kim, Michael Pieper, Sarath Chandar, Nan Rosemary Ke, Sai Rajeshwar, Alexandre de Brebisson, Jose M. R. Sotelo, Dendi Suhubdy, Vincent Michalski, Alexandre Nguyen, Joelle Pineau and Yoshua Bengio, International Conference on Learning Representations (ICLR), 2017 arXiv We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language generation and retrieval models, including template-based models, bag-of-words models, sequence-to-sequence neural network and latent variable neural network models. By applying reinforcement learning to crowdsourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble. The system has been evaluated through A/B testing with real-world users, where it performed significantly better than many competing systems. Due to its machine learning architecture, the system is likely to improve with additional data.
	Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations David Krueger, Tegan Maharaj, Janos Kramar, Mohammad Pezeshki, Nicolas Ballas,Nan Rosemary Ke, Anirudh Goyal Yoshua Bengio, Aaron Courville Chris Pal International Conference on Learning Representations (ICLR), 2017 arXiv / code We propose zoneout, a novel method for regularizing RNNs. At each timestep, zoneout stochastically forces some hidden units to maintain their previous values. Like dropout, zoneout uses random noise to train a pseudo-ensemble, improving generalization. But by preserving instead of dropping hidden units, gradient information and state information are more readily propagated through time, as in feedforward stochastic depth networks.