Towards mechanistically constrained Recurrent Neural Network models of population dynamics

Abstract

Recurrent Neural Networks (RNNs) trained on neuroscience tasks are promising models of population dynamics of their biological counterparts [1, 2, 3, 4]. In this approach, RNNs are only constrained by task definition, and can potentially find many solutions to the same task. This is because training RNNs (or any highly parameterized function) by input-output examples can suffer from underspecification, as different algorithms or mechanisms can solve the same input-output examples. In light of this, how do we ensure that our RNN models are faithful to the biological computation being modelled? To systematically approach this question, we need a ground truth model. We therefore use a student-teacher framework in which both the teacher and the student are RNNs. In particular, a teacher RNN is trained to solve a task – much like an animal in laboratory – and the student is constrained to match either the behavior or the neural data of the teacher. We then compare mechanistic similarity by invoking the concept of stress tests – inputs not from the task related training distribution. We recognise that both behavioral and neural constraints have weaknesses in this regard and that in order to find some kind of guarantee, information about the teacher’s response to perturbations to the inputs is crucial for the student. Motivated by the abundance of behavioral data, we propose a novel method of training RNNs that we call ‘Jacobian constraint’ , wherein we constrain not only the RNNs input-output behavior but also the sensitivity – the behavioral response to infinitesimal stress tests. We find that RNNs obtained by this method replicate stress test behavior better than those obtained by constraining RNNs to neural data.[5, 6]