Publications

see CV for complete and up-to-date list

Preprints

  • Untangling tradeoffs between recurrence and self-attention in neural networks, G. Kerg, B. Kanuparthi, A. Goyal, K. Goyette, Y. Bengio, G. Lajoie, under review (2020), [preprint: https://arxiv.org/abs/2006.09471].

  • Advantages of biologically-inspired adaptive neural activation in RNNs during learn- ing, V. Geadah, G. Kerg, S. Horoi, G. Wolf, G. Lajoie, under review (2020), [preprint: arxiv.org/abs/2006.12253].

  • On Lyapunov Exponents for RNNs: Understanding Information Propagation Using Dynamical Systems Tools, R. Vogt, M. Puelma Touzel, E. Shlizerman, G. Lajoie, under review (2020), [preprint: https://arxiv.org/abs/2006.14123].

  • Gradient Starvation: A Learning Proclivity in Neural Networks with Cross-Entropy Loss, M. Pezeshki, G. Lajoie, Y. Bengio, A. Courville, D. Precup, under review (2020), [preprint: https://arxiv.org].

  • Lagrangian-based Dynamics for Game Optimization, R. Askari, A. Mitra, G. Lajoie, I. Mitliagkas, under review (2020), [preprint: https://arxiv.org].

  • Implicit Regularization in Deep Learning: A View from Function Space A. Baratin, T. George, C. Laurent, V. Thomas, D. Hjelm, G. Lajoie, P. Vincent, S. Lacoste-Julien, under review (2020), [preprint: https://arxiv.org/abs/2008.00938].

  • Dynamic compression and expansion in a classifying recurrent network, Matthew Farrell, Stefano Recanatesi, Guillaume Lajoie and Eric Shea-Brown, under review (2020). [preprint: https://www.biorxiv.org/content/10.1101/564476v1].

  • Transcranial DC stimulation affects population dynamics and single cell firing rate but not tuning in macaque sensorimotor cortex, Andrew R. Bogaard, Guillaume Lajoie, Hayley Boyd, Andrew Morse, Stavros Zanos, Eberhard E. Fetz, under review (2020) [preprint: https://www.biorxiv.org/content/10.1101/516260v2].

Selected Articles

Untangling tradeoffs between recurrence and self-attention in neural networks

Giancarlo Kerg, Bhargav Kanuparthi, Anirudh Goyal, Kyle Goyette, Yoshua Bengio, Guillaume Lajoie, Accepted at Neural Information Processing Systems (NeurIPS), (2020).

Samuel Laferrière, Marco Bonizzato, Numa Dancause, and Guillaume Lajoie, IEEE Brain BrainInsight, Issue 2, (2020).

Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Net- works with Attention over Modules

S. Mittal, A. Lamb, A. Goyal, V. Voleti, M. Shanahan, G. Lajoie, M. Mozer, Y. Bengio, International Conference of Machine Learning (ICML), (2020).

Learning complex motor control with neurostimulation: a hierarchical and adaptive algorithm to optimally explore neural maps

Samuel Laferrière, Marco Bonizzato, Sandrine Côté, Numa Dan- cause, and Guillaume Lajoie, IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 28, no. 6, pp. 1452-1460, June 2020, doi: 10.1109/TNSRE.2020.2987001.

Low-dimensional dynamics of encoding and learning in recurrent neural networks

Stefan Horoi, Victor Geadah, Guy Wolf, and Guillaume Lajoie, Advances in Artificial Intelligence (proceedings of CanadianAI 2020), Chapter No: 27, DOI:10.1007/978-3-030-47358-7_27

Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics

Giancarlo Kerg, Kyle Goyette, Max- imilian Puelma Touzel, Gauthier Gidel, Eugene Vorontsov, Yoshua Bengio, and Guillaume Lajoie, 33rd Conference on Neural Information Processing Systems (NeurIPS), (2019), arxiv.org/abs/ 1905.12080.

Correlation-based model of artificially induced plasticity in motor cortex by a bidirectional Brain-Machine Interface

Guillaume Lajoie, Nedialko Krouchev, John F. Kalaska, Adrienne Fairhall, Eberhard E. Fetz, PLoS Comput. Biol., (2017), Vol. 13, No. 2, Pages e1005343- 34, DOI:10.1371/journal.pcbi.1005343

Encoding in Balanced Networks: Revisiting Spike Patterns and Chaos in Stimulus- Driven Systems

Guillaume Lajoie, Kevin K. Lin, Jean-Philippe Thivierge, Eric Shea-Brown, PLoS Comput. Biol., (2016), Vol. 12, No. 12, Pages e1005258-30, DOI: 10.1371/journal.pcbi. 1005258

Dynamic signal tracking in a simple V1 spiking model

Guillaume Lajoie, Lai-Sang Young, Neural Computation, (2016), Vol. 28, No. 9, Pages 1985-2010, DOI: 10.1162/NECO_a_00868

Driving reservoir models with oscillations: a solution to the extreme structural sensitivity of chaotic networks

Philippe Vincent-Lamarre, Guillaume Lajoie, Jean-Philippe Thivierge, J Comput Neurosci, (2016), DOI: 10.1007/s10827-016-0619-3

Structured chaos shapes spike-response noise entropy in balanced neural networks

Guillaume Lajoie, Jean-Philippe Thivierge, Eric Shea-Brown, Frontiers in Computational Neuro- science, (2014), vol. 8, pp 1-10, DOI: 10.3389/fncom.2014.00123

Chaos and reliability in balanced spiking networks with temporal drive

Guillaume Lajoie, Kevin K. Lin, Eric Shea-Brown, Phys. Rev. E, (2013), vol. 87 (5), p. 052901, DOI: 10.1103/ PhysRevE.87.052901

Shared Inputs, Entrainment, and Desynchrony in Elliptic Bursters: From Slow Pas- sage to Discontinuous Circle Maps

Guillaume Lajoie, Eric Shea-Brown, SIAM Journal of Applied Dynamical Systems, (2011), vol. 10, p. 1232, DOI: 10.1137/100811726