Applications of Transformer Neural Networks in Correlated Matter

Abstract

The thesis explores in great detail how the advancements in machine learning could be leveraged to further fundamental research in physics. We use the transformer architecture to classify different phases of matter with interpretable models. We propose a modified architecture, the correlator transformer, that provides full interpretability in terms of correlation functions. We also show that transformers could be a good variational ansatz generator for quantum many-body systems and find good approximations for ground state energies for different Heisenberg systems.

A significant portion of the thesis is dedicated to applying vision transformers for phase classification in different systems. Considerable work has been done in regard to phase classification using neural networks, and with great success. But, despite their effectiveness, they tend to operate as "black boxes". This work seeks to shed some light on these black boxes in terms of physically relevant observables. By training vision transformers on a range of systems, from lattice models like the Ising and $Z_2$ gauge models to continuous systems such as the Fermi gas, the thesis provides new insights into how a neural network "perceives" these phases of matter. The goal is not only to classify these phases accurately but also to understand the underlying physical correlations the models use to make their decisions, thus addressing the critical challenge of interpretability in deep neural networks.

Deep neural networks have the potential to serve as universal function approximators, and this capability has been used to successfully solve quantum many-body systems variationally. Thus, the second part of the thesis focuses on this aspect of the transformer. We show that a vision transformer can parameterise the quantum many-body wave functions and approximate their ground state and other observables accurately.