New Material Discovery Using Physics-Inspired Reinforcement Learning Techniques

Abstract

The accelerated discovery of novel materials with targeted properties is crucial for advancing technological fields such as photovoltaics, catalysis, and energy storage. Traditional methods for material discovery often rely on experimental intuition or computational brute-force screening, both of which become increasingly inefficient as the complexity and dimensionality of the search space increase. To overcome these limitations, this thesis introduces a novel computational framework based on physics-inspired reinforcement learning (RL), specifically leveraging Stochastic Hamiltonian Dynamics (SHD), to efficiently explore high-dimensional search spaces for materials optimization. The properties of materials can be calculated via computational techniques like Density Functional Theory (DFT) calculations, which, although accurate, are computationally demanding. To address this challenge, we incorporate local surrogate modeling to approximate expensive DFT computations rapidly and cost-effectively. These inherently differentiable surrogate models significantly accelerate property evaluations, enabling efficient gradient-based optimization within both structural and chemical spaces. Additionally, to seamlessly navigate categorical variables associated with chemical compositions, we employ the Gumbel-Softmax reparameterization technique, transforming discrete choices into differentiable, continuous variables. Optimal control theory, particularly the Stochastic Pontryagin Maximum Principle (PMP), underpins our approach, providing a systematic framework for stable convergence and global optimization. By adapting SHD within this control-theoretic framework, we combine deterministic momentum-driven exploration with controlled stochastic perturbations, effectively avoiding local minima and promoting efficient global optimization. Ultimately, this thesis advances computational methodologies in materials science by establishing a robust, scalable, and versatile discovery framework. By integrating physical insights, reinforcement learning strategies, surrogate modeling, and optimal control, we provide a transformative approach that addresses the critical computational barriers and accelerates the discovery of new materials with optimal properties.