Partial differential equations (PDEs) are used in domains such as physics and engineering to represent complex physical processes, providing insight into how some of the world’s most complex physical and natural systems function.
To solve these complicated equations, researchers employ high-fidelity numerical solvers, which can be time-consuming and computationally costly to run. The current simpler alternative, data-driven surrogate models, compute only the goal attribute of a PDE solution, not the entire solution. These are trained on a set of data generated by the high-fidelity solver to anticipate the PDE outputs for new inputs. This is data-intensive and costly because complicated physical systems necessitate a large number of simulations to produce sufficient data.
In a new paper, “Physics-enhanced deep surrogates for partial differential equations,” published in December in Nature Machine Intelligence, a new method is proposed for developing data-driven surrogate models for complex physical systems in fields such as mechanics, optics, thermal transport, fluid dynamics, physical chemistry, and climate models.
The paper was authored by MIT’s professor of applied mathematics Steven G. Johnson, along with Payel Das and Youssef Mroueh of the MIT-IBM Watson AI Lab and IBM Research, Chris Rackauckas of Julia Lab, and Raphaël Pestourie, a former MIT postdoc who is now at Georgia Tech. The authors refer to their technology as a “physics-enhanced deep surrogate” (PEDS), which combines a low-fidelity, explainable physics simulator, and a neural network generator. The neural network generator is fully trained to match the high-fidelity numerical solver’s output.
“I aspire to substitute the inefficient process of trial and error with systematic, computer-aided simulation and optimization,” Pestourie says in a statement. “Recent advancements in AI, such as ChatGPT’s big language model, rely on hundreds of billions of parameters and require massive amounts of resources to train and assess. PEDS, on the other hand, is affordable to everyone because it is extremely efficient in computing resources and has a very low barrier in terms of the infrastructure required to use it.
The article demonstrates that PEDS surrogates can be up to three times more accurate than an ensemble of feedforward neural networks with limited data (about 1,000 training points) and reduce training data requirements by at least a factor of 100 to attain a goal error of 5%. This scientific machine-learning method, developed with the Julia programming language designed by MIT, is so efficient in both computing and data.
According to the authors, PEDS provides a broad, data-driven technique for bridging the gap between a vast array of reduced physical models and the related brute-force numerical solvers that simulate complicated systems. This technique provides precision, speed, data efficiency, and physical insight into the process.
Pestourie states, “Since the 2000s, as processing power grew, the trend of scientific models has been to increase the number of parameters to better match the data, often at the expense of reduced predicted accuracy. PEDS performs the opposite by selecting its parameters carefully. It uses automatic differentiation technology to train a neural network, resulting in an accurate model with few parameters.
“The main challenge that prevents surrogate models from being used more widely in engineering is the curse of dimensionality—the fact that the needed data to train a model increases exponentially with the number of model variables,” Pestourie said.
“PEDS reduces this curse by incorporating information from the data and the field knowledge in the form of a low-fidelity model solver.”
According to the researchers, PEDS has the potential to reinvigorate a large body of pre-2000 literature devoted to minimum models—intuitive models that PEDS might improve in accuracy while also being predictive for surrogate model applications.
“The application of the PEDS framework is beyond what we showed in this study,” Das explains. “Complex physical systems regulated by PDEs are everywhere, from climate modeling to seismic modeling and beyond. Our physics-inspired rapid and explainable surrogate models will be extremely useful in many applications, complementing other emerging techniques such as foundation models.”
