Real-time Physics Simulations and Machine Learning
“The underlying physical laws necessary for the mathematical theory of a large part of physics and the whole of chemistry are thus completely known, and the difficulty is only that the exact application of these laws leads to equations much too complicated to be soluble”, said the renowned British quantum physicist Paul Dirac in 1929 [1]. Dirac implied that all physical phenomena can be simulated down to the quantum, from protein folding to material failures and climate change. The only problem is that the governing equations are too complex to be solved at realistic time-scales.
Does this mean that we can never achieve real-time physics simulations? Well, physicists have a knack for developing models, methods, and approximations to achieve the desired results in shorter timescales. With all the advancements in research, software, and hardware technology, real-time simulation has only been made possible at the continuum scale which is most evident in video game physics.
Simulating physical phenomena such as collisions, deformations, fracture, and fluid flow are computationally intensive, yet models have been developed that simulate such phenomena in real-time within games. Of course there have been a lot of simplifications and optimizations of different algorithms to make it happen. The fastest method is rigid body physics. This is what most games are based on where objects can collide and rebound without deforming. Objects are represented by convex collision boxes which surround the object, and when two objects collide, the collision is detected in real-time and appropriate forces are applied to simulate the impact. There are no deformations or fractures in this representation. The video game ‘Teardown’ is potentially the pinnacle of rigid body physics.
Although rigid body physics is good for simulating non-deformable collisions, it is not suitable for deformable materials such as hair and clothes which games heavily rely on. This is where soft-body dynamics comes in. Below, you can see four methods for simulating deformable objects in the order of complexity:
Spring-Mass Model:
The name is totally self-explanatory. Objects are represented by a system of point masses that are connected to each other via springs. You can think of it as a network of one-dimensional Hooke’s law in a 3D setup. The main drawbacks of this model is that it requires a lot of manual work in setting up the mass-spring network, and there isn’t a rigorous relationship between material properties and model parameters. Nonetheless, the model has been implemented exceptionally well in ‘BeamNG.Drive’, a real-time vehicle simulator that is based on spring-mass model to simulate vehicle deformations.
Position-based Dynamics (PBD):
The methods of simulating kinematics are generally based on force-based models where the particle accelerations are calculated from Newton’s second law, and then integrated to obtain the velocities and positions at every time step. In position-based dynamics, the positions are computed directly through solving a quasi-static problem involving a set of equations that include constraints. PBD is less accurate but faster than a forced-based approach, making it ideal for applications in games, animation films, and visual effects. The movement of hair and clothes in games are generally simulated through this model. PBD is not limited to deformable solids, but can also be used to simulate rigid body systems and fluids. Here is an excellent survey on PBD methods [2].
Finite-Element Method (FEM):
The finite element method of computing deformations in materials is based on numerically solving the stress-strain equations based on the elastic field theory. It is essentially solving the 3D Hookes law in 3D. The material is divided into finite elements, usually tetrahedra, and the stress and strain on vertices are calculated at every time step through solving a linear matrix equation. FEM is a mesh-based approach to simulating soft-body dynamics. It is very accurate and the model parameters are directly related to material properties such as Young’s modulus and Poisson ratio. FEM simulations for engineering applications are generally not real-time, but recently AMD, one of the largest semiconductor companies, released its multi-threaded FEM library for games called FEMFX that simulated material deformations in real-time.
Material Point Method (MPM):
MPM is a highly accurate mesh-free method which is much more suitable than mesh-based methods for simulating large deformations, fractures, multi-material systems and viscoelastic fluids because of its improved efficiency and resolution.. MPM is currently the state-of-the-art of mesh-free hybrid Eulerian/Lagrangian methods, developed as a generalization to older methods such as Particle in Cell (PIC) and Fluid Implicit Particle (FLIP). MPM simulations are not real-time, and state-of-the art simulations take about half a minute per frame for systems involving about a million points. Here is a comprehensive course notes on MPM [3].
Machine Learning and Physics Simulations
So what does Machine Learning have to do with all this? Well you have probably already noticed that there is always a trade-off between computation speed and accuracy/resolution. With physics solvers having been optimized enormously over the past few decades, there is little room left for step-change improvements.
Here is where Machine Learning comes in. Recent research by Oxford [4], Ubisoft La Forge [5], DeepMind [6,7], and ETH Zurich [8] demonstrate that a deep neural network can learn physics interactions and emulate them multiple orders of magnitude faster. This is done through generating millions of simulation data, feeding them through the neural network for training, and using the trained model to emulate what a physics solver would do. Although the offline process would take a lot of time in generating data and training the model, the trained neural network model is much faster at simulating the physics. For instance, the researchers at Oxford [4] developed a method called Deep Emulator Network Search (DENSE) that accelerates simulations up to 2 billion times, and they demonstrated this in 10 scientific case studies including astrophysics, climate, fusion, and high energy physics.
In the gaming sector, Ubisoft La Forge’s team used a simple feed-forward network that trains on the vertex positions of 3D mesh objects at three subsequent time frames and learns to predict the next frame [5]. The model essentially compares the predictions with the known positions from the simulated datasets, and back-propagates to adjust the model parameters to minimize the error in making predictions. The team used Maya’s nCloth physics solver to generate simulation data which is an advanced spring-mass model optimized for cloths. They also implemented a Principal Component Analysis (PCA) to only train on the most important bases. The results were astounding. The neural network could emulate the physics up to 5000 times faster than the physics solver.
Another recent work by Peter Battaglia’s team at DeepMind achieved astonishing results with graph networks [6]. Unlike traditional neural networks where each layer of nodes is connected to every node in the next layer, a graph neural network has a graph-like structure. With this model, they managed to simulate a wide range of materials including sand, water, goop, and rigid solids. Instead of predicting the positions of particles, the model predicts the accelerations, and the velocities and positions are computed using an Euler integration. The simulation data were generated using a range of physics solvers including PBD, SPH (smoothed-particle hydrodynamics) and MPM. The model was not optimized for speed and therefore it was not significantly faster than the physics solvers, but certainly it demonstrated what can be made possible when Machine Learning meets physics.
This field is still in its infancy, but certainly we will be observing new ML-based technologies that enhance physics simulations. There are just so many models for simulating any physical phenomena at all scales and complexities, ranging from quantum mechanics and molecular dynamics to microstructure and classical physics, and the potential opportunities to create value from the duo of Machine learning and Physics are immense.
[1] Paul Dirac, Quantum Mechanics of many-electron systems, Proc. R. Soc. Lond. A 123, 714 (1929)
[2] J. Bender et al., A Survey on Position Based Dynamics, EUROGRAPHICS (2017)
[3] Chenfanfu Jiang et al., The Material Point Method for Simulating Continuum Materials, SIGGRAPH courses (2016)
[4] J. Wolper et al., CD-MPM: Continuum Damage Material Point Methods for Dynamic Fracture Animation, ACM Trans. Graph. 38, 119 (2019)
[5] M. Kasim et al., Building high accuracy emulators for scientific simulations with deep neural architecture search, arXiv (2020)
[6] D. Holden et al., Subspace Neural Physics: Fast Data-Driven Interactive Simulation, SCA Proc. ACM SIGGRAPH (2019)
[7] A. Sanchez-Gonzalez et al., Learning to Simulate Complex Physics with Graph Networks, Proc. 37th Int. Conf. ML, PMLR, 119 (2020)
[8] T. Pfaff et al., Learning Mesh-based Simulations with Graph Networks, arXiv (2021)[9] B. Kim et al., Deep Fluids: A Generative Network for Parameterized Fluid Simulations, Computer Graphics Forum, 38, 59 (2019)
