More detailed technical article: "Physics considerations for a simulated reality", pdf
Though graphics have improved by leaps and bounds, modern game engines still feel very far from enabling an open-ended simulated reality in the sense of the The Matrix. Specifically, what I think they are lacking is a materials system, part of a larger missing physics engine.
If my only possible interactions with the environment are those contemplated ahead of time by the designers, and these are conceptually high-level, it feels like the expressive capacity of the world is going to be very limited. For example, if I open doors by walking to within some defined distance and triggering a discrete first-class event (“press E”), this is a red flag about the world’s asymptotic depth.
Of course, there is no bright line for what I’m thinking of here when I say things like “conceptually high level”: at some point, the possible interactions have to be determined ahead of time, just as they are in our universe. But just because the line has to be drawn somewhere I don’t think precludes us from seeing the qualitative difference between a physics in which a car moves because the model’s location in space is directly updated by the game engine, versus one in which it moves because the wheels impart a force on the road, versus one in which chemical reactions propagate force down a cam shaft to a transmission which in turn imparts force onto the axle, then the wheel, then the material of the road surface, and the car is propelled forward via second law reaction. Even this is still very high level! But we can see the gain in expressiveness as the level of the “physics” is moved down.
When I started thinking about this, one of the things I quickly realized is that there is a deep connection between relativity and multiplayer capability. That is, while a game can obviously simulate travel through space for multiple players, it cannot simulate travel through time in a way that has any meaningful consequences, and relativity tells us that there is a fundamental connection between these things in our physics. A multiplayer simulated reality for humans in our universe cannot be relativistic. If we want to engineer something that looks like our world, we likely need to start from scratch rather than simply trying to encode the interactions of our own universe.
Another thing that jumped out is that there are significant performance optimizations available if we use quantum computers instead of classical ones. I’m not even talking about simulating quantum effects - just the problem of maintaining and processing the state of a large universe. Updating the state of every point at every moment quickly becomes intractable. (And, not that it matters, has relativity problems.) A quantum computer might enable lazy evaluation, allowing us to defer many computations until some observer (player) wants to use them.
The more I thought about the challenge of building a highly expressive virtual world, the more interesting physics questions seemed to fall out of it. It’s obvious that there are many years worth of problems hidden in this “reverse physics” category, and while I found a little bit of prior work on the idea, I was surprised not to find a thriving field already deep into these questions. I hope that is where we are heading.
I’ve written a longer preprint-style article teasing out some of the first things I encountered while falling down this rabbit hole, which is probably beyond the scope of a post here, but I would love to hear from anyone working in this area already or who finds it as interesting as I do.
Full article: Physics considerations for a simulated reality, pdf