October 07, 2024
In terms of technology, I’ll paste here what I wrote as an answer to a similar question.
The first principle is stereoscopic vision: the image that is projected to each eye is slightly shifted to the right and left, like in real life. This gives you a 3D image.
Lenses are used to shift the focus of the image that is very close to your eye on your retina so you can see clearly.
Sensors (accelerometer, giroscope and magnetometer) are used to capture the relative movement of the headset, so the image is updated correctly when you move your head around. It gives you 3 degrees of freedom (called 3DoF) through rotations around the X, Y and Z axes. Desktop and mobile headsets both offer 3DoF.
For desktop headsets (e.g. Oculus Rift, HTC Vive, Playstation VR), you also have optical position tracking so you can move in space and the image is updated accordingly. So you can look around objects, crouch, lie down and walk around. This give you an extra 3 degrees of freedom, translations (movement) along the X, Y & Z axes. It’s called 6 degrees of freedom (6DoF) VR.
Now, in terms of the experience, mobile give you a fun experience for a low price. But the lack of position tracking makes the experience a bit gimmicky. Also, limited controls make it less fun. Fortunately, controls are improving, with Google Daydream for example. Also, solutions are investigated and developed right now to offer mobile position tracking.
When you have 6DoF and room scale VR, you’re transported to a whole new dimension. It’s a clear difference. And to me, this is how VR should be experienced.
There’s resolution differences between headsets, which are a bit noticeable, but are not a big deal. Limited resolution on even the best headsets, is an issue though. You want 6k+ resolution so that you can start forgetting about pixels in your field of view.
There’s a lot of companies focusing on building headsets today, which will help pushing progress forward towards much better headsets.
One of the limiting factors is the hardware needed to run high-resolution headsets at a high framerate (90+ fps). It will slow down progress. Also note that desktop headsets offer better framerates compared to mobile, which helps fight motion sickness.