October 07, 2024
A near-total remake. While some of the material in the original footage might be salvageable, it wouldn't be much. Extrapolating a heightmap to convert a 2D movie into a 3D one is easily accomplished via AI. The results aren't always great and objects generally look like flat cardboard standouts in a diorama, but it's enough for most purposes. Converting the same footage to VR requires is another thing entirely because a simple height map isn't going to be enough. That kind of 3D effect of only convincing when viewed head-on. For VR you need to be able to represent objects, people, and environments from every conceivable angle, and do so with realistic geometry. 2D footage simply doesn't contain all the information you need to extrapolate all that.
So, given that letting an AI just automagically turn the footage into a complete volumetric representation of the movie is off the table, what could you actually salvage from the original?
This is probably the easiest thing to preserve in a VR remake. You couldn't do a straight lift of the original soundtrack, unfortunately. Individual lines of dialog, Foley work, and musical scores would need to be isolated and remixed to allow for the creation of a 3D soundscape within the VR version of the movie, as just piping in the original single-channel audio would make for a highly disorienting experience. But sound engineers have been doing this for a long time, and have gotten very good at it.
These are probably a complete wash. I know it's tempting to say that you could just use photogrammetry, but it's hard to imagine you could ever get an acceptable result unless you still somehow had access to the original set. Creating a static mesh through photogrammetry requires very careful lighting, camera placement, and most importantly coverage from as many different angles as you can manage. Film sets in that era generally only got shot from 3 or 4 angles and were usually mostly obscured by the cast. And the lack of color information means the algorithms that stitch it all together to produce a final mesh are going to be at a disadvantage, to begin with. There might be one or two examples where it would work, but since most wouldn't you might as well just redo everything from scratch.
Again, you're probably going to have to recreate the cast from the ground up. But in this case, you may actually be able to use photogrammetry to at least get you started. The technique doesn't work for objects in motion, but the way they framed reaction shots in the black and white days means you're going to have a wealth of tight angle close-ups of faces, eyes, and lips against a fairly indistinct background. Mix that in with some publicity shots and reference photos from the hair and makeup department, and if you don't have enough data to do a full photogrammetric mesh of the actors' heads you should at least be able to provide photorealistic textures for your model.
Additionally, while this is again a bit of a long shot, once you've got the mesh rigged for animation you might be able to use those close-up shots to do some rudimentary motion capture to try and capture the exact expressions and performances used in pivotal scenes. You'd still probably want a human animator to come in and clean it up since you'd be working from a single source (ideally you'd want 3 at least for facial work) but at least it's something. Or, if you're really determined to get that exact performance, you could stick with the Myst V approach and just use the video as a texture map. Because what has having a nose you can see in profile ever really done for humanity?
This one's going to be a near-total loss. In VR the audience has complete control over the camera, which means you can't restrict or distort their vision to achieve a specific effect. After all, if you take away the ability to use Dutch angles then this famously evocative shot from The Third Man just becomes a guy running down an empty street:
Reworking the blocking and how information is imparted to the audience, in general, is pretty much obligatory if you want anything within the story to carry the necessary dramatic weight, which will likely require taking a page from how video games seek to set the scene.
It would be an interesting project to be sure - but the work you ended up with at the end of it would be wholly distinct from the one that inspired it, and I suspect most would find it to be an inferior experience to the original.
