Nine problems with Vision Pro passthrough: ‘VR trying to be AR’

Reviewers have highlighted nine problems with Vision Pro passthrough – the use of external cameras to project an image of your surroundings.

These issues result from Apple’s attempt to simulate an augmented reality experience in what is actually a virtual reality device …

Vision Pro passthrough

It’s widely believed that Apple’s long-term goal is a Glasses product which allows you to see the real world while overlaying information: true augmented reality (AR).

Vision Pro, in contrast, is a fully-enclosed virtual reality (VR) headset. Apple addresses this by using a set of high-quality external cameras, allowing it to project an image of your surroundings in real time.

This isn’t a new idea: Meta does this even on its consumer VR headsets. But Apple’s goal is to make what you see indistinguishable from direct sight.

Nine problems identified

Reviewers are generally very impressed by this, but as The Verge’s Nilay Patel notes, you can’t escape the limitations of cameras. He identifies no fewer than nine problems:

• Motion blur
• Distortion (warped straight lines, and flickering displays like a microwave clock)
• Blurriness in low light caused by high-ISO noise-reduction
• Wavering brightness levels
• Limited color reproduction
• Limited field of view
• Vignetting at the edges of the viewable area
• Color fringing
• Highlights reflecting in the lenses

Here’s what he has to say:

The Vision Pro cannot overcome the inherent nature of cameras and displays. You can easily see motion blur when you move your head in the Vision Pro — motion blur that increases in low light and leads to some weird warping of straight lines. Low light also causes the overall sharpness of the video passthrough to drop as noise reduction kicks in: my iPhone screen got noticeably blurrier when the sun set.

If you’re in a medium-lit room halfway immersed in a dark virtual environment with a bright window open — say, sitting in your kitchen at night with the lights on while writing a review in a Google Docs window floating on a dark beach — you will notice the display brightness slowly ramp up and down as the system tries to average out the brightness of everything you’re looking at. The LCD clock on my microwave flickers when I look at it through these cameras. And Apple’s specs say the display supports 92 percent of the DCI-P3 color gamut, which means the Vision Pro can only show you 49 percent of the colors your eyes can actually see.

The displays have other limitations: the field of view isn’t huge, and the essential nature of looking at tiny displays through lenses makes that field of view feel even smaller. Apple won’t tell me the exact number, but the Vision Pro’s field of view is certainly smaller than the Quest 3’s 110 horizontal degrees. That means there are fairly large black borders around what you’re seeing, a bit like you’re looking through binoculars. 

On top of that, there’s a little bit of distortion and vignetting around the edges of the lenses, and you’ll see some green and pink color fringing at the edges as well, especially in bright environments. All of this makes the usable field of view feel even smaller. If you’re looking at something bright or otherwise high contrast — a white text window floating above a dark desert landscape, for example — you’ll see highlights reflecting in the lenses.

9to5Mac’s Take

As Patel says, these are inherent limitations of video cameras, especially in low light. Apple can beat the competition here, but it can’t beat physics.

All the same, the company has been careful to carry out hands-on demos in very controlled surroundings, and it’s only now the device is being used in real-world conditions that we get an idea of how these limits impact the Vision Pro experience.

Do these issues have any impact on the appeal the device has for you? Let us know in the comments.

Add 9to5Mac to your Google News feed.