The Apple Vision Pro Will Be Amazing for Entertainment

The Apple Vision Pro Will Be Amazing For Entertainment

Forgive me for committing the same sin that many pundits have before me. That’s commenting on a device – the Apple Vision Pro – that I haven’t actually touched yet. But enough is known to say a few sensible words about what keeps me busy: the quality of live and on-demand video VR and the utility of the new device for entertainment.

Apple may not mention “VR” in any of its marketing around the Vision Pro, but it’s obviously a device that can be used to consume VR video. And, from the specs, a very nice one at that. In the Vision Pro, the productions that we have been involved with – e.g., the Beijing Games – would have looked a lot better than they already did. More on that below.

Opposites Attract – Maybe

In many respects, Meta and Apple have taken opposite approaches. The most important one: the Oculus started as a VR device and now starts adding AR capabilities like pass-through video. The Vision Pro is an AR device at heart (although again, Apple won’t call it that, but all the examples are AR) that will be capable of quite amazing VR.

The Oculus Rift “Development Kit” was an experimental device when then-Facebook bought Oculus almost 10 years ago. It slowly grew into an ecosystem via the Go to the Quest (Pro) over the last 10 years. With perhaps the exception of the Quest Pro, all these devices were affordable to the enthusiast user.

Quest (né Oculus) was and still is essentially a gaming device. Meta’s recent launch of a gaming subscription confirm this. And even though Meta commissioned some stunning VR360 productions (e.g., The Soloist), VR video on the Quest remains a second-class citizen.

Apple starts at the other end of the spectrum, with a very high-end and expensive device. Expensive, but at launch already with a complete set of developer tools, a polished UX, a plethora of iOS apps to use on it, so the new device is ready to roll. This is the way Apple always does it: they don’t so much invent new concepts as execute extremely well on what others have pioneered. (Although 5,000 patents is nothing to sneeze at.)  The Vision Pro, even though billed as a Spatial Computer, is really an entertainment device at heart. There’s a reason the only launch partner on the podium was Disney.

Playing Catch-up

I am sure the differences will become less over time. Apple’s headsets can’t keep starting at $3,499 plus VAT; there will be models that lose the “Pro” and they will come down in price. Hopefully this won’t take too long. The price of the Vision Pro is a red herring; it’s actually fully in line with some other initial product introductions by Apple. The Vision Pro should be judged on its merits, not its price. “Make it good before you make it cheap” says Brad Lynch (SadlyItsBradley) in his live stream on the launch. And I can only agree.

The device is a stake in the ground, a start, the beginning of a new ecosystem, and I can’t wait to get my hands on one.

It provides a new benchmark for the industry, which has already started to play catch-up. Quest development will no doubt take a few usability cues from the Vision Pro and Meta has conducted a very significant body of research on visual quality. They call it the Visual Turing Test.. The pass-through video of these devices will keep improving as well. Apple’s new device will also inspire other headset makers, like Pico. Samsung will follow Apple back into the XR game after abandoning the GearVR 5 years ago. “Samsung, Google and Qualcomm will team up to develop advanced hardware and software to make this immersive technology a reality” according to the company’s report  of its Galaxy Unpacked event.

Pixels Per Degree

Which brings us back to video quality. I hope and believe that the Vision Pro will set a new bar for VR video. Again, Apple didn’t bill the device as a VR headset, even though they announced tooling to create 3D memories. (Which I found rather gimmicky, by the way – who’s going to run around at a kids’ party with a space helmet? But these clips might actually work very well when you could record them with an iPhone. And why not? That device is easily wide enough to house two cameras at the proper eye distance.)

The Vision Pro has higher resolution than any other headset and its micro OLED panels have better contrast than any backlit LCD panel will ever have, even with local dimming. How much of a resolution improvement will the Vision Pro provide over existing consumer headsets? The useful measure is pixels per degree (ppd), which is a function of display resolution and field of view. The Quest 2, for example, has a resolution of about 20.5 pixels per degree according to this Wikipedia page. (The Quest 3 should be a little higher, but probably not more than 10% of an increase when measured in one dimension.)

Apple announced the total resolution of the Vision Pro as 23 Megapixels across both screens, which is 11.5 Megapixel per screen. The pictures release by Apple seem to suggest an aspect ratio of 1,165:1, which gives us 3,660 pixels across – give or take a few. Let’s further assume a field of view per eye of about 90 degrees, then we get slightly more that 40 pixels per degree. It’ll be a bit lower if the single-eye field of view turns out to be wider, but it’s a good ballpark. It’s significantly more than the Quests have, and still lower than the Varjo headsets’ resolution in the central display.

Perfect for 8K VR180

That ~40 ppd is a very interesting data point. When we distributed the Beijing Games (see our IBC paper for details), Cosm produced the VR180 feed in 8K. This equates to an average of 41.5 ppd across the screen; a little higher (maybe 48 ppd) in the center of the image where you want that resolution, and a little lower towards the edges of the fisheye source video. It looked amazing on the Quest 2, better then any live VR I’d seen before.

But to date nobody, not even us, has ever been able to enjoy the full, glorious 8K resolution. We didn’t even get close. It must look completely stunning in the Vision Pro – the difference between 22.5 and 40-something ppd is very significant.

Obviously VR360 will benefit as much from increased resolution as VR 180, but the production will be more challenging. 8K bodies exist (e.g. the RED V-Raptor or the BlackMagic URSA) and there are matching fisheye lenses (e.g. the Canon 8-15 mm fisheye zoom lens). When doing VR360, the ERP (EquiRectangular Projection) resolution of the camera would need to be some 14.5K to fully exercise the Vision Pro’s resolution. A resolution of 8K is already at the lower limit of what current headsets need to fully show their resolution, let alone what a device like the Vision Pro needs. It will certainly make sense to move to a 12K production for the Vision Pro, and higher if possible.

We can’t wait to get one – porting ClearVR to it should be completely straightforward.

Perfect for Multiview, Too

Many of the use cases and examples that Apple showed in its keynote presentation used multiple screens. There seemed to be a hint of multiple videos for at least one brief moment. Doing a multiview sports experience on a Vision Pro should be mind-blowing. Imagine seeing MotoGP, Formula 1 or the Tour de France, and keeping track of the director cut and all your favorite athletes at the same time. Imagine combining this with data overlays and separate data screens, that can be fully personalised. And then imagine driving all of that, intuitively, with just a few flicks of your fingers.

Imagine tracking not just a football (soccer :~) game on a Sunday afternoon, but also that tennis match, the golf tournament and the cycling race. Imagine having a virtually unlimited number of TV sets floating around in your home, or having access to a virtually unlimited number of cameras from your favourite sports. And imagine getting these screens at almost the same resolution (again, in pixels per degree) that you’d get from your HD TV set at home.

It’s precisely these new ways of content consumption that Tiledmedia is exploring with our Mosaic Multiview solution, and from what we know today porting should be a breeze. Trying that tech on the Vision Pro is just another reason were dying to get one in the office!

In Short, Perfect for Entertainment

I don’t believe in the Vision Pro as a head-worn computer that is used hours on end – not even with all of its pass-through capabilities. Not this bulky, and certainly not with an external battery. But I’ve seen enough entertainment in VR to know that it will catch on as an entertainment device. Headset use will shift from primarily gaming to all-round entertainment as devices get better: better ergonomics and better audio-visual quality. The Vision Pro is a huge step towards that quality. It seems the perfect match for today’s state of the art in VR180 production and distribution.

New services will emerge that take advantage of the affordances of this new device. Live VR, which is really starting to approach “as if you were there” and novel services that utilise many virtual screens. I’ll say it one last time: just can’t wait to try it myself!