How the new VR screen could end the smartphone

A smartphone screen is a wonder of the world. It’s not just that it’s bright and colorful and sharp. In some ways, it’s as good as human biology allows. We’ve packed so many pixels into such a small space that any more would be lost on us. We can’t make the screens themselves bigger, because then they’d become too large to hold. The only way to get more information from a smartphone screen is to bring the pixels closer to our eyes, with the device somehow mounted on our heads rather than holding it in our hands. Instead of a phone as we usually think of it, it would be more like a pair of glasses.

Sound unlikely? In fact, many smart CE companies (Apple, Microsoft, Google, HTC) are already working on this new screen. When it arrives, experiences you’ve only seen in the movies will become the stuff of everyday life.

The “diffraction limit” for the human eye

When you look through a small hole, things on the other side appear blurry. This is because when a ray of light passes through a hole on its way to your retina, it spreads out a little. Think of watching a wave from the ocean hit a sea-wall with a narrow opening: The way the straight wave becomes a ripple spreading out on the other side is the same as what happens to rays of light when they go through a hole.

You can play with this by making your own “pinhole camera” and looking through it at text far away. The smaller the hole, the blurrier. And your iris is, of course, also a hole.

Source: Getty Images: Carmelo Geraci/EyeEm

This means that our eyes, given their size, have only a certain ability to see detail. For the human pupil, at about 5 millimeters across, we can express this limit in pixels-per-degree, and the number is around 60. So, for example, if you hold a quarter at arm’s length away from your eye, it’s going to take up about 2.5 degrees of your view, which means that a little square display of 150 pixels across will look “perfect” to your eye. Any more pixels will be a waste because you wouldn’t be able to see them.

Starting in about 2010, our smartphone displays reached that level of quality, where we could no longer see the pixels, even held as close to our face as comfortable — a change that you may remember Apple aptly branding the “Retina” display. Even big-screen TVs have now reached that same limit . Anything beyond 4K is a waste of money because you can’t see the difference unless you are sitting so close you can feel the heat from the screen!

This means that a 6-by-3-inch phone held in our hands is never going to take up more than a tiny part of our field of view, and will never be able to show us more than the few dozen lines of text we can read on it today.

Insatiable appetite

But both our appetite and capacity for digesting visual information is tremendous. We love screens, and the bigger the better. We would love it, for example, if our laptops could magically unfold to have four screens instead of just the one (like those super-cool folding ones on Westworld).

Ideally, we’d be able to see screens in every direction, with the option to turn them off when we want to pay the closest attention to the real world. And these will be a far cry from early prototypes like Google Glass, which had an even smaller field of view and text/information capacity than your smartphone display.

The biggest possible screen

That is exactly what is being worked on: If you could fix the screen in front of your eyes with lenses that make it comfortable to look at, and which also have the ability to precisely sense the rotation of the head, you could create a magic new “screen” that completely surrounds you with pixels too small to be seen. Wherever you turn your head, the pixels right in front of your eyes would change to display the part of the virtual screen where you are looking.

This new screen will appear really, really big — about the equivalent of 16 4K monitors, with about 200 million pixels. Imagine being able to snap your fingers, any time, and be surrounded by 16 monitors containing any content you’d like — ones for email, text messages, browsing, video and whatever else you’d like to remember to check on. No-one will be able to see the content but you, and they come with you everywhere, just like your smartphone.

Trillion-dollar market

Would you wait in line at the Apple store for a $500 headset that surrounds you with 16 magic floating 4K monitors that don’t weigh anything and no one else could see? Of course you would — and you will. And by the way, you can keep the keyboard and mouse on your old desk, too . You just won’t need the monitor anymore.

Source: Hoxton/Paul Bradbury

This is exactly why so many great companies —  Apple, Microsoft, HTC, Google and startups like Magic Leap, Avegant and ODG — are working on trying to build this screen. The worldwide market for screens is about a trillion dollars, so whoever gets this new screen right will reap the rewards.

Accessible to all

Because they are self-contained and therefore cannot consume much in the way of computing power, these new devices will be less expensive than their predecessors — about the same price as a smartphone. And this means that, like the smartphone in comparison to the PC, they will be widely accessible  — in the hands of billions of us within the next 10 years.

So it is possible that this change will be empowering for the many people worldwide who currently have access to simple smartphone screens but not the more powerful work and learning opportunities offered by the expensive desktop and wall-mounted screens found in only more affluent homes and offices today. These inexpensive devices could give everyone worldwide the equivalent of a gigantic Bloomberg terminal.

VR and AR are lucky benefactors, not the killer apps

Thus far I haven’t focused on 3D VR worlds or AR objects superimposed on the real world, because these new screens won’t need them to still be wildly successful. Like camera apps were to smartphones, VR and AR applications will be the lucky beneficiaries of the race to a new screen. Once you have a device with such a screen, you can display 3D content or superimpose 3D content on the real world or travel to virtual worlds or communicate across the world as an avatar. Incredible VR applications that enhance human connections and experiences despite great distances are in the works now. But you won’t have to rush out to buy a headset — you’ll already have one for browsing and email.

Multiple companies will release headsets and glasses in the coming couple of years that will replace the smartphone screen as the way we take in visual information from our computers. Our first use of these screens will be to do all the things we struggle to do on our smartphones. Following that, virtual worlds, VR and AR will begin to make use of these screens, allowing us to augment or substitute entirely new worlds for the real one.

Everyone’s searching for the new “killer app” for VR and AR headsets, but you are already looking at it by reading this article.