HOT CHIPS: A Display in Your Contact Lens

Is augmented reality (AR) going to be the next big thing? Tim Cook, CEO of Apple, thinks it will pervade our entire lives. But it certainly won't be until we don't have to wear big, heavy, clunky headsets. I was impressed at CES a couple of years ago when I tried on the first AR glasses that seemed acceptably comfortable. See my post What's New in Consumer Electronics? CES 2020 for more on that.

But going a step further was the presentation by Mike Wiemer and Renaldi Winoto of Mojo titled Mojo Lens—AR Contact Lenses for Real People. That's right, they are putting a display and all the associated electronics into a contact lens.

There are lots of reasons to like this, some of which are the same reasons as many people who need vision correction prefer contact lenses to glasses. Mojo have been doing this since their first lens in 2017 (which admittedly just turned on a single LED to trailblaze the technology). This year they have a feature-complete version that has fast wireless data, eye tracking, and a microLED display (black and white).

The first question everyone has is where is the display, and how do you see through it. You might think that the display would be off to the side and use some weird optical effect so you could see it. But no, the answer is that it is right in the center of your pupil. They can do that because it is tiny, only reducing the light arriving on your retina by a few percent, which is not noticeable. This is the world's tiniest display. Its pixel pitch is 18.7um, giving 14,000 pixels per inch. You know how sometimes photographers put something in a photograph for scale, like a banana or a baseball. Well, that picture above shows you the pixel pitch with a red blood cell for scale.

Having built the world's smallest display, you have to put it right in front of the pupil so it can be seen. It doesn't block your view. Roughly it is like your pupil being a bit smaller or putting on a pair of normal (not sun-) glasses. But you need optics, too. A display this close to the eye will not project an image onto the retina and so you won't be able to see it. They use reflective projection optics to project a sharp image onto the retina. The whole thing, that they call a Femto-projector, fits into the thickness of a contact lens. And, as discussed above, it doesn't affect your view of the world.

But that's not enough, you need eye-tracking. Your eye is not used to having a static image projected onto when your eyes move. They are used to the image moving. So a static image appears to move with the eye. Eye-tracking follows your eye so that the image moves just the way you would expect so you get the opposite effect, that it looks static even though it is actually moving. They consider that they have the world's best eye-tracker. And it is inside a contact lens.

System Partition

Everyone at HOT CHIPS wants to know how the system is put together, and the system is partitioned as in the above diagram. The lens contains the display and the sensing. There is ultra-low-power wireless to link the lens to the relay device, which handles generating the data or getting it from the cloud. So where does the ultra-low-power wireless go? In the lens, too. This is a multi-die design!

This is actually an advantage since chips are flat and contact lenses are rounded. This means that it is easier to fit several small die into a lens than one big one. The wireless SoC, the power management, the imager, the micro-display, and the motion sensors are all separate die embedded in the lens.

It wouldn't be HOT CHIPS without an obligatory block diagram so there you go. This is the control processor for the whole system.

And here's the power management (left). The system is charged wirelessly overnight. There is also an image sensor (right) since AR needs to know what you are looking at. This is a through-the-back CMOS Image Sensor (CIS) with 256x256 resolution at 41 frames per second at 61 microwatts. The die area is 1mm x 1.3mm.

Putting it all together allows the user to look at a scene, like a tractor, and see not just the scene (via normal light) but also the AR annotation (in this case the tractor outline, maybe so that you can repair it).

One of the questions was about how it works in bright light. If it is too bright, then the display gets overwhelmed, just like looking at your smartphone in bright light. There are two things you can do. One, put on sunglasses as you normally would. Less intuitive, second, you can shut your eyes.

It was Arthur C. Clarke who said any sufficiently advanced technology is indistinguishable from magic. Well, putting an entire display and camera system, including wireless interfaces and battery charging into a contact lens qualifies to me! It's not quite a smartphone in your eye but close.

Sign up for Sunday Brunch, the weekly Breakfast Bytes email.

.