Seeing Things Your Way
Hear Director of Sensor Systems Ramin Mirjalili describe how he tracks eye movement without cameras
Part of your job at Mojo is overseeing eye tracking solutions. What is that?
Eye tracking is the capability to follow what object(s) a person is looking at or what direction they’re looking—where their gaze is headed.
Why is everyone in AR/VR talking about eye tracking right now?
There are two reasons eye tracking is so popular right now. First, you need to lock AR content onto the real world. Let’s say you want to put a digitally rendered sign on a real tree. If the user walks around and comes back, you want the sign to stay in the same place in the real world. You have to know where someone is looking so you can keep putting AR content in the same place, even if they look away.
Second, you want to make sure you project higher resolution images onto the fovea—the sharpest area of your vision versus the entire display which requires more processing power that drive larger batteries and can be warmer to the user. You need to know where people are looking to display high-res images where they’re looking.
What approach is most common for the headsets and glasses used today?
Most of today’s solutions are what we call “video system”. These head-mounted devices have cameras that look for your iris and pupil and track those through a series of video frames that they process. This approach has come a long way, but these systems do have some challenges: The power to process all those frames is still computationally intensive and tends to slowdown as data accrues. If your goggles move as your head moves, they lose accuracy and when you blink or close your eyes briefly, they lose line of sight altogether.
A video tracking system on a contact lens doesn’t sound particularly feasible given size and power requirements. So how does Mojo Lens perform eye-tracking?
We are “instrumenting” the eye, which means we have sensors embedded within the lens that measure and track motion changes.
Why did you decide to go with motion sensors?
Because of our form factor, we needed a solution that had better resolution/translation of movement, lower latency, and a lower power budget. We realized we could measure eye motion from the inside out, which means we needed to instrument the eye. From an engineering perspective, it just makes sense to measure motion by sitting on the motion platform itself, using the proper set of sensors. We started by looking at different sensing architectures, and worked with sensor manufacturers to customize, and integrate mature sensor technologies already out there, similar to the ones in your cell phones.
What’s the biggest challenge in eye tracking?
Well, your eye moves around a lot, more often than you’d think. Think several times a second at least when you’re awake. Of course you’re always looking around at your environment, but even when you’re staring straight ahead, there are small, fast motions that the eye does that you are not aware of called microsaccades. And because your eye movement is controlled by muscles, those are never perfectly still either.
So the problem isn’t that our sensors can’t pick up all that movement—they can. The problem is deciding how small of a motion you need to capture and analyze, now that becomes challenging. At what point do tiny eye movements stop being significant data and turn into noise? The signal/noise threshold might be higher for AR applications where text is being displayed, but lower for something like a stop sign that requires only recognition.
How did you get started on eye tracking, and what fascinates you about it?
My PhD research was on the idea of an AR contact lens. I was probably the only person in this company—apart from the work that Mojo co-founders were already doing independently—who’d already theorized approaches to integrating a display inside a contact lens. It was the most sophisticated problem I could find to solve—it's a beautiful engineering challenge to work on miniaturization of optics and electronics for a smart contact lens. Ultimately, I fell in love with the concept.
I continued the research during my post-doctorate work by working on motion sensors, micromechanics, and micro-magnetics. My passion for sensors and transducers grew as well, and I went into learning about health and wellness applications using medical, optical, and other types of sensors.
The combination of my love of the smart contact lens idea and my knowledge and background in sensors is what gave me the opportunity to join Mojo and continue working on solving hard and meaningful problems.