I drove a car using only eye tracking and a phone at MWC 2024

At Mobile World Congress, there are always a few bizarre demonstrations of technology. HONOR had one of its own in which it allowed journalists to control a car using only eye movements. This was made possible by eye-tracking tech inside the company’s newly launched HONOR Magic 6 Pro smartphone.

Before you assume I was driving a car through the streets of Barcelona just by looking at things, let me make some details clear. I could only do four things: start the car, stop the car, move it forward a few meters, and move it backward a few meters. Additionally, the car was in a warehouse the entire time, with a safety crew monitoring the situation to ensure no one was hurt. Despite the lack of a wild Barca joyride, it was still pretty impressive.

So how does this work? Let’s jump in!

Eye tracking in smartphones is nothing new. Way back in 2013, Samsung launched its Smart Scroll feature inside the Galaxy S4. Using eye tracking, you could scroll a page up or down simply by looking at the top or bottom of the phone. It worked surprisingly well — but few people liked it because using your finger was both faster and easier. Eventually, Samsung abandoned Smart Scroll.

A lot has changed in eleven years, though, and HONOR is ready to give eye-tracking tools another chance. On the Chinese version of the HONOR Magic 6 Pro, eye-tracking tools already enable people to expand and open notifications just by looking at them. This requires much more finesse than Smart Scroll could have ever provided, and that finesse is made possible by AI processing on the device itself.

To track eye movements, the phone uses its front-facing 3D sensor, which lives in the pill-shaped display cutout. To set it up, you need to calibrate the phone so that it knows where your eyes are and how your eye movement works. It only takes a few seconds to do.

I was astonished at how accurately the system could detect what I was looking at once I had calibrated the phone — but it certainly wasn’t flawless. This makes sense, though: this calibration setup is heavily skewed at the moment because all the data is based on Asian eyes. I also had some trouble with my glasses, which seemed to confuse the system a bit. I took them off, and things worked much better.

These are some of the reasons why the global version of the Magic 6 Pro isn’t launching with this capability. HONOR wants to collect more data on a wider range of eyes before it pushes the feature to global products.

Once you’re set up, you can look at a screen element (such as a button or notification) and stay looking at it for a second or two, and that’s all it takes to activate it.

So that’s how the eye-tracking tech works, but now let’s get into driving the car!

In HONOR’s warehouse space, an unidentified car (likely an Alfa Romeo Giulia Quadrifoglio) was hooked up to a self-driving system. This allowed the car to move forward and backward without anyone touching anything inside it. The car’s remote key was inside, which allowed it to auto-start and stop through smartphone controls.

HONOR built a simple app for this demo. The app had four buttons to control the four actions, with plenty of space between each button to avoid eye-tracking confusion. After all, you don’t want someone accidentally moving the car forward when they meant to move it backward, for example.

Like I said earlier, I did face issues with my glasses when I first tried the demo. But, once I removed them, I could “push” each button accurately and quickly. HONOR let us do it a few times (and even ride in the car one of those times, which was cool).

Overall, the demo was simple but effective. The impracticability of the demo notwithstanding, it did prove that smartphone eye tracking could be used for more than just scrolling your screen.

If you’re concerned about the security of any data obtained from eye tracking, HONOR says that it is taking that very seriously. All eye-tracking data is kept on-device, not in the cloud, and the selfie camera isn’t used at all for the tracking, so there’s never a full photo of your eyes. In a sense, it’s not too dissimilar from a phone’s fingerprint scanner.

The big question, though, is why is HONOR doing this? No one will be driving a car with just eye movements any time soon, after all. While that’s true, the idea of using eye movements to interact with technology is a sound concept. For example, if your phone were propped on your kitchen counter while cooking, using your eyes to move around a recipe would be incredibly advantageous.

What HONOR has really done here is put some theatrics behind a simple idea: “pushing” a button using your eyes. It’s a bit over the top, sure, but no less exciting.

HONOR says the global version of the Magic 6 Pro will get eye-tracking features at some point this year. It also said it might come to last year’s Magic 5 Pro. That device has the necessary hardware for the feature, so it’s just a matter of HONOR putting in the work to make it happen.

What do you think about this? Do you see any advantages to eye-tracking tech? Let us know in the comments!