The archive · Product Ideas · Product decision · 2021
Android's Camera Switches let eye and face movements operate the phone
Android makes its front camera a switch: on-device ML reads face and eye gestures so people with motor and speech disabilities can control and communicate.
Google (Android)
What it had to solve
Most people navigate their phones with voice commands or their hands, but that is not always possible for people with severe motor and speech disabilities. Android wanted a way for them to control the phone and communicate using face and eye movement alone.
How it works
Most people navigate phones by voice command or by hand, but that is not always possible for people with severe motor and speech disabilities. Android set out to let them control the phone and communicate with their face alone.
It built Camera Switches, a feature inside Switch Access in the Android Accessibility Suite. The phone's front-facing camera, combined with on-device machine learning, detects face and eye gestures — looking up, left or right, smiling, raising the eyebrows, opening the mouth — and maps each to an on-screen action.
The feature began rolling out in September 2021. Because recognition happens on the device, the video never leaves the phone, and because the camera is already there, no extra hardware has to be bought or installed.
Why it lands
- On-device machine learning keeps the video private and makes the switch work offline, with no cloud round-trip in the accessibility loop.
- Using a sensor that already ships in every phone removes the cost and availability barriers of dedicated assistive hardware.
- Mapping several distinct facial gestures to different actions lets users not only select items but compose messages, so communication becomes possible, not just navigation.
What it did
A camera that already ships in every phone replaced dedicated assistive hardware: users gained a free, on-device input method for navigating and communicating, processed locally with no cloud round-trip. The work is filed in the D&AD awards archive under Digital Experience Design.
What you can take
When users cannot operate a device the usual way, use the sensors already inside it: on-device ML made the camera an input channel, removing the cost barriers of special assistive hardware.
Since then
Camera Switches began rolling out within the Android Accessibility Suite in September 2021 and became fully available to Android users by the end of that month. The work is filed in the D&AD awards archive under Digital Experience Design, and it stands as an example of accessibility shipped as a platform default rather than as a separate product.
Sources
spotted an error? The archive wants to know.
Your turn
You just read one. Describe the brief you are staring at, and see who has been given the same problem.
Free account · 3 free questions · no card