Android's Camera Switches let eye and face movements operate the phone

Most people navigate their phones with voice commands or their hands, but that is not always possible for people with severe motor and speech disabilities. Android wanted a way for them to control the phone and communicate using face and eye movement alone.

Make the phone's front camera an assistive switch: on-device machine learning turns face and eye gestures — looking, smiling, raising eyebrows — into on-screen actions.

Android built Camera Switches into Switch Access, part of the Android Accessibility Suite: the front-facing camera, combined with on-device machine learning, detects face and eye gestures such as looking left or right, smiling or raising the eyebrows, and maps each one to actions on the screen. The feature began rolling out in September 2021 and needs no extra hardware.

A camera that already ships in every phone replaced dedicated assistive hardware: users gained a free, on-device input method for navigating and communicating, processed locally with no cloud round-trip. The work is filed in the D&AD awards archive under Digital Experience Design.

Most people navigate phones by voice command or by hand, but that is not always possible for people with severe motor and speech disabilities. Android set out to let them control the phone and communicate with their face alone.

It built Camera Switches, a feature inside Switch Access in the Android Accessibility Suite. The phone's front-facing camera, combined with on-device machine learning, detects face and eye gestures — looking up, left or right, smiling, raising the eyebrows, opening the mouth — and maps each to an on-screen action.

The feature began rolling out in September 2021. Because recognition happens on the device, the video never leaves the phone, and because the camera is already there, no extra hardware has to be bought or installed.

On-device machine learning keeps the video private and makes the switch work offline, with no cloud round-trip in the accessibility loop.

Using a sensor that already ships in every phone removes the cost and availability barriers of dedicated assistive hardware.

Mapping several distinct facial gestures to different actions lets users not only select items but compose messages, so communication becomes possible, not just navigation.

When users cannot operate a device the usual way, use the sensors already inside it: on-device ML made the camera an input channel, removing the cost barriers of special assistive hardware.

Camera Switches began rolling out within the Android Accessibility Suite in September 2021 and became fully available to Android users by the end of that month. The work is filed in the D&AD awards archive under Digital Experience Design, and it stands as an example of accessibility shipped as a platform default rather than as a separate product.