How to use the new Live Captions in iOS 16

Apple is launching a new suite of accessibility features in the soon-to-be-unveiled iOS 16, and (finally) adding a Live Captions feature for all audio content across devices.

The new option lets users easily turn on automatic captioning in their Settings menu, which will apply to any audio played within the device, from phone calls, to FaceTime sessions, to videos. The captions appear at the top of the screen while audio is playing in real time, and users can also customize the captions' font size to better accommodate their needs.

It's a great addition to the multitude of accessibility features on Apple devices, which include customizable adjustments across vision, mobility, and even cognitive needs. But the company trails behind other brands and devices who already offer accessible captioning directly in-device, including the caption and real-time text-to-speech capabilities of Google Pixel phones announced earlier this year.

In its iOS 16 accessibility preview, Apple described the new Live Captions hearing feature as atool for the deaf and hard of hearing community to better navigate their devices, calls, video conferencing and social media apps with device-only, live generation of text designed to protect the privacy and safety of the speakers' conversations.

The best part — for both deaf users and active iPhone users trying to stay connected to a multitude of social connections — is that the captions account for multiple speakers and sounds. Devices will automatically detect and attribute different speech to each person on the call, using your friends' already-saved contact information.

Via Giphy(opens in a new tab)

The Live Captions aren't just restricted to iPhone users either, but can be turned on in the Settings menus of iPad and Mac devices. For Mac users, the update also includes a live text-to-speech feature for calls, similar to the Google Pixel update, where users can type a response and have it spoken aloud to others on the call.

How Live Captions work

While in an audio or video call, or watching a video on your device, Live Captions will automatically appear at the top of your screen in a separate pop up window, similar to a push notification. For calls, Facetime sessions, or conference meetings, speakers' names are determined by the contact information you have saved already, including contact images and first names.

For videos, Live Captions don't include the same audio descriptions as general closed captioning, only transcribing live speaker voices rather than all sounds playing on screen.

If you're an iPhone user interested in the new Live Captions, here's how to make sure the feature is turned on and customized to your liking:

Go to "Settings" and select the "Accessibility" menu

Go to your phone settings first, then select "accessibility" one-third of the way down the menu options. Credit: Apple / Mashable

Find all Apple Accessibility settings in your iPhone's Settings menu.

Scroll to the section labeled "Hearing" and select "Live Captions"

Once in your accessibility settings, scroll about halfway down to find the Live Captions option. Credit: Apple / Mashable

You'll probably also see a selection for Closed Captioning settings (different from Live Captions) which control how already-generated captioning services appear on your devices.

Turn on captions system-wide or in-app

The current Live Captions Beta shows options to turn on captions for your entire device or for calling apps specifically. Credit: Apple / Mashable

To turn on Live Captions across your entire device, toggle the top button to the "On" position. For calling apps, such as FaceTime, you can manually select which app allows Live Captions. Note that turning these on will alert the people you are speaking with that their audio is being transcribed to text.

Adjust to your liking using "Appearance"

You can adjust the font size and the color of both the Live Caption text and pop-up window. Credit: Apple / Mashable

You can also adjust the way Live Captions appear on your device by using the Appearance settings. In this menu, you'll find customization tools for font size, font color, and the background color of caption text windows.

Adjust the captions to your desired font size. Credit: Apple / Mashable

Credit: Apple / Mashable

The new feature is only available in English, for now, and is accessible on iPhone 11 models and up, iPad with A12 Bionic and later, and Mac with Apple Silicon.