Tech Behind ThingsHow the ordinary machinery actually works

Data & Privacy

Your television is quietly identifying what is on its own screen

It does not need to know the source. It samples what it displays, matches the sample against a reference library and reports what it found.

Wooden blocks spelling 'Cyber Security' on a wooden grid background.
Photograph by Ann H via Pexels
Editorial note. Independent reporting and analysis. Nothing here is sponsored or paid for. How we work.

These are listed in the order worth acting on, which with automatic content recognition is not the order they are usually presented in.

What matters most

  • Samples of picture and sound are reduced to compact fingerprints.
  • It works on any input, including consoles and physical media.
  • The setting is usually enabled during setup under an unrelated name.

Fingerprinting what is displayed

The television periodically samples the picture it is currently showing and the audio it is playing out through its speakers. Those samples are reduced to compact fingerprints, small summaries designed to survive compression and scaling. The fingerprints are sent to a service that compares them against a library built from broadcast and streaming sources.

A match identifies the programme, the channel or the advertisement, along with the moment it was displayed. The picture itself is not transmitted, which is why manufacturers describe this as recognition rather than as recording.

The input does not matter

Because recognition happens at the panel, it applies equally to broadcast television, a streaming application or an external device. A games console, a set-top box or a disc player connected over the display cable is sampled exactly the same way.

This is what distinguishes it from a streaming service knowing what you watched inside its own application. It means viewing habits across every source converge into a single record held by the television's manufacturer. Content with no online component at all, including physical media, is identified whenever it appears in the library.

What the data is used for

Audience measurement is the traditional purpose, and it produces genuinely useful aggregate information about what large numbers of people watch. The commercially valuable application is advertising, where knowing what you watched informs which advertisements you are shown. Some arrangements match the television to other devices in the household by their shared network address.

That matching allows an advertisement seen on the television to influence what appears on a phone in the same house. Manufacturers frequently earn more from this than from the margin on the hardware, which shapes how the feature is presented.

Where the setting hides

The feature is usually presented during initial setup as part of a bundle of agreements accepted with a single confirmation. Its name rarely mentions recognition, appearing instead as viewing information, interest-based advertising or a similarly neutral phrase.

The short version: it can normally be turned off afterwards, in a privacy or terms section rather than in the picture settings. A factory reset restores it, so any set that has been reset needs the setting checked again. Software updates have occasionally reintroduced or renamed the option, which makes an occasional check worthwhile.

What else the set can observe

A connected television sees the other devices on the local network and the address the household presents to the internet. Applications on the set carry their own analytics, which are separate from the manufacturer's recognition feature. Sets with microphones for voice control listen for a wake word locally and send audio after it is detected.

In practice, built-in cameras are uncommon now, and where they do exist they should be treated exactly like any other connected camera. The practical point is that a television is a general purpose computer with a network connection and an audience.

Implementations differ, and vendors are not obliged to document the differences.

Reducing what it reports

Turning off the recognition setting is effective for that feature and does not affect the analytics inside individual applications. Keeping the set off the network entirely stops all of it, at the cost of applications and firmware updates. Using an external streaming device and treating the television as a display moves the data collection to that device instead.

The short version: blocking specific destinations at the router works until the addresses change, which they periodically do. Local rules on consent for this kind of processing differ considerably, and enforcement action has occurred in several jurisdictions.

Everything above, in order of what to do first

  1. Fingerprinting what is displayed. The television periodically samples the picture it is currently showing and the audio it is playing out through its speakers.
  2. The input does not matter. Because recognition happens at the panel, it applies equally to broadcast television, a streaming application or an external device.
  3. What the data is used for. Audience measurement is the traditional purpose, and it produces genuinely useful aggregate information about what large numbers of people watch.
  4. Where the setting hides. The feature is usually presented during initial setup as part of a bundle of agreements accepted with a single confirmation.
  5. What else the set can observe. A connected television sees the other devices on the local network and the address the household presents to the internet.
  6. Reducing what it reports. Turning off the recognition setting is effective for that feature and does not affect the analytics inside individual applications.

The takeaway

It is not watching the room; it is reading its own screen and telling somebody.

The constraint is almost always physical, and marketing rarely mentions which one.

Questions readers ask

Does this mean my television is recording my viewing?

It generates fingerprints of what is displayed rather than storing video. The resulting record of what you watched is the point.

Will turning it off break anything?

Recommendations may become less tailored. Playback, applications and picture quality are unaffected by the setting.

Data & Privacytelevisionprivacymeasurementhome
Mikkel Aas
Editor, Tech Behind Things

Mikkel edits Tech Behind Things and has taken apart more devices than he has successfully reassembled.

Also by Mikkel Aas