VR headsets give enough data for AI to accurately guess ethnicity, income and more

Blending virtual reality with artificial intelligence could turn into a privacy nightmare.

By analysing how people moved while wearing virtual reality headsets, researchers said, a machine learning model accurately predicted their height, weight, age, marital status and more the majority of the time. The work exposes how artificial intelligence could be used to guess personal data, without users having to directly reveal it.

In one study at the University of California, Berkeley, in February, researchers could pick out a single person from more than 50,000 other VR users with more than 94% accuracy. They achieved that result after analysing just 200 seconds of motion data. In a second June study, researchers figured out a person’s height, weight, foot size and country with more than 80% accuracy using data from 1,000 people playing the popular VR game Beat Saber. Even personal information like marital status, employment status and ethnicity could be identified with more than 70% accuracy.

The researchers used a machine learning model to analyse data uploaded to virtual reality headsets, such as eye or hand movements. “The easy ones for the model are age, gender, ethnicity, country,” said lead researcher Vivek Nair at UC Berkeley. To figure out someone’s age, for instance, the model could guess based on how quickly they hit a virtual target. Having a faster reaction time is correlated with having better eyesight and being younger in age. “But there are even things like your level of income, your disability status, health status, even things like political preference can be guessed,” he said.

Nearly half of the participants in both studies used Meta Platforms Inc’s Quest 2, 16% used the Valve Index and the remaining participants used other headsets such as the HTC Vive or Samsung Windows Mixed Reality. Virtual reality headsets capture data that wouldn’t be available through a traditional website or app, such as a user’s gaze, body language, body proportions and facial expressions, said Jay Stanley, senior policy analyst at the American Civil Liberties Union. “It brings together a whole bunch of other privacy issues, but also intensifies them.”

Already, Meta, which makes most of its money off of advertising based on user data, has been relying on machine learning to fill in the gaps of what it knows about people, though it’s unclear how much VR data is in the mix. In 2021 Apple made changes to its privacy policy that limited the amount of data Meta could track on iPhones, wiping out US$10bil (RM45.71bil) of revenue for the social media giant. That forced the company to invest in AI. This year, Meta returned to double-digit revenue growth, after improving its AI to predict what content and ads people want to see.

Meta has been running limited ads in VR headsets since 2021, and said at the time that it wouldn’t use data processed and stored on the devices, such as images of hands, to target ads. When asked for more detail on the policy for its headset-derived data now, Meta pointed Bloomberg to its Quest Safety Center, where the company explains how wearers can set their avatar, profile picture, name and username to private, providing some control over who else can see it. The company also explains that “data sent to and stored on our servers will be disassociated from your account when we no longer need it to provide the service or improve the eye tracking feature.”

Meta has come under scrutiny in the past for collecting sensitive personal data on its users. In 2021 Meta shut down its facial recognition system and removed more than one billion facial images after facing regulatory pressure. Biometric data like facial images are particularly sensitive because they can’t change and can easily identify a specific individual. Nair said that VR headsets capture equally sensitive data, but because the technology is newer, users and regulators don’t understand it yet, making it potentially more dangerous.

Since VR headsets need to collect data such as eye and hand movements to work, privacy controls are much harder to build than for websites or apps. There are a few ways, like encrypting the information VR headsets collect or limiting the amount of data that’s being stored, Stanley said. But the companies that make these headsets also “have incentives to gather information about people for marketing,” he said.

Privacy controls and consumer awareness about how much data VR headsets collect is low, according to researchers. Combined with powerful AI extrapolations, “I don’t think it’s reasonable to expect consumers to defend themselves here,” Stanley said. “The knowledge gaps are just too large and the technology moves too fast.” – Bloomberg