A. Market Developments

In 2022, Meta, the company behind Facebook, WhatsApp and Instagram, was the global market leader in VR headsets, with a share of 81 per cent. Chinese DPVR and ByteDance, the company behind TikTok, both had a seven per cent share in 2022, making them the second and third biggest market players.Footnote ² Apple made its XR debut in February 2024. The number of VR headsets sold in 2022 amounted to 9.1 million worldwide, according to statistical research firm Statista.Footnote ³ By comparison, 1.4 billion smartphones were sold that year.Footnote ⁴

There is a large gap between the optimistic predictions about the XR market in previous years and its actual size in 2022: $30 billion.Footnote ⁵ In 2017, Consulting firm McKinsey & Company predicted that the XR market would yield $60 billion in profits by 2022. Earlier, XR analytics firm Digi-Capital estimated it to be $150 billion by then.Footnote ⁶ Nevertheless, predictions for the future XR market remain substantial: the European Commission quotes predictions of a €800 billion market by 2030. Statista, however, now estimates it to be worth $62 billion in 2029, so predictions vary.Footnote ⁷

The impact of immersive technologies on human rights and society in general is strongly related to a large-scale consumer breakthrough, which has yet to happen. If it does, however, the human rights risks could be significant, both for consumers and non-consumers.

B. Use Cases

XR is already being applied and experimented with in sectors such as healthcare, training and education, entertainment, infrastructure, industry and art. Use cases include VR police training, trauma therapy, experiments with AR for anatomy lessons, 3D virtual environments for bridge design in international teams and AR assistance for order picking in distribution centres.Footnote ⁸

These applications build on two main benefits of XR experiences, which stem from the strong feeling of presence and emotion: ‘being’ together despite physical distance, and life-like, seamless experiences through (partly) virtual scenarios. A third advantage often mentioned is efficiency, especially in the logistics sector, for instance, by delineating an optimal route using virtual signage within distribution centres, for employees to efficiently fulfil order picking.Footnote ⁹ However, available studies on applications often used a small number of participants; therefore, conclusions should be taken with caution, and additional research is needed.

C. Data Collection and Analysis in XR

To understand the impact of XR on human rights, it is crucial to understand the interaction between users, software and devices. The Cybernetic Feedback Loop, a model developed by the Rathenau Instituut, describes this interaction in three steps: data collection, data analysis and application (see Figure 2 for an example).Footnote ¹⁰

Figure 2.

Cybernetic Feedback Loop.

All XR devices need to collect and analyse data from users and their surroundings to run an immersive experience; that is, to render and constantly adjust virtual elements based on user behaviour. Within the scope of our research, it is impossible to say with certainty what kind of data are collected by current XR devices. However, it was feasible to develop an overview (Table 1) of possible types of analysis and application of physical and behavioural data collected through different XR devices, based on information about sensors on existing XR devices, prototypes in development and existing data processing practices on smartphones and smartwatches.

Table 1.

Possible types of physical and behavioural data processed through XR devices

A. Datafication

Immersive technology is accelerating data collection and analysis. More and more information, of an increasingly intimate nature, is being processed. As Table 1 shows, hand movements, pupil reflexes and potentially brain activity can be used to match the XR experience to the physical environment, as well as to infer states of mind, characteristics and preferences.

Besides click behaviour or search history collected from desktop browsers and smartphones, companies behind XR systems can profile users based on information about physical and behavioural activities that is collected predominantly unconsciously, e.g. movements, pupil reflexes, heartbeat, et cetera, can be monitored by measuring systems in XR headsets. Technology companies are also developing (experimental) combinations with brain computer interfaces to make XR experiences more interactive.Footnote ¹¹ Measuring neuroactivities can lead to the processing of even more sensitive types of data, also known as neurodata.

B. Hyper-personalization

The (prototype) XR glasses, Galea, analyse brain and muscle activity. According to the company, the purpose is to measure emotions, stress levels and arousal. As a possible application, Finnish XR company Varjo mentions changing the colour of a virtual environment to alert users of their stressed or fatigued mental state.Footnote ¹² Such technological developments contribute to the immersiveness of the virtual world and make user experiences hyper-personal. The integration of generative AI into XR also furthers hyper-personalization, for example, with personalized advertisements in XR environments, or with applications such as Replika, which provides users with a digital ‘friend, partner or mentor’ that tailors itself to each user and with whom users can share AR experiences.Footnote ¹³ This far-reaching form of personalization can further exacerbate the personal ‘bubbles’ people are already experiencing as a consequence of social media algorithms, and complicate shared perceptions of reality.

C. Manipulation

Finally, technical advancements in XR make it easier to manipulate user experiences in both virtual and physical environments. This can be illustrated with Pokémon Go, which manipulated users’ perception and behaviour. The AR game managed to move players from public spaces to commercial locations in the physical world by planting game components there, nudging them to spend money. User movement in public spaces sometimes causes public nuisances due to the gathering of many Pokémon Go players, and causes harm when people are playing the game while driving.Footnote ¹⁴

A. Privacy

Article 12 of the Universal Declaration of Human Rights (UDHR) covers the rights to private life and correspondence, freedom from surveillance and data protection. Although certain types of data processing are necessary for XR to function, the amount of data currently collected and the sensitive information that can (potentially) be derived from it are not justified from a privacy rights perspective.

First, some of the big technology companies developing XR base their revenue models on data analysis and advertisements. This incentivizes them to keep adding new sensors to the already large variety their devices contain, resulting in data processing on a much larger scale beyond that of current smartphones.

Second, as illustrated by Table 1, an increasing amount of intimate information is or could be derived from the data collection, compromising subjects’ privacy. Data thought to be anonymous proves to be far from it in practice. For instance, a 2023 study showed that based on five minutes of head and hand motion collected by VR devices, a user could be uniquely identified with 94.33% accuracy in less than two minutes, out of a pool of more than 55,000 users. With this, the scientists demonstrated VR biometrics’ comparability to facial recognition and fingerprint identification systems.Footnote ¹⁵

Other research shows physical and behavioural activities that are monitored, for example, measuring eye-tracking, can be linked to age, gender, ethnicity, sexual preference and medical diagnoses such as attention deficit hyperactivity disorder (ADHD) and autism.Footnote ¹⁶

Third, collected data may be used for other purposes than initially collected, creating ‘function creep.’ Examples from current widely used technologies illustrate the risk of function creep when data are collected. For example, the app Muslim Pro reminded users when to pray and showed them Mecca’s direction based on their location data, but then the US military accessed and used these data to inform special operations abroad.Footnote ¹⁷ Companies that process more and more information should realize that the more data that is collected, the higher the aforementioned risks are.

Lastly, risks that apply to users of immersive technologies may also apply to non-users, since sensors of immersive devices also analyse the user’s surroundings.

B. Autonomy

Human rights instruments protect both physical and mental autonomy. More generally, Article 3 of the UDHR refers to the right to life, liberty and security (or inviolability) of a person. Through intuitive and covert influence, XR can be used to manipulate users, putting pressure on their mental privacy and self-determination.

According to visual information researcher Bibri and sustainability researcher Allam, the combination of digital technology and biotechnology makes it increasingly possible to predict people’s thoughts, preferences and behaviour, and thus how they respond to certain cues.Footnote ¹⁸ This means the sensitive data and inferences discussed above could be used to hyper-personalize (partly) virtual environments and nudge users towards consumption, but also to influence their ideas and views, for instance via computer-generated avatars.Footnote ¹⁹

C. Non-discrimination

Article 2 of the UDHR states that everyone is entitled to the rights and freedoms set out in the Declaration, without distinction. Examples of protected discrimination grounds are race, sex, language, religion and political opinion.

First, widespread adoption of immersive technologies could amplify the risk of digitalization in general: the growing ‘digital divide,’ a widening gap between those who benefit from digital technology and those who do not, because they have unequal access to the technology, or are impacted more by its drawbacks.

Secondly, online harassment can be exacerbated. Harassment of groups already considered marginalized in gaming and VR, such as ethnic minorities, women and the LGBTQIA+ community, is an emerging (but understudied) problem. It ranges from racist and sexist comments to virtual violence and virtual touching without consent. In a study based on personal experiences of participants, it is noted that, unlike in other (two-dimensional [2D]) online social spaces, because of the focus on embodiment and immersion, these behaviours in social VR resemble offline harassment.Footnote ²⁰

Thirdly, XR devices and environments can be non-inclusive. For example, cybersickness appears to be experienced considerably more by women than men, possibly because XR headsets are designed for a male target group, which has a wider eye distance.Footnote ²¹ Furthermore, XR experiences sometimes lack diversity in avatar options, making it difficult for underrepresented users to translate their identity through virtual visual cues (such as wheelchairs or accurate portrayals of race). This could even lead to minorities renouncing XR, furthering the lack of representation and the digital divide.Footnote ²² Currently, insufficient research exists on how discrimination manifests itself in XR. An ethnographic study that included interviews with industry experts showed more knowledge is needed to design safe, accessible and inclusive immersive environments.Footnote ²³

D. Clean and Healthy Environment

In a general comment from 2019, the International Covenant on Civil and Political Rights (ICCPR) stated that ‘environmental degradation, climate change and unsustainable development constitute some of the most pressing and serious threats to the ability of present and future generations to enjoy the right to life,’ as protected by Article 6 of the ICCPR and Article 3 of the UDHR.Footnote ²⁴ In 2022, the United Nations General Assembly adopted a resolution encompassing ‘the human right to a clean, healthy and sustainable environment.’Footnote ²⁵

The data collection and processing for XR requires energy from data centres. Large-scale XR use will lead to the creation of many more data centres. Some innovators have claimed the opposite by speculating that XR applications could reduce travel CO2 emissions by improving remote collaboration, or replace part of the pollution of the fashion industry with a digital, more sustainable one.Footnote ²⁶ But whether this outweighs the impact of additional data centres is unclear and needs to be studied.