Organic data have the potential to enable innovative measurements and research designs by virtue of capturing human behavior and interactions in social, educational, and organizational processes. Yet what makes organic data valuable also raises privacy concerns for those individuals whose personal information is being collected and analyzed. This chapter discusses the potential privacy threats posed by organic datasets and the technical tools available to ameliorate such threats. Also noted is the importance for educators and research scientists to participate in interdisciplinary research that addresses the privacy challenges arising from the collection and use of organic data.

Despite promising early evidence for the validity of well-designed game-based assessments (GBAs) for employee selection, the interaction between the complexity of games and their use in international and cross-cultural contexts is unknown. To address this, this paper presents a descriptive, qualitative study examining the perspectives of both GBA vendors and organizational stakeholders related to cross-cultural issues unique to GBAs related to 1) privacy, 2) legality, and 3) applicant reactions. Overall, privacy and legality concerns appeared similar for GBAs as with other assessment methods, although certain common characteristics of GBAs amplify common concerns. Applicant reactions appeared more positive to GBAs across national borders and cultures than traditional assessments, although some international differences were reported. Other cross-cultural topics raised included international differences in the conflation of GBA and artificial intelligence, in the importance of mobile-first design, and in the ability of GBAs to provide a more language-agnostic experience than other assessment types.

Social media is an ever-increasing aspect of the internet presence and daily life. Despite certain challenges in defining the construct, researchers have realized the possibility that social media can allow for the measurement and assessment of a wide variety of variables. Throughout the ever-growing number of social media sites and apps across countries and languages, there is an abundance of formats that researchers can utilize, such as photo, text, location, video, and more. In this book chapter, we conducted a literature search and identified four constructs that are most frequently studied using social media (i.e., personality, emotion/affect/mood, life satisfaction, and political views). We then summarized a list of studies that use social media to investigate these four constructs. Additionally, social media offers unique opportunities for researchers to assess various cross-cultural data, which can present its own challenges. We also provide examples of the potential opportunities and challenges, as well as ethical and technical considerations for researchers to keep in mind.

Technology, that is, the output of human innovation, has always been central to human progress worldwide. Early on, the ancients developed the wheel, concrete, calculus, and paper, which led to advances in transportation, construction, and communication. Today, the incarnation of technology falls in the realm of the digital and computational, and its progress has been rapid, even arguably exponential. In his chapter, “The Law of Accelerating Returns,” Ray Kurzweil writes, “An analysis of the history of technology shows that technological change is exponential, contrary to the common-sense ‘intuitive linear’ view. So we won’t experience 100 years of progress in the 21st century – it will be more like 20,000 years of progress (at today’s rate)” (Kurzweil, 2004, p. 381).

Behavioral measurement is the hallmark of research in the field of computational social science. We are witnessing innovative as well as clever use of existing and novel, commercial, or research-grade “sensors” to measure various aspects of human behavior and well-being. Passive sensing, a version of measurement where data is gathered and tracked unobtrusively using pervasive and ubiquitous sensors, is increasingly recognized and utilized in organizational science research. This chapter presents an overview of where passive sensing has been successful in workplace measurement, ranging from assessing worker personality and productivity, to their well-being, and understanding the overall organizational pulse. A range of passive sensing infrastructures are described (e.g., smartphones, wearable devices, social media) and several machine-learning-based predictive approaches are noted in this body of research. The chapter then highlights outstanding challenges as this field matures, which include issues of limited generalizability in computational measurement of workplace behaviors, gaps and limitations of gold standard assessment, model simplicity and sophistication tradeoffs, and, importantly, privacy risks. The chapter concludes with recommendations on important areas that need further or altogether new investments, so as to fully realize the potential of passive sensing technologies in more accurate, actionable, and ethical workplace measurement.

The ubiquity of mobile devices allows researchers to assess people’s real-life behaviors objectively, unobtrusively, and with high temporal resolution. As a result, psychological mobile sensing research has grown rapidly. However, only very few cross-cultural mobile sensing studies have been conducted to date. In addition, existing multi-country studies often fail to acknowledge or examine possible cross-cultural differences. In this chapter, we illustrate biases that can occur when conducting cross-cultural mobile sensing studies. Such biases can relate to measurement, construct, sample, device type, user practices, and environmental factors. We also propose mitigation strategies to minimize these biases, such as the use of informants with expertise in local culture, the development of cross-culturally comparable instruments, the use of culture-specific recruiting strategies and incentives, and rigorous reporting standards regarding the generalizability of research findings. We hope to inspire rigorous comparative research to establish and refine mobile sensing methodologies for cross-cultural psychology.

The quality of psychological assessment processes in talent management is influenced by our choices about which measurement technologies to use. Technology with relevance to assessing talent is also advancing at great speed in many domains. These advances include processing power and speed, human computer interaction research, and machine learning and artificial intelligence. Given these rapid developments, it is an appropriate time to pause and take stock of how emerging assessment approaches (e.g., game-based assessment) that leverage these new developments are used, relative to more traditional approaches such as questionnaires and interviews. To achieve this objective, we report here on a survey of European assessment practitioners. We ask about the technology they use for psychological assessment, the constructs they measure with those approaches, and the levels of organisations they are used at. We also asked about how traditional approaches are being enhanced with technology and about practitioner perceptions of the reliability, validity and adverse impact and privacy of their technological choices.

In this essay, we review the Technology and Measurement around the Globe chapters with an eye toward integration and synthesis. We primarily focus on implications for testing, and then make connections to the broader world of nontest assessment. We identify themes of privacy, fairness, workplace applications, and emerging technologies, and offer a research agenda for future investigations that seek to understand culture, technology, and measurement.

An overview of testing and measurement in North America is provided, covering topics related to privacy laws and regulations, online proctoring, artificial intelligence, accommodations, accessibility, and the “opt out of testing” movement that are currently defining measurement in North America. This is not to say that these challenges are unique to North America; in fact, the challenges related to these topics are being faced all over the world in varying degrees and the same opportunities exist, but these topics are of particular importance when it comes to measurement and assessment in North America. Building on these observations, a discussion of how advances in technology and computing power provide an opportunity to challenge the status quo related to assessment; these advancements will allow assessment of skills in more authentic ways that will provide better insight into someone’s knowledge, skills, and abilities. The question we should be asking and attempting to answer is “How can assessment developers leverage the power of the cloud and technology to measure skills more accurately and create higher fidelity in the assessment process?”