1. Representational harms in the motion capture imaginary

Within the movement computing community, there is an understanding of what Laura Karreman (Reference Karreman2017) calls the “motion capture imaginary.” In a practical sense, this has to do with the way external tools of measurement are used to extrapolate what is going on inside a body at the level of muscles, bones and joints. Motion capture includes both markered and markerless systems for digitally recording movement. Optical motion capture uses cameras positioned around a perimeter (a “volume”) to track and record the movement of reflective markers attached to key points on the exterior of a subject’s body. The resulting data is triangulated to calculate the markers’ locations in three-dimensional space, and thus estimate the position and angle of joints. Markerless motion capture employs computer vision and pose estimation to process real-time or pre-recorded video from one or more viewpoints, translating visual features such as body surface and shape into three-dimensional motion data. These two dominant methods of data capture map bodies in three ways: kinematic (a skeletal outline), planar (dividing a body into discrete surface shapes) and/or volumetric (a digital mannequin or point cloud) (see Zheng et al. Reference Zheng, Wenhan, Chen, Yang, Zhu, Shen, Kehtarnavaz and Shah2023). Regardless of approach, extracted 3D motion data comes into being through the predictive inference and approximation of automated processes, and can then be analyzed and manipulated, usually by being “retargeted” to a digital rendering of another body. In other words, motion capture data is not visible in itself; it must be attached to a form it animates. From stick figures to fully realized gaming avatars, those forms each come with bodily logics that they inherit from longer histories of schematizing and abstracting motion to make it legible and modifiable. For example, precedents include the motion studies of Marey and Muybridge in biomechanical analysis; rotoscoping in cartoon animation as a form of motion transfer; and Labanotation and other systems that have informed how movement is documented and analyzed in as diverse fields as factory efficiency, anthropology and HCI (Karreman Reference Karreman2017; Laemmli Reference Laemmli2026).

The way motion capture encodes embodiment is thus an epistemology of the body that is consolidated and perpetuated across multiple domains as a shared template. Reducing human motion to data narrows the possibilities of movement to attributes that are understood to be quantifiable (see Turmo Vidal et al. Reference Turmo Vidal, Segura and Waern2023). Philosopher and dance scholar Erin Manning critiques the limited imaginary at work in “motion-detecting technology”: because the parameters for registering bodily motion are established beforehand, “you had to know in advance what could happen” (Reference Manning2009, pp. 62, 65). The body that is imagined by the apparatus of capture becomes the body that is knowable – an idea that crosses disciplines from dance to critical race studies in technology (Benjamin Reference Benjamin2019). The transformation of bodies into “quantum media” has a long history of expressing and producing Western cultural values through the governance and surveillance of bodies (Wernimont Reference Wernimont2019, p. 1; see also Braun Reference Braun2014; Browne Reference Browne2015). When bodies do not conform to the dominant norms, for example, in terms of race, gender or ability, they risk misclassification or dismissal (Buolamwini Reference Buolamwini2023).

Over the past few years, there has been increasing recognition of the representational harms embedded in AI-based systems. As Gebru and Denton summarize, “Representational harms can occur when computer vision systems reinforce social hierarchies, often by influencing perceptions of and attitudes towards particular social groups” (Reference Gebru and Denton2024, p. 223). Critiques of AI often focus on the harmful effects of stereotyping in underlying data annotation, which contributes to the dehumanization of already minoritized populations. Erasures and omissions belong to representational harms as well, “when a system fails to recognize, and appropriately label, culturally specific artifacts, activities, or attributes” (Gebru and Denton Reference Gebru and Denton2024, p. 226). For example, Katzman et al. (Reference Katzman, Wang, Scheuerman, Blodgett, Laird, Wallach and Barocas2023) point to the occlusion of meaning-making properties in terms of social groups. They note the difference between applying generic labels like “people,” “walking,” and “street” to a photograph of suffragists, versus utilizing terms more specifically connected to the historical and political contexts of women protesting the injustice of being denied voting rights in the U.S. during the late 19th and early 20th centuries. The generality of people walking in the street does not adequately reflect the specific activity documented, and the labels applied thus not only fail in their description, but they also mischaracterize the activities taking place. Klein and D’Ignazio point out in “Data Feminism for AI,” that a key issue is how data science pipelines rest on “the erroneous idea that curating, labeling, and documenting data is unskilled labor,” resulting in “naive and ill-considered data” that create issues downstream (Reference Klein and D’Ignazio2024, pp. 107, 133). As Kate Crawford summarizes: “training sets often build on each other, creating genealogies of data where problems and historical particularities persist over time” (Reference Katzman, Wang, Scheuerman, Blodgett, Laird, Wallach and Barocas2023, p. 1369).

In the field of dance, representational harms also occur when captured and processed motion data is not sufficiently nuanced to document the precision that dancers have honed in their movements. Whereas dancers move with expertise cultivated through the bodily logics of their specific practices, their motion is computationally represented by a generic or generalizable figure. If dancers’ experiences are remediated through onscreen digital doubles, then it is particularly important to understand the capaciousness and limits of the technological bodily imaginary, and the forms of appearance that are facilitated and foreclosed by such socio-technical systems. To be made visible, optical motion capture’s positional data is processed through automatic solvers or retargeted to existing “rigs,” while pose estimation models are used for markerless motion data. These models, which predict and then “recognize” bodily position and motion on the basis of limited training data, tend to make expert dancers look like amateurs while also reinforcing the social and cultural biases encoded in the training data. These observations are in dialogue with more systematic studies. Harvey et al.’s survey looks across 278 papers published from the 1930s to 2023 in terms of whose measurements define understandings of the human body in anthropometry and later motion capture. They argue that historically, the normative assumption has been “those who are male, white, ‘able-bodied,’ and of unremarkable weight,” and make the case that “these assumptions continue to be implicit in modern motion capture due to practices of validation that entrench, rather than challenge, them” (Reference Harvey, Sandhaus, Jacobs, Moss and Sloane2024, p. 1.1; see also Rajko Reference Rajko2022). Our own research with dancers and teachers working in the lineage of Katherine Dunham Technique highlights specific examples of bias, such as how the highly articulate mobility of the spine associated with African and Afro-diasporic dance practices is not effectively parsed by standard motion models (Bench and Elswit Reference Bench and Elswit2024a). A limited number of marker points in place of the spine’s thirty-three vertebrae will not capture the serpentine motion of yonvalou,Footnote ⁶ invisibilizing both the craft of a dancer’s precision, including practices of hyperextension and hyperflexion (or even simultaneous flexion and extension), as well as the cultural knowledge it activates (see Figure 1).

Figure 1.

Photograph of Demonstrator Celia Benvenutti, certified Dunham Technique Teacher, coached by Rachel Tavernier, Master Dunham Technique Teacher, courtesy of the Institute for Dunham Technique Certification, with Harmony Bench and Kate Elswit. For Artificial Intelligence for Creative Movement Analysis and Synthesis in collaboration with Visceral Histories, Visual Arguments: Dance-Based Approaches to Data. Recorded at Motion Lab, under the supervision of Senior Creative Technologist and collaborator Vita Berezina-Blackburn, the Advanced Computing Center for the Arts and Design, the Ohio State University. January 26, 2023. Photograph by Logan Wallace.

In this sense, there is a productive conjunction between the critical AI literature addressing representational harms, which recognizes the cascade effects of choices made in labeling data, and the archival principle of representativeness, which understands that not everything will be captured, and instead emphasizes the active curation of materials that represent the most salient elements of activity.Footnote ⁷ And yet, even such nuanced questions of representativeness cannot exist outside their relationship to uses, benefits and harms, leading to calls to move away from “building predictive tools” toward “valuing and prioritizing in-depth and contextual understanding” (Birhane Reference Birhane2021, p. 7). In discussion with dance communities, we find that it is not expected, or even desirable, to train or fine-tune models that will be able to perfectly reflect the types of nuances that are visible to dancers. At one of our gatherings on Race, Motion Data, and AI, in which we shared our ongoing research on spinal articulation, dancer and professor Bernard Brown expressed this contradiction of experiencing the representational harms of motion capture vis à vis his dance expertise in the following way: “It can’t see my spine! I worked hard for my spine. But also, don’t take my spine!!”Footnote ⁸ The question then follows: what is necessary in order to hold enough of what matters from movement’s specificity? Under what circumstances might the deformations of motion capture data be more valuable for archiving and communicating particular aspects of movement than failed photorealism?

2. What can motion capture archive?

While performance archivists and scholars have grappled with how bodily practices can be an uneasy fit for conventional archives, embodied knowledge is sometimes recognized in the GLAM (galleries, libraries, archives and museums) sector under the UNESCO category of “intangible cultural heritage” (ICH). Multiple fields contributed to the guidelines in UNESCO’s 2003 Convention for the Safeguarding of Intangible Cultural Heritage, which covers the performing arts alongside oral traditions, festivals and so on. Notably, the Convention suggests that in order to be “safeguarded,” such practices must be inscribed through a process of transformation into archival objects (UNESCO 2024, pp. 30-31; see also “Views” 2004). Performance scholar Diana Taylor, who was involved in creating the ICH framework for UNESCO, critically reflects on the inscription process: “The way to safeguard live practices, apparently, was by turning them into something they are not – primarily documentation: photographs, recordings, and categorization. Producing a record of performance is not the same as performance” (Reference Taylor2016, p. 151).Footnote ⁹

In and beyond the field of intangible cultural heritage, a substantial number of recent research projects explore the potential of motion capture in “safeguarding” embodied practices such as performance. The literature on motion capture in ICH often reveals a tension between an ideal of three-dimensional data as “omniscient” or “unbiased” compared to the more perspectival two-dimensional screen media (see Chao et al. Reference Chao, Delbridge, Kenderdine, Nicholson, Shaw, Whatley, Cisneros and Sabiescu2018; Delbridge Reference Delbridge2015), and concerns about the practical challenges of producing usable data. Some studies are premised on the claim that “generating motion data is no longer considered a technological barrier” (Aristidou et al. Reference Aristidou, Chalmers, Chrysanthou, Loscos, Multon, Sarupuri and Stavrakis2022) and that “advances in artificial intelligence and computer vision will resolve one or more challenges in preserving dance traditions” (Reshma et al. Reference Reshma, Balakrishnan Kannan and Shailesh2023, p. 19).Footnote ¹⁰ Others are more cautious about the mismatch between movement knowledge and what is possible to archive with existing technologies versus what may be available in the future. Still others ask what alternatives may exist to the limited framework of safeguarding, and conversations regarding the potential for “experimental preservation” join other approaches that honor the particularities of past practices, while bypassing the project of fixing them into a singular form (Pataranutaporn et al. Reference Pataranutaporn, Mano, Bhongse-Tong, Chongchadklang, Archiwaranguprok, Hantrakul, Eaimsa-ard, Maes and Klunchun2024). Each of the recent projects we describe in this section grapples with strategies for better sourcing and analyzing historical motion data through image classification and motion capture. Because they retain entanglement with people who are expert in their own physical practices, the projects surface a wider variety of logics, sensibilities and types of information than we have come to expect from motion data.

Given the complexity of acquiring meaningful motion data from dancers, automated processes are often called upon to compensate for inevitable occlusions and data loss, extrapolating from available data to fill in erasures and omissions. A 2024 article presents an edge case of motion capture in relation to the Nigerian Eyo masquerade, in which participants’ bodies are not directly visible due to layers of white cloth covering them from head to toe – including their faces, hands and feet. Assessing the many challenges, the researchers ultimately propose that the most “efficient pipeline for digitizing folk dances with complex and bulky costumes” is to use optical motion capture of the dancer’s body without costumes, denoise that data via deep learning, and then animate a costume in post-processing, using cloth modeling and physics simulation (Ami-Williams et al. Reference Ami-Williams, Serghides and Aristidou2024, p. 146). When surveyed, most individuals with expert knowledge of African masquerade forms preferred the simulated costumes to kinematic renderings of the underlying mover alone. Whereas the technological solution presumed that embodied performances were extricable from the material, affective, symbolic and ritual properties of the masquerade costume, domain experts reasserted its importance, even in simulated form.Footnote ¹¹ This project highlights the cultural specificity of motion data, including the very question of what constitutes movement.

Responding to emerging ethical priorities regarding inclusive data and community-engaged research, a number of projects have arisen that derive more substantive principles of motion data by incorporating expert knowledge-holders directly into the research process. One particularly long-running project, The Hong Kong Martial Arts Living Archive (HKMALA), uses the specific movement of martial arts as a foundation to experiment with the living nature of intangible cultural heritage, in particular with respect to the knowledge that is “dwelling in and enacted through the body” (Hou et al. Reference Hou, Kenderdine, Picca, Egloff and Adamou2022, 1:3). Among a broader range of activities, including multimodal documentation to contextualize motion capture of martial arts masters’ performances, HKMALA recently created “a comprehensive domain ontology and the first annotated ontological resource dedicated to traditional Chinese martial arts” (Hou and Kenderdine Reference Hou and Kenderdine2024, p. 589). Their multifaceted process incorporates the annotation of different intangible dimensions of martial arts, which goes through multiple levels of review, including with knowledge experts. In this way, HKMALA experiments with computational inference and interoperability from a place of practice-based specificity, rather than generalizability. Other projects engage in commitments toward artistic and cultural heritage preservation primarily through the promise of re-engagement. For example, a small-scale project recently worked to encode principles of Thai Classical dance into a generative AI model. Choreographer Pichet Klunchun identified the form’s 59 major postures in his work No. 60, and this information provided a baseline for an AI-driven “improvisatory framework that ultimately enables present-day practitioners to creatively and respectfully interact with computational manifestations of ancestral choreographic knowledge” (Pataranutaporn et al. Reference Pataranutaporn, Mano, Bhongse-Tong, Chongchadklang, Archiwaranguprok, Hantrakul, Eaimsa-ard, Maes and Klunchun2024).

In artistic spaces, the concept of movement portraiture is also gaining currency as a means to gesture to the inadequacy of motion capture data alone when representing and sharing the full dimensions of a person. Grisha Coleman’s The Movement Undercommons project proposed the use of motion capture for everyday movement, with particular focus on experiences of migration to produce “kinesthetic data ‘portraits’” that visualize both individual and group narratives. Countering the idea that dance and pedestrian movement data are generalizable, The Movement Undercommons proposes that it may be possible to identify less represented movement vernaculars by developing a “situated approach” for sourcing and capturing movement data as always tied to “legacies of expulsion, displacement, migration, diaspora, and dense cultural diversity” (Coleman and McCaffrey Reference Coleman and McCaffrey2018, p. 348). Similarly, LaJuné Macmillian’s Black Movement Library is based on a series of “movement portraits” that combine motion data with personal storytelling, and thus “ritualizes the archival process of data collection, inviting movement to be re-represented as a digital memory of life through motion” (Macmillian, n.p.). Macmillian offers the portrait as an alternative to “capture,” asking what it might mean to invent new ways for computational technologies to “witness” subjects more holistically.Footnote ¹² Both Coleman and Macmillian invoke the concepts of the “repository” and “library,” respectively, in which gathering and storing more holistic motion data from minoritized subjects points toward future reuse.

Such possibilities of future access are tethered in various implicit and explicit ways to questions of performance data as part of memory-keeping. This includes projects that situate themselves within a broader landscape of “embodied heritage preservation” such as Practicing Odin Teatret’s Archives (2021-25), for which the use of motion capture to produce a “virtual archive of embodied theatre techniques” is inseparable from the reactivation of that training system in a virtual environment through XR formats (Marouda et al. Reference Marouda, Selva and Maes2023, p. 54). Another example concerned with how to activate data for future renderings is Loops (2001-2011) by Marc Downie, Shelley Eshkar and Paul Kaiser (the OpenEndedGroup), which is based on Merce Cunningham’s 1971 solo of the same name. Loops encompasses iterations extrapolated from Cunningham’s motion capture data from 2000, when he was already at an advanced age. The OpenEndedGroup applied visual effects as well as behaviors to each datapoint, resulting in an animation that evolves constantly. The artists note that both Cunningham’s performance and their digital rendering “provoke similar challenges to preservation. Both are always performed live, never quite repeating from one performance to the next.” Furthermore, Loops can only exist in iterations since “every time the work is exhibited, its code must be carefully adjusted to accommodate even small updates in commercial display drivers” (OpenEndedGroup, n.p.).

These projects are all variously concerned with processes by which motion capture can engage with expert knowledge to archive performance, making it accessible to the future. And yet they also draw out fault lines, particularly around the aspects of performance and embodied knowledge that current motion capture data is incapable of representing. Of particular resonance across these examples is the status of a singular motion contributor versus a community of practice, in terms of who is recognized as a knowledge-holder across individual, communal and ancestral epistemic strata. Whereas Cunningham is representative of his own hyper-individualized postmodernist physicality, those who steward dance practices often struggle to have their inheritances, legacies and innovations recognized within what is often characterized as folk or vernacular dance (Kraut Reference Kraut2015). The stakes of representation, including potential harms and questions of ownership or extraction, are uneven and often disprivilege minoritized subjects. How, then, might movement be understood in a more expansive and holistic way?

3. Data as the remains of performance

Whereas many of the projects above demonstrate reflexive partnerships between dance artists and computer scientists, much of the data that has been extracted for AI training is stored behind computational systems that are “too complex or too technical to be understood by the people whose lives are implicated in them” (Cifor et al. Reference Cifor, Garcia, Cowan, Rault, Sutherland, Chan, Rode, Hoffmann, Salehi and Nakamura2019, para. 7). Moreover, such systems are transformative; as Jussi Parikka notes, “our gestures are already integrated into datasets that feed into training sets and model what gestures become meaningful” (Reference Parikka2023, p. 76). Making the case that dance might have more protection as data on the blockchain than as copyrighted choreography, legal scholars Poveda Yánez and Fraisse caution that there are important power dynamics to consider: “there is a great risk of well-versed professionals in computerized methods of human movement recording to end up hoarding every dance that enters the digital space in the form of data” (Reference Poveda Yanez and Fraisse2022, p. 88). While scholars most often turn to the dichotomy of extraction versus stewardship, Poveda Yánez and Fraisse’s use of “hoarding” is conspicuous and points to how residues of online engagement in the form of viral videos and digital texts are scraped and aggregated into massive datasets without the consent of content creators. When working in collaboration with dance experts, data can be generated by willing partners, but when similar content is harvested online, there is no meaningful participation. To put it another way, the archive can’t consent.

In this section, we propose that understanding motion data as bodily remains, in line with performance studies’ understanding of remaining more broadly, can shift alignments by acknowledging that motion data arises from and belongs to individuals, whose bodies are also archives of practices that are enmeshed with communities. Even when working in contexts of more careful data contribution, the motion models that researchers have available to them may not have been created under the same obligations of care. In the example with which we opened this essay, Berezina-Blackburn is trying to steward older motion capture datasets as best as possible, but once the datasets have been disentangled from their contributors, it is harder to reassemble them, to supplement them, to repatriate them or decommission them. Bodily data is a remainder, and as Tonia Sutherland has argued, often a lucrative remainder that is also subject to reanimation. Sutherland critiques what she calls the “resurrection practice” (Reference Sutherland2023, p. 88) of using digital media to animate the dead – something she observes in the widely contested holographic appearance of deceased rapper Tupac Shakur in concert. While such resurrection practices have a longer media history, they have proliferated with the creation of chatbots trained on the writing and speech patterns of the dead, particularly celebrities whose visual representation and sonic identity continue to be profitable posthumously. “Dead celebrities represent a growing enterprise,” writes Sutherland (Reference Sutherland2023, p. 95), echoing an argument put forward in an earlier essay by Jason Stanyek and Benjamin Piekut about posthumous duets. They called for considerations of “deadness” alongside performance scholarship’s fascination with liveness, in order to understand how the recorded voices of dead artists were generating capital in conjunction with the living through the technological advances of sound segmentation: “being recorded means being enrolled in futures (and pasts) that one cannot wholly predict nor control” (Stanyek and Piekut Reference Stanyek and Piekut2010, p. 18).

Yet bodies participate in remaining too, as themselves archives. The field of performance studies has come to understand that, while singular performance events may seem to disappear, performance also “remains differently,” as Rebecca Schneider puts it, in a manner that exceeds the conventional logic of the archive. This includes both explicit moments of restaging or reenactment, as well as how technique, repertory and so on pass through communities of practice. These facilitate saving performance, but in a manner “not invested in identicality” with regard to its reproduction: “remains do not have to be isolated to the document, to the object, to bone versus flesh. Here the body […] becomes a kind of archive and host to a collective memory” (Schneider Reference Schneider2011, p. 101). In information science, Caswell and Cifor propose a corresponding shift from a legalistic, individual rights-based framework to a feminist ethics of care, that charges archivists with “affective responsibility to engage in radical empathy with others, seen and unseen. It acknowledges that relationships change over time, that while the record may be fixed, our obligations to it – its creator, its subject, its users, its community – are constantly evolving in ways unforeseen” (Reference Caswell and Cifor2016, p. 42).

In this sense, new methods of collecting motion data are required that are capable of amplifying the archival quality of bodies themselves, rather than presuming that performance remains through inscription alone. As part of the project Visceral Histories, Visual Arguments: Dance-Based Approaches to Data, we explored the potential to gather motion data from punctuated documentation of Master Dunham Technique Teacher Rachel Tavernier learning and teaching the same exercise across 40 years: first as a student being coached on the exercise Fall and Recovery by Katherine Dunham at her home in Haiti in 1983, then explaining that same exercise to dancer Yasmine Lee in the early 2000s during filming for the Katherine Dunham Legacy Project, and finally teaching it with demonstrator Celia Benvenutti in the Motion Lab at The Ohio State University in 2023 (Figure 2). We were interested that digital remains might be created and used in ways that make visible how an individual like Tavernier holds multiple temporalities in her body simultaneously (Bench and Elswit Reference Bench and Elswit2023a). Tavernier was one of Dunham’s key demonstrators who codified much of the technique as it is taught today, and yet this example also underscores how embodiment is a collective project; no single body can hold the history alone.

Figure 2.

Forty years of embodied knowledge. Image composite. Above: Katherine Dunham (teacher) with Rachel Tavernier (demonstrator). From “Katherine Dunham on Dunham Technique.” Video. https://www.Loc.Gov/item/ihas.200003814/. Recorded in Haiti in 1983. Middle: Rachel Tavernier (teacher) with Yasmine Lee (demonstrator). From “Dunham Technique: Fall and recovery with body roll.” Video. https://www.Loc.Gov/item/ihas.200003854/. Recorded in New York in 2001. Below: Rachel Tavernier (teacher) with Celia Benvenutti (demonstrator). Photograph documents motion capture session in Columbus in 2023.

The digital and corporeal are already interconnected, from the ways bodies archive technologies by forming habits for their use (Chun Reference Chun2016, p. xi) to how bodily affects may likewise remain in media objects (Schneider Reference Schneider, Jucan, Parikka and Schneider2019, p. 54). At the same time as modes of operation are installed as “techniques of the body” through repeated use (Mauss, Reference Mauss[1935] 1973), media themselves require constant upkeep. If they are not updated, digital files “are no longer used, useable, or cared for, even though,” Chun points out the paradox, “updates often ‘save’ things by literally destroying – that is, writing over – the things they resuscitate” (Reference Chun2016, p. 2). Scholars of performance have made a similar observation that repertory is not fixed as a stable object, but is renewed and transformed as it travels from performer to performer, and it is this constant evolution and reinterpretation – rather than documentation alone – that enables performance practices to be preserved. Dance must be performed to remain, and, like media, that means it must continue to be updated.

In dance, we understand that projects of reenactment, recreation and reinvention offer means to engage with and activate the past. Reenactment unsettles historical linearity by amplifying the time lags inherent to all performance, explicitly creating new contexts for the past in the present (Franko Reference Franko2017; Schneider Reference Schneider2011). While some practices shore up or reimagine dance histories that feel known, others demonstrate “the potential, but also the very precarious nature, of working with such creative strategies at the intersection of multiple contested legacies” (Elswit Reference Elswit2014, p. 6). Such repetition is also prominently visible in the dance challenges that circulate through popular digital media platforms. As dancers repeatedly embody the gestures of viral media, they both amplify the popularity of certain choreographies and evolve them by incorporating their own styles (Bench Reference Bench, Bales and Eliot2013). Whether looking at examples of historical reconstruction or how dances spread virally across the bodies of online participants, repetition both stabilizes and changes choreography. Repetition creates endurance in what has been understood to be ephemeral. Chun describes this phenomenon as the enduring ephemeral of new media: “a battle of diligence between the passing and the repetitive” (Reference Chun2008, p. 167). Iterative performances both preserve choreography and change it. Better, they preserve it because they change it.

That 2023 motion capture session with Tavernier arose out of an equitable participatory design research process (Harrington et al. Reference Harrington, Erete and Piper2019) with facilitators Penny Godboldo, Patricia Wilson and Tavernier – all of whom are current or former co-directors of the Institute for Dunham Technique Certification – through which we explored what an oral history dataset might look like that emerges from how dance passes from body to body in a studio setting.Footnote ¹³ Because Dunham’s repertory is rarely performed, the history is located primarily in documents and film footage on the one hand, and in the bodies of educators who continue to promote the technique on the other. This leads Sutherland (Reference Sutherland2023) to cite Dunham Technique as exemplifying the value of community custodial relationships in contemporary archival practice. The interview questions for that oral history dataset project circulated around two strands of Dunham Technique that our interlocutors selected as having identifiable regional variations as well as physiological and pedagogical changes over time.Footnote ¹⁴ During interviews, the ten selected legacy-holders discussed who learned which versions from whom, and also demonstrated the exercises and their qualitative properties in various ways, from singing rhythms, to gesturing with their hands, to engaging in full-body actions when words proved insufficient. Through our conversations, we began to see the physical demonstrations, the codified terminology, and the individual memories as all forms of motion data arising from these dance experts’ bodily archives.

We simultaneously began to apply pose estimation techniques to historical film footage of these same exercises, to see whether any complementary 3D motion data might emerge. However, the derived data could not match the nuance of the studio narratives (see Plant Reference Plant2023). That sharpened our investigation into the potential of motion capture data to align with a broader understanding of historical motion data. While motion capture tends to be anchored in the present, as dancers we know that the motion we see in front of us is both specific and collective: gathering together a dancer, their teachers and their teachers’ teachers, all made up of repeated scenes of training and performance experiences, as Figure 2 illustrates. We asked how motion data could account not just for the present, but also what we call deep bodily time. We staged pedagogical encounters in the motion capture studio that focused on teaching and cueing movement, in an attempt to access different registers and layers of movement knowledge and its transmission. And we also experimented with different ways of meaningfully visualizing all of these forms of motion data. For example, we discovered that retargeting Benvenutti and Tavernier’s motion capture data to human-like figures was both disappointing from a representational standpoint and insufficient from an analytical one. By contrast, we found we could show “enough” of the nuanced spinal articulation we were investigating by creating an emergent network of edges that connected optical marker points only at the moments in which they came within a certain distance of one another (Figure 3). This dynamically changing network imagery not only bypassed the solidity of a dissatisfying avatar, but also reinforced the mesh of pedagogical relationships between teachers and students, gesturing toward the density of all of these points within historical legacies that assemble and refashion dancing bodies over and over again.

Figure 3.

Screengrab from “Layering Embodied Data.” 2024 VR piece, featuring Rachel Tavernier with Celia Benvenutti – see Figure 1 caption for full details. 3D movement visualization in Unity by creative technologist Nicola Plant for VHVA. See https://vimeo.com/1003315448.

This trajectory of experimentation flagged up for us a number of critical concerns regarding the practical problems of motion data as historical evidence embedded in bodily archives. First, our focus not on human movement or dance in general, but on the physical execution of specific exercises within a codified technique called attention to the inadequacy of existing motion models. Tavernier was interested in the potential of motion data to document the nuances of Dunham Technique that sometimes get lost, especially as the earlier generations of teachers are becoming ancestors. One of her hopes was that our research might make that aspect of the pedagogy more visible, and we were inspired by that impetus at the same time as we were frustrated at the challenges of realizing it through motion capture. Second, there were the real concerns that we navigated with dancers, over and over, regarding how to set constraints on potential futures of their motion data, in terms of where it would live, who would own and/or steward it, who would be entitled to access and what knowledge or permission future users would need to access it. There are no one-size-fits-all answers to these questions. In the gatherings we co-convened on Race, Motion Data, and AI, we heard participants repeatedly turn to the phrase “working at the speed of trust,” popularized by adrienne maree brown (Reference Brown2017). Trust is fragile. Despite our best intentions in setting up rigorous dancer-centric agreements to handle motion capture data, photography and video, we still witnessed those items being disentangled, circulating separately through secondary and tertiary parties without the required attribution. Even good-faith agreements, fully carried through, will not prevent dancers from being confronted with their motion data appearing in unexpected and even unsettling ways. A co-researcher who participated in designing a related project nonetheless experienced unease at seeing their motion data aggregated with others as part of a (failed) experiment to automate movement annotation with machine learning. This connects to the third concern, which is how to anticipate the futures of any data that might be made publicly available, in particular at a moment in which generative AI is rapidly expanding capacities in synthetic video. Scraped data from online dance videos remains a hot commodity with which to demonstrate a model’s technological capacities – a problem that is predicated on misunderstanding the relationship of motion data to dance-based knowledge, as we elaborate in the next section.Footnote ¹⁵ Motion data is all too often caught in an economic regime that encourages users to separate themselves from their data, accruing value to platforms and displacing social costs from corporations to communities. By contrast, expanding understandings of motion data can offer an opportunity to trouble reductive, presentist and solutionist practices of AI, by imagining how else motion data might come into being and what it can do.

4. Expanding motion data

The projects described above show that a singular formulation of motion data is insufficient for the questions of archiving and sharing movement knowledge that arise in relation to dance. We elaborated how we were engaging in motion capture research at the same time as we were interviewing Dunham legacy-holders; although we first understood those interviews and motion data research as independent of each other, we quickly learned they were deeply intertwined. During the interviews, our interlocutors accompanied their stories with physical demonstrations, and the conversations gathered together multiple, intergenerational memories; similarly, in the motion capture studio, Tavernier and Benvenutti rehearsed the stories that Dunham Technique teachers share with their students. These examples further hint at the ways artists are also entwined. As both dance scholars and practitioners ourselves, we understand that people carry many generations of movement in their bodies, and any documentation of dance likewise contains these layered histories. How dancers interpret a role or perform an exercise is also a physical record of the different teachers with whom they have trained, threading together distinct geographies and histories as they manifest in a dancer’s movements. No single body archives this history alone, rather dance histories are held in relation by many bodies simultaneously. In this way, motion data must account for embodiment as a collective project, including how dancers perpetuate and transmit memory, as well as how media practices support both the endurance and reactivation of such bodily practices.

Across our broader set of projects that put forward a set of principles and practices for visceral data analysis, we pick up the question of how dance forces a reconsideration of data practices in terms of all the specific yet expansive ways that “bodies are experiential, arrange knowledge, are repositories of memory, are in process, and produce relationality” (Bench and Elswit Reference Bench, Elswit, Gold and Klein2023b). Here, we propose that a more expansive concept of motion data rooted in an understanding of bodily archives can offer a fuller understanding of the past while also demanding that we sit with the ethical problems and representational harms that arise when data is treated as detachable from its human sources. In addition to motion capture data then, we contend that motion data also includes the instructions and terms used to teach or cue movement, as well as affective language that accompanies oral storytelling and sharing memories. Motion data also encompasses actual movements in the form of choreography and repertory transmitted from body to body, as well as dancers’ use of “marking,” which is both a physical shorthand and memory aid used when rehearsing or learning new movement. Motion data also includes film and video documentation, which suspend the temporality of historically and culturally specific embodiments. Each of these contributes to representing the deep bodily time of embodied archives, in which a single person can hold multiple generational and regional variations of a practice accrued over their lifetime and the lifetimes of their teachers. Whereas motion capture data emphasizes the individual, an expanded register of motion data better illustrates how embodied knowledge is also cumulative and shared.

Such expanded understandings of motion data can unsettle conventions of data capture. Dancers’ practice of marking, for example, exemplifies how expertise already gives rise to principles of abstraction that could inform computation rather than being subjected to it. Similarly, communities of practice already establish protocols for transmitting, sharing and owning movement knowledge. In the course of doing this research, we got into a tussle with a senior computer science professor over the use agreements for dancers’ motion capture data. Even though we were focused on specialized movements from dance experts rather than pedestrian motion, they wanted our collaborators to forfeit all current and future rights to their motion data, asserting that: “If they aren’t willing we can surely find others who will.” This was precisely the extractive form of data capture we were working so hard to avoid. As we have argued throughout this essay, such flattening of the expert into the generic misses how dance archives its own histories in the bodies of dancers, and thus exposes participants to further representational as well as epistemic harms. It abdicates responsibility for finding ways to process and render that motion data in ways that feel meaningful to its movers, and also places future use in the hands of technologists rather than the people from whom the movement arose.

It has always been striking to us that dancers are the best critics of motion data models because they are attuned to the nuances of finely detailed movement. Better processes for gathering and analyzing motion data will only emerge when we innovate alternatives in a manner that educates and empowers stakeholders with expert motion knowledge to engage on their own terms. We think of this as working toward a “dancer-in-the-loop” method. What methods of motion data curation and analysis exist or can be created that are not based on a politics of capture and extraction but are rather produced in collaboration with dance-based custodial communities? Through what processes can we better represent deep bodily time, and how might the engagement with such movement legacies and the communities that sustain them inform the development of equally deep crediting practices for movement computing?

Over this essay, we have argued that when applying AI to dance archives, the motion capture imaginary embedded in standard motion models risks reproducing representational harms; that nonetheless there exist a variety of projects that are experimenting with ways to incorporate movement expertise into uses of motion capture for archiving embodied practices; that such data needs to be understood as bodily remains in a manner that is inextricable from performance’s remains; and that both AI-based motion models and approaches to archiving can be better guided by an expanded understanding of motion data. Inspired by guidance on data curation and stewardship, as well as our own research experiences and conversations, we argue that motion data should be handled as a kind of visceral data. We offer the following as reflective questions and guiding principles for research that centers the body as archive in an era of AIFootnote ¹⁶:

• - Whose motion data is being archived and how is that recognized? A visceral approach to motion data demands both individuality without individualism and a gathering of what is shared without the generality of aggregation. How does such motion data tether individuals to dance’s many communities of practice, and the legacies each carry? What protocols for access, respect and attribution govern use within those various spaces?

• - Which knowledges and knowledge-holders are represented by this archived motion data, and which are obscured? A visceral approach understands motion data as created and creative, interconnected with interpretation, and negotiated at the intersections of artistic, computational and cultural expertise. What conditions of mutual respect allow each domain to be accountable to the others?

• - How does motion data interface with bodies as always themselves archives? A visceral approach to motion data recognizes that movement carries histories, ways of knowing, inheritances and legacies. What practices of motion data collection, gathering and sharing can open out to other forms of memory-keeping?

• - Whose interests are served in collecting and distributing this motion data? A visceral approach insists that motion data must be curated and cultivated for specific purposes, including for the benefit of those who share their data. Under what conditions can contributors feasibly withdraw their data from perpetual circulation, or even lay it to rest?