2.1 Study design

The experiment utilized the Minecraft Memory and Navigation task (MMN), which is composed of four distinct terrain environments (Simon et al., Reference Simon, Clemenson, Zhang, Sattari, Shuster, Clayton, Alzueta, Dulai, de Zambotti, Stark and Baker2022). Relevant data from this task have been reported in the referenced study. This study was approved by the Institutional Review Board at the University of California Irvine. All participants provided verbal and written consent prior to participation. Participants received either course credits or monetary compensation for their time.

2.2 Participants

In total, 182 participants completed the Minecraft Memory and Navigation Task (see Table 1a and 1b for details). All participants were healthy adults with no reported medical, mental health, or sleep disorders. For each participant, in addition to their performance in the game, we also recorded a set of baseline covariates to better understand individual differences and their potential influence task performance. Table 1a summarizes all categorical covariates. In Table 1a, Env1, Env2, Env3, and Env4 refer to the specific environment to which participants were randomly assigned. Video game experience refers to participants’ usual self-reported gaming type. First-person games have an immersive perspective in which players view the world of the game through the eyes of the character. Third-person games, on the other hand, offer an external perspective of the character, providing a broader view of the surroundings. We note that more than half of the participants have experience in both game types. Participants reported neither if they did not have experience with either type of video game. Minecraft skill refers to self-reported skill level of the participants specific to Minecraft. More than half of the participants do not have prior game experience in Minecraft. For variable sex, female participants are more than double the male participants. Six participants identified themselves as transgender or bisexual. Table 1b shows the numerical covariates. Gaming hours record the self-reported weekly playing time, and age refers to the participants’ age in years. These covariates allow for detailed exploration of factors influencing performance and behavior within the experiment.

Table 1 Descriptive statistics of participant covariates across virtual environments

2.3 Minecraft memory and navigation task

Each of the four environments was a fully immersive 3D space, providing participants with a realistic and interactive experience. These 3D environments were designed to ensure consistent complexity and layout while maintaining distinct spatial configurations to minimize potential carryover effects between environments. Each environment contained 12 distinct objects, along with obstacles and impassable tiles, such as fences, rivers, cacti, and trees, which added an additional layer of challenge and realism. Participants were randomly assigned to one of the environments to ensure balanced conditions throughout the study. Participants were only allowed to walk or run during the task. Screenshots for each of the environments are shown in Figure 1.

Figure 1 Screenshots of four game environments.

The MMN task had two primary aims. The first aim was to familiarize participants with the virtual environment and evaluate their ability to search through an environment and encode the spatial relationships between objects, landmarks, and boundaries. This phase assessed participants’ capacity to form a cognitive map through free exploration. The second aim was to test and measure participants’ spatial memory and navigation strategies by requiring them to recall and accurately place objects within the environment. Each participant completed two training sessions and one test session. The methodology of the MMN task is briefly described below; for more detailed information, please refer to Simon et al. (Reference Simon, Clemenson, Zhang, Sattari, Shuster, Clayton, Alzueta, Dulai, de Zambotti, Stark and Baker2022).

Training. During the training session, the participants began at a central starting point within their assigned environment and were instructed to explore freely. Their task was to locate all 12 objects, hidden within chests in the ground and marked by a single pink block above, and memorize their locations. The object locations were not provided to the participants during training or test. The training phase ended when participants either found all 12 objects or the allotted time of 10 minutes per training expired. Participants could have up to 20 minutes of free exploration. Immediately after completing the first training session, participants proceeded to the second training session and repeated the same procedure.

Test. Following the two training sessions, the participants were immediately tested on their knowledge of the location of the 12 objects. The order in which the objects were presented during the test was randomized. Furthermore, to increase the task’s difficulty, there were four potential starting points, on the outer edge of the environment in the test phase, which differed from the start location in the training phase. For each test trial, the participant was teleported randomly to one of the four test start locations and instructed to place the testing object at the location they remembered from the training. After placing the object, the participant was again teleported to a new location and the next object to be tested was provided. Participants were given a maximum of 3 minutes to place each object during a test trial.

2.4 Spatial memory and navigation task data

During both training and test sessions, participants’ in-game 3D coordinates were recorded each second to track their movements. In addition, a log file was maintained to capture specific timestamps for key events. In the training sessions, the timing began as soon as the participants entered the environment. Each time a participant found an object, the exact time was recorded, creating a detailed log of their progress. For the test session, for each of the 12 objects, we recorded the exact time participants entered the environment after being presented with the specific object, the time, and location of their object placement.

3.1 Navigation in 3D environments: Past work and challenges

The 3D environment in Minecraft provides significantly richer information compared to traditional two-dimensional (2D) spaces due to its distinct vertical dimension. Unlike the horizontal plane, vertical movement is heavily constrained, as participants are generally bound to the ground of the Minecraft world. This limitation means that the vertical dimension cannot simply be treated as an extension of the other two dimensions, as free movement upward or downward is not possible. The rules of the Minecraft game further emphasize these restrictions, e.g., players cannot ascend cliffs or vertical surfaces higher than three blocks without the aid of tools or structures, which introduces a layer of strategic planning in navigation. Similarly, falling from significant heights results in health penalties, making the vertical dimension not only a physical but also a consequential element of game play.

These constraints also create a distinct cognitive experience for the players. Just like in the real world, individuals naturally favor flat or gently sloping terrain over steep or rugged landscapes, reflecting an intuitive understanding of effort and risk. Moreover, unique or striking topographical features tend to leave lasting impressions, enhancing memory and shaping navigation patterns within the environment. The interplay between verticality, terrain, and cognition highlights how Minecraft’s 3D environment offers a more complex and immersive experience than its 2D counterpart.

We use Figure 2 to demonstrate the advantage of considering 3D paths over their 2D versions. For each pair of pictures, we note that 3D paths provide more information and naturally help with better understanding study participant’s in-game behavior. For example, in 2D pictures, participants often take seemingly indirect or circuitous routes despite the availability of straightforward paths. However, by checking the 3D pictures, it becomes evident that the participants are actually attempting to avoid steep inclines or declines by taking alternative routes, thereby minimizing the slope and reducing the effort required for movement.

Figure 2 An example of comparisons between 2D and 3D paths.

We highlight a few statistical challenges when analyzing 3D path data in our study. First, 3D paths are inherently high-dimensional. For example, a participant who spends 10 minutes in one training session would result in 600 longitudinal measurements of $x,y,$ and z coordinates. This has not even counted for additional variables, such as time stamps and speed information. Moreover, the movement data are highly auto-correlated in both space and time, which makes common dimension reduction techniques, such as PCA and t-SNE (Nathan et al., Reference Nathan, Getz, Revilla, Holyoak, Kadmon, Saltz and Smouse2008; Troje, Reference Troje2002) not directly applicable. Third, one needs to take into account the game environment and rules when analyzing the data. For example, the constraints of the third dimension, such as limited vertical mobility and penalties for steep slopes or falls, differ significantly from the first two dimensions. For this reason, several popular approaches in spatial statistics, such as state-space models (Howard et al., Reference Howard, Green, Kelly and Ferguson2008), Gaussian process-based approaches (Nardi & Stachniss, Reference Nardi and Stachniss2019), and spatial–temporal approaches (Lu et al., Reference Lu, Ruan and Huang2022) are not directly applicable. Lastly, path lengths depend on several factors, such as task difficulty (e.g., paths between different pairs of objects may have different difficulty levels) and participant heterogeneity (e.g., different navigation strategies, and game skill levels). Classical curve alignment methods, such as Dynamic Time Warping (Müller, Reference Müller2007) are not directly applicable due to these factors.

3.2 Defining optimal paths

Navigating and analyzing 3D data pose unique challenges that differ significantly from those in 2D environments. In 3D spaces, relying solely on 2D data oversimplifies the environment, failing to capture critical aspects of player behavior. Moreover, applying standard 3D Euclidean distances is insufficient, as constraints on the third dimension—such as limited vertical mobility and penalties for steep slopes or falls—often diverge from those of the horizontal dimensions.

To address these challenges, it is crucial to use a pathfinding algorithm that can flexibly model cost under environmental constraints. A natural choice is Dijkstra’s algorithm (Dijkstra, Reference Dijkstra and Smith2022), a classic method in graph theory known for computing the shortest path in a weighted network. More details of Dijkstra’s algorithm are provided in Appendix A. By explicitly modeling movement costs and constraints, such as elevation or hazardous terrain, Dijkstra’s algorithm can be readily adapted to Minecraft’s 3D environment. This flexibility makes it invaluable for solving navigation problems where standard Euclidean metrics are insufficient. Its applications span GIS, robotics, and spatial behavior modeling (Saab & VanPutte, Reference Saab and VanPutte1999), providing both accuracy and computational efficiency in complex, constrained spaces.

Our work builds on prior research that uses graph-based models to study trajectory data (Spaccapietra et al., Reference Spaccapietra, Parent, Damiani, de Macedo, Porto and Vangenot2008). These models, which represent movement paths as graphs with nodes for locations and edges for movements, have been successfully applied in domains like motorized transportation and unmanned vehicles (Del Mondo et al., Reference Del Mondo, Rodríguez, Claramunt, Bravo and Thibaud2013). One of the key advantages is the flexibility to incorporate additional spatio-temporal constraints, as well as the ability to model and analyze spatio-temporal evolutions (Elayam et al., Reference Elayam, Ray and Claramunt2022; Martin et al., Reference Martin, Wiedemann, Reck and Raubal2023). By leveraging these graph-based techniques, we aim to address the complexities of 3D navigation while ensuring accurate modeling of player movement in dynamic environments.

For our Minecraft data, we use the algorithm to evaluate all feasible (walkable) paths and pinpoint the shortest route between them. We begin by introducing the concept of a block. In Minecraft, a block is a cube that measures one unit on each side. It is the fundamental unit of the game world as it serves as the basic building and interaction element. Blocks are aligned to a grid, meaning that their positions are fixed to integer coordinates. Consequently, the entire game world is structured as a 3D grid of blocks.

To describe the application of Dijkstra’s algorithm in our study, we start with a 3D environment terrain map $ E \subset \mathbb {R}^3 $ . A Minecraft block $ b \in E $ is represented as $ b = (b_x, b_y, b_z) $ , where $ b_x \in \mathbb {Z} $ and $ b_y \in \mathbb {Z} $ are integers corresponding to the block’s $ x $ - and $ y $ -coordinates, respectively, and $ b_z \in \mathbb {Z}^+ $ is a positive integer uniquely determined by $ b_x $ and $ b_y $ . Together, these coordinates define the block’s location in the 3D environment. In our model, blocks serve as nodes in a graph, with edges connecting adjacent blocks based on specific criteria that reflect the mechanics of Minecraft’s game. The edge from block $ b_1 $ to block $ b_2 $ is considered walkable if the following conditions are satisfied:

1. 1. Both blocks must be valid and reachable: $ b_1, b_2 \in E $ . This ensures that both blocks belong to the terrain map and are not obstructed by elements, such as water, trees, or fences, which would render them inaccessible within the game environment.

2. 2. Blocks must be adjacent in a 2D sense:

   $$\begin{align*}|b_{1x} - b_{2x}| + |b_{1y} - b_{2y}| = 1 \quad \text{or} \quad |b_{1x} - b_{2x}| = |b_{1y} - b_{2y}| = 1, \end{align*}$$

   i.e., adjacency is defined as direct neighbors (in x or y direction) or diagonal adjacency.

3. 3. Vertical height changes must be within a permissible range:

   $$\begin{align*}-4 < b_{2z} - b_{1z} < 3. \end{align*}$$

   This constraint indicates that, in the Minecraft environment, players cannot climb cliffs higher than two blocks or descend falls exceeding three blocks in a single step.

These rules ensure that the graph accurately represents the navigational constraints and possibilities within the 3D Minecraft environment, forming the basis for applying Dijkstra’s algorithm to find the optimal paths. With these rules in place, we can now define the distance or edge weight between two walkable nodes. The distance is defined as follows:

(1) $$ \begin{align} d(b_1, b_2) = \sqrt{(b_{2x} - b_{1x})^2 + (b_{2y} - b_{1y})^2} + |b_{2z} - b_{1z}| \times (2 + 0.5 \times \text{sgn}(b_{2z} - b_{1z})). \end{align} $$

The first term in (1) is the standard 2D Euclidean distance between the two blocks, while the second term corresponds to the additional cost associated with vertical movement. Here, sgn $(\cdot )$ is the sign function. Definition (1) ensures that the 2D distance remains consistent with the standard Euclidean distance in two dimensions, while the cost increases with the height difference to reflect the greater effort required to navigate steeper vertical transitions. Additionally, moving upward incurs a higher cost than moving downward, aligning with the physical and cognitive challenges associated with climbing.

For a given environment E, we defined a weighted graph based on the terrain and its properties discussed above. Using this graph, for any pair of blocks $b_{start}$ and $b_{end}$ in E, we can apply Dijkstra’s algorithm to identify the shortest path between them. The distance or cost of this shortest path, as computed by Dijkstra’s algorithm, is referred to as Dijkstra’s distance, denoted as $D(b_{start}, b_{end})$ . This distance captures the minimal cost required to traverse from $b_{start}$ to $b_{end}$ .

3.3 Cost difference during object–location learning

Although Dijkstra’s algorithm provides a connected terrain map, analyzing individual behavior reveals additional complexities that require further interpretation. For example, Figure 3 illustrates an inconsistency in a player’s training session while learning object locations. In some object-to-object movement, such as moving from the Bed object to the String object or moving from the Pants object to the Roses object, the player appears to walk along a clear, efficient path toward a distinct destination. In contrast, in other movements, such as from the Pie object to the Record object or from the Roses object to the String object, the player exhibits wandering behavior, deviating from a straightforward path and covering significantly more distance than necessary. This variability suggests the need to distinguish and classify different search behavior patterns within a single player’s movements during learning.

Figure 3 Example of paths from participant no. 2025 during training session 1.

We now segment the training path—the players complete movements during learning—into distinct segments to better analyze individual behavior. Each segment is defined as starting at one unique object and ending at the next, allowing for a structured breakdown of the path. Once segmented, each segment can be analyzed independently for key metrics, such as cost, distance, and elevation changes. These metrics provide a basis for further analysis, including clustering and association studies.

For a segment $ s = (s_1, s_2, s_3, \dots , s_T) $ of participant i, with $ s_t $ representing the 3D coordinate at time $ t $ starting from a certain object, and $ T $ representing the total time spent moving from one object to another, we define the actual cost of the segment $ s $ from time $ t $ as

$$\begin{align*}AC_{is}(t) = \sum_{k = t}^{T-1} D(s_k, s_{k+1}), \end{align*}$$

where $ D(s_k, s_{k+1}) $ represents the cost of moving from coordinate $ s_k $ to $ s_{k+1} $ . We also define the optimal cost of the segment $ S $ from time $ t $ as

$$\begin{align*}OC_{is}(t) = D(s_t, s_T), \end{align*}$$

which represents the minimum possible cost of moving directly from $ s_t $ to $ s_T $ .

For any segment $ s $ of participant i and time $ t $ , it can be seen that $AC_{is}(t) \geq OC_{is}(t).$ This is because $OC_{is}(t)$ represents the smallest possible cost of traveling from $ s_t $ to $ s_T $ , as it corresponds to the distance computed using Dijkstra’s algorithm; while $ AC(S, t) $ refers to the cost of a specific path taken to move between the same points, which may involve additional deviations or detours. As a result, it follows naturally that $AC_{is}(t) \geq OC_{is}(t).$

Now, we define the cost difference of a segment s of participant i at time $ t $ as

$$\begin{align*}\Delta C_{is}(t) = AC_{is}(t) - OC_{is}(t), \end{align*}$$

where $\Delta C_{is}(t)$ represents the difference between the actual cost and the optimal cost of moving from $ s_t $ to $ s_T $ . Here, $\Delta C_{is}(t)$ can be viewed as a non-negative function of time observed at $1,2,\ldots ,T$ . This definition helps isolate segment-specific behavioral patterns, decoupling them from the original terrain and object location data. This decoupling simplifies the data structure while standardizing comparisons across segments, making it easier to identify patterns and trends independent of the original environment.

To ensure consistency across all segments, we normalize the cost difference curves. This step involves aligning all curves to the same length, as the duration of movement between objects can vary substantially across segments. More specifically, let $\Delta C_{is}(t)$ be the cost difference curve of a segment $s = (s_1, s_2, \ldots , s_{T_s})$ with total duration $T_s$ for participant i. We normalize this curve by interpolating it to a fixed time domain $[0,T]$ , where T is a unified time scale set as $100$ in our study. We denote the normalized curve by $\Delta C^{\prime }_{is}(t)$ , which is a function of rescaled time $t \in [0,T]$ . It is important to note that normalizing the curves does not result in significant information loss. Although the original lengths of the curves may reflect different behavioral patterns—shorter curves often represent more direct paths and longer curves indicate more circuitous routes—the process of normalization retains the essential shape and variation of the cost difference. The underlying patterns of behavior, such as whether a participant followed an efficient or wandering path, remain embedded in the structure of the normalized curves. This ensures that critical behavioral information is preserved for subsequent analysis.

3.4 Clustering cost difference curves

We employ a functional clustering approach to analyze the normalized cost difference curves. This methodology follows a widely used framework in functional data analysis that combines basis function representation with centroid-type clustering methods. In particular, prior work has integrated B-spline or wavelet bases with K-means or fuzzy clustering algorithms to cluster functional trajectories (Abraham & Cornillon, Reference Abraham and Cornillon2003; Giacofci et al., Reference Giacofci2013; Jacques & Preda, Reference Jacques and Preda2014). Our approach adopts a similar strategy by using cubic B-spline basis functions to extract coefficient vectors from each normalized curve, followed by K-means clustering applied to the estimated coefficients. This setup enables the identification of prototypical navigation patterns across segments while maintaining computational simplicity. To capture common patterns, we combined all cost difference curves, regardless of environment or training phase, into a single dataset prior to clustering. We first smooth the curves using spline analysis to reduce noise while preserving key structural features. Splines are particularly suited to this task because we are primarily interested in the turning points within the curves, which represent critical behavioral patterns. We use cubic spline interpolation (Ramsay & Dalzell, Reference Ramsay and Dalzell1991) for convenience (we have tried higher-order splines but the results are largely similar so we decide to keep the spline order at $3$ ). The interval $[0,T]$ is divided into sub-intervals with equally spaced knots. We denote the corresponding spline basis functions by $P_1(t),\ldots ,P_K(t)$ . A total of $K=10$ basis functions were used. We have conducted a sensitivity analysis on this choice in Appendix B and the results suggest that the clustering results are not sensitive to this choice. For each normalized cost difference curve $\Delta C^{\prime }_{is}(t)$ , we can approximate it by a linear combination of spline basis functions:

(2) $$ \begin{align} \Delta C^{\prime}_{is}(t) \approx \sum_{j=1}^K \beta_j P_j(t). \end{align} $$

The spline-based approach highlights transitions and curvature, aligning with our goal of understanding individual behavioral variability. We estimate the coefficients $\boldsymbol {\beta } = (\beta _1,\ldots ,\beta _K)$ using least squares fitting for each of the normalized cost difference curves and then perform K-means clustering (Jacques & Preda, Reference Jacques and Preda2014) to the estimated spline coefficients. It is important to note that each study participant has about $30$ curves corresponding to 30 path segments from two training sessions combined. Clustering is performed at the curve level instead of the participant level because we are mainly interested in understanding the heterogeneity among the path segments. In the next section, we will conduct a participant-level analysis.

Through our clustering process, we have identified four distinct clusters representing different types of trajectories based on their cost difference curves. The number of clusters is determined by the Silhouette score (an elbow plot is provided in Appendix E). Each cluster captures unique behavioral patterns, shedding light on the variability in navigation strategies among participants. Figure 4a shows the mean and standard deviation curves for each cluster. There are a total of 5,572 curves, and 4,499 curves (80.74%) in Cluster 1, 826 curves (14.82%) in Cluster 2, 121 curves (2.17%) in Cluster 3, and 126 curves (2.26%) in Cluster 4. To check the stability of our clustering result, we repeat the same clustering analysis 1,000 times with different random seeds and calculate the adjusted Rand index (ARI) between these cluster results and the current one. The average ARI is 0.9973, with a standard deviation of 0.0059. This confirms the stability of our cluster results.

Cluster 1 (Figures 4b and 5a) represents the best-performing trajectories, characterized by minimal cost differences. This cluster dominates, encompassing over 80% of the cost difference curves. These trajectories reflect the most efficient behavior, where participants move directly from one object to the next with a clear target in mind. The paths in this cluster suggest a high level of decisiveness and effective navigation, indicating that for most segments, participants follow an optimal or near-optimal route.

Cluster 2 (Figures 4c and 5b) captures the second-best trajectories, which exhibit moderate improvements in efficiency over time. Although these paths are less direct than those in Cluster 1, they show evidence of participants adjusting their behavior as they navigate. The moderate cost differences suggest that participants may initially deviate from the optimal path but eventually correct their course. This cluster likely represents learning or exploratory behavior, where participants refine their routes as they gain more information about the environment.

Figure 4 Functional clustering analysis results.

Clusters 3 and 4, on the other hand, represent poor-performing trajectories with significant inefficiencies. These trajectories deviate substantially from the optimal path, indicating struggles in navigation. Cluster 4 (Figures 4e and 5d), in particular, highlights a pattern in which participants initially struggle to find the correct path, but quickly adjust and align with the optimal trajectory. This cluster suggests a reactive behavior, where participants make abrupt course corrections after recognizing inefficiencies in their initial navigation choices. The cost differences drop sharply before the middle of the segment, reflecting this rapid adaptation.

Figure 5 Comparison of sample curves to their optimal paths for each of the four clusters.

Note: Each subfigure represents a cluster and it contains four different environments.

In contrast, Cluster 3 (Figures 4d and 5c) displays prolonged inefficiency. Participants in this cluster take significantly longer time to identify the correct path, resulting in consistently higher cost differences for the most part of the segment. This cluster may reflect hesitation, confusion, or repeated errors in decision-making. The trajectories in Cluster 3 highlight the challenges some participants face in navigating the environment effectively, providing an interesting contrast to the adaptive behavior seen in Cluster 4.

To explore the impact of training on cluster distributions, we analyzed the proportions of cost difference curves in each cluster for two training datasets. From Table 2, we observe a notable difference between Training 1 and Training 2, particularly in Cluster 1 and Cluster 2. Cluster 1, which represents the best-performing trajectories, shows a higher proportion in Training 2 (82.9%) compared to Training 1 (78.6%). In contrast, Cluster 2, which captures moderately efficient trajectories, decreases from 16.9% in Training 1 to 12.7% in Training 2. This suggests that some trajectories from Cluster 2 in Training 1 may have switched to Cluster 1 in Training 2, indicating improved navigation efficiency. Clusters 3 and 4, representing less efficient trajectories, exhibit minimal changes across the two trainings. Cluster 4 shows very low proportions in both datasets, with a slight decrease in Training 2 (2.3% vs. 2.0%), while Cluster 3 remains nearly constant.

Table 2 Cluster distribution for two training sessions

These findings support the hypothesis that Training 2, conducted immediately after Training 1, benefits from participants’ increased familiarity with the environment. The higher proportion of efficient trajectories in Cluster 1 and the reduction in moderately efficient trajectories in Cluster 2 demonstrate improved performance and adaptation over time. This improvement highlights the impact of prior exposure on the game and demonstrates a learning benefit between consecutive training sessions.

The data in Table 3 also highlight the impact of different environments on participants’ behavior. Notably, the fourth environment shows a marked difference compared to the first three environments. In Environment 4, the proportion of trajectories in Cluster 1 drops to 76.8%, compared to over 81% in the other environments. This indicates that participants were less likely to adopt the most efficient behaviors in this environment. The results imply that the distinct characteristics of Environment 4 may have posed additional challenges or encouraged exploration, leading to a slight shift in behavior patterns and small decrease in efficient movement.

Table 3 Cluster distribution for four game environments

Table 4 provides insight into the difference in navigation patterns between objects first found during training compared to at the end. The “first segment” refers to the segment where a participant begins the training session, starting from the initial point and moving toward the first object. In contrast, the “last segment” represents the transition from the penultimate object to the final object in the session. After completing the “last segment,” the training session concludes. These segments are particularly significant as they mark the temporal boundaries of the training process and may exhibit distinct behavioral patterns compared to the intermediate segments.

Table 4 Cluster distributions for all segments (all), path segments to the first subject (first), and segments to the last subject (final)

The first segment, where participants are just beginning to navigate the environment, shows a significantly higher proportion of trajectories in Cluster 1 (90.6%). This suggests that participants tend to make direct and efficient choices when selecting their initial target, likely opting for the nearest object. The high efficiency of these paths reflects the relatively straightforward decision-making process early in the task, as participants have not yet been influenced by exploration or challenges introduced by subsequent targets. In contrast, the last segment, where participants are finding the final remaining object, which is often more challenging to locate. This difficulty is reflected in the higher proportions of Cluster 3 (4.0%) and Cluster 4 (4.2%) trajectories in the last segment. These clusters represent less efficient navigation patterns, with greater wandering or prolonged decision-making likely due to the increased difficulty in identifying and reaching the last object. These results highlight distinct behavioral patterns associated with the first and last segments. The first segment is characterized by a preference for efficiency and directness, while the last segment reflects greater difficulty and variability in navigation strategies.

In summary, the functional clustering of standardized cost difference curves revealed distinct behavioral patterns in navigation strategies. The results showed that the majority of trajectories (Cluster 1) were characterized by highly efficient and direct paths, suggesting that participants predominantly displayed optimal or near-optimal navigation. Cluster 2 highlighted moderate improvements and adaptive learning, indicating participants’ ability to refine their routes with increased familiarity. Clusters 3 and 4, on the other hand, captured less efficient behaviors, with Cluster 4 reflecting reactive adjustments and Cluster 3 exhibiting prolonged inefficiencies due to hesitation or confusion.

Further analysis demonstrated the influence of training, environmental complexity, and segment characteristics on cluster distributions. Notably, Training 2 resulted in a higher proportion of efficient trajectories, underscoring the role of learning and adapting navigation strategies. The variations across environments revealed that specific environmental features posed some additional challenges, encouraging exploratory behavior. Lastly, the first and last segment effects highlighted distinct temporal-based decision-making strategies during individual training sessions: initial segments showed a tendency for efficiency, while last segments reflected increased difficulty and variability. Together, these findings provide a comprehensive view of navigation dynamics, shedding light on how participants balance efficiency, adaptation, and exploration under varying conditions.

We also repeat our clustering analysis of cost difference curves using functional principal component analysis scores. The resulting clusters are provided in Appendix C and they are very similar to the ones obtained by the B-spline basis. These findings suggest that our analysis is not sensitive to the choice of basis functions.

3.5 Functional regression analysis

To address the open question of how training behaviors influence performance during testing, we aim to extend our analysis from individual segments to a participant-level perspective, which requires aggregating multiple cost difference curves for each participant. This is not a trivial task because of the high level of heterogeneity in these curves. Our strategy is to first perform a classification step on the associated segments by accounting for their intrinsic difficulty levels, i.e., curves from “easy” segments should not be directly combined with curves from “difficult” segments. For each segment, we calculate the cost associated with the optimal path as the Dijkstra distance between the start and end points. In Figure 6a, we show the histogram of optimal path costs for all segments in the training. We decide to use a cutoff of 70 to distinguish between low-cost and high-cost segments. In other words, segments with an optimal cost below 70 are typical paths we expect participants to pick, while segments with an optimal cost above 70 represent inefficient or challenging paths (e.g., two objects are far apart from each other or this segment belongs to a challenging terrain). The cutoff value of 70 is chosen for three reasons: (1) the histogram suggests that most of the segments have an optimal path cost below 70; (2) in all four environments, it is possible to connect between all object pairs with optimal costs less than 70 across. As shown in Figure 6b, these low-cost segments are sufficient to cover the entire map in a relatively direct and efficient manner. (3) 53 of 182 (29.12%) participants pick only path segments with an optimal cost below 70.

Figure 6 Analysis of optimal path costs across different environments.

For each participant, we calculate the average curve of the normalized cost difference curves associated with low-cost segments (optimal costs below 70) (see Figure 7 for the average curves of all participants). We also count the number of high-cost segments (optimal cost $> 70$ ) for each participant and call it “Removed Curve Count.” This metric provides a complementary perspective, capturing how often participants engaged in inefficient navigation behaviors. By separating the analysis of low-cost and high-cost segments, we aim to better understand the factors contributing to these deviations and their potential impact on overall performance. More discussions about this choice are provided in Appendix D.

Figure 7 Average cost difference curves for low-cost segments from 182 study participants.

To evaluate the test performance of the participants, we define three outcome variables: $ Y_1 $ , $ Y_2 $ , and $ Y_3 $ , each capturing a different aspect of navigation and placement behavior. The outcome $ Y_1 $ is the total number of objects correctly located during the test. Here, an object is considered successfully located if its placement is within 10-block distance to the object’s true location. This variable measures overall memory, where larger values indicate better performance. The second variable, $ Y_2 $ is the average time taken to correctly locate the objects, calculated only for the objects that were successfully placed. This variable reflects efficiency, with smaller values indicating quicker navigation. Finally, $ Y_3 $ is the average distance from the correct location, measured using the Dijkstra distance, for objects that were correctly placed. This variable represents precision, where smaller values correspond to more accurate placements. By analyzing these target variables, we explore how training behaviors influence test performance and gain insights into the relationship between navigation strategies and outcomes during the test phase.

To model the relationship between training behaviors and test performance, we employ a scalar-on-functional regression (SoFR) model (Crainiceanu et al., Reference Crainiceanu, Goldsmith, Leroux and Cui2024; Goldsmith et al., Reference Goldsmith, Bobb, Crainiceanu, Caffo and Reich2011). This approach integrates both scalar predictors, such as participant-specific features, and functional predictors, such as the average cost difference curves. The model can be written as follows:

(3) $$ \begin{align} Y_{k, i} = \beta_{k0} + \boldsymbol{Z}_i^\top \boldsymbol{\beta}_{k1} + \int X_i(t) \beta_k(t) \, dt + \epsilon_{ki},~~k=1,2,3, \end{align} $$

where $ Y_{k, i} $ represents the k-th test performance outcome for participant $ i $ , $ \boldsymbol {Z}_i $ is the set of scalar covariates, including age, sex, video game experience, Minecraft skill, weekly gaming hours, removed curve count, cluster proportions, and average segment time. Cluster proportion is the proportion of clusters 2–4 in each participant’s total segments. The average segment time is the average time a participant spends on all segments during training. The term $ \boldsymbol {\beta }_{k1} $ is the vector of associated coefficients. The term $ X_i(t) $ represents the average curve of the normalized cost difference curves associated with low-cost segments for participant i, providing dynamic information about a participant’s navigation behavior over time. The functional coefficient $ \beta _k(t) $ describes how the influence of $ X_i(t) $ changes over time, enabling us to capture the detailed relationship between the shape of the cost difference curve and the test performance. The error terms $\epsilon _{ki}$ ’s are assumed to be independently and identically distributed (i.i.d.) following a normal distribution $N(0, \sigma _k^2) $ . The model (3) essentially fits separate regressions to each of the $Y_1,Y_2,Y_3$ . It is possible to extend this model by considering a joint modeling approach to account for the dependence in the outcomes. We do not pursue this direction here because the correlations among them are small, e.g., Corr $(Y_1,Y_2) = -0.15$ , Corr $(Y_1,Y_3) = -0.25$ , and Corr $(Y_2,Y_3) = 0.10$ .

We fit functional regressions using the fRegress function from the fda R package. The functional covariate is represented using a B-spline basis with ten basis functions, and the functional coefficients are estimated with a roughness penalty on the second derivative. The penalization is implemented via the fdPar function, using Lfdobj = int2Lfd(2) to penalize curvature and a smoothing parameter of lambda = 0.1. The number of basis functions was set to 10 to allow sufficient flexibility while maintaining smoothness through penalization.

For $Y_1$ , the total number of objects correctly located, the estimated coefficients for scalar covariates are summarized in Table 5. The model explains a moderate proportion of the variance ( $ R^2 = 0.2958 $ ). The results suggest that sex may influence performance in locating objects. Specifically, males tended to identify more objects compared to females and individuals in the “Other” sex category (estimate = 0.9536, p = 0.0569). Additionally, Minecraft skill level significantly affected performance $Y_1$ . Participants in the Intermediate skill group tended to locate fewer objects than those in the Expert group (estimate = $-$ 1.2844, p = 0.0678), while participants in the Novice group performed significantly worse (estimate = $-$ 1.7253, p = 0.0263). These findings align with intuitive expectations that higher Minecraft skill levels likely correspond to better familiarity with the game mechanics and require lower cognitive demands of video game controls while participating in the task than would be required for a novice player. The proportions of specific clusters also show a notable impact on $Y_1$ . A higher proportion of segments in Cluster 3 (estimate = $-$ 11.6734, p = 0.0969) and Cluster 4 (estimate = $-$ 12.3578, p = 0.0716), which correspond to less efficient navigation during training, were associated with poorer performance in locating objects during testing. This suggests that these clusters during training capture patterns associated with suboptimal test performance and are indicative of underlying learning factors that negatively affected recall performance.

Table 5 Summary of scalar regression coefficients for $ Y_1 $ , $ R^2 = 0.2958 $

The influence of video game experience on $Y_1$ , however, is somewhat unexpected. Compared to participants who played both game types or only third-person games, those who exclusively played first-person games performed worse (estimate = $-$ 1.1099, p = 0.0594). Unsurprisingly, the “Neither” group performed the worst overall (estimate = $-$ 1.8533, p = 0.0069).

The scalar estimated coefficients for the functional regression analysis on $ Y_2 $ , the average time taken to correctly locate objects, are summarized in Table 6. The model explains a moderate proportion of the variance ( $ R^2 = 0.2602 $ ). However, most covariates were not statistically significant, which may be attributed to the specific definition of $ Y_2 $ —the average time calculated to successfully locate objects. For individuals who located only a few objects at test, their performance might be influenced by chance rather than by an intentional or efficient navigation strategy. In such cases, their successful placements may result from random or lucky guesses rather than a clear memory or deliberate navigation strategy, thereby diminishing the predictive power of some covariates on $ Y_2 $ . The Removed Curve Count, however, does exhibit a notable relationship with $ Y_2 $ . This variable captures the number of training segments with optimal costs exceeding 70, representing atypical or inefficient navigation behaviors in training. Interestingly, a higher Removed Curve Count is associated with shorter average times for locating objects during test (estimate = $-$ 2.7311, p = 0.0246). One possible explanation is that taking non-optimal or unconventional routes during training may inadvertently provide participants with a deeper, more comprehensive understanding of the environment. This enhanced familiarity could then translate into more efficient navigation during testing, despite the seemingly inefficient behavior observed in training.

Table 6 Summary of scalar regression coefficients for $Y_2$ , $ R^2 = 0.2602 $

Notably, average segment time emerged as a significant predictor of $ Y_2 $ , with a positive association (estimate = 1.0805, p = 8.18e-06). This suggests that longer time during training was strongly linked to longer average durations per correct placement. One interpretation is that this variable reflects a participant’s inherent pacing or navigational style. For instance, some individuals may adopt a fast-paced approach in both training and testing phases, while others naturally proceed more slowly and cautiously. These consistent behavioral patterns could explain the strong association between total time spent and average time per successful placement, implying that individual differences in game style—such as speed, thoroughness, or hesitancy—may systematically influence performance efficiency.

However, both the functional covariates of $Y_1$ and $Y_2$ were not significant. Figure 8a and 8b shows the estimated $\beta (t)$ with 95% confidence interval for $Y_1$ and $Y_2$ obtained by bootstrap. We can see that most part of the 95% confidence interval for both plots contains zero, which means that they are not statistically significant.

The scalar estimated coefficients for the functional regression analysis on $Y_3$ , the average distance from the correct location, are summarized in Table 7. The model explains a moderate proportion of the variance ( $ R^2 = 0.2523 $ ). A notable effect of Minecraft skill level is observed. Participants in the Intermediate skill group tend to place objects farther from the correct location compared to those in the Expert group (estimate = 0.9158, p = 0.0605), while participants in the Novice group perform even worse (estimate = 1.2548, p = 0.0199). This finding aligns with expectations, as higher skill levels likely correspond to better navigation and spatial awareness, leading to more accurate placements.

Table 7 Summary of scalar regression coefficients for $ Y_3 R^2 = 0.2523 $

There is also an environment effect. Compared to Environment 1, which serves as the baseline, participants in Environment 4 place objects farther away (estimate = 1.1712, p = 0.0056). This difference across environments is consistent with the findings in the functional clustering analysis (Table 3). Specifically, the proportion of Cluster 1, which represents the best-performing cluster during training, is significantly lower in Environment 4 compared to other environments.

A cluster effect is also observed. A higher proportion of segments in Cluster 2 is associated with more accurate object placement (estimate = $-$ 3.1647, p = 0.0493). Cluster 2 represents paths that, while less optimal than those in Cluster 1, still maintain relatively straightforward navigation during training. This result can be interpreted through the characteristics of Cluster 2. Paths in this cluster deviate slightly from the optimal trajectory, allowing participants more time to explore the environment while still keeping the target object in mind. This behavior likely helps participants develop a better understanding of the terrain and landmarks surrounding the object’s true location, thereby improving their accuracy during testing.

Figure 8 Functional coefficient curves and associated 95% confidence bands for $Y_1$ , $Y_2$ , and $Y_3$ .

A notable effect of the functional predictor is observed in the regression on $ Y_3 $ . From the figure of the functional predictor $\beta (t)$ (Figure 8c), we can observe that, although most parts of $\beta (t)$ are not statistically significant, the last segment of the functional predictor does exhibit a significant effect. Specifically, the larger the last segment of the average curve, the closer the participant tends to place the objects. This result is somewhat unexpected, as during training, participants who quickly locate and approach the objects are typically considered better performers. However, the behavior represented in the final 20% of the training paths might differ. Participants could move directly and efficiently toward the target object, resulting in a cost difference curve closer to zero. However, they could also take smaller, more tentative steps in the last few blocks before reaching the object. In some cases, they may even deliberately deviate from the optimal route, exploring the area around the object more thoroughly. Such behavior would produce a larger final 20% in the cost difference curve. Participants exhibiting this second type of behavior are likely to develop a better understanding of the specific region surrounding the target object’s true location. This enhanced familiarity with the immediate vicinity of the object could explain their improved accuracy in placing objects closer to their true locations during testing.

In summary, this section extends the analysis of navigation behaviors from segment-level clustering to a participant-level perspective, incorporating functional regression to investigate how training influences test performance. The introduction of the cost difference curve and its functional representation provide a systematic way to capture and quantify navigation strategies, allowing for the identification of key patterns across training and testing phases.