1. Introduction
On 18 September 2013, the Maltese-flagged chemical tanker Ovit ran aground in Dover Strait. Subsequent investigation found that two of the contributing factors related to the use of Electronic Chart Display and Information System (ECDIS) onboard. Firstly, the safety contour settings were inappropriate for the vessel's draught. Secondly, the safety zone featureFootnote 1 was incorrectly configured and the alarm, which could have alerted the officer of the watch of the shallow waters ahead, was disabled (MAIB, 2014).
Three years later, on December 3, 2016, the Spanish-flagged general cargo vessel Muros ran aground on the Haisborough Sand in the North Sea. Similar to the case of the Ovit, it was also found that the safety zone feature was not activated (MAIB, 2017). It should be noted that both the Ovit and Muros, at the time of their respective accidents, were equipped with the Maris ECDIS 900.Footnote 2
A number of similar accidents, where incorrect use of ECDIS was involved, gave rise to the term ‘ECDIS-assisted accident’ (Nielsen, Reference Nielsen2016). Reports of such ‘ECDIS-assisted accidents’ often list the ECDIS operator as the main culprit as he/she has not operated the system in the manner intended by the manufacturers and the vessels’ managers (MAIB, 2008, 2012, 2015). However, a study by researchers from Lund University found operators’ errors to be connected to design issues, especially the complexity of system interfaces and information overload. Information is not intuitively organised, and operators must navigate through complex system menus and sub-menus to locate required features. Furthermore, there is too much information presented and unnecessary data are often included, which can potentially become cluttered (Nielsen, Reference Nielsen2016). Together, these design issues make it difficult for the operators to work effectively and increase the chance of error.
To help mitigate these design issues, the authors conducted research to develop a pattern to organise contents on the displays into groups so that seafarers/operators can access important information with ease. In this paper, the term grouping pattern is used to refer to a way to organise the different information, control functions and settings available on the displays of navigation systems into various groups. Given the limited resources available to the research team, the scope of this research is limited only to Radar and ECDIS displays, and their equivalent components in integrated navigation systems (INS) or integrated bridge systems. Also, this research only concerns equipment on vessels within the scope of the International Convention for the Safety of Life at Sea (SOLAS).
This paper details the selected methods and the findings, and provides implications for future research.
2. Previous studies and findings
There have already been academic studies and regulatory documents concerning the organisation of information on navigation displays, although the number is limited. Specifically, standard IEC 62288:2014 (IEC, 2014) together with document MSC.191(79) (IMO, 2004a) require data and control functions on the interfaces of navigation systems to be logically grouped based on their functions or tasks. The documents also provide examples of grouping patterns for main Radar functions, but such groups are only suggestive, and the authors could not find any source validating or evaluating the usability of such patterns. Additionally, these documents do not specify exactly how a grouping pattern can be considered ‘logical’ and only state that such criteria shall be confirmed by ‘analytical evaluation’ (IEC, 2014, p. 16).
Document MSC.1/Circ.1609 (known unofficially as the ‘S-mode guidelines’) introduces another grouping pattern for essential information. Compared with IEC 62288:2014, the pattern introduced in the S-mode guidelines contains more generic data and can be applied to a wider range of navigation equipment. Also, unlike the IEC pattern, this S-mode pattern is mandatory, and equipment manufactured after 2024 must follow MSC.1/Circ.1609 (IMO, 2019). Still, the S-mode grouping pattern only includes a limited number of navigational data, and the usability of the grouping pattern has not been evaluated.
Regarding academic studies, the few research studies on the organisation of contents on navigation displays all employ eye-tracking devices to record the eye movements of the operator while navigating, either on a real ship or in a simulated environment (IMO, 2018; Hareide, Reference Hareide2019). All these studies share common drawbacks. With eye-tracking data, it is only possible to identify the screen areas that users most often gaze upon, without understanding the reasoning and/or intention of the user. Additionally, there is a difference between unintentional gazing upon a screen and actively searching the screen for information, and it is difficult with eye-tracking data to distinguish between those two actions.
In summary, there are both industry guidelines and academic studies on the grouping of contents on navigation displays. However, all these studies have weaknesses and none of their findings were validated with users. Also, all existing studies fail to capture the reasoning/intention of users. As a result, there is a need for a research approach that can develop a grouping pattern, collect data on users’ reasoning/intention, and validate the grouping pattern using appropriate usability methodologies.
3. Methods
As this research aims to improve the usability of an interactive system, a user-centred design approach is deemed most appropriate as similar design projects have found the approach to be effective in creating systems with high usability (Petersen, Reference Petersen2010). The authors designed this research to follow standard user-centred design activities, as prescribed in standard ISO 9241-210 (ISO, 2010). The research contains three phases, each with its own methods.
3.1 Phase 1 – Identifying the context of use
The first step of this research is identifying the context of use, which involves identifying characteristics of the users, the tasks to be performed, and the working environment (ISO, 2010). Specifically, in this case, it is necessary to identify the tasks involved when seafarers navigate ships and the functions on Radar and ECDIS required to support each task.
To achieve the objective of this phase, the authors performed three activities:
• Review performance standards of navigation systems to identify minimum mandatory functions.
• Review existing analyses of navigation tasks, addressing the weaknesses of each analysis, and merge results of all available analyses together into a single analysis of navigation tasks.
• Conduct a survey to collect data from seafarers on the frequency of use (FOU) of each function of a standard INS, together with the purpose of each function.
The first activity involves reviewing performance standards for marine navigation systems to identify mandatory functions. As per the scope of this research, only performance standards for Radar (IMO, 2004b), ECDIS (IMO, 2006), and INS (IMO, 2007) were considered. This action was taken to account for the variety in design and functionality of systems between different manufacturers.
The second activity involves reviewing analyses of navigation tasks. The research team originally planned to perform a new task analysis but eventually changed the plan after considering two factors. Firstly, several analyses of navigation tasks have been conducted both in academia and in the industry, and all of them have limitations. Secondly, conducting a new comprehensive analysis is time- and resource-consuming, even more so if the authors attempt to account for the limitations of previous analyses. As a result, the research team decided not to conduct a new analysis, but rather to review existing analyses, accommodate their shortcomings and combine the results together. Given the advancement of navigation techniques over time, the analyses by Sanquist et al. (Reference Sanquist, Lee and Rothblum1994), Van Westrenen (Reference Van Westrenen1999), Koester et al. (Reference Koester, Anderson and Steenberg2007), Røed (Reference Røed2007), and Procee et al. (Reference Procee, Borst, van Paassen, Mulder and Bertram2017) were considered. The research team has taken the following measures to address the shortcomings of these analyses:
• The analyses were conducted at different times over a long period. As a result, some of the identified tasks reflect out-dated practices, many of which have been automated or modified. To address this issue, the research team identified such out-dated practices and revised them using literature on contemporary nautical practices (Swift, Reference Swift2004; IMO, 2004b, 2006, 2007) as well as by reflecting on the experience of the researchers, all of whom were seafarers. An example is the task of computing Target Relative Motion and True Motion Vectors by plotting on the Radar PPI, as described in the analysis by Sanquist et al. (Reference Sanquist, Lee and Rothblum1994), which has long been automated by Automatic Radar Plotting Aid (ARPA).
• Many existing task analyses have limited scope, such as area of observation or people observed, or having specific operational contexts, such as high-speed craft (Røed, Reference Røed2007). To address this issue, the research team identified such specific tasks and removed or adapted them to conventional navigation scenarios. This step was taken using two sources of reference. The first one is, once again, literature on contemporary nautical practices and the experience of members of the research team. The other reference is results of the survey on FOU for standard functions of INSs, which will be discussed in subsequent sections
• All existing analyses lack details on some of the identified tasks while giving sufficiently detailed descriptions of others. To address this issue, the research team merged all analyses together, allowing the merits of one analysis to accommodate for the shortcomings of another. The results of the FOU survey were also used to add details to under-specified tasks
The third step involves a survey on the frequency with which seafarers use each function of a standard INS when performing navigation duties. The survey also collected data on the purpose of each function and scenarios in which the functions are utilised, with results collected from 601 seafarers worldwide. Details of the survey and the results are published in an article by Vu et al. (Reference Vu, Lützhöft and Emad2019).
In the end, the results of the three activities described above were combined to create a comprehensive picture of how seafarers operate navigation systems. The results help to identify the functions of Radar and ECDIS that are most essential for safe navigation and which should be readily available on the displays.
3.2 Phase 2 – Developing an initial grouping pattern
Once the context of use is established, the next step should be identifying users’ requirements. In this case, it is necessary to understand how seafarers perceive and categorise information when engaging in navigation duties and organise the contents, which were identified at the end of phase 1, into groups in a manner so that seafarers can access necessary information with ease.
To these ends, the researchers used a method called ‘card sorting’, which originated from the Q methodology developed by Stephenson (Reference Stephenson1935) to study subjectivity and is now commonly used by web designers and software developers to develop information architectures. The method involves providing research subjects with a set of cards representing relevant concepts and having the subjects sort the cards into groups that are similar in certain ways (Wood and Wood, Reference Wood and Wood2008). This method has been proven to be a simple yet useful method to gain insight into how users categorise and organise information (Faiks and Hyland, Reference Faiks and Hyland2000; Gatsou et al., Reference Gatsou, Politis and Zevgolis2012; Doubleday, Reference Doubleday2013).
In this project, the card sorting was conducted in the form of unmoderated open sorting using an online platform at www.usabilitest.com. Each research subject was given 49 cards representing 49 functions of Radar and ECDIS that are essential for safe navigation. The term ‘functions’ is used here to refer to a piece of information, a control function, or a setting that is presented on the displays of Radar and ECDIS or their equivalent modules on an INS. These 49 functions were selected based on the results of phase 1, and the number of 49 cards was selected after several pilot studies to achieve the maximum completeness of data while also preventing the research subjects from experiencing fatigue. A complete list of these cards is attached in Annex 1.
The card sorting activity was advertised with the support of the Nautical Institute, which is a London-based international organisation for maritime professionals with consultative status at the IMO, and the research subjects thus recruited (n = 63) were all members of the Institute, who are also maritime professionals. Grouping patterns created by these research subjects were merged using a statistical classification technique called advanced merge method to create a single grouping pattern. See Vu and Lutzhoft (Reference Vu and Lutzhoft2018) for details.
The grouping pattern that resulted after phase 2, here referred to as the ‘initial grouping pattern’, represents a collective mental model of all 63 research subjects regarding the classification of the 49 selected Radar and ECDIS functions. In this grouping pattern, the 49 selected Radar and ECDIS functions are arranged into 12 groups. This initial grouping pattern, however, cannot be taken as a reliable final result due to the weaknesses of the card-sorting method used to create the pattern. Specifically, card sorting has the following weaknesses that must be considered:
• In card-sorting experiments, participants put content into groups (classifying). In actual usage scenarios, people look for information. There are differences between classifying and finding content (Spencer, Reference Spencer2009).
• Unless participants think out loud while sorting the cards, card sorting cannot capture the rationale behind the grouping patterns (Maiden and Hare, Reference Maiden and Hare1998).
• Card sorting does not produce concretely defined categories. It is very unlikely that the research participants will agree on everything and there will be disagreement to different extents in many cases. As a result, there is a certain degree of intuition involved in the data analysis process (Brucker, Reference Brucker2010).
Considering these factors, the researchers developed a method to validate the initial grouping pattern, the details of which are presented in Section 3.3.
3.3 Phase 3 – Validating the initial grouping pattern
Phase 3 involves validating the initial grouping pattern created at the end of phase 2. This step corresponds to the action of evaluating the initial design against user requirements in a standard user-centred design process (ISO, 2010).
Considering the limitations with card sorting discussed in Section 3.2, the researchers decided to evaluate the initial grouping pattern using the following procedure:
1. The initial grouping pattern resulting from the card-sorting study in phase 2, which contained 12 groups, was graphically illustrated as segments of Radar and ECDIS displays. The illustration can be seen in Figure 1 with the groups numbered from 1 to 12. This illustration was designed not to follow the design style of any specific manufacturer while still being sufficiently detailed to be recognisable by any person familiar with Radar and ECDIS.
2. The researchers randomly selected seven from the 12 available groups, and deleted one item from each.
3. The resulting 12 groups were shown to a new sample of research subjects (n = 35), who were maritime professionals. The subjects were asked seven questions, each of which involved finding one of the seven missing pieces of content. In specific, the subjects were instructed to point out, among the 12 available groups, the one they believed most likely to contain the missing items. Such groups were referred to as ‘selected groups’. Each missing item could belong to several selected groups as different research subjects could have different mental models. The number of seven questions was selected after several pilot tests to ensure the research subjects would stay engaged and maintain the thorhoughness of their answers throughout the process. The researchers also applied the following two principles:
i. All subjects were asked to explain their answers.
ii. If a subject believed they could not find the missing information anywhere among the 12 available groups, there was an option to select a ‘New Box’.
4. The research subjects in phase 3 were randomly selected from among the professional networks of the researchers, and none of them had participated in phase 2 of this study. In fact, all but one of the subjects had never heard of this study until being recruited. In total, 35 subjects participated in phase 3 of this study. Demographic data of these research subjects are presented in Table 1.
5. Data collected from these research subjects contained two types of information: the selected group(s) for each of the 49 Radar and ECDIS functions involved, and the reasons why these groups were selected. The researchers conducted thematic analysis following the inductive approach (Boyatzis, Reference Boyatzis1998) on the answers collected from the research subjects to identify common themes among the reasoning of the subjects. The goal of this step was to determine, for each of the 49 selected functions, the most suitable group, here referred to as the ‘logical group’. The term ‘logical’ is used to conform with documents IEC 62288:2014 (IEC, 2014) and MSC.191(79) (IMO, 2004a). However, since these documents do not specify this ‘logical’ criterion, the authors set the criteria for a selected group to be considered the logical group as having the agreement of at least 80% of the research subjects. Subsequently, the identified logical groups would be compared with the initial grouping pattern to determine necessary revisions. Given this validation criterion, there were, for each of the 49 selected functions, three possible outcomes:
i. The arrangement in the initial grouping pattern matched the logical group and no revision was needed.
ii. There was a logical group for the function in question, but it differed from the arrangement in the initial grouping pattern. In this case, the initial grouping pattern would be modified to align with the new results.
iii. Research subjects did not agree on a logical group for the function in question. In this case, the researchers would analyse the reasoning behind the subjects’ decisions, combined with results of the card-sorting experiment in phase 2 and usability principles set out in MSC.1/Circ.1609 (IMO, 2019) to determine the logical group and compare this result with the initial grouping pattern.
In the end, the work in phase 3 resulted in a revised edition of the initial grouping pattern. The authors labelled this revised edition as the ‘revised grouping pattern' to differentiate it with the initial grouping pattern. The following section will present and discuss details of this revised grouping pattern.
A graphical illustration of the 12 groups in the initial grouping pattern
Demographic data of the 35 subjects participating in phase 3
4. Results
Following the revisions made after phase 3 of this study, the original 49 Radar and ECDIS functions were reduced to 48, as one function was found to be incompatible with the rest. The removed function is the function to display time-labels at a selected interval along the ship's track, which is required under clause 11.4.12 of the performance standards for ECDIS (IMO, 2006). Results from both the validation study in phase 3 and the card-sorting study in phase 2 found this function to be incompatible with any of the remaining functions and that it should be placed in a separated group.
The remaining 48 functions were organised into 13 groups, listed in Table 2 below. Details of these groups are presented in the following sections.
The revised grouping pattern
The following sections will discuss details of each of the 13 groups in the revised grouping pattern.
4.1 Universal Presentation Settings (Fundamental)
This group contains two fundamental settings that configure the presentation of the Operational Display Area, as defined in MSC.191(79) (IMO, 2004a). Specifically:
• Multifunction Display (MFD) Mode Indicator – this function is required under clause 7.1.1 of resolution MSC.191(79) (IMO, 2004a). It is only applicable for displays capable of presenting multiple functions, which are often found on INS.
• Brightness Configuration.
The settings contained in this group are ‘universal’, in the sense that they are applicable to all displays of an INS including the Conning Display. In the case of conventional non-integrated bridges, the MFD Mode Indicator function is not applicable. The function for Brightness Configuration, on the other hand, works similarly for both integrated and non-integrated systems.
There was no consensus among the subjects in phase 3 regarding the logical group for the MFD Mode Indicator. At the same time, 80% of the subjects placed Brightness Configuration in the same group with Rain and Anti-clutter Controls. The main reason is because the MFD Mode Indicator is not available on non-integrated bridges, and it is rarely adjusted, while Brightness is available on all systems, is related to settings that adjust the presentation of the Operational Display Area, and is adjusted more frequently. However, the subjects in phase 3 also reported that seafarers often follow a common routine when setting up equipment, which involves the following steps:
• Turning on a system/Switching a system from standby to operational mode/ Selecting an MFD Mode
• Adjust Brightness.
• Configure other main system settings, which depend on the specific system in use.
Considering this common set up routine, and the fact that 56% of the research subjects in the card-sorting study in phase 2 placed these two functions together in one group, the authors decided that it would be beneficial to group the MFD Mode Indicator and Brightness Configuration into the group Universal Display Settings (Fundamental) and place this group near other presentation settings, which are discussed in the next section.
4.2 Universal Presentation Settings (Operational)
This group contains seven settings that configure the Operational Display Area. The term ‘universal’ used in labelling this group has a narrower sense as the settings of this group are only applicable to the Route Monitoring, Route Planning and Collision Avoidance modules of an INS. On non-integrated systems, these settings are applicable only to Radar and ECDIS displays. The seven settings included in this group are:
• Range Scale/Chart Scale
• Range Ring Scale
• Centred/Off-centred Display
• Orientation Mode
• Motion Mode
• True Motion reset
• Stabilisation Mode & Source.
Six of these functions: Range Scale/Chart Scale, Centred/Off-centred Display, Orientation Mode, Motion Mode, True Motion Reset, and Stabilisation Mode & Source, were grouped together by at least 80% of the research subjects in phase 3. This grouping pattern was due to the similarity in their functionalities, FOU and the natural workflow of most users.
Range Ring Scale was placed together with Range Scale/Chart Scale by 60% of research subjects in phase 3. One subject grouped Range Ring Scale next to Electronic Bearing Line (EBL). Their argument was that they frequently use Range Ring Scale as a substitute for Variable Range Marker (VRM), and Range Ring Scale would be combined with EBL to roughly estimate the positions of objects. However, this was not the mainstream opinion. Since Range Ring Scale is dependent on the actual Range Scale/Chart Scale in use and these two features were also grouped together by 60% of the subjects in phase 2, the researchers decided that it is logical to have Range Ring Scale next to Range Scale/Chart Scale.
One subject of the validation study in phase 3 grouped Stabilisation Mode & Source together with Own-ship Speed, as the Stabilisation Mode & Source determines whether Speed Over Ground or Speed Through Water will be displayed, and that was the arrangement on the subject's most recent vessel. However, this was not the mainstream opinion: 60% of the subjects placed Stabilisation Mode & Source in the group that contained Orientation & Motion Settings, while another 20% grouped this function with Brightness Settings, as both are related to the Operational Display Area and both are rarely adjusted. Considering the mainstream opinion of subjects in phase 3 and the fact that Stabilisation Mode & Source was also grouped together with Orientation Mode, Motion Mode, True Motion Reset, and Centred/Off-centred Display by 60% of the subjects in phase 2, the researchers decided that it is logical to have Stabilisation Mode & Source in the same group with Orientation and Motion Settings.
Considering the common workflow mentioned in Section 4.1, this group Universal Presentation Settings (Operational) should be placed near group Universal Presentation Settings (Fundamental).
4.3 Radar Settings
This group contains two settings to configure the Operational Display Area of Radar systems, which are:
• System operational status (Standby/Run)
• Status (automatic/manual) and level of Gain and Anti-clutter Control functions.
This group contains fundamental Radar settings, which are usually set up right after the Radar is switched on. To facilitate the common workflow mentioned in Section 4.1, this group should be placed next to both the Universal Presentation Settings (Operational) and Universal Presentation Settings (Fundamental) groups.
The label ‘Radar Settings’ is selected for this group to follow the standard set out in table 6 of document MSC.1/Circ.1609 (IMO, 2019).
4.4 Chart Display Settings
This group contains two settings for the presentation of the chart on ECDIS systems. These two settings are:
• Chart Display mode (Display Base/Standard Display/All other information) – this function allows users to quickly select a chart display mode. The Standard Display mode selector is mandatory, as required by clause 5.3 of performance standards for ECDIS (IMO, 2006).
• Chart Layer configuration (add or remove chart layers) – this function allows users to manually select which chart information is to be displayed. The function is required by clause 5.5 of performance standards for ECDIS (IMO, 2006).
The settings in this group are often set up right after the ECDIS is switched on. To facilitate common workflow, this group should be placed next to both the Universal Presentation Settings (Operational) and Universal Presentation Settings (Fundamental) groups.
The label ‘Chart Display Settings’ is selected for this group to follow the standard set out in table 6 of document MSC.1/Circ.1609 (IMO, 2019).
4.5 Chart Safety Settings
This group contains four chart-related safety parameters. The label ‘Chart Safety Settings’ is selected to align with table 6 of document MSC.1-Circ.1609 (IMO, 2019). The four features included in this group are:
• Look-ahead settings – this function raises an alarm if the vessel, given her course and speed, will enter dangerous waters. The function is mandatory, as required by clauses 11.4.3, 11.4.4, 11.4.6 of ECDIS performance standards (IMO, 2006). There is no standard name for this feature and different manufacturers use different terms, including Own-ship check, Safety Frame or Anti-grounding. In this paper, the term ‘Look-ahead’ is used.
• Safety contour settings.
• Safety depth settings.
• Under-Keel Clearance [UKC].
The results of the validation study during phase 3 indicate that the subjects categorised settings of navigation systems based on the purpose of the setting, the technical nature of the setting, and the frequency of adjustment. An example is with Safety Contour. The subjects’ opinions were divided into two approaches: based on the use of the setting (to avoid grounding) and based on the technical nature of this setting (as an Electronic Navigational Chart (ENC) layer). The first approach was adopted by 60% of the subjects, who selected and grouped Safety Contour Settings with other depth-related settings, especially Safety Depth Settings. The other 40% took the second approach and grouped Safety Contour with other chart layer settings, placing it in the Chart Display Settings group discussed in Section 4.4.
The researchers decided to keep Safety Contour Settings in this group Chart Safety Settings instead of Chart Display Settings because Safety Contour, as required by MSC.232(82) is a part of Base Display and must be permanently shown on the screen. Users cannot remove this information from the chart. Furthermore, Safety Contour Settings and Safety Depth Settings are rarely adjusted. Chart Layer Settings, on the other hand, are adjusted more frequently. Users actively enable or disable certain layers to create the most suitable chart display for the prevailing situation. Also, Safety Contour Settings and Safety Depth Settings were grouped together by 92% of the subjects in the card-sorting study in phase 2.
Another example is with UKC. Technically, this is a part of own-ship data as it depends on the vessel's draught. However, this information is also depth-dependent. In the validation study in phase 3, 80% of the subjects placed UKC with Safety Depth and Safety Contour settings, while 20% placed UKC among other own-ship data such as Course, Speed and Geographical Coordinates. The argument was that users would often need to check UKC in a scenario where the depth might be a safety concern.
4.6 Target Settings
This group contains two settings that configure the presentation of targets on the Radar displays. The label ‘Target Settings’ is selected to align with document MSC.1-Circ.1609 – Table 6 (IMO, 2019). The two features included in this group are:
• Target Vector Time and Mode settings
• Target Trail Time and Mode settings
These settings are directly related to Targets but, in a broader term, are also related to both the configuration of the Operational Display Area and to other safety settings.
This group should be placed near group Universal Presentation Settings (Operational) discussed in Section 4.2 as it is a common workflow to adjust Target Settings after setting up the Operational Display Area.
This group should also be placed near Target Data as users often adjust values for Target Trailing Length based on data of concerning Targets.
4.7 Automatic Identification System (AIS) Settings
This group contains settings that configure the display of targets retrieved from the automatic identification system (AIS), which is the dominant source of Target Data on ECDIS. These settings include:
• AIS Processing Status (On/Off)
• AIS filtering criteria
The setting for target association, while not included in this study, was also mentioned by research subjects to be a part of this group. Subjects reported a common setup workflow starting with switching on AIS processing, followed by adjusting filtering criteria, and finally setting up Target Association settings.
This group should be placed near group Target Settings discussed in Section 4.6 as the settings in both groups are related to the presentation of Targets on the displays.
There already exists a standard grouping pattern for AIS presentation status, set out in resolution MSC.191(79) (IMO, 2004a), which corresponds with this AIS Settings group. These results indicate that the standard grouping pattern for AIS settings, set out in MSC.191(79), matches the common workflow of users.
4.8 Own-ship Data
This group contains Own-ship Data generated from shipboard sensors, which include:
• Own-ship Heading (HDG)
• Own-ship Course (COG/CTW)
• Own-ship Rate of Turn (ROT)
• Own-ship Speed (SOG/STW)
• The source of own-ship's speed
• Own-ship coordinates (LAT/LON)
The content of this group received strong consensus among research subjects. All subjects in the validation study in phase 3 support this grouping pattern. With the card-sorting study in phase 2, HDG, COG/CTW, ROT, SOG/STW, and LAT/LON were grouped together by 84% of the research subjects. The source of own-ship's speed was included in the same group by 60% of the subjects.
4.9 Target Primary Data
Target Data are available from both ARPA and AIS. Results from both the card-sorting study in phase 2 and the validation study in phase 3 indicate that, for each target, some data are more important than others and should be readily available at all times while others are less critical and not always required.
This section presents Target Primary Data, which are essential for safe navigation and should be readily available. These six pieces of data are:
• Target Range
• Target Bearing
• Target CPA
• Target TCPA
• Target Course
• Target Speed
• Source of Primary Target Data
Target Primary Data can be calculated using ARPA or fetched from AIS. All subjects involved in the validation study in phase 3 commented that ARPA would be the preferable source for Primary Data. Target Secondary Data, discussed in Section 4.10, originate solely from AIS.
4.10 Target Secondary Data
Target Secondary Data are those that are not essential for collision avoidance and do not need to be readily available. These five pieces of data all originate from AIS:
• Target Identification
• Target Heading
• Target Rate of Turn
• Target Navigational Status
• Target LAT/LON
Among Target Secondary Data, Target Identification and Target Heading received diverse feedback from research subjects.
With Target Identification, 60% of the subjects in phase 3 grouped it in Target Secondary Data while the other 40% grouped this information among Target Primary Data. The argument of the latter group was that, for ease of communication, the operator would require the name and/or call sign of the target vessel to be readily available. The counter-argument of the former group was that contacting other ships for collision avoidance would be inadvisable and rarely carried out. Also, most systems do display the name of AIS-activated ships next to the AIS icon of that ship on both ECDIS and Radar displays. Considering these arguments, the researchers decided to place Target Identification among Target Secondary Data.
Similarly, with Target Heading, 60% of the subjects in phase 3 grouped Target Heading among Secondary Data while the other 40% grouped it among Primary Data. The latter group argued that Target Heading determines the applicable rule(s) of the road and, therefore, is important information. The former group counter-argued that Target Heading information is transmitted from the AIS of the target ship, and it is not possible to verify the integrity of the information received. In addition, Target Heading is not essential for determining the rule of the road to be used as the officer of the watch can do so by looking out of the windows and observing the target ship visually. Considering these arguments, the researchers decided to place Target Heading among Target Secondary Data.
4.11 Tools
This group contains tools to aid with navigation and collision avoidance. The five functions included in this group are:
• EBL readout
• Bearing Reference Indication (True/Relative Bearing)
• VRM readout
• Cursor readouts
• Line of Position (LOP) controls
Within this group, the grouping of VRM, EBL and Cursor readouts received a general consensus from at least 80% of the research subjects in phase 3. On the other hand, there were diverse opinions regarding the grouping of Bearing Reference Indication and LOP controls.
With Bearing Reference Indication, the subjects’ opinions were divided into two approaches. The first approach, selected by 60% of the subjects, was to group this function with EBL and cursor readouts as all these features are related to Bearing and users often need to adjust Bearing mode only when operating EBL. The second approach, selected by 40% of the subjects, was to group Bearing Reference Indicator with other presentation settings in the group Universal Presentation Settings (Operational). The argument was that Bearing Reference Indication usually serves also as the control to switch between True and Relative Bearing Mode, and such setting is considered a presentation setting.
Both approaches have no effect on the way EBL is used, as users can always recognise whether True Bearing or Relative Bearing is being used by looking at the value of EBL readouts. It was decided that Bearing Reference Indication should be placed together with EBL and Cursor readouts in this Tools group because this approach received more support from research subjects (60% of the subjects in the validation study in phase 3 and 61% in the card-sorting study in phase 2).
With LOP controls, 60% of the subjects in phase 3 placed this function together with EBL and VRM due to the similarity between these two. The other 40% of the subjects placed this function in group Chart Display Settings discussed in Section 4.5. However, these subjects commented that they were familiar with such arrangement on their previous vessels and there was no other reason for their answers. As a result, these answers could have been influenced by recency bias. Considering this factor, the researchers decided to place LOP controls together with EBL and VRM in this ‘Tools’ group.
It should be noted that, in document MSC.1/Circ.1609, the Cursor Readouts is placed in a separate group called Cursor Location while EBL and VRM are grouped together in a group called Measurement Info (IMO, 2019, p. 29). Still, there is no part in this document where it is specified that these two groups Cursor Location and Measurement Info cannot be placed together in the same head group. As a result, the results of this study do not contradict document MSC.1/Circ.1609.
4.12 Route Plan
This group contains functions used for making/adjusting routes by creating/editing Waypoints or by browsing existing routes in the Route Database. This group received a general consensus from the research subjects and contains the following items:
• Route plan management/ Route Database
• Waypoint management tools
• Route validation (checking the planned route against hazards)
All subjects in phase 3 selected this grouping pattern. In the card-sorting study in phase 2, this grouping pattern was also agreed by 84% of the subjects.
The label ‘Route Plan’ is selected for this group to stay consistent with document MSC.1/Circ.1609 (IMO, 2019).
4.13 Alerts
Replies from all subjects involved in the validation study in phase 3 indicate that there should always be a separate area on the display where all alerts associated with a system are displayed. This result reconfirms the findings of the card sorting study in phase 2. At the same time, feedback from research subjects also indicate that a mechanism for quickly tracing an alert to the source that triggers the alert will allow users to understand the messages quickly and react more effectively.
5. Discussion
Before being implemented, the results presented in Section 4 must be considered in terms of reliability and applicability. This section will discuss the methods used in terms of validity and reliability, as well as factors to consider when applying the results of this research project.
5.1 Methodological discussion
As discussed in Section 3, this study consists of three phases, each with different methods. The methods used in each phase will be discussed in the following subsections.
5.1.1 The three sources of data in phase 1
The first phase involves establishing a general picture of the use of different functions of standard shipboard navigation systems. The authors combined existing cognitive analysis of navigation tasks with the results of a survey of the FOU of each function on a standard INS and subsequently compared these data with performance standards for Radar, ECDIS and INS. As discussed in Section 3, each of these three data sources has strengths and limitations, and by combining the three sources together, the authors minimised the weaknesses of each individual source to create the most reliable overview of the way seafarers operate navigation systems in practice.
The second phase involves a study employing the card-sorting method which resulted in an initial pattern to group together 49 data/control functions on navigational displays that are most essential for the safe operation of any vessel. The quality of this ‘initial grouping pattern’ depends on the validity and reliability of the card-sorting method used.
5.1.2 Card-sorting validity
Considering construct validity, the validity of card sorting as a method relies on the extent to which the way people cognitively categorise information is connected to their performance when working with interactive systems. Although card sorting has been widely used in developing information architectures, there are few studies that verify the connection between card sorting and improved system usability in a manner that can be scientifically evaluated.
One of the earlier studies in this topic was the study of internal structure of categories by Rosch and Mervis (Reference Rosch and Mervis1975). The authors conducted experiments using a set of artificially constructed strings of letters and digits. These strings were purposely constructed to form families (groups of strings that share common letters and/or digits). The level of family resemblance (the number of common attributes that strings of the same family share with each other) differs between families. Research participants learnt the strings and subsequently performed category recognition tests by identifying categories for certain strings. Results of the experiments showed that family resemblance and the lack of overlapping attributes with contrasting string families were correlated with ease of learning and user performance in identifying items after learning. Accordingly, classifying contents into groups with exclusive attributes can make it easier for users to memorise content structure and locate relevant items more accurately.
There are also studies that specifically evaluate the value of card sorting in improving the usability of existing information structures. One such study is the redesign of indexes for the University of Arizona Library by a team of librarians (Dickstein and Mills, Reference Dickstein and Mills2000). At the time of the redesign attempt, the indexes page, which was organised alphabetically, had already been criticised for lack of usability. The librarian team decided to group the indexes in subjects to make it easier for users to locate relevant items. To that end, the team conducted an open card-sorting study using a set of 82 cards. The result of the card-sorting experiment was a large number of index groups, which did not match the expectations of the librarian team. As a consequence, the team discarded those results and developed the index groups based on their perception of academic subjects. After several months, however, many library users complained that the organisation and terminologies used on the indexes page were confusing. The librarian team subsequently restructured the indexes page using results from the card-sorting experiment. The revised page allowed users to locate information more easily. This study is an example of designers and users having different mental models and the potential usability issues when designers ignore users’ mental models (Dickstein and Mills, Reference Dickstein and Mills2000).
Card sorting was also used to revise the structure for the Google AdWords Help Centre. Results of these card-sorting exercises were developed into mock-up webpages and tests were conducted to compare user performance between the old webpage and the revised one. The experiment results showed that users performed tasks significantly faster with the revised Google AdWords Help Centre (Nakhimovsky et al., Reference Nakhimovsky, Schusteritsch and Rodden2006).
Besides traditional card sorting methods, a varied form called Modified-Delphi Card Sorting was employed in the redesign of the library website for National Taiwan Normal University. Following this method, a grouping pattern for contents of the library website was developed by a research participant. This ‘initial grouping pattern’ was then reviewed and modified by eight other participants, one after another. Subsequently, another 20 subjects took part in findability testing for the pattern. Results of the findability tests indicated that the Modified-Delphi Card Sorting method led to improved findability for the website contents (Shieh and Wu, Reference Shieh and Wu2010).
In summary, there have been studies investigating the validity of card sorting and their results indicate that grouping content in a logical manner makes it easy for users to find and access information, and card sorting helps to develop such logical grouping patterns.
5.1.3 Card-sorting reliability
There are different types of card-sorting methods, and the reliability may differ between them. However, since this study employed open card sorting, it is only important to consider the reliability of open card sorting. To this end, Katsanos et al. (Reference Katsanos, Tselios, Avouris, Demetriadis, Stamelos and Angelis2019) conducted a study comparing data from six open card-sorting experiments involving 140 participants. These six experiments were divided into two sets of three. The first set of three was completed by 82 participants, using the same set of cards, and the number of participants for each experiment ranged from 16 to 34. The second set of three was completed by 58 participants, using the same set of cards but different from the cards used in the first three experiments, and with a similar number of participants per experiment as the first set. All six experiments concerned general topics and did not require domain-specific knowledge. Correlation tests using Spearman's correlation and Mantel tests on distance matrices for studies of the same set found significant correlations in all cases. These results indicate that similar grouping patterns were produced by participants with similar profiles performing open card-sorting exercises for the same content. Comparisons of dendrograms produced from the six card-sorting experiments found high similarity in first-level categories (90⋅9%–95⋅3% for the first three experiments and 92⋅9%–96⋅5% for the second set). These results showed that different card sorting sessions of the same content produced highly similar dendrograms in terms of first-level categories. Although there are limitations in terms of scope (only six card-sorting experiments were considered) and the lack of consideration of qualitative data such as category labels, the study by Katsanos et al. (Reference Katsanos, Tselios, Avouris, Demetriadis, Stamelos and Angelis2019) provides support for the cross-study reliability of card sorting.
In summary, there are both academic studies and industrial applications to demonstrate the validity and reliability of the card-sorting method as a suitable tool for developing effective information architecture.
5.1.4 The validation study in phase 3
The validation study in phase 3 was conducted to simulate the way users search for information on navigational displays. There are, however, three issues with the way this phase was conducted.
Firstly, the research subjects were not shown complete screens but rather several fragments of Radar and ECDIS displays mixed together. In a real-life scenario, a subject would interact with a complete screen and would be given cues through the use of colours, the screen layout and other contextual information, all of which could not be replicated in the study. Consequently, the method used in phase 3 lacks ecological validity. However, by showing only fragments instead of complete Radar and ECDIS displays, the study avoided the possibility that research subjects could be biased towards the interfaces they are most familiar with or had used most recently.
Secondly, there is an issue with the number of research subjects. There were 49 pieces of data/control functions in the initial grouping pattern, and each research subject only answered seven questions concerning seven functions. As a result, it required seven research subjects for one complete evaluation of all 49 functions. With 35 subjects participating, it resulted in only five complete evaluations of the whole grouping pattern. In other words, each of the 49 pieces of data in the initial grouping pattern was evaluated by five research subjects.
The number of participants needed for a reliable usability study depends on several factors such as the type and purposes of the study or the characteristics of the user population. In this case, the aim is to identify issues with the initial grouping pattern that resulted from the card-sorting study in phase 2, which fits the description of formative tests (Scholtz, Reference Scholtz2000). With formative tests, Nielsen and Landauer (Reference Nielsen and Landauer1993) reviewed 11 studies and suggested a number of 15 participants divided in three iterations of testing and redesigning. When applied to this study, since it required seven research subjects for one complete test, the total number of required research subjects would be 115, divided in three rounds of 35 subjects for each. However, due to logistical issues associated with the Covid-19 pandemic, it was not possible to achieve that number of research subjects. Consequently, only a third of the desired number of research subjects were recruited. Additionally, while the subjects were randomly selected, they were selected from among the researchers’ professional networks, which posed a potential issue with sampling bias. Considering all of these factors, there is a potential reliability issue with the results of phase 3.
The third issue is with the data analysis method, especially with functions/information that received diverse opinions from research subjects. As discussed in Section 3.3, if there was no selected group agreed by at least 80% of the subjects, the researchers would have to decide the logical group themselves after considering the subjects’ answers, the results of the card-sorting study in phase 2, and general usability principles. All three researchers have backgrounds in seafaring and maritime human factors. This procedure meant that there was a degree of subjectivity involved. The researchers did attempt to minimise personal biases by discussing the results of the analyses together to reach a group consensus. Still, it was not possible to eliminate subjectivity entirely.
5.2 Applicability consideration
The revised grouping pattern that resulted after phase 3 can be used to develop information architecture for Radar and ECDIS systems or their equivalent modules on an INS. However, given the limitations discussed in Section 5.1, there are several factors to consider when applying the results of this study.
Firstly, the study is not comprehensive, in the sense that it does not consider the whole range of functionalities available on a standard Radar and ECDIS or their respective equivalent modules on an INS. The revised grouping pattern only contains 48 functions, arranged in 13 groups. When applied to an actual system, it is necessary to consider the functions not included in this study. It would be possible for future research to expand and include more functions to this grouping pattern.
Secondly, this study does not involve formal usability tests where users’ performance is quantitatively measured. As a result, any attempt to apply the results of this study should include formal usability tests using high-fidelity prototypes and, ideally, with a comparison to an existing design.
Finally, the results of this study do not contradict any existing industry standards. Rather, these results complement and can be applied in conjunction with existing standards.
6. Conclusion
This paper documents the backgrounds, methods, and results of a study consisting of three phases aiming to develop a logical pattern to group contents on the displays of Radar and ECDIS and their equivalent modules of an INS. The aim of such a grouping pattern is to allow seafarers to access essential information with minimal cognitive effort.
Phase 1 of the study involved establishing an overview of how seafarers use each function of the bridge system when engaging in navigation duties. To this end, the researchers gathered data from three sources, namely, cognitive analyses of navigation tasks, performance standards of navigation systems, and a survey on the FOU for each function of a standard INS.
Phase 2 of the study involved selecting, based on the results of phase 1, 49 functions of Radar and ECDIS that are essential for safe navigation and conducting a card-sorting study involving 63 seafarers to develop an initial grouping pattern to arrange these 49 functions into 12 groups.
Phase 3 of the study involved conducting a validation study with 35 seafarers to see whether the initial grouping pattern developed at the end of phase 2 corresponded with the mental model of seafarers when processing information available on navigation displays.
In the end, the revised grouping pattern contained 48 functions organised into 13 groups. One function was removed from the study as it was found to be incompatible with the rest. The resulting 13 groups are presented in Table 3 below.
The revised grouping pattern
The results of this study do not contradict but rather complement existing industry standards and can be applied in developing graphical user interfaces for future systems. However, the study does not involve all available functions of navigation systems and there were no formal usability tests where users’ performance could be quantitatively measured. These two factors must be considered when applying this grouping pattern. Also, it is possible for future research to build upon the results of this study and develop a more comprehensive information architecture for shipboard navigation systems.
Acknowledgements
The researchers would like to show their gratitude to all the seafarers and maritime professionals who took part in the study, with special thanks to the Nautical Institute, Optimal Workshop, and Capt. Sriram Rajagopal of Anglo Eastern.
Ethical standards
This project was conducted following standards set out by the Norwegian National Committee for Research Ethics in the Social Sciences and the Humanities. All personal data were anonymised and managed under project number 60814, managed by the Norwegian Centre for Research Data.
Annex 1
Table A1 below contains contents of the 49 cards used in the card-sorting study in phase 2 of this research.
Contents of the 49 cards used in the card sorting study forming phase 2 of this research
Open access
