2.1 Success likelihood index method (SLIM)

Embrey et al. (Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984a, Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984b) developed SLIM for measuring the probability of human error that emerges during the practice of a particular task. SLIM is an expert-based HRA method. Expert-based HRA methodologies are used when there is a lack of statistical, historical or recorded data related to a particular task. This is the case for many cyber-related tasks, as cybersecurity is a very recent issue with a small set of historical data.

The most commonly used expert-based HRA methodologies are SLIM, the technique of human error rate prediction (THERP), and the human error assessment and reduction technique (HEART) (Kayisoglu et al., Reference Kayisoglu, Gunes and Besikci2022). These are all the first-generation methods developed for HRA. Therefore, they only consider the skill-based and rule-based activities, and their disadvantage is not to consider other factors, such as organisational factors, the impact of context and errors of commission (Norazahar, Reference Norazahar2020). However, their critical advantages shine when applied to operational, maintenance and incident analysis for specific tasks or scenarios. In this case, human behaviour is based on skills, procedures and knowledge. In particular, SLIM depends on knowledge of experts that is a group for arguing specific tasks, and performance-shaping factors (PSFs) affect the success of tasks. The framework of the PSF is created on human factors performance based on expert opinion. The algorithm of SLIM is easy to apply and allows fast human error probability (HEP) for any specific task or scenario (Calixto, Reference Calixto2016). In this context, the most useful method can be considered as SLIM for understanding overall HEP of any particular task sequence, which have not been tacked before and lack data, and their PSF effects. Moreover, at the end of SLIM method, the possibility of errors which can happen in a specific operation can be decreased by developing and arranging PSFs within the system that procured an improvement in all safety levels. Contrary to this, the THERP method is more useful for understanding an event tree model and complex graphic representation for complex tasks. However, it omits the human PSFs, which impacts human error positively or negatively. HEART has nominal human unreliability table, standard generic tasks and error-producing condition table. Accordingly, experts have the option to choose proper generalised generic tasks and related error producing conditions from tables for any specific task instead of developing such items themselves. Moreover, it is a straightforward technique for solving and performing natural human reliability analysis cases thanks to the standard tables. However, the specific task sometimes cannot be fit with generic tasks in the table. It is more suitable for petrochemical and nuclear industries. It must be developed and changed for appropriate use in other fields.

Conversely, in this study, SLIM method is selected to determine the HEP values of the tasks related to ECDIS cybersecurity because of the advantages, such as applicability for specific tasks, usability in case of lack of data due to expert opinion and arrangement of the system safety level with PSF functions.

2.2 Process of the SLIM method

Kayisoglu et al. (Reference Kayisoglu, Gunes and Besikci2022) presented a flow diagram as shown in Figure 1 for the SLIM method in their study, which handles the HEP for bunkering operation in the maritime. Accordingly, the SLIM method is performed according to the process shown in Figure 1.

Figure 1.

Flow diagram for SLIM (Kayisoglu et al., Reference Kayisoglu, Gunes and Besikci2022)

The benefits and advantages of the step-by-step SLIM roadmap are as follows (Kayisoglu et al., Reference Kayisoglu, Gunes and Besikci2022). The problem statement step procures a perspective from the author while determining the PSFs for the tasks in the SLIM method. In addition to defining and explicating the tasks, experts must provide great perspective and systematic approaches when weighting and rating the PSFs for each tasks provided. Moreover, defined PSFs functionally help specify operational achievements and strategic and financial activities to achieve success in any operation or task. Although expert judgement includes subjectivity, experienced and skilled experts do possess enough knowledge for the considered tasks and procure high-level information to overcome inaccessible and restrictive data for the relevant scenario. The weighting of PSFs for a specific scenario provides an initial sight about the criticality of the performance factors for the success of an operation as a whole. Procuring an initial view for the success of an operation is one of the advantages of the SLIM approach over other existing HRA methods. In the step of rating PSFs for each task, each task in an operation is evaluated specifically to define the steps where errors may occur. Hence, the risks related to the operation are systematically decreased by identifying strategic targets and taking required measures specific to each step. After achieving the SLI values and HEP values that are derived from SLI values, inter-judgement, sensitivity and rating analyses through analysis of variance (ANOVA) tests are carried out to check reliability and validity of the considered case study.

3.1 Problem statement

As mentioned in the introduction, ECDIS is an electronic device that includes software, hardware, data and a human–machine interface. While ECDIS has a wide range of strengths for navigational officers in terms of design intent and functions, it also has a range of weaknesses, such as entry points that can be exploited by attackers across IT and OT configuration systems, communication interfaces with other cyber–physical systems onboard ships and human error. It is necessary to focus on the human aspect, especially navigating officers onboard a ship, when considering maritime cyber threats, as they interact with these systems and their vulnerabilities. Digital technologies must properly introduced on the bridge onboard a ship in a safe and efficient way, because if they are managed poorly, this can damage the safety of the ship and main root of maritime knowledge, skills and expertise. In this regard, the steps mentioned in the flow diagram of the SLIM in Figure 1 are followed and enforced to find human error probabilities for ECDIS cybersecurity specifically in this study.

3.2 Data collection

According to Figure 1, experts are identified according to having adequate and effective training, reasonable experience, and high level and related position on the considering tasks, scenarios or operation.

In this study, the considering tasks are related to the ECDIS operations pursued cybersecurity. These tasks are performed by navigation officers and controlled by a master mariner onboard. Five such experts evaluate the tasks and PSFs in this study. All of them are deck officers and masters onboard their ships. According to the International Convention on Standards of Training, Certification and Watch keeping for Seafarers (STCW) (Safety4Sea, 2019), all our experts have to have taken qualified training and have at least one year of sea experience before becoming a deck officer; therefore these experts do have a high level experience in the sea as a deck officer or master. In addition, these experts are in key positions for performing ECDIS navigation practices on the bridge. In terms of ECDIS cybersecurity, as mentioned before, IMO introduced an amendment of MSC.428 resolution, which is ‘Marine Cyber Risk Management in SMS onboard ships,’ in 2017. Pursuant to this change, all ships must consider cybersecurity to be implemented in their SMS by 2021. For this reason, since 2017, shipping companies have to show the necessary care for the crew they have sent to their ships, especially the navigational officers and masters, to create an infrastructure on cybersecurity, because since the beginning of the 2021, all ships are inspected by the port state controls according to cybersecurity strategies stated in their SMSs. This is a regulatory indicator that masters and navigating officers should have cybersecurity awareness onboard ships.

Detailed demographic information about this study's experts is provided in Table 3. The reliability and validity of the obtained data and performed analysis are ensured in the reliability analysis section. According to the reliability analysis, the obtained data is reliable and applicable for the analysis.

Table 3.

Demographic information of experts

For obtaining the weighting and rating of PSFs data that impacts ECDIS task cybersecurity, questionnaires were prepared with two sections. One section obtains expert rating scores of the PSFs in the view of considering each specific tasks about ECDIS cybersecurity. The other one obtains weights of the PSFs in view of the overall operation of ECDIS cybersecurity. The scoring of experts is performed by using a 1–9 point scale in both questionnaires, as SLIM requires (Embrey et al., Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984a, Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984b). The obtained data are firstly extended to 0–100 scale by multiplying 10 as requirements of the original SLIM, then used in equations standardised for SLIM in the following sections.

3.3. Tasks identification

According to the original SLIM, tasks or error modes related to operation are identified by discussion with a panel of experts, as well as by performing other steps of SLIM, such as identifying, weighting and rating PSFs as one panel (Embrey et al., Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984a, Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984b). However, in this study, the tasks related with ECDIS are determined by referencing the guide of operational handbook for ECDIS (Weintrit, Reference Weintrit2009), and OCIMF's guide related to the recommendations on usage of ECDIS and preventing incidents (OCIMF, 2020). Then, these determined ECDIS tasks are integrated with the cybersecurity by referencing ECDIS cyber incidents (Svilicic et al., Reference Svilicic, Brčić, Žuškin and Kalebić2019a, Reference Svilicic, Rudan, Jugović and Zec2019d, Reference Svilicic, Kristić, Žuškin and Brčić2020; Karahalios, Reference Karahalios2020; Kristic et al., Reference Kristic, ŽuŁkin, Brčic and Car2021; Tam et al., Reference Tam, Hopcraft, Moara-Nkwe, Misas, Andrews, Harish, Giménez, Crichton and Jones2022), various recommended international IT and OT cybersecurity standards for maritime such as International Organization for Standardization (ISO), IEC standards and NIST framework (ISO/IEC, 2010; BSI, 2011, 2017, 2018, 2021; Cichonski et al., Reference Cichonski, Millar, Grance and Scarfone2012), and other codes of cybersecurity for ships (ICS and other organizations, 2016; Boyes and Isbell, Reference Boyes and Isbell2017; IACS, 2020). Moreover, the developed final ECDIS tasks integrated with cybersecurity are checked by referencing a safety management manual of a shipping company, which is required by the IMO Maritime Safety Committee (MSC) according to the Resolution MSC.428(98) ‘Maritime Cyber Risk Management in Safety Management Systems’ (IMO, 2017a). The resolution requires that maritime cyber risks are appropriately considered in existing safety management systems (as defined in the ISM Code), as of 1 January 2021. Although there are similar tasks on ECDIS cybersecurity in the safety management manual of the shipping company, as one example, it is understood that there is a need for further development of these manuals, which must be applied on ships with the additional tasks developed within the scope of this study. As a result, in Table 4, the ECDIS cybersecurity tasks are considered in four different specific categories: ‘relationship between ECDIS manufacturers and navigating officers for ENC and ECDIS software update’, ‘responsibilities of navigating officers and company officers for ENC and ECDIS software updates’, ‘navigation responsibilities of deck officers on ECDIS’ and ‘company and vessel officers’ awareness on cybersecurity technical requirements onboard’.

Table 4.

Tasks for ECDIS cybersecurity

3.4 Identification of performance-shaping factors (PSFs)

The term ‘performance-shaping factor’ (PSF) stands for the items that impact the success of considered tasks or operation. In other words, PSFs provide information on preventing human errors/faults for any specific tasks or operation (Embrey et al., Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984a, Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984b). For identifying PSFs for ECDIS cybersecurity, this study used research in the literature, guidelines for ships cybersecurity, international IT and OT standards.

According to Kristic et al. (Reference Kristic, ŽuŁkin, Brčic and Car2021), for deck officers, ensuring the safety of the ship always takes precedence over meeting operational commitments and carrying out the ship's routine. In this context, their primary responsibility is navigation. Navigation onboard involves a range of several processes, which some of them are carried out in a specific order, some almost constantly, some randomly, and some rarely. The lack of established rules regarding the optimum use of navigation systems and techniques adds to the difficulty of this issue. Optimum use of navigation systems depends on the type of ship, the quality of navigational equipment on board, and the experience and skills of the seafarer (Bolat and Kayişoğlu, Reference Bolat and Kayişoğlu2019).

At the same time, deck officers take the necessary measures to oversee all facilities of the procurement. For this purpose, they and other members on the bridge team ensure that they have appropriate training and preparedness for training and that all competence and systems are adequate. They should also ensure that all digital charts and publications are updated with the information provided by the Notice to Mariners (NtMs) and that all essential equipment is correctly outfitted. Since a ship has a dynamic lifecycle at sea and in ports, it is highly possible to make serious mistakes while performing a voyage plan. To plan a route carefully, the crew must use the necessary charts and monitor the ship's position for a comprehensive voyage, all of which are signs of a professional seafarer (ECE/TRANS/SC, 2013).

To ensure the safety of ships, any uncertainty about the location of the ship poses a hazard and should be cleared without delay. The best way to do this is by cross-checking data to avoid ambiguity around ship positions by utilizing all available tools and constantly controlling various sources of location information. The cross-checking method consists in using many different navigational techniques to maintain both operational controls and capabilities that may be required in an emergency situation. Any single navigational system creates a single point of failure; therefore, they must be backed up by another source to ensure the ship's safety (Nielsen, Reference Nielsen2016).

On the other hand, Kristic et al. (Reference Kristic, ŽuŁkin, Brčic and Car2021) has highlighted the overreliance of the seafarer on ECDIS and the equipment it is tied to. Overreliance on ECDIS and other IT and OT assets create cyber-vulnerabilities in terms of human behaviour. The MSC has stated that unsecure tendencies of the seafarers, such as overreliance to technologic information onboard, is preventable not only training, but also with proper navigational culture, rigorous ships procedures, safety management rules, information sharing about related topics and awareness (IMO, 2017b).

All of the state-of-the-art guidelines, codes and regulations (ICS and other organizations, 2016; Boyes and Isbell, Reference Boyes and Isbell2017; PaSea, 2018; National Cybersecurity Centre, 2020) mention that these navigation tasks are performed on ECDIS or other aids to navigation equipment by considering cybersecurity measurements. These measurements are required both technical measurements onboard and having perception of the usage of these technical measurements. Accordingly, a cautious attitude about a potential cyber incident occurring suddenly, with the perception of cyber-attacks against ECDIS and other assets, engenders an attitude of waiting on alarm and good decision-making skills for safe navigation and secure systems. In addition, adequate time availability with these skills for error detection and correction also prevents human errors in the case of any cyber-attacks against ECDIS.

In the context of the references and after the controls by experts, the PSFs are specified as in Table 5.

Table 5.

Performance shaping factors for ECDIS cybersecurity

3.5 Weighting of PSFs

According to SLIM, after determining the tasks related to ECDIS cybersecurity and defining PSFs, experts weight the PSFs by considering which factor is more important to achieve the navigating officers the overall ECDIS cybersecurity tasks without any failure.

After experts give weighted PSFs scores of individually for the success of overall ECDIS cybersecurity tasks, according to the SLIM method, the arithmetic means of the individual weights for each PSFs and their normalised weights are calculated, as shown in Table 6. This step performed takes into consideration the process of aggregating the individual expert opinions to obtain an overall HEP value for each human task (Embrey et al., Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984b). The process requires that the arithmetic means of the individual weights and individual ratings of the PSFs are calculated and then used for gain an overall success likelihood index (SLI).

Table 6.

Normalised weights

3.6 Rating of PSFs

Experts give scores for PSFs by taking each task into consideration separately. Like in Section 3.4, all individual expert ratings are extended to a scale of 0–100. After that, the arithmetic means of the individual experts’ ratings are calculated for achieving the overall SLI, as in Table 7.

Table 7.

Means of PSFs ratings

3.7 Calculation of success likelihood index (SLI)

According to SLIM (Embrey et al., Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984a, Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984b), the SLI values are obtained according to Equation (1) for each task:

(1)\begin{equation}\textrm{SLI} = \left\{ {\mathop \sum \limits_{i = 1}^n \textrm{Weigh}{\textrm{t}_i} \times \textrm{Ratin}{\textrm{g}_i}\textrm{|}i = \# \textrm{PSF}} \right\}\end{equation}

The resulting SLI values for each task are presented in Table 8. The multiplications of weight values with rating values (Weight  × Rating) are demonstrated in the related column (see ‘Product’ column) in Table 8 for each task. In the last column, all product values for each task are summed up and SLI values for each task are achieved.

Table 8.

SLI values of tasks

3.8 Transformation SLI to HEP

According to SLIM (Embrey et al., Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984a, Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984b), the HEP value is obtained by using SLI values and anchor values (a and b) corresponding to each task in accordance with the Equation (2):

(2)\begin{equation}\log \,\textrm{of}\ \textrm{Probability}\,\textrm{of}\,\textrm{Success} = aSLI + b\end{equation}

The value obtained from Equation (2) represents the success probability of a considered task, and it is this value that allows us to determine the human error probability of each task. The value of success probability should be subtracted from 1 after the anti-log of it is taken. The constant values of ‘a’ and ‘b’ in Equation (2) are the special items to SLIM and they are calculated by using SLIs of any two tasks related to considered operation and their HEP values, which have already been known or established before (Embrey et al., Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984a). For ECDIS cybersecurity, the two tasks, the HEP values of which are already known, are not encountered in the literature or from any risk assessment that have made by ships or in the shipping company. Hence, in this study, the absolute probability judgement method is used to find ‘a’ and ‘b’ endpoints for each of the four different task categories developed for ECDIS cybersecurity. The method of absolute probability judgement provides data for most rare-event operations or scenarios when the data frequency may not exist by the way of absolute probability judgments of experts on the best and worst cases for the evaluated operation (Kayisoglu et al., Reference Kayisoglu, Gunes and Besikci2022).

In this study, experts focus on the four different categories of ECDIS cybersecurity and estimate the human error probability for the best and worst cases in these categories separately by taking into consideration the situation where PSFs are as bad as possible. These are the situations which navigating officers must have in order to successfully complete ECDIS cybersecurity missions, and vice versa. After obtaining HEP values of the two reference tasks for each task category thanks to the absolute probability judgement method, the values of ‘a’ and ‘b'are found by using ‘0’ and ‘100’ values as SLI values in Equation (2) for the worst- and best-case scenarios, respectively. The obtained reference HEP values are supposed to be reasonable and reliable for the ECDIS cybersecurity tasks when trusting to experts’ experience and knowledge as in all research sourced by expert opinion (Embrey et al., Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984a, Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984b; He et al., Reference He, Wang, Shen and Huang2008; Hakam and Ratriwardhani, Reference Hakam and Ratriwardhani2013) and examining the cyber incidents or attacks against ECDIS in literature. Estimated HEP values for best and worst cases regarding ECDIS cybersecurity and constant values derived from them are detailed in Table 9. In Table 10, unknown HEP values for evaluated tasks are obtained by using SLI values in Table 8 and ‘a’ and ‘b’ constant values in Table 9 in the Equation (2).

Table 9.

Estimated HEP for the best and worst case of ECDIS cyber-security task categories

Table 10.

Human error probability for each tasks

3.9 Reliability analyses

According to Embrey et al.'s sustainable method requirements, the reliability analyses of SLIM specifically include analysis of rating data, sensitivity analysis and inter-judge consistency analysis. These reliability analyses are carried out through the tests of two-way ANOVA by considering intended purposes (Embrey et al., Reference Embrey, Humphreys, Rosa, Kirwan and Rea1984a).

The aim of the ANOVA tests is to compare the means of two groups or more. It is a collection of statistical models used to analyse differences between means and their correlated variation between and within groups. ANOVA is based on the law of total variance, in which the observed variance in a given variable is divided into components attributable to different sources of variation (Emerson, Reference Emerson2017). According to the ANOVA test, when the significance values (p) in ANOVA test results are less than 0 · 05, it means that considered dependent variable significantly differ between considered independent variables or groups (Gamage and Weerahandi, Reference Gamage and Weerahandi1998).

In this study, for analysis of rating data, the intended purpose is that the PSF ratings significantly become different between PSF categories and evaluated tasks. With the sensitivity analysis, it is aimed that weight values significantly become different between PSF categories, but they do not significantly become different between tasks. For inter-judge consistency analysis, it is given that individual log HEP values significantly become different between evaluated tasks, but they do not significantly become different between experts. If these purposes are confirmed by the analysis results, it is ensured that the obtained data for this study is valid and SLIM analysis results are reliable.

In this context, the results of ANOVA tests for each reliability analysis are presented in Tables 11–13. According to the inter-judge consistency analysis in Table 11, the results showed that individual log HEP values significantly become different between evaluated tasks (p < 0 · 05) but they do not significantly become different between experts (p > 0 · 05). The results of inter-judge consistency analysis are consistent with intended purpose in this study.

Table 11.

ANOVA results for inter-judge consistency

^a R Squared = 0 · 777 (Adjusted R Squared = 0 · 774).

The interclass correlation coefficient, which states the mean correlation between the experts’ opinions, is obtained via Equation (3). ‘F’ value in Equation (3) represents ‘F’ value of Expert in Table 9 and ‘n’ shows number of experts. The result of Equation (3) (0 · 998 – the closer it is to ‘1’, the higher-level agreement between experts it is) evidences very high-level agreement between experts:

(3)\begin{equation}r = \frac{{F - 1}}{{F + ({n + 1} )}} = \frac{{579 - 1}}{{579 + ({5 + 1} )}} = 0\;.\;988\end{equation}

According to the sensitivity analysis in Table 12, weight values significantly become different between PSF categories (p < 0 · 05) but they do not significantly become different between tasks (p > 0 · 05). These results are in line with the desired expectations of this study. In addition, recommendations are designed in the findings and discussions by interrupting the weights of PSF categories to identify which PSFs have the greatest effect on the probability of success or failure about ECDIS cybersecurity. This is a significant advantage of SLIM over other approaches.

Table 12.

ANOVA results for sensitivity analysis

^a R Squared = .010 (Adjusted R Squared = .006)

According to the rating analysis in Table 13, PSF ratings significantly become different between PSF categories and evaluated tasks (p < 0 · 05). The result of rating analyses are consistent with the intended purpose in this study. Accordingly, the mean of the PSF ratings can be approved as a measure of the overall quality of the ECDIS cybersecurity with regard to the tasks under consideration.

Table 13.

ANOVA results for rating analysis

^a R Squared = .046 (Adjusted R Squared = .042)

In accordance with the results of the reliability analysis, the obtained data from experts for this study is valid and the results of SLIM analysis are reliable and commendable.

3.10 Findings and discussions

The results of SLIM analyses in Table 10, Figures 2 and 3 allow us better to understand the human error probabilities for ECDIS cybersecurity and the impact of PSFs on ECDIS cybersecurity, respectively.

Figure 2.

HEP graph for ECDIS cybersecurity

Figure 3.

The impact assessment of PSFs on ECDIS cybersecurity

Among the task categories considered for ECDIS cybersecurity the fourth category, which includes tasks related to company and vessel officers awareness on cybersecurity technical requirements onboard, has the highest possibility for failure. In this category, navigating officers mostly fall into error related to performing removal of data and software licenses or any inconsistent equipment related to ECDIS, including storage media. At the same time, all tasks in the fourth category involve very close failure probabilities. Regarding tasks in the fourth category, navigating officers should consciously restrict access to software on the ECDIS computer, including the operating system, by password protection for ensuring cybersecurity. Additionally, they should only be provided with access to the network and network services that they have been specifically authorised to use for ECDIS cybersecurity. They should not remove any ECDIS equipment (including information and software) from the bridge or the ship without prior approval. Finally, crew should be aware that remote and wireless access to ECDIS should be controlled by using an encryption protocols. These tasks include critical level of failure possibilities according to the analysis results. To prevent these failures, navigating officers should have and understand policies, regulations, standards, checklists and code of practices related to ECDIS cybersecurity, according to the analysis results as detailed in Table 7. Additionally, these technical security processes and procedures measures (e.g. software/hardware, internet security systems, alarms of warning messages) related to ECDIS cybersecurity should have a place in the Safety Management System (SMS) and Safety Management Manuals, including detailed professional prepared cybersecurity clauses, especially ECDIS cybersecurity. Essentially, navigating officers should have constant awareness of ECDIS cybersecurity and then adequate ECDIS cybersecurity education and training in order to search and understand these process and procedures in the SMS and other sources and perform them onboard a ship in the scope of SMS.

In accordance with the analysis results, the second highest failure probabilities for ECDIS cybersecurity relates to the responsibilities of navigating officers and company officers to update ENC and ECDIS software . The most likely failure in this context is either ENC updates are not sent by the manufacturer on a secure program or this secure program does not address in the whitelisting application of the ship or company. Whitelisting has capability of defining the secure items, applications, network and access points, such as whitelisted access control for portable storage devices including limiting documents to write, read and operate on removable medias; only on given permission for the use of encrypted devices and the use of drives with particular serial numbers. For this reason, whitelisting is one important technical measures that should be established and adopted by companies to ensure all ship cybersecurity, including ECDIS. Then, unclosed ECDIS ports for external portable drives such as USB are also a highly possible failure onboard a ship. In addition to this, navigating officers generally do not use unique and pre-scanned USB drives for receiving the ENC updates from the program and uploading them to the ECDIS, and the USBs used by navigating officers are not to be recorded in the whitelisting application of the ship or company. Such mitigations are useful for tasks in the fourth category, for avoiding these failures. Firstly, the requirements related to closing of ECDIS ports should take place in SMS onboard a ship, and ENC and ECDIS software updates protocols and communications should be secured, which includes strategies such as whitelisting. If ECDIS ports are not closed, the requirement of usage of unique and pre-scanned USB drives recorded in the whitelisting application should be involved in SMS, as well. Navigating officers should have the adequate education to understand related technical measures and awareness of these requirements to provide ECDIS cybersecurity.

When evaluating the tasks in the third category, all tasks have a very similar probability of failure. This category consists of navigation responsibilities for navigating officers on ECDIS along with on the cybersecurity. After navigating officers make and save the passage plan in ECDIS, they do not generally ensure the availability, integrity and confidentiality of it by making sure no deletions, changes or lack of information can be made to the passage plan. Additionally, to understand the integrity of the passage plan, officers should maintain the correct level of zoom on ECDIS charts to ensure safety critical information is displayed on ECDIS. However, according to the results, this has one of the highest possible failure in this category. Moreover, before taking over a navigational watch, the incoming navigating officers generally error by not confirming the ECDIS configuration against the passage plan requirements, such as safety settings, chart display and alarm system management. The outbound officer does not highlight any changes made to the ECIDS configuration, which is except for the passage plan parameters, if there is one. Before that, they should use cross-checking methods with such as radar or visual fix to confirm the accuracy of information displayed on the ECDIS. However, although this task is a low-probability error in this category, it is still an important task that may not be performed and may affect other tasks. Finally, following a cyber-attack and especially when approaching any waypoint or important area (e.g. canal, TSS, narrow channel, pilot station, etc), officers can err by not rechecking the passage plan. According to these results, the most essential factors affecting these failures are effective usage of ECDIS in compliance with good navigation practice along with the perception knowledge of ECDIS cybersecurity. In this scope, navigating officers should have good navigation culture and behaviour integrated with cybersecurity and keep themselves on the alarm that cyber-attack can occur suddenly and at any moment.

Finally, among the task categories considered for ECDIS cybersecurity, the first category, which includes tasks related to relationship between ECDIS manufacturers and navigating officers for ENC and ECDIS software update, has the lowest possibility for failure. In this category, the highest possible failure is that if manufacturers detect any inconsistency in ECDIS performance, they do not generally issue technical bulletins with mitigating measures to all ship owners/operators who manage ships equipped with their systems to highlight issues and to correct inconsistencies. When this happens, the masters and navigating officers cannot be aware of the issues and cannot develop any protective measures for future plans, and most importantly they cannot determine whether the technical issue is derived from a cyber-attack or not. This is followed by that a navigating officer not able to detect the future errors of ECDIS manufacturers’ safety bulletins or software upgrades or future inconsistencies in ECDIS-related data or functionality the next time. To prevent these failures, it is recommended that cyber information-sharing policies between persons, company, ship and other stakeholders in the maritime sector, such as ECDIS manufacturers, should be developed and adopted by related parties and should be addressed in the SMS of shipping companies.

All in all, evaluating overall ECDIS cybersecurity, the most important factors to ensure effective cybersecurity for it are (i) effective usage of ECDIS in compliance with good navigation practice along with cybersecurity perception; (ii) adequate ECDIS cybersecurity training; (iii) developing technical security measures, and addressed them in safety management system of shipping companies; and (iv) developing cyber information sharing between related stakeholders in the maritime sector.