2.1 Similarity metrics and preprocessing strategies

SANOM takes advantages of the well-known SA to discover the shared concepts between two ontologies in question (Mohammadi et al., to appear). A potential alignment is modeled as a state in the SA whose evolution would result in a more reliable matching between ontologies. The evolution requires a fitness function in order to gauge the goodness of the intermediate solutions to the ontology matching problem. We first define the fitness of an alignment.

Definition 1 (Fitness). The fitness of an alignment A, shown as F(A), between ontologies O and Oʹ is the aggregate fitness of its correspondences and is defined as

\begin{align*}F(A) = \sum_{c \in A}\, f(c),\end{align*}

where $f: A \rightarrow R$ is the fitness of correspondence $c\in A$ .

To compute the fitness of an alignment, we need to compute the fitness of its correspondences. There are two ways of computing such fitness. The first approach is to consider the names (e.g., URI, label, comments) of the entities in question and calculate a similarity measure. SANOM distinguishes the similarity between classes and properties and consequently uses different methods.

Another class of similarity computation is structural, where the positions of entities in their ontologies are used for similarity computation. Overall, the fitness of a correspondence is computed as the sum of sting and structural similarity metrics, that is, $f(c) = f_{\rm lexical}(c) + f_{\rm strucutural}(c)$ , where $f_{\rm lexical}(c)$ and $f_{\rm strucutural}(c)$ are the string and structural metrics, respectively.

Prior to introducting the similarity metrics, a preprocessing step is employed that has been shown to improve the quality of alignment (Cheatham & Hitzler, Reference Cheatham and Hitzler2013).

2.1.3 Structural similarity metric

The foregoing similarity metrics yields high score to the concepts with a high lexical likeness. Another similarity of two concepts can be obtained based on their positions in the given ontologies.

We consider two structural similarity metrics for the current version of SANOM:

• The first metric is obtained by the subsumption relation of classes. Let $O_1$ and $O_2$ are two ontologies, and $c_1 \in O1$ and $c_2 \in O2$ are two classes whose superclasses are $s_1 \in O1$ and $s_2 \in O2$ , then the aligning of classes $s_1$ and $s_2$ would increase the chance of matching $c_1$ and $c_2$ . If s is a correspondence mapping $s_1$ to $s_2$ , then the increased similarity of $c_1$ and $c_2$ is gauged by

(1) \begin{align}f_{\rm structural}(c_1,c_2) = f(s).\end{align}

• Another structural similarity is calculated from the object and data properties. The alignment of two properties would indicate the fact that their corresponding domain and/or ranges are also identical. By the same token, if two properties have the similar domain and/or range, then the chance that they match as well increases.

A recent study (Cheatham & Hitzler, Reference Cheatham and Hitzler2014) has investigated that the mapping of properties based solely on their names would lead to both high false-positive and false-negative rates. Following the recommendation of Cheatham and Hitzler (Reference Cheatham and Hitzler2014), we further use the core concept of the properties as one of their names.

The current version of SANOM treats the object and data properties differently. For the object properties $op_1$ and $op_2$ , their related domains and ranges are obtained as the appending their set of ranges and domains, respectively. Then, Soft TF-IDF computes the similarity of their names, domains, and ranges. The final similarity of two properties is then taken as the average of top two fitness scores obtained by Soft TF-IDF. The fitness of data properties is also calculated as the similarity average of names and their corresponding domains.

On the other hand, mapping properties would increase the chance of mapping their related classes. Let $e_1$ and $e_2$ be classes, $op_1$ and $op_2$ be the object properties, and $R_1$ and $R_2$ be the ranges, then the fitness of the correspondence $c = (e_1,e_2)$ is obtained as

(2) \begin{align}f_{\rm structural}(c) = \frac{f_{\rm string}(R_1,R_2) + f_{\rm string}(op_1,op_2)}{2}.\end{align}

2.2 Warm initialization

Since the SA operates on one single state only, the speed of its convergence could be quite low. A good initial state would pave the way of fast convergence. To this end, a randomized greedy algorithm is used to find a near-optimal solution as the initial state. A random number r in the interval [1, n] is selected, and the corresponding element in that position is chosen. Afterward, the entity $e_r$ is matched with the entity $e_{j_r}$ , where $e_{j_r}$ is the most similar entity to $e_r$ . The procedure is iterated until the last concept of the first ontology finds a corresponding mapping in the second. This method is evidently greedy and not optimal, but it can provide a good starting point for the SA.

Algorithm 1 is the complete procedure of this greedy technique, which also takes into account the one-to-one mapping constraints.

Algorithm 1

Randomized greedy technique for initialization (Mohammadi et al., to appear)

2.3 SA adaptation

To find the optimal solution, the SA needs to move to a new state with higher fitness values. One needs to first devise a methodology to create a new state, called successor, and then transition to it based on the underlying idea of the SA.

In the following, the successor creation and transition to a new state are discussed.

2.3.1 Successor generation

A successor is created by swapping the elements of the current state. In the current version of SANOM, we swap q elements of the current state, where $q = \lceil \%5 * |S|\rceil$ and $|S|$ is the length of the current state. The alteration of elements is done by first finding q distinct number between 1 and n, called k, and then exchanging the elements s(k(i)) and $s(k(i+1))$ where i is an index. Algorithm 2 summarizes the procedure of creating a successor based on the current one.

Algorithm 2

Generating successors of the current state (Mohammadi et al., to appear)

2.3.2 Transition

The transition to a new state is probabilistic in the SA. If the fitness value of the successor is bigger than that of the current state, then the move to the successor certainly happens. Otherwise, the move is reliant on the fitness value difference between two states and the temperature. Let S and Sʹ be the current state and successor while the temperature is T, and f(S) and f(Sʹ) be their corresponding fitness value. Thus, the probability to transition to Sʹ is

(3) \begin{align}P_{\rm move} = \rm min\left(e^{\frac{\Delta E}{T}},1\right),\end{align}

where $\Delta E = f(S') - f(S)$ . If $\Delta E > 0$ , then the move to the next state certainly happens since $P_{\rm move} = 1$ . If $\Delta E < 0$ , then the transition happens with the probability $P_{\rm move} < 1$ , where $P_{\rm move}$ is proportionate to $\Delta E$ and $\frac{1}{T}$ .

2.4 SANOM

SANOM starts with calculating the similarity of each concept of the first ontology to those in the second. Then, an initial alignment is obtained by the greedy technique in Algorithm 1. The initial alignment is then enhanced by the SA by generating a new state, computing its fitness, and then moving to it. Such an enhancement is repeated for some number of iterations.

The number of iterations is a parameter determined by the user. The temperature used in each iteration of SANOM can be computed merely based on the number of iterations. Given the number of iterations $k_{\rm max}$ and the initial temperature t (by default $t=1$ ), the temperature at the iteration k is obtained as

\begin{align*}t_k = \left(1 - \frac{k}{k_{\rm max}}\right)t.\end{align*}

The overall ontology matching algorithm is summarized in Algorithm 3.

Algorithm 3

SANOM (Mohammadi et al., to appear)

4.1 Anatomy track

The anatomy track is one of the earliest benchmarks in the OAEI. The task involves aligning the Adult Mouse anatomy and a part of National Cancer Institute thesaurus containing the anatomy of humans. Each of the ontologies has approximately 3000 classes that are designed carefully and are annotated in technical terms.

MapPSO was applied to the anatomy track, but its generated alignment had precision and recall of less than 0.05. Thus, we do not compare SANOM with MapPSO in this track. The best-performing systems in this track use biomedical background knowledge, where the top system is AML (Faria et al., Reference Faria, Balasubramani, Shivaprabhu, Mott, Pesquita, Couto and Cruz2017). Among other systems, LogMap (Jiménez-Ruiz & Grau, Reference Jiménez-Ruiz and Grau2011) is the best one with no use of background knowledge.

Table 1 tabulates precision, recall, and F-measure of the participating systems in the anatomy track. Since LogMap does not use any biomedical background knowledge, the performance of SANOM could be impartially compared with it. According to this table, the recall of SANOM is slightly higher than LogMap which means that it could identify more correspondences than that of LogMap. However, precision of LogMap is better than that of SANOM with the margin of 3%. The overall performance of SANOM is quite close to LogMap since their F-measure has only 1% difference.

Table 1

The precision, recall, and F-measure of the participating systems in the OAEI anatomy track

In addition, the systems are compared based on McNemar’s test, and the outcome of the test is visualized using a directed graph (Mohammadi et al., Reference Mohammadi, Atashin, Hofman and Tan2018a). In this regard, every two systems could be paired, and the McNemar’s test can be applied to verify if two systems are significantly different. There are two ways of using McNemar’s test for comparison. The first approach is to ignore the false positives generated by each of the methods. Figure 1 displays the directed graph from comparing the participating systems with ignoring false positives. The nodes in this graph are the participating systems, and each edge $A\rightarrow B$ indicates that the system in origin is significantly better than that at the other end. Based on this graph, AML is the best-performing system and is followed by LogMapBio and XMap. SANOM has also outperformed LogMap in this case, which could also be justified based on recall of the two systems.

The second way of using the McNeamr’s test is to consider the false positives as well. Figure 2 displays the resulting directed graph from the comparison of each two systems with considering false positive. In this case, AML has the best performance again, but XMap outperforms LogMapBio in contrast to the case that false positives are ignored. LogMap is also better than SANOM in this respect.

Figure 3 plots the fitness function value, precision, and recall curve for different iterations of SANOM for the anatomy track. According this figure, the fitness value is in aggregate increasing, although there are some drops as well that get back to the nature of SA in moving to states with lower fitness values. Precision is higher at the beginning and it decreases during the SANOM iterations. This is because the initial alignment is conservative and precision is high as expected, as well as the fact that EAs evolve randomly that potentially increase the false positives. In terms of recall, in contrast, the quality of alignment has been significantly improved. This corroborates the effectiveness of SA.

Figure 1

The comparison of participating systems in the OAEI 2018 anatomy track based on the McNemar’s test with considering false positives. The nodes in the graph are the participating systems, and each directed edge $A \rightarrow B$ means that A is superior to B

Figure 2

The comparison of participating systems in the OAEI 2018 anatomy track based on the McNemar’s test while the false positives are ignored. The nodes in the graph are the participating systems, and each directed edge $A \rightarrow B$ means that A is superior to B

Figure 3

Precision, recall, and fitness function value computed by generated alignments in different iterations for the anatomy track. In order to be able to display with the fitness function, precision and recall are multiplied by 10

4.2 Conference track

The conference track involves the pairwise alignment of seven ontologies. Table 2 shows precision, recall, and F-measure of SANOM, LogMap, and AML (Faria et al., Reference Faria, Balasubramani, Shivaprabhu, Mott, Pesquita, Couto and Cruz2017), and MapPSO (Bock & Hettenhausen, Reference Bock and Hettenhausen2012) on the conference track. AML and LogMap have been the top two systems in terms of precision and recall for several years.

According to Table 2, recall of SANOM is superior to both LogMap and AML. SANOM’s average recall is 7% and 14% more than those of AML and LogMap, respectively, but its precision is 10% less than both of the systems. In addition, SANOM significantly outperforms MapPSO in terms of precision, recall, and F-measure. To show that SANOM is much faster than MapPSO, we compare these system based on execution time on the conference track. Table 3 displays the execution time of SANOM and MapPSO for 21 tasks in the conference track. In sum, MapPSO completed all 21 tasks in 747 seconds, while SANOM completed them in 58 seconds. Thus, SANOM is not only superior to MapPSO in terms of precision, recall, and F-measure, but it also identifies the alignment much faster compared to MapPSO.

We also conducted the Wilcoxon signed-rank test recommended in Mohammadi et al., (Reference Mohammadi, Hofman and Tan2018b) to verify if the difference between systems is significant. Based on this analysis, SANOM is significantly different from AML and LogMap in terms of recall and is statistically better than LogMap regarding F-measure. The F-measure of SANOM is not statistically different form AML. In addition, SANOM is significantly superior to MapPSO with respect to precision, recall, and F-measure. Overall, the performance of SANOM is quite competitive with the top-performing systems in the conference track.

4.3 The disease and phenotype track

We also applied SANOM to the OAEI disease and phenotype track (Harrow et al., Reference Harrow, Jiménez-Ruiz, Splendiani, Romacker, Woollard, Markel, Alam-Faruque, Koch, Malone and Waaler2017) that consists of matching various disease and phenotype ontologies. For this experiment, we consider the alignment of the human phenotype (HP) to the mammalian phenotype (MP), and the human disease ontology (DOID) and the orphanet and rare diseases ontology (ORDO). Since these ontologies have approximately 15 000 entities, the alignment is challenging. MapPSO could not find the alignment of ontologies in this track within more than 24 hours. Thus, we omit it for comparison in this experiment. For the reference alignment, we use a voted alignment that was created based on the outputs of the alignment systems in last few years.

Fewer systems could generated an alignment in this track. The participating systems in this track use biomedical background knowledge: LogMap uses normalizations and spelling variants the SPECIALIST Lexicon, XMAP uses a dictionary of synonyms extracted from the Unified Medical Language System Metathesaurus (Bodenreider, Reference Bodenreider2004), and AML has three background resources, one of which is selected automatically (Faria et al., Reference Faria, Pesquita, Santos, Cruz and Couto2014). However, SANOM does not use any background knowledge specifically for biomedical ontologies.

Table 2

The precision, recall, and F-measure of SANOM, AML, and LogMap on various datasets on the conference track. The highest score of each performance metric for each task is in boldface

Table 3

Execution time of SANOM and MapPSO on 21 tasks in the conference track

Table 4 shows the result of alignment systems for matching DOID to ORDO. Based on this table, SANOM outperforms AML, LogMap, and XMap in terms of precision and is competitive with LogMapLite. Concerning recall, in contrast, AML and LogMap have better alignments, and SANOM outperforms XMap and is comparable to LogMapLite. In terms of F-measure, LogMap is superior in this track, followed by LogMapLite and SANOM. Overall, SANOM outperforms XMap and AML, even though it does not use any biomedical background knowledge.

Table 5 tabulates the performance of systems for HP and MP matching. With regard to precision, SANOM outperforms all systems in this task. LogMap and AML are the top two systems in terms of recall, and SANOM outperforms XMap and is comparable to LogMapLite in this regard. In terms of F-measure, LogMap and AML are the best systems, followed by LogMapLite and SANOM.