Hostname: page-component-5db58dd55d-ggg9q Total loading time: 0 Render date: 2026-05-25T11:53:03.478Z Has data issue: false hasContentIssue false
  • English
  • Français

A consensual dataset for complex ontology matching evaluation

Part of: Ontology Alignment: Algorithms and Evaluation

Published online by Cambridge University Press:  07 July 2020

Elodie Thiéblin ,
Michelle Cheatham ,
Cassia Trojahn Open the ORCID record for Cassia Trojahn [Opens in a new window]  and
Ondrej Zamazal
Elodie Thiéblin
Affiliation:
IRIT & Université de Toulouse 2 Jean Jaurès, Toulouse, France, e-mail: elodie.thieblin@irit.fr, cassia.trojahn@irit.f
Michelle Cheatham
Affiliation:
Wright State University, Dayton, USA, e-mail: michelle.cheatham@wright.edu
Cassia Trojahn
Affiliation:
IRIT & Université de Toulouse 2 Jean Jaurès, Toulouse, France, e-mail: elodie.thieblin@irit.fr, cassia.trojahn@irit.f
Ondrej Zamazal
Affiliation:
University of Economics, Prague, Czech Republic, e-mail: ondrej.zamazal@vse.cz
Get access Rights & Permissions [Opens in a new window]

Abstract

Simple ontology alignments, largely studied in the literature, link one single entity of a source ontology to one single entity of a target ontology. One of the limitations of these alignments is, however, their lack of expressiveness, which can be overcome by complex alignments, which are composed of correspondences involving logical constructors or transformation functions. While most work on complex ontology matching has been dedicated to the development of complex matching approaches, there is still a lack of benchmarks on which the complex approaches can be systematically evaluated. The aim of this paper is to present the process of constructing the consensual complex Conference dataset, describing the design choices and the methodology followed for constructing it. We discuss the issues the experts were faced with during the process and discuss the lessons learned and perspectives in the field.

Information

Type
Research Article
Information
The Knowledge Engineering Review , Volume 35 , 2020 , e34
Copyright
© Cambridge University Press, 2020

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Abacha, A. B. & Zweigenbaum, P. 2014. Means: une approche sémantique pour la recherche de réponses aux questions médicales. TAL 55(1), 71–104.Google Scholar
Algergawy, A., Cheatham, M., Faria, D., Ferrara, A., Fundulaki, I., Harrow, I., Hertling, S., Jiménez-Ruiz, E., Karam, N., Khiat, A., Lambrix, P., Liand, H., Montanelli, S., Paulheim, H., Pesquita, C., Saveta, T., Schmidt, D., Shvaiko, P., Splendiani, A., Thiéblin, E., Trojahn, C., Vataščinová, J., Zamazal, O. & Zhou, L. 2018. Results of the ontology alignment evaluation initiative 2018. In OM-2018: Proceedings of the Twelfth International Workshop on Ontology Matching.Google Scholar
Cheatham, M. & Hitzler, P. 2014. Conference v2. 0: an uncertain version of the OAEI conference benchmark. In International Semantic Web Conference, 33–48. Springer.CrossRefGoogle Scholar
Dekhtyar, A. & Hayes, J. H. 2006. Good benchmarks are hard to find: toward the benchmark for information retrieval applications in software engineering. In Proceedings of the 22th International Conference on Software Maintenance.Google Scholar
Dhamankar, R., Lee, Y., Doan, A., Halevy, A. & Domingos, P. 2004. iMAP: discovering complex semantic matches between database schemas. In Proceedings of the 2004 ACM SIGMOD International Conference on Management of Data, 383–394. ACM.CrossRefGoogle Scholar
Do, H.-H., Melnik, S. & Rahm, E. 2002. Comparison of schema matching evaluations. In Net. ObjectDays: International Conference on Object-Oriented and Internet-Based Technologies, Concepts, and Applications for a Networked World, 221–237. Springer.CrossRefGoogle Scholar
Doan, A. (2005). Dataset of complex correspondences. Illinois Semantic Integration Archive. Computer Science Department, University of Illinois. http://pages.cs.wisc.edu/~anhai/wisc-si-archive/.Google Scholar
Euzenat, J., Ferrara, A., Hollink, L., Isaac, A., Joslyn, C., Malaisé, V., Meilicke, C., Nikolov, A., Pane, J., Sabou, M., Scharffe, F., Shvaiko, P., Spiliopoulos, V., Stuckenschmidt, H., Sváb-Zamazal, O., Svátek, V., dos Santos, C. T., Vouros, G. A. & Wang, S. 2009. Results of the ontology alignment evaluation initiative 2009. In Proceedings of the 4th International Workshop on Ontology Matching (OM-2009) collocated with the 8th International Semantic Web Conference (ISWC-2009) Chantilly, USA, October 25, 2009.Google Scholar
Euzenat, J. & Shvaiko, P. 2013. Ontology Matching. Springer.CrossRefGoogle Scholar
Faria, D., Pesquita, C., Balasubramani, B. S., Tervo, T., Carrio, D., Garrilha, R., Couto, F. M. & Cruz, I. F. 2018. Results of AML participation in OAEI 2018. In OM-2018: Proceedings of the Twelfth International Workshop on Ontology Matching.Google Scholar
Halpin, H., Hayes, P. J., McCusker, J. P., McGuinness, D. L. & Thompson, H. S. 2010. When owl:sameAs isn’t the same: an analysis of identity in linked data. In The Semantic Web – ISWC 2010, Patel-Schneider, P. F., Pan, Y., Hitzler, P., Mika, P., Zhang, L., Pan, J. Z., Horrocks, I. & Glimm, B. (eds), 305–320. Springer.CrossRefGoogle Scholar
He, B., Chang, K. C.-C. & Han, J. 2004. Discovering complex matchings across web query interfaces: a correlation mining approach. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 148–157. ACM Press.CrossRefGoogle Scholar
Hollink, L., Van Assem, M., Wang, S., Isaac, A. & Schreiber, G. 2008. Two variations on ontology alignment evaluation: methodological issues. In 5th European Semantic Web Conference, 388–401.Google Scholar
Isaac, A., Matthezing, H., van der Meij, L., Schlobach, S., Wang, S. & Zinn, C. 2008. Putting ontology alignment in context: usage scenarios, deployment and evaluation in a library case. In 5th European Semantic Web Conference, 402–417.Google Scholar
Jiang, S., Lowd, D., Kafle, S. & Dou, D. 2016. Ontology matching with knowledge rules. In Transactions on Large-Scale Data-and Knowledge-Centered Systems XXVIII, Hameurlain, A., Küng, J., Wagner, R. & Chen, Q. (eds). Springer, 75–95.Google Scholar
Maedche, A., Motik, B., Silva, N. & Volz, R. 2002. Mafra—a mapping framework for distributed ontologies. In International Conference on Knowledge Engineering and Knowledge Management, 235–250. Springer.CrossRefGoogle Scholar
Makris, K., Bikakis, N., Gioldasis, N. & Christodoulakis, S. 2012. SPARQL-RW: transparent query access over mapped RDF data sources. In 15th International Conference on Extending Database Technology, 610–613. ACM.CrossRefGoogle Scholar
Maltese, V., Giunchiglia, F. & Autayeu, A. 2010. Save up to 99% of your time in mapping validation. In On the Move to Meaningful Internet Systems, OTM 2010 - Confederated International Conferences: CoopIS, IS, DOA and ODBASE, Hersonissos, Crete, Greece, October 25–29, 1044–1060.Google Scholar
Mathur, S. N., O’Sullivan, D. & Brennan, R. 2018. Milan: automatic generation of R2RML mappings. In Proceedings of the 26th Irish Conference on Artificial Intelligence and Cognitive Science, Dublin, Ireland, 7th December, 1–12.Google Scholar
Meilicke, C. 2011. Alignment Incoherence in Ontology Matching. PhD thesis, Universität Mannheim.Google Scholar
Meilicke, C. & Stuckenschmidt, H. 2008. Incoherence as a basis for measuring the quality of ontology mappings. In 3rd International Conference on Ontology Matching, 431, 1–12.Google Scholar
Meilicke, C., Stuckenschmidt, H. & Šváb-Zamazal, O. 2009. A reasoning-based support tool for ontology mapping evaluation. In European Semantic Web Conference, 878–882. Springer.CrossRefGoogle Scholar
Nunes, B. P., Mera, A., Casanova, M. A., Breitman, K. K. & Leme, L. A. P. 2011. Complex matching of RDF datatype properties. In 6th ISWC Workshop on Ontology Matching.Google Scholar
Oliveira, D. & Pesquita, C. 2018. Improving the interoperability of biomedical ontologies with compound alignments. Journal of Biomedical Semantics 9(1), 1:1–13:13.CrossRefGoogle ScholarPubMed
Parundekar, R., Knoblock, C. A. & Ambite, J. L. 2010. Linking and building ontologies of linked data. In ISWC, 598–614. Springer.CrossRefGoogle Scholar
Parundekar, R., Knoblock, C. A. & Ambite, J. L. 2012. Discovering concept coverings in ontologies of linked data sources. In ISWC, 427–443. Springer.CrossRefGoogle Scholar
Pinkel, C., Binnig, C., Jiménez-Ruiz, E., Kharlamov, E., May, W., Nikolov, A., Sasa Bastinos, A., Skjœveland, M. G., Solimando, A., Taheriyan, M., Heupel, C. & Horrocks, I. 2017. RODI: benchmarking relational-to-ontology mapping generation quality. Semantic Web 9(1), 25–52.CrossRefGoogle Scholar
Qin, H., Dou, D. & LePendu, P. 2007. Discovering executable semantic mappings between ontologies. In On the Move to Meaningful Internet Systems, 832–849. Springer.CrossRefGoogle Scholar
Ritze, D., Meilicke, C., Šváb Zamazal, O. & Stuckenschmidt, H. 2009. A pattern-based ontology matching approach for detecting complex correspondences. In 4th ISWC Workshop on Ontology Matching, 25–36.Google Scholar
Ritze, D., Völker, J., Meilicke, C. & Šváb Zamazal, O. 2010. Linguistic analysis for complex ontology matching. In 5th Workshop on Ontology Matching, 1–12.Google Scholar
Scharffe, F. 2009. Correspondence Patterns Representation. PhD thesis, Faculty of Mathematics, Computer Science and University of Innsbruck.Google Scholar
Serpeloni, F., Moraes, R. & Bonacin, R. 2011. Ontology mapping validation. International Journal of Web Portals 3(3), 1–11.CrossRefGoogle Scholar
Sim, S. E., Easterbrook, S. & Holt, R. C. 2003. Using benchmarking to advance research: a challenge to software engineering. In Proceedings of the 25th International Conference on Software Engineering, ICSE’03, 74–83, Washington, DC, USA. IEEE Computer Society.CrossRefGoogle Scholar
Singh, A., Debruyne, C., Brennan, R., Meehan, A. & O’Sullivan, D. 2017. Extension of the M-Gov ontology mapping framework for increased traceability. In Proceedings of the 12th International Workshop on Ontology Matching co-located with the 16th International Semantic Web Conference (ISWC 2017), Vienna, Austria, October 21, 49–60.Google Scholar
Solimando, A., Jimenez-Ruiz, E. & Guerrini, G. 2017. Minimizing conservativity violations in ontology alignments: algorithms and evaluation. Knowledge and Information Systems 51(3), 775–819.CrossRefGoogle Scholar
Solimando, A., Jiménez-Ruiz, E. & Pinkel, C. 2014. Evaluating ontology alignment systems in query answering tasks. In ISWC 2014 International Conference on Posters & Demonstrations, 301–304.Google Scholar
Stapleton, G., Howse, J., Bonnington, A. & Burton, J. 2014. A vision for diagrammatic ontology engineering. In International Workshop on Visualizations and User Interfaces for Knowledge Engineering and Linked Data Analytics, 1–13.Google Scholar
Stevens, R., Lord, P., Malone, J. & Matentzoglu, N. 2018. Measuring expert performance at manually classifying domain entities under upper ontology classes. Journal of Web Semantics 57, 1–13.Google Scholar
Šváb, O., Svátek, V., Berka, P., Rak, D. & Tomášek, P. 2005. Ontofarm: towards an experimental collection of parallel ontologies. In Poster Track of ISWC, 2005.Google Scholar
Thiéblin, E., Amarger, F., Hernandez, N., Roussey, C. & Trojahn, C. 2017. Cross-querying LOD datasets using complex alignments: an application to agronomic taxa. In Research Conference on Metadata and Semantics Research, 25–37. Springer.CrossRefGoogle Scholar
Thiéblin, É., Cheatham, M., dos Santos, C. T., Zamazal, O. & Zhou, L. 2018a. The first version of the OAEI complex alignment benchmark. In Proceedings of the ISWC 2018 Posters & Demonstrations, Industry and Blue Sky Ideas Tracks co-located with 17th International Semantic Web Conference (ISWC 2018), Monterey, USA, October 8–12, 2018.Google Scholar
Thiéblin, É., Haemmerlé, O., Hernandez, N. & Trojahn, C. 2018b. Task-oriented complex ontology alignment: two alignment evaluation sets. In The Semantic Web - 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, Proceedings, 655–670.Google Scholar
Thieblin, E., Haemmerle, O., Hernandez, N. & Trojahn, C. to appear. Survey on complex ontology matching. Semantic Web Journal.Google Scholar
Thiéblin, E., Haemmerlé, O. & Trojahn, C. 2018. Complex matching based on competency questions for alignment: a first sketch. In Ontology Matching Workshop.Google Scholar
Tordai, A., van Ossenbruggen, J., Schreiber, G. & Wielinga, B. 2011. Let’s agree to disagree: on the evaluation of vocabulary alignment. In Proceedings of the Sixth International Conference on Knowledge Capture, K-CAP’11, 65–72. ACM.CrossRefGoogle Scholar
Van Hage, W. R., Isaac, A. & Aleksovski, Z. 2007. Sample evaluation of ontology-matching systems. In EON, 2007, 41–50.Google Scholar
Visser, P. R., Jones, D. M., Bench-Capon, T. J. & Shave, M. 1997. An analysis of ontology mismatches; heterogeneity versus interoperability. In AAAI 1997 Spring Symposium on Ontological Engineering, Stanford CA., USA, 164–172.Google Scholar
Walshe, B., Brennan, R. & O’Sullivan, D. 2016. Bayes-recce: a bayesian model for detecting restriction class correspondences in linked open data knowledge bases. International Journal on Semantic Web and Information Systems (IJSWIS) 12(2), 25–52.CrossRefGoogle Scholar
Zamazal, O. & Svátek, V. 2017. The ten-year ontofarm and its fertilization within the onto-sphere. Web Semantics: Science, Services and Agents on the World Wide Web 43, 46–53.CrossRefGoogle Scholar
Zhou, L., Cheatham, M., Krisnadhi, A. & Hitzler, P. 2018. A complex alignment benchmark: Geolink dataset. In International Semantic Web Conference, 273–288. Springer, Cham.CrossRefGoogle Scholar