Skip to main content Accessibility help
×
Home

User validation in ontology alignment: functional assessment and impact

Abstract

User validation is one of the challenges facing the ontology alignment community, as there are limits to the quality of the alignments produced by automated alignment algorithms. In this paper, we present a broad study on user validation of ontology alignments that encompasses three distinct but inter-related aspects: the profile of the user, the services of the alignment system, and its user interface. We discuss key issues pertaining to the alignment validation process under each of these aspects and provide an overview of how current systems address them. Finally, we use experiments from the Interactive Matching track of the Ontology Alignment Evaluation Initiative 2015–2018 to assess the impact of errors in alignment validation, and how systems cope with them as function of their services.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      User validation in ontology alignment: functional assessment and impact
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      User validation in ontology alignment: functional assessment and impact
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      User validation in ontology alignment: functional assessment and impact
      Available formats
      ×

Copyright

This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.

References

Hide All
Abu Helou, M. & Palmonari, M. 2017. Multi-user feedback for large-scale cross-lingual ontology matching. In 9th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, 5766.
Aumüller, D., Do, H. H., Maßmann, S. & Rahm, E. 2005. Schema and ontology matching with COMA++. In ACM SIGMOD International Conference on Management of Data, 906908.
Aurisano, J., Nanavaty, A. & Cruz, I. F. 2015. Visual Analytics for Ontology Matching Using Multi-linked Views. In International Workshop on Visualizations and User Interfaces for Ontologies and Linked Data, CEUR 1456, 2536.
Beisswanger, E. & Hahn, U. 2012. Towards valid and reusable reference alignments-ten basic quality checks for ontology alignments and their application to three different reference data sets. Journal of Biomedical Semantics 3(S-1), S4:1S4:14.
Conroy, C., Brennan, R., O’Sullivan, D. & Lewis, D. 2009. User evaluation study of a tagging approach to semantic mapping. In The Semantic Web: Research and Applications – 6th European Semantic Web Conference, ESWC 2009, Heraklion, Crete, Greece, May 31–June 4, 2009, Proceedings, LNCS 5554, 623637.
Cruz, I. F., Antonelli, F. & Stroe, C. 2009. AgreementMaker: efficient matching for large real-world schemas and ontologies. Proceedings of the VLDB Endowment 2(2), 15861589.
Cruz, I. F., Palmonari, M., Loprete, F., Stroe, C. & Taheri, A. 2016. Quality-based model for effective and robust multi-user pay-as-you-go ontology matching. Semantic Web 7(4), 463479.
Cruz, I. F., Stroe, C. & Palmonari, M. 2012. Interactive user feedback in ontology matching using signature vectors. In 28th IEEE International Conference on Data Engineering, 13211324.
Cruz, I. F., Sunna, W., Makar, N. & Bathala, S. 2007. A visual tool for ontology alignment to enable geospatial interoperability. Journal of Visual Languages & Computing 18(3), 230254.
Curino, C., Orsi, G. & Tanca, L. 2007. X-SOM: A flexible ontology mapper. In 18th International Workshop on Database and Expert Systems Applications, 424428.
Dabrowski, J. & Munson, E. 2011. 40 years of searching for the best computer system response time. Interacting with Computers 23(5), 555564.
da Silva, J., Revoredo, K. & Baiao, F. 2018a. ALIN results for OAEI 2018. In 13th International Workshop on Ontology Matching, CEUR 2288, 117124.
da Silva, J., Revoredo, K., Baiao, F. & Euzenat, J. 2018b. Interactive ontology matching: using expert feedback to select attribute mappings. In 13th International Workshop on Ontology Matching, CEUR 2288, 2536.
Dhamankar, R., Lee, Y., Doan, A., Halevy, A. & Domingos, P. 2004. iMAP: discovering complex semantic matches between database schemas. In ACM SIGMOD International Conference on Management of Data, 383394.
Diallo, G. 2014. An effective method of large scale ontology matching. Journal of Biomedical Semantics 5, 44:144:19.
Djeddi, W. E., Ben Yahia, S. & Khadir, M. T. 2018. XMap: Results for OAEI 2018. In 13th International Workshop on Ontology Matching, CEUR 2288, 210215.
Djeddi, W. E. & Khadir, M. T. 2014. A novel approach using context-based measure for matching large scale ontologies. In 16th International Conference on Data Warehousing and Knowledge Discovery, LNCS 8646, 320331.
Dragisic, Z., Ivanova, V., Lambrix, P., Faria, D., Jiménez-Ruiz, E. & Pesquita, C. 2016. User validation in ontology alignment, The Semantic Web – ISWC 2016 – 15th International Semantic Web Conference, Kobe, Japan, October 17–21, 2016, Proceedings, Part I, LNCS 9981, 200217.
Dragisic, Z., Ivanova, V., Li, H. & Lambrix, P. 2017. Experiences from the anatomy track in the ontology alignment evaluation initiative. Journal of Biomedical Semantics 8(1), 56:156:28.
Duan, S., Fokoue, A. & Srinivas, K. 2010. One size does not fit all: Customizing ontology alignment using user feedback. In The Semantic Web – ISWC 2010 – 9th International Semantic Web Conference, ISWC 2010, Shanghai, China, November 7-11, 2010, Revised Selected Papers, Part I, LNCS 6496, 177192.
Dudas, M., Lohmann, S., Svatek, V. & Pavlov, D. 2018. Ontology visualization methods and tools: a survey of the state of the art. The Knowledge Engineering Review 33, e10:1e10:39.
Ehrig, M. & Sure, Y. 2005. Foam – framework for ontology alignment and mapping – results of the ontology alignment evaluation initiative. In Proceedings of the K-CAP 2005 Workshop on Integrating Ontologies, CEUR 156, 7276.
Euzenat, J., Meilicke, C., Shvaiko, P., Stuckenschmidt, H. & Trojahn, C. 2011. Ontology alignment evaluation initiative: six years of experience. Journal on Data Semantics XV, 158192.
Euzenat, J. & Shvaiko, P. 2013. User Involvement. Ontology Matching, 353375.
Euzenat, J. 2017. Interaction-based ontology alignment repair with expansion and relaxation. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI), 185191.
Falconer, S. & Noy, N. 2011. Interactive techniques to support ontology matching. Schema Matching and Mapping, 2951.
Falconer, S., Noy, N. & Storey, M.-A. 2006. Towards understanding the needs of cognitive support for ontology mapping. In First International Workshop on Ontology Matching, CEUR 225.
Falconer, S., Noy, N. & Storey, M.-A. 2007. Ontology mapping – a user survey. In 2nd International Workshop on Ontology Matching, CEUR 304, 4960.
Falconer, S. & Storey, M.-A. 2007. A cognitive support framework for ontology mapping. In The Semantic Web. 6th International Semantic Web Conference – 2nd Asian Semantic Web Conference, ISWC 2007 + ASWC 2007, Busan, Korea, November 11–15, 2007. Proceedings, LNCS 4825, 114127.
Faria, D., Pesquita, C., Santos, E., Palmonari, M., Cruz, I. F. & Couto, F. M. 2013. The AgreementMakerLight ontology matching system. In OTM 2013 Conferences, LNCS 8185, 527541.
Faria, D., Martins, C., Nanavaty, A., Oliveira, D., Sowkarthiga, B., Taheri, A., Pesquita, C., Couto, F. M. & Cruz, I. F. 2015. AML results for OAEI 2015. In 10th International Workshop on Ontology Matching, CEUR 1545, 116123.
Fu, B., Noy, N. & Storey, M.-A. 2017. Eye tracking the user experience – an evaluation of ontology visualization techniques. Semantic Web 8(1), 2341.
Gennari, J. H., Musen, M. A., Fergerson, R. W., Grosso, W. E., Crubézy, M., Eriksson, H., Noy, N. & Tu, S. W. 2003. The evolution of Protégé: an environment for knowledge-based systems development. International Journal on Human-Computer Studies 58(1), 89123.
Granitzer, M., Sabol, V., Onn, K. W., Luckose, D. & Tochtermann, K. 2010. Ontology alignment – a survey with focus on visually supported semi-automatic techniques. Future Internet 2(3), 238258.
Guedes, A., Baiao, F. A. & Revoredo, K. 2014. On the identification and representation of ontology correspondence antipatterns. In 8th International Workshop on Modular Ontologies, CEUR 1248.
Hamdi, F., Safar, B., Niraula, N. & Reynaud, C. 2009. TaxoMap in the OAEI 2009 alignment contest. In 4th International Workshop on Ontology Matching, CEUR 551, 230237.
Hamdi, F., Reynaud, C. & Safar, B. 2010. Pattern-based mapping refinement. In 17th International Conference on Knowledge Engineering and Management, LNCS 6317, 115.
Harrow, I., Balakrishnan, R., Jimenez-Ruiz, E., Jupp, S., Lomax, J., Reed, J., Romacker, M., Senger, C., Splendiani, A., Wilson, J. & Woollard, P. 2019. Ontology mapping for semantically enabled applications. Drug Discovery Today.
Hu, W. & Qu, Y. 2008. Falcon-AO: A practical ontology matching system. Journal of Web Semantics 6(3), 237239.
Hu, W., Qu, Y. & Cheng, G. 2008. Matching large ontologies: a divide-and-conquer approach. Data & Knowledge Engineering 67, 140160.
Ivanova, V., Bach, B., Pietriga, E. & Lambrix, P. 2017. Alignment cubes: towards interactive visual exploration and evaluation of multiple ontology alignments. The Semantic Web – ISWC 2017 – 16th International Semantic Web Conference, Vienna, Austria, October 21–25, 2017, Proceedings, Part I, LNCS 10587, 400417.
Ivanova, V., Bergman, J. L., Hammerling, U. & Lambrix, P. 2012. Debugging taxonomies and their alignments: the ToxOntology-MeSH use case. In First International Workshop on Debugging Ontologies and Ontology Mappings, 2536.
Ivanova, V. & Lambrix, P. 2013. A unified approach for aligning taxonomies and debugging taxonomies and their alignments. In The Semantic Web: Semantics and Big Data – 10th International Conference, ESWC 2013, Montpellier, France, May 26-30, 2013. Proceedings, LNCS 7882, 115.
Ivanova, V., Lambrix, P. & Åberg, J. 2015. Requirements for and evaluation of user support for large-scale ontology alignment. In The Semantic Web. Latest Advances and New Domains – 12th European Semantic Web Conference, ESWC 2015, Portoroz, Slovenia, May 31–June 4, 2015. Proceedings, LNCS 9088, 320.
Ivanova, V., Lambrix, P., Lohmann, S. & Pesquita, C. 2019. Visualization and interaction for ontologies and linked data – editorial. Journal of Web Semantics 55, 145149.
Jean-Mary, Y. R., Shironoshita, E. P. & Kabuka, M. R. 2009. Ontology matching with semantic verification. Journal of Web Semantics 7(3), 235251.
Jiménez-Ruiz, E., Cuenca Grau, B., Horrocks, I. & Berlanga, R. 2009. Ontology integration using mappings: towards getting the right logical consequences. The Semantic Web: Research and Applications. ESWC 2009, LNCS 5554, 173187.
Jiménez-Ruiz, E., Cuenca Grau, B., Horrocks, I. & Berlanga, R. 2011. Logic-based assessment of the compatibility of UMLS ontology sources. Journal of Biomedical Semantics 2(S-1), S2:1S2:16.
Jiménez-Ruiz, E. & Cuenca Grau, B. 2011. LogMap: Logic-based and scalable ontology matching. The Semantic Web – ISWC 2011 - 10th International Semantic Web Conference. LNCS 7031 pp 273288.
Jiménez-Ruiz, E., Cuenca Grau, B., Zhou, Y. & Horrocks, I. 2012. Large-scale interactive ontology matching: algorithms and implementation. In 20th European Conference on Artificial Intelligence (ECAI), 444449.
Jiménez-Ruiz, E., Payne, T. R., Solimando, A. & Tamma, V. 2016. Limiting logical violations in ontology alignnment through negotiation. In Principles of Knowledge Representation and Reasoning (KR), 217226.
Jiménez-Ruiz, E., Agibetov, A., Samwald, M. & Cross, V. 2018. Breaking-down the ontology alignment task with a lexical index and neural Embeddings. arXiv1805.12402.
Jirkovský, V. & Ichise, R. 2013. MAPSOM: user involvement in ontology matching. In Semantic Technology. Third Joint International Conference, JIST 2013, Seoul, South Korea, November 28–30, 2013, Revised Selected Papers, LNCS 8388, 348363.
Kachroudi, M., Diallo, G. & Ben Yahia, S. 2017. OAEI 2017 results of KEPLER. In 12th International Workshop on Ontology Matching, CEUR 2032, 138145.
Katifori, A., Halatsis, C., Lepouras, G., Vassilakis, C. & Giannopoulou, E. G. 2007. Ontology visualization methods – a survey. ACM Computing Surveys 39(4), 10:110:43.
Kensche, D., Quix, C., Li, X. & Li, Y. 2007. Geromesuite: A system for holistic generic model management. In 33rd International Conference on Very Large Data Bases, 13221325.
Kheder, N. & Diallo, G. 2015. ServOMBI at OAEI 2015. In 10th International Workshop on Ontology Matching, CEUR 1545, 200207.
Kirsten, T., Groß, A., Hartung, M. & Rahm, E. 2011. GOMMA: a component-based infrastructure for managing and analyzing life science ontologies and their evolution. Journal of Biomedical Semantics 2, 6:16:24.
Laadhar, A., Ghozzi, F., Megdiche, I., Ravat, F., Teste, O. & Gargouri, F. 2018. OAEI 2018 results of POMap++. In 13th International Workshop on Ontology Matching, CEUR 2288, 192196.
Laera, L., Tamma, V., Euzenat, J., Bench-Capon, T. & Payne, T. 2006. Reaching Agreement over Ontology Alignments. In The Semantic Web - ISWC 2006, 5th International Semantic Web Conference, ISWC 2006, Athens, GA, USA, November 5–9, 2006, Proceedings, LNCS 4273, 371384.
Laera, L., Blacoe, I., Tamma, V., Payne, T., Euzenat, J. & Bench-Capon, T. 2007. Argumentation over Ontology Correspondences in MAS. In 6th International Joint Conference on Autonomous Agents and Multiagent Systems, 228:1228:8.
Lambrix, P., Dragisic, Z. & Ivanova, V. 2012. Get my pizza right: Repairing missing is-a relations in ALC ontologies. Semantic Technology. Second Joint International Conference, JIST 2012, Nara, Japan, December 2–4, 2012. Proceedings, LNCS 7774, 1732.
Lambrix, P. & Edberg, A. 2003. Evaluation of ontology merging tools in bioinformatics. In Pacific Symposium on Biocomputing, 589600.
Lambrix, P. & Kaliyaperumal, R. 2017. A session-based ontology alignment approach enabling user involvement. Semantic Web 8(2), 225251.
Lambrix, P. & Liu, Q. 2009. Using partial reference alignments to align ontologies. In The Semantic Web: Research and Applications, 6th European Semantic Web Conference, ESWC 2009, Heraklion, Crete, Greece, May 31–June 4, 2009, Proceedings, LNCS 5554, 188202.
Lambrix, P. & Tan, H. 2006. SAMBO - a system for aligning and merging biomedical ontologies. Journal of Web Semantics 4(3), 196206
Lambrix, P. & Tan, H. 2007. A tool for evaluating ontology alignment strategies. Journal Data Semantics VIII, 182202.
Lanzenberger, M., Sampson, J. & Rester, M. 2010. Ontology visualization: tools and techniques for visual representation of semi-structured meta-data. Journal of Universal Computer Science 16(7), 10361054.
Lanzenberger, M., Sampson, J., Rester, M., Naudet, Y. & Latour, T. 2008. Visual ontology alignment for knowledge sharing and reuse. Journal of Knowledge Management 12(6), 102120.
Li, J., Tang, J., Li, Y. & Luo, Q. 2009. Rimom: A dynamic multistrategy ontology alignment framework. IEEE Transactions on Knowledge and Data Engineering 21(8), 12181232.
Li, Y., Stroe, C. & Cruz, I. F. 2015. Interactive visualization of large ontology matching results. In International Workshop on Visualizations and User Interfaces for Ontologies and Linked Data, CEUR 1456, 3748.
Massmann, S., Raunich, S., Aumüller, D., Arnold, P. & Rahm, E. 2011. Evolution of the COMA match system. In 6th International Workshop on Ontology Matching, CEUR 814, 4960.
McCann, R., Shen, W. & Doan, A. 2008. Matching schemas in online communities: a web 2.0 approach. In 2008 IEEE 24th International Conference on Data Engineering, 110119.
Meilicke, C., Stuckenschmidt, H. & Tamilin, A. 2008. Supporting manual mapping revision using logical reasoning. In 23rd AAAI Conference on Artificial Intelligence, 12131218.
Melnik, S., Garcia-Molina, H., & Rahm, E. 2002. Similarity flooding: a versatile graph matching algorithm and its application to schema matching. In Proceedings 18th IEEE International Conference on Data Engineering, 117128.
Nagy, M., Vargas-Vera, M. & Stolarski, P. 2009. DSSim results for OAEI 2009. In 4th International Workshop on Ontology Matching, CEUR 551, 160169.
Ngo, D. & Bellahsene, Z. 2016. Overview of YAM++ - (not) Yet Another Matcher for ontology alignment task. Journal of Web Semantics 41, 3049.
Nielsen, J. 1993. Usability Engineering, Morgan Kaufmann.
Noy, N. F., Mortensen, J., Alexander, P. & Musen, M. 2013. Mechanical Turk as an ontology engineer? ACM Web Science, 262271.
Noy, N. & Musen, M. 2000. Algorithm & tool for automated ontology merging and alignment. In 17th AAAI Conference on Artificial Intelligence, 450455.
Otero-Cerdeira, L., Rodríguez-Martínez, F. J. & Gómez-Rodríguez, A. 2015. Ontology matching: a literature review. Expert Systems with Applications 42(2), 949971.
Patterson, R. E., Blaha, L. M., Grinstein, G. G., Liggett, K. K., Kaveney, D. E., Sheldon, K. C., Havig, P. R. & Moore, J. A. 2014. A human cognition framework for information visualization. Computers & Graphics 42, 4258.
Paulheim, H., Hertling, S. & Ritze, D. 2013. Towards evaluating interactive ontology matching tools. In The Semantic Web: Semantics and Big Data, 10th International Conference, ESWC 2013, Montpellier, France, May 26–30, 2013. Proceedings, LNCS 7882, 3145.
Pesquita, C., Ivanova, V., Lohmann, S. & Lambrix, P. 2018. A framework to conduct and report on empirical user studies in semantic web contexts. In Knowledge Engineering and Knowledge Management. 21st International Conference, EKAW 2018, Nancy, France, November 12–16, 2018, Proceedings, LNCS 11313, 567583.
Pesquita, C., Faria, D., Santos, E., Neefs, J. & Couto, F. M. 2014. Towards visualizing the alignment of large biomedical ontologies. In Data Integration in the Life Sciences. 10th International Conference, DILS 2014, Lisbon, Portugal, July 17–18, 2014. Proceedings, LNBI 8574, 104111.
Quix, C., Geisler, S., Kensche, D. & Li, X. 2008. Results of GeRoMeSuite for OAEI 2008. 3rd International Workshop on Ontology Matching, CEUR 431, 160166.
Reul, Q. & Pan, J. Z. 2010. KOSIMap: Use of description logic reasoning to align heterogeneous ontologies. In 23rd International Workshop on Description Logics, CEUR 573, 497508.
Ritze, D. & Paulheim, H. 2011. Towards an automatic parameterization of ontology matching tools based on example mappings. In 6th International Workshop on Ontology Matching, CEUR 814, 3748.
Sarasua, C., Simperl, E. & Noy, N. F. 2012. Crowdmap: Crowdsourcing ontology alignment with microtasks. The Semantic Web – ISWC 2012 – 11th International Semantic Web Conference, Boston, MA, USA, November 11–15, 2012, Proceedings, Part I, LNCS 7649, 525541.
Schwichtenberg, S. & Engels, G. 2015. RSDL workbench results for OAEI 2015. In 10th International Workshop on Ontology Matching, CEUR 1545, 192199.
Shneiderman, B. 1996. The eyes have it: a task by data type taxonomy for information visualizations. IEEE Symposium on Visual Languages, 336343.
Shi, F., Li, J., Tang, J., Xie, G. & Li, H. 2009. Actively Learning Ontology Matching via User Interaction. In The Semantic Web - ISWC 2009, 8th International Semantic Web Conference, ISWC 2009, Chantilly, VA, USA, October 25–29, 2009. Proceedings, LNCS 5823, 585600.
Shvaiko, P. & Euzenat, J. 2013. Ontology matching: state of the art and future challenges. In Knowledge and Data Engineering 25(1), 158176.
Shvaiko, P., Giunchiglia, F., Da Silva, P. & McGuinness, D. 2005. Web explanations for semantic heterogeneity discovery. In The Semantic Web: Research and Applications, Second European Semantic Web Conference, ESWC 2005, Heraklion, Crete, Greece, May 29–June 1, 2005, Proceedings, LNCS 3532, 303317.
Smith, E. & Kosslyn, S. 2013. Cognitive Psychology: Mind and Brain, Pearson.
Solimando, A., Jiménez-Ruiz, E. & Guerrini, G. 2017. Minimizing conservativity violations in ontology alignments: algorithms and evaluation. Knowledge and Information Systems 51(3), 775819.
Tan, H. & Lambrix, P. 2007. A method for recommending ontology alignment strategies. The Semantic Web. 6th International Semantic Web Conference, 2nd Asian Semantic Web Conference, ISWC 2007 + ASWC 2007, Busan, Korea, November 11–15, 2007. Proceedings, LNCS 4825, 494507.
Wang, P. & Xu, B. 2008. Debugging ontology mappings: a static approach. Computing and Informatics 27(1), 2136.
Wang, P. & Xu, B. 2009. An effective similarity propagation method for matching ontologies without sufficient or regular linguistic information. In The Semantic Web. Fourth Asian Conference, ASWC 2009, Shanghai, China, December 6–9, 2009. Proceedings, LNCS 5926, 105119.
Wang, P., Zhou, Y. & Xu, B. 2011. Matching large ontologies based on reduction anchors. 22nd International Joint Conference on Artificial Intelligence, 23432348.
Wei-Kleiner, F., Dragisic, Z. & Lambrix, P. 2014. Abduction framework for repairing incomplete ELontologies: complexity results and algorithms. In 28th AAAI Conference on Artificial Intelligence, 11201127.

User validation in ontology alignment: functional assessment and impact

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed