Skip to main content Accessibility help
×
Home

A learning to rank approach for cross-language information retrieval exploiting multiple translation resources

  • Hosein Azarbonyad (a1), Azadeh Shakery (a2) (a3) and Heshaam Faili (a2) (a3)

Abstract

Cross-language information retrieval (CLIR), finding information in one language in response to queries expressed in another language, has attracted much attention due to the explosive growth of multilingual information in the World Wide Web. One important issue in CLIR is how to apply monolingual information retrieval (IR) methods in cross-lingual environments. Recently, learning to rank (LTR) approach has been successfully employed in different IR tasks. In this paper, we use LTR for CLIR. In order to adapt monolingual LTR techniques in CLIR and pass the barrier of language difference, we map monolingual IR features to CLIR ones using translation information extracted from different translation resources. The performance of CLIR is highly dependent on the size and quality of available bilingual resources. Effective use of available resources is especially important in low-resource language pairs. In this paper, we further propose an LTR-based method for combining translation resources in CLIR. We have studied the effectiveness of the proposed approach using different translation resources. Our results also show that LTR can be used to successfully combine different translation resources to improve the CLIR performance. In the best scenario, the LTR-based combination method improves the performance of single-resource-based CLIR method by 6% in terms of Mean Average Precision.

Copyright

Corresponding author

*Corresponding author. Email: shakery@ut.ac.ir

References

Hide All
Agirre, E., Di Nunzio, G.M., Ferro, N., Mandl, T. and Peters, C. (2009). CLEF 2009: ad hoc track overview. In Proceedings of the 2009 International Conference on Multilingual and Multimodal Information Access Evaluation: Cross-language Evaluation Forum, CLEF ‘09, pp. 1537. Berlin, Germany: Springer.
AleAhmad, A., Amiri, H., Darrudi, E., Rahgozar, M. and Oroumchian, F. (2009). Hamshahri: A standard persian text collection. Knowledge-Based Systems 22(5), 382387.
Aljlayl, M. and Frieder, O. (2001). Effective Arabic-English cross-language information retrieval via machine-readable dictionaries and machine translation. In Proceedings of the Tenth International Conference on Information and Knowledge Management, CIKM ‘01, pp. 295302. New York, NY, USA: ACM.
Azarbonyad, H., Shakery, A. and Faili, H. (2012). Using learning to rank approach for parallel corpora based cross language information retrieval. In Proceedings of the 20th European Conference on Artificial Intelligence, ECAI’ 12, pp. 7984. Amsterdam, the Netherlands: IOS Press.
Azarbonyad, H., Shakery, A. and Faili, H. (2013). Exploiting multiple translation resources for English-Persian cross language information retrieval. In Information Access Evaluation. Multilinguality, Multimodality, and Visualization, CLEF ‘13, pp. 9399. Berlin, Germany: Springer.
Azarbonyad, H., Shakery, A. and Faili, H. (2014). Learning to exploit different translation resources for cross language information retrieval. International Journal of Information and Communication Technology Research 6(1), 5568.
Berger, A. and Lafferty, J. (1999). Information retrieval as statistical translation. In Proceedings of the 22Nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ‘99, pp. 222229. New York, NY, USA: ACM.
Brown, P.F., Pietra, V.J.D., Pietra, S.A.D. and Mercer, R.L. (1993). The mathematics of statistical machine translation: parameter estimation. Computational Linguistics 19(2), 263311.
Cao, Y., Xu, J., Liu, T.Y., Li, H., Huang, Y. and Hon, H.W. (2006). Adapting ranking SVM to document retrieval. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ‘06, pp. 186193. New York, NY, USA: ACM.
Chen, A. and Gey, F.C. (2004). Multilingual information retrieval using machine translation, relevance feedback and decompounding. Information Retrieval 7(1-2), 149182.
Chen, A., Jiang, H. and Gey, F. (2000). Combining multiple sources for short query translation in Chinese-English cross-language information retrieval. In Proceedings of the Fifth International Workshop on Information Retrieval with Asian Languages, IRAL ‘00, pp. 17–23.
Darwish, K. and Oard, D.W. (2003). Probabilistic structured query methods. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ‘03, pp. 338344. New York, NY, USA: ACM.
Da San Martino, G., Romeo, S., Barroón-Cedeño, A., Joty, S., Maàrquez, L., Moschitti, A. and Nakov, P. (2017). Cross-language question re-ranking. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ‘17, pp. 11451148. New York, NY, USA: ACM.
Ferro, N. and Peters, C. (2010). CLEF 2009 ad hoc track overview: TEL and Persian tasks. In Proceedings of the 2010 International Conference on Multilingual and Multimodal Information Access Evaluation: Cross-language Evaluation Forum, CLEF ‘10, pp. 1335. New York, NY, USA: ACM.
Finkel, J.R., Grenager, T. and Manning, C. (2005). Incorporating non-local information into information extraction systems by Gibbs sampling. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, ACL ‘05, pp. 363370. Stroudsburg, PA, USA: Association for Computational Linguistics.
Gao, W., Niu, C., Zhou, M. and Wong, K.F. (2009). Joint ranking for multilingual web search. In Proceedings of the 31st European Conference on IR Research on Advances in Information Retrieval, ECIR ‘09. Berlin, Germany: Springer.
Ghanbari, E. and Shakery, A. (2018). Query-dependent learning to rank for cross-lingual information retrieval. Knowledge and Information Systems. https://doi.org/10.1007/s10115-018-1232-8
Godavarthy, A. and Fang, Y. (2016). Cross-language microblog retrieval using latent semantic modeling. In Proceedings of the 2016 ACM International Conference on the Theory of Information Retrieval, ICTIR ‘16, pp. 303306. New York, NY, USA: ACM.
Gollins, T. and Sanderson, M. (2001). Improving cross-language retrieval with triangulated translation. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ‘01, pp. 9095. New York, NY, USA: ACM.
Hashemi, H.B. (2011). Using comparable corpora for English–Persian cross-language information retrieval. Dissertation, University of Tehran, Iran.
Hashemi, H.B. and Shakery, A. (2014). Mining a Persian–English comparable corpus for cross-language information retrieval. Information Processing and Management 50(2), 384398.
Hashemi, H.B., Shakery, A. and Faili, H. (2010). Creating a Persian–English comparable corpus. In Proceedings of the 2010 International Conference on Multilingual and Multimodal Information Access Evaluation: Cross-language Evaluation Forum, CLEF’10, pp. 2739. Berlin, Germany: Springer.
Jabbari, F., Bakhshaei, S., Mohammadzadeh Ziabary, S. and Khadivi, S. (2012). Developing an open-domain English-Farsi translation system using AFEC: Amirkabir bilingual Farsi-English corpus. In Proceedings of 10th Biennial Conference of the Association for Machine Translation in the Americas.
Jones, G.J.F. and Lam-Adesina, A.M. (2002). Exeter at clef 2001: Experiments with machine translation for bilingual retrieval. In: Revised Papers from the Second Workshop of the Cross-Language Evaluation Forum on Evaluation of Cross-Language Information Retrieval Systems, CLEF ‘01, pp. 5977. Berlin, Germany: Springer.
Kadri, Y. and Nie, J. (2007). Combining resources with confidence measures for cross language information retrieval. In Proceedings of the ACM First Ph.D. Workshop in CIKM, PIKM ‘07, pp. 131138. New York, NY, USA: ACM.
Lavrenko, V., Choquette, M. and Croft, W.B. (2002). Cross-lingual relevance models. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ‘02, pp. 175182. New York, NY, USA: ACM.
Li, H. (2011) Learning to Rank for Information Retrieval and Natural Language Processing. San Rafael, California: Morgan & Claypool Publishers.
Lu, W.H., Chien, L.F. and Lee, H.J. (2004). Anchor text mining for translation of web queries: a transitive translation approach. ACM Transactions on Information Systems 22(2), 242269.
Mansouri, A. and Faili, H. (2012). State-of-the-art English to Persian statistical machine translation system. In 2012 16th CSI International Symposium on Artificial Intelligence and Signal Processing (AISP), pp. 174–179.
McNamee, P. and Mayfield, J. (2002). Comparing cross-language query expansion techniques by degrading translation resources. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ‘02, pp. 159166. New York, NY, USA: ACM.
Metzler, D. and Croft, B. (2007). Linear feature-based models for information retrieval. Information Retrieval 10(3), 257274.
Mosavi Miangah, T. (2009). Constructing a large-scale English-Persian parallel corpus. Meta: Journal des traducteursMeta: Translators’ Journal 54(1), 181188.
Nallapati, R. (2004). Discriminative models for information retrieval. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ‘04, pp. 6471. New York, NY, USA: ACM.
Nie, J. (2010). Cross-Language Information Retrieval. San Rafael, California: Morgan & Claypool Publishers.
Nie, J., Isabelle, P., Plamondon, P. and Foster, G. (1998). Using a probabilistic translation model for cross-language information retrieval. In 6th Workshop on Very Large Corpora, pp. 1827.
Pirkola, A., Hedlund, T., Keskustalo, H. and Järvelin, K. (2001). Dictionary-based cross-language information retrieval: problems, methods, and research findings. Information Retrieval 4(3-4), 209230.
Rahimi, R. and Shakery, A. (2013). A language modeling approach for extracting translation knowledge from comparable corpora. In Proceedings of the 35th European Conference on Advances in Information Retrieval, ECIR’13, pp. 606617. Berlin, Germany: Springer.
Rahimi, Z. and Shakery, A. (2011). Topic based creation of a Persian-English comparable corpus. In Proceedings of the 7th Asia Conference on Information Retrieval Technology, AIRS’11, pp. 458469. Berlin, Germany: Springer.
Rahimi, R. and Shakery, A. (2017). Online learning to rank for cross-language information retrieval. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ‘17, pp. 10331036. New York, NY, USA: ACM.
Ren, F. and Bracewell, D.B. (2009). Advanced information retrieval. Electronic Notes on Theoretical Computer Science 225, 303317.
Robertson, S.E., Walker, S. and Beaulieu, M. (2000). Experimentation as a way of life: Okapi at TREC. Information Processing and Management 36(1), 95108.
Shakery, A. and Zhai, C. (2013). Leveraging comparable corpora for cross-lingual information retrieval in resource-lean language pairs. Information Retrieval 16(1), 129.
Talvensaari, T., Pirkola, A., Järvelin, K., Juhola, M. and Laurikkala, J. (2008). Focused web crawling in the acquisition of comparable corpora. Information Retrieval 11(5), 427445.
Tao, T. and Zhai, C. (2005). Mining comparable bilingual text corpora for cross-language information integration. In Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, KDD ‘05, pp. 691696. New York, NY, USA: ACM.
Tsai, M.F., Wang, Y.T. and Chen, H.H. (2008). A study of learning a merge model for multilingual information retrieval. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ‘08, pp. 195202. New York, NY, USA: ACM.
Türe, F., Lin, J.J. and Oard, D.W. (2012). Combining statistical translation techniques for cross-language information retrieval. In Proceedings of the 24th International Conference on Computational Linguistics, COLING ‘12, pp. 26852702.
Usunier, N., Amini, M.R. and Goutte, C. (2011). Multiview semi-supervised learning for ranking multilingual documents. In Proceedings of the 2011 European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part III, ECML PKDD’11, pp. 443458. Berlin, Germany: Springer.
Vulić, I. and Moens, M.F. (2015). Monolingual and cross-lingual information retrieval models based on (bilingual) word embeddings. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ‘15, pp. 363372. New York, NY, USA: ACM.
Wu, D. and He, D. (2012). Exploring the further integration of machine translation in English-Chinese cross language information access. Program: Electronic Library and Information Systems 46(4), 429457.
Xu, J. and Li, H. (2007). AdaRank: a boosting algorithm for information retrieval. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ‘07, pp. 391398. New York, NY, USA: ACM.

Keywords

A learning to rank approach for cross-language information retrieval exploiting multiple translation resources

  • Hosein Azarbonyad (a1), Azadeh Shakery (a2) (a3) and Heshaam Faili (a2) (a3)

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed