In this chapter, we discuss word-based models. The models stem from the original work on statistical machine translation by the IBM Candide project in the late 1980s and early 1990s. While this approach does not constitute the state of the art anymore, many of the principles and methods are still current today.

Reviewing this seminal work will introduce many concepts that underpin other statistical machine translation models, such as generative modeling, the expectation maximization algorithm, and the noisy-channel model. At the end of the chapter, we will also look at word alignment as a problem in itself.

Machine Translation by Translating Words

We start this chapter with a simple model for machine translation that is based solely on lexical translation, the translation of words in isolation. This requires a dictionary that maps words from one language to another.

Lexical Translation

If we open a common bilingual dictionary, say, German—English, we may find an entry like

Most words have multiple translations. Some are more likely than others. In this example, the translation house will often be correct when translating Haus into English. Others are common as well – building, home – while some are used only in certain circumstances. For instance, the Haus of a snail is its shell.