Introduction and motivation

What is a syntactic parameter? An initial response is that it is a unit of syntactic variation, but surely this is an unsatisfying answer. Syntactic variation is exactly what the theory of parameters seeks to explain. Furthermore, it is far from obvious that parameters are units in any primitive sense. An explanatory theory of language variation and language learnability should try to provide an account of the content of the notion of parameter. This chapter is an invitation to a branch of mathematics – information theory and, more specifically, Kolmogorov complexity – that may provide the foundations for a full-blown linguistic theory of parameters. The very nature of the topic requires that we cover a broad range of mathematics. We will start with a brief overview of probability theory and then turn to a discussion of some of the fundamental elements of information theory, along with applications to linguistics. We will then consider a computational approach to (linguistic) descriptions and introduce some computational mathematics needed for a complexity-theoretic approach to descriptions. Using some results from data compression, we will then combine probability and information theory, on the one hand, with computational mathematics on the other hand to give a theory of optimal descriptions. Finally, we will show how optimal descriptions are related to likely descriptions, a relationship of deep significance to studies of linguistic variation and typology.