In this chapter, I will try to answer the following three questions. What do we mean by ‘theory’ in the applied discipline of social policy? How do we distinguish between scientific theory, normative theory and ideology? Is it possible – or desirable – to design and implement rational social policies in the fractious world of democratic party politics where partisanship and passion are more evident than scholarly impartiality and rationality is in short supply?

The role of theory in social research

The development and testing of theories is an essential element in the conduct of social research. Theories set out explanatory and predictive propositions about the causal relationships between phenomena, such as the characteristics and incidence of poverty and the processes by which people become poor, remain poor and escape from poverty.

Deductivists and inductivists hold differing views about the sequence in which scientific research should be conducted. Deductivists start by formulating a theory and then proceed from this general proposition to a consideration of particular cases in order to test their theory. Inductivists start by drawing inferences from particular cases from which they proceed towards the formulation of a theory.

The central assumption of the inductivist approach is that ‘scientific knowledge grows out of simple unbiased statements reporting the evidence of the senses’ (Medawar, 1984: 98). ‘In real life’, however, as Peter Medawar suggests, ‘discovery and justification are almost always different processes’, and researchers seldom, if ever, start their enquiries with a clean sheet. They begin with a review of the relevant literature, and some provisional ideas, or hypotheses, about the subject they wish to investigate. Medawar's reductivist approach to the conduct of research shares much in common with Karl Popper's hypothetico-deductive method of scientific enquiry.

In Popper's approach, we must rely ‘upon the best tested of theories, which are the ones for which we have good rational reasons, not of course good reasons for believing them to be true, but for believing them to be the best available … the best among competing theories, the best approximations to the truth’ (Popper, 1978: 95; emphasis added).