A considerable body of evidence suggests that early caregiving may affect the short-term functioning and longer term development of the hypothalamic–pituitary–adrenocortical axis. Despite this, most research to date has been cross-sectional in nature or restricted to relatively short-term longitudinal follow-ups. More important, there is a paucity of research on the role of caregiving in low- and middle-income countries, where the protective effects of high-quality care in buffering the child's developing stress regulation systems may be crucial. In this paper, we report findings from a longitudinal study (N = 232) conducted in an impoverished periurban settlement in Cape Town, South Africa. We measured caregiving sensitivity and security of attachment in infancy and followed children up at age 13 years, when we conducted assessments of hypothalamus–pituitary–adrenocortical axis reactivity, as indexed by salivary cortisol during the Trier Social Stress Test. The findings indicated that insecure attachment was predictive of reduced cortisol responses to social stress, particularly in boys, and that attachment status moderated the impact of contextual adversity on stress responses: secure children in highly adverse circumstances did not show the blunted cortisol response shown by their insecure counterparts. Some evidence was found that sensitivity of care in infancy was also associated with cortisol reactivity, but in this case, insensitivity was associated with heightened cortisol reactivity, and only for girls. The discussion focuses on the potentially important role of caregiving in the long-term calibration of the stress system and the need to better understand the social and biological mechanisms shaping the stress response across development in low- and middle-income countries.