Latent infection from Toxoplasma gondii (T. gondii) is widespread worldwide and has been associated with cognitive deficits in some but not all animal models and in humans. We tested the hypothesis that latent toxoplasmosis is associated with decreased cognitive function in a large cross-sectional dataset, the National Health and Nutrition Examination Survey (NHANES). There were 4178 participants aged 20–59 years, of whom 19·1% had IgG antibodies against T. gondii. Two ordinary least squares (OLS) regression models adjusted for the NHANES complex sampling design and weighted to represent the US population were estimated for simple reaction time, processing speed and short-term memory or attention. The first model included only main effects of latent toxoplasmosis and demographic control variables, and the second added interaction terms between latent toxoplasmosis and the poverty-to-income ratio (PIR), educational attainment and race-ethnicity. We also used multivariate models to assess all three cognitive outcomes in the same model. Although the models evaluating main effects only demonstrated no association between latent toxoplasmosis and the cognitive outcomes, significant interactions between latent toxoplasmosis and the PIR, between latent toxoplasmosis and educational attainment, and between latent toxoplasmosis and race-ethnicity indicated that latent toxoplasmosis may adversely affect cognitive function in certain groups.