Health-related quality of life (HRQoL) in the context of COVID-19 is not fully understood. We assessed HRQoL using Patient-Reported Outcomes Measurement Information System® measures among 559 former COVID-19 patients and 298 non-infected individuals. HRQoL was captured once up to 2 years after the initial test. Additionally, we described associations of characteristics with impaired HRQoL. Overall, HRQoL scores were inferior among former patients. A meaningful group difference of at least three T-score points was discernible until 12 months after testing for fatigue (3.1), sleep disturbance (3.5), and dyspnoea (3.7). Cognitive function demonstrated such difference even at >18 months post-infection (3.3). Following dichotomization, pronounced differences in impaired HRQoL were observed in physical (19.2% of former patients, 7.3% of non-infected) and cognitive function (37.6% of former patients, 16.5% of non-infected). Domains most commonly affected among former patients were depression (34.9%), fatigue (37.4%), and cognitive function. Factors that associated with HRQoL impairments among former patients included age (OR ≤2.1), lower education (OR ≤5.3), and COVID-19-related hospitalization (OR ≤4.7), among others. These data underline the need for continued attention of the scientific community to further investigate potential long-term health limitations after COVID-19 to ultimately establish adequate screening and management options for those affected.