This study proposes two novel time-varying model-averaging methods for time-varying parameter regression models. When the number of predictors is small, we propose a novel time-varying complete subset-averaging (TVCSA) procedure, where the optimal time-varying subset size is obtained by minimizing the local leave-h-out cross-validation criterion. The TVCSA method is asymptotically optimal for achieving the lowest possible local mean squared error. When the number of predictors is relatively large, we propose a factor TVCSA method to reduce the computational burden by first reducing the dimension of predictors by extracting a few factors using principal component analysis and then obtaining the TVCSA forecasts from time-varying models with the generated factors. We show that the TVCSA estimator remains asymptotically optimal in the presence of generated factors. Monte Carlo simulation studies have provided favorable evidence for the TVCSA methods relative to the popular model-averaging methods in the literature. Empirical applications to equity premiums and inflation forecasting highlight the practical merits of the proposed methods.