Factor rotation is a crucial step in interpreting the results of exploratory factor analysis. Several rotation methods have been developed for simple structure solutions, but their extensions to bi-factor analysis are often not well established. In this article, we propose a mathematical framework that incorporates customized factor structure as a regularization to produce the optimal orthogonal or oblique rotation. We demonstrate the utility of the framework using examples of simple structure rotation and bi-factor rotation. Through detailed simulations, we show that the new method is accurate and robust in recovering the factor structures and latent correlations when bi-factor analysis is applied. The new method is applied to a test data and a Quality of Life survey data. Results show that our method can reveal bi-factor structures that are consistent with the theories.