The research of early evolution stages and formation of galaxies requires creation of appropriate non-linear theory. The formation process of globular cluster systems (GCS) covers the long period of time as probably one begins from a dark matter state. So far it is difficult to simulate formation of GCS and galaxies ab initio. That’s why it is necessary to construct exact analytically solvable models of non-linear non-stationary early stages of evolution of self-gravitating systems and revealing the instabilities on a background of non-equilibrium states. The formation and evolution of GCS at early stage of collapsing dark matter (or protogalaxy) can be explained by instability of the modes of the high oscillation degrees (Nuritdinov et al., 2000), which corresponds to rather small-scale perturbations of the density of collapsing system. The mode degree defines on the average the number of clusters.