The suitability of milk for specific dairy products might be improved by changing milk fat or milk protein composition. In the RobustMilk project, we showed that milk fat composition is determined by genetic factors. In addition, recent studies indicate that milk protein composition is strongly affected by genetic factors. This suggests that there are opportunities to change milk composition by means of selective breeding. Traditional selection is based on large-scale phenotyping and not all analytical methods are suited for this purpose. The RobustMilk project team has shown that several fatty acids can be predicted on the basis of IR spectra. Accuracy of predicting individual milk proteins based on IR spectra is low. In addition to phenotypic records, selection might be based on genotypic information. DGAT1 and SCD1 genotypes are strongly associated with fat composition. β-Lactoglobulin, β-casein and κ-casein protein variants are strongly associated with protein composition. We conclude that tools are now available for changing detailed milk fat or milk protein composition by means of selective breeding.