The spectral eddy viscosity (SEV) concept is a handy tool for the derivation of large-eddy simulation (LES) turbulence models and for the evaluation of their performance in predicting the spectral energy transfer. We compute this quantity by filtering and truncating fully resolved turbulence data from direct numerical simulations (DNS) of neutrally and stably stratified homogeneous turbulence. The results qualitatively confirm the plateau–cusp shape, which is often assumed to be universal, but show a strong dependence on the test filter size. Increasing stable stratification not only breaks the isotropy of the SEV but also modifies its basic shape, which poses a great challenge for implicit and explicit LES methods. We find indications that for stably stratified turbulence it is necessary to use different subgrid-scale (SGS) models for the horizontal and vertical velocity components. Our data disprove models that assume a constant positive effective turbulent Prandtl number.