We present a model for velocity fluctuations of dilute sedimenting spheres at low Reynolds number. The central idea is that a vertical stratification causes the fluctuations to decrease below those of an independent uniform distribution of particles, such a stratification naturally occurring from the broadening of the sedimentation front. We use numerical simulations, scaling arguments, structure factor calculations, and experiments to show that there is a critical stratification above which the characteristics of the density and velocity fluctuations change significantly. For thin cells, the broadening of the sediment front (and the resulting stratification) is small, so the velocity fluctuations are predicted by independent-Poisson-distribution estimates. In very thick cells, the stratification is significant, leading to persistent decay of the velocity fluctuations for the duration of the experiment. Estimated stratifications quantitatively agree with the simulations, and indicate the likelihood that previous experimental measurements were also affected by stratification. The velocity fluctuations in sedimentation are therefore not universal but instead depend on both the cell shape and developing stratification.