This paper studies the validity of nonparametric tests used in the regression discontinuity design. The null hypothesis of interest is that the average treatment effect at the threshold in the so-called sharp design equals a pre-specified value. We first show that, under assumptions used in the majority of the literature, for any test the power against any alternative is bounded above by its size. This result implies that, under these assumptions, any test with nontrivial power will exhibit size distortions. We next provide a sufficient strengthening of the standard assumptions under which we show that a version of a test suggested in Calonico, Cattaneo, and Titiunik (2014) can control limiting size.