The classical definitions of GARCH-type processes rely on strong assumptions on the first two conditional moments. The common practice in empirical studies, however, has been to test for GARCH by detecting serial correlations in the squared regression errors. This can be problematic because such autocorrelation structures are compatible with severe misspecifications of the standard GARCH. Numerous examples are provided in the paper. In consequence, standard (quasi-) maximum likelihood procedures can be inconsistent if the conditional first two moments are misspecified. To alleviate these problems of possible misspecification, we consider weak GARCH representations characterized by an ARMA structure for the squared error terms. The weak GARCH representation eliminates the need for correct specification of the first two conditional moments. The parameters of the representation are estimated via two-stage least squares. The estimator is shown to be consistent and asymptotically normal. Forecasting issues are also addressed.