This paper questions whether it is possible to derive consistency and asymptotic normality of the Gaussian quasi-maximum likelihood estimator (QMLE) for possibly the simplest multivariate GARCH model, namely, the multivariate ARCH(1) model of the Baba, Engle, Kraft, and Kroner form, under weak moment conditions similar to the univariate case. In contrast to the univariate specification, we show that the expectation of the log-likelihood function is unbounded, away from the true parameter value, if (and only if) the observable has unbounded second moment. Despite this nonstandard feature, consistency of the Gaussian QMLE is still warranted. The same moment condition proves to be necessary and sufficient for the stationarity of the score when evaluated at the true parameter value. This explains why high moment conditions, typically bounded sixth moment and above, have been used hitherto in the literature to establish the asymptotic normality of the QMLE in the multivariate framework.