We calculate the moments of the characteristic polynomials of $N\times N$ matrices drawn from the Hermitian ensembles of Random Matrix Theory, at a position t in the bulk of the spectrum, as a series expansion in powers of t. We focus in particular on the Gaussian Unitary Ensemble. We employ a novel approach to calculate the coefficients in this series expansion of the moments, appropriately scaled. These coefficients are polynomials in N. They therefore grow as $N\to\infty$, meaning that in this limit the radius of convergence of the series expansion tends to zero. This is related to oscillations as t varies that are increasingly rapid as N grows. We show that the $N\to\infty$ asymptotics of the moments can be derived from this expansion when $t=0$. When $t\ne 0$ we observe a surprising cancellation when the expansion coefficients for N and $N+1$ are formally averaged: this procedure removes all of the N-dependent terms leading to values that coincide with those expected on the basis of previously established asymptotic formulae for the moments. We obtain as well formulae for the expectation values of products of the secular coefficients.