We demonstrate a close analogy between a viscoelastic medium and an electrically conducting fluid containing a magnetic field. Specifically, the dynamics of the Oldroyd-B fluid in the limit of large Deborah number corresponds to that of a magnetohydrodynamic (MHD) fluid in the limit of large magnetic Reynolds number. As a definite example of this analogy, we compare the stability properties of differentially rotating viscoelastic and MHD flows. We show that there is an instability of the Oldroyd-B fluid that is physically distinct from both the inertial and elastic instabilities described previously in the literature, but is directly equivalent to the magnetorotational instability in MHD. It occurs even when the specific angular momentum increases outwards, provided that the angular velocity decreases outwards; it derives from the kinetic energy of the shear flow and does not depend on the curvature of the streamlines. However, we argue that the elastic instability of viscoelastic Couette flow has no direct equivalent in MHD.