On a smooth Banach manifold $M$, the equivalence classes of curves that agree up to acceleration form the second order tangent bundle $T^{2}M$ of $M$. This is a vector bundle in the presence of a linear connection $\nabla$ on $M$ and the corresponding local structure is heavily dependent on the choice of $\nabla$. In this paper we study the extent of this dependence and we prove that it is closely related to the notions of conjugate connections and second order differentials. In particular, the vector bundle structure on $T^{2}M$ remains invariant under conjugate connections with respect to diffeomorphisms of $M$.