A complete seismic inversion would solve for the physical parameters in the scattering potential. Most imaging algorithms stop short of that, solving instead for a reflectivity function. To produce the physical model behind the reflectivity, further analysis is required. Similarly, to synthesize seismic data one often starts from a reflectivity rather than a scattering potential.
Inversion for or modeling from a reflectivity model is legitimate, if incomplete, though one does have to ask, just what is reflectivity? The reflection coefficient is a well defined concept for plane waves impinging upon an abrupt boundary of no curvature. Asymptotically, the concept extends to waves far from source or receiver impinging upon boundaries of reasonable curvature. Consequently, when we think about reflectivity we are usually thinking about discrete specular reflections produced at reflecting boundary surfaces. However, changes in real-Earth properties may or may not be abrupt, so confining reflectivity to a boundary surface is at best an idealization, at worst wrong. Moreover, imaging algorithms normally produce images of volumes, not surfaces. That is, they produce a value at every point within a volume representing the Earth's subsurface, not just along selected surfaces within the volume. Imaging algorithms solve for a pointwise reflectivity function, not a specular reflectivity function.
This is as it should be. Specular reflectivity is the integrated response of a boundary surface, not the response of a single point on that surface.