The James Webb Space Telescope (JWST) hosts a non-redundant Aperture Masking Interferometer (AMI) in its Near Infrared Imager and Slitless Spectrograph (NIRISS) instrument, providing the only dedicated interferometric facility aboard — magnitudes more precise than any interferometric experiment previously flown. However, the performance of AMI (and other high resolution approaches such as kernel phase) in recovery of structure at high contrasts has not met design expectations. A major contributing factor has been the presence of uncorrected detector systematics, notably charge migration effects in the H2RG sensor, and insufficiently accurate mask metrology. Here we present AMIGO, a data-driven calibration framework and analysis pipeline that forward-models the full JWST AMI system — including its optics, detector physics, and readout electronics — using an end-to-end differentiable architecture implemented in the JAX framework and in particular exploiting the ∂LUX optical modelling package. AMIGO directly models the generation of up-the-ramp detector reads, using an embedded neural sub-module to capture non-linear charge redistribution effects, enabling the optimal extraction of robust observables, for example kernel amplitudes and phases, while mitigating systematics such as the brighter-fatter effect. We demonstrate AMIGO’s capabilities by recovering the AB Dor AC binary from commissioning data with high-precision astrometry, and detecting both HD 206893 B and the inner substellar companion HD 206893 c: a benchmark requiring contrasts approaching 10 magnitudes at separations of only 100mas. These results exceed outcomes from all published pipelines, and re-establish AMI as a viable competitor for imaging at high contrast at the diffraction limit. AMIGO is publicly available as open-source software community resource.