Critiques of End-to-End Vision: A Case for Retrieval-Augmented Object Identification (RAOI)

The dominant paradigm in computer vision is to train large Convolutional Neural Networks (CNNs) to map pixels directly to labels. This approach has produced systems with super- human benchmark performance but suffers from serious structural weaknesses. As shown in my prior work, The Weaponization of Imperfection, these models do not truly identify objects; they optimize for statistical correlations in continuous feature space. This inherent reliance on high-dimensional probability optimization leaves them topologically vulnerable to imperceptible adversarial noise that can arbitrarily force a catastrophic misclassification, such as turning an ambulance into a tank (Akbar, 2025). In this proposal, drawing inspiration from the Generative Latent Prediction framework of World Models and the success of Retrieval-Augmented Generation (RAG) in multimodal systems, I critique the reliance on monolithic classification. I argue that the primary goal of a robust vision system should not be classification, which is fundamentally guessing, but deterministic identification, which is verifiable retrieval from memory. I propose RAOI (Retrieval-Augmented Object Identification), a new modular architecture that structurally separates the mechanism of visual encoding from semantic memory. By combining principled background isolation, a robust discrete visual hashing mechanism, and an external retrieval memory (VRMS) like DNS, RAOI offers a path toward systems that are more robust, adaptable, and intrinsically interpretable than current end-to-end networks. Keywords: Retrieval-Augmented Vision, Adversarial Robustness, Discrete Hashing, Open-World Learning, Object Detection