Skip to main content Accessibility help
×
Home

Building on prior knowledge without building it in

  • Steven S. Hansen (a1), Andrew K. Lampinen (a1), Gaurav Suri (a2) and James L. McClelland (a1)

Abstract

Lake et al. propose that people rely on “start-up software,” “causal models,” and “intuitive theories” built using compositional representations to learn new tasks more efficiently than some deep neural network models. We highlight the many drawbacks of a commitment to compositional representations and describe our continuing effort to explore how the ability to build on prior knowledge and to learn new tasks efficiently could arise through learning in deep neural networks.

Copyright

References

Hide All
Bartunov, S. & Vetrov, D. P. (2016) Fast adaptation in generative models with generative matching networks. arXiv preprint 1612.02192.
Fodor, J. A. & Pylyshyn, Z. W. (1988) Connectionism and cognitive architecture: A critical analysis. Cognition 28(1–2):371.
Gülçehre, Ç. & Bengio, Y. (2016) Knowledge matters: Importance of prior information for optimization. Journal of Machine Learning Research 17(8):132.
Johnson, M., Schuster, M., Le, Q. V., Krikun, M., Wu, Y., Chen, Z. & Hughes, M. (2016) Google's multilingual neural machine translation system: Enabling zero-shot translation. arXiv preprint 1611.04558. Available at: https://arxiv.org/abs/1611.04558.
Krizhevsky, A., Sutskever, I. & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. Presented at the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, December 3–6, 2012. In: Advances in Neural Information Processing Systems 25 (NIPS 2012), ed. Pereira, F., Burges, C. J. C., Bottou, L. & Weinberger, K. Q., pp. 1097–105. Neural Information Processing Systems Foundation.
Marr, D. (1982) Vision: A computational investigation into the human representation and processing of visual information. MIT Press.
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D. & Riedmiller, M. (2013) Playing Atari with deep reinforcement learning. arXiv preprint 1312.5602. Available at: https://arxiv.org/abs/1312.5602.
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., & Lillicrap, T. (2016). One-shot learning with memory-augmented neural networks. arXiv preprint 1605.06065. Available at: https://arxiv.org/abs/1605.06065.
Vinyals, O., Blundell, C., Lillicrap, T. & Wierstra, D. (2016) Matching networks for one shot learning. Vinyals, O., Blundell, C., Lillicrap, T. Kavukcuoglu, K. & Wierstra, D. (2016). Matching networks for one shot learning. Presented at the 2016 Neural Information Processing Systems conference, Barcelona, Spain, December 5–10, 2016. In: Advances in Neural Information Processing Systems 29 (NIPS 2016), ed. Lee, D. D., Sugiyama, M., Luxburg, U. V., Guyon, I. & Garnett, R., pp. 3630–38. Neural Information Processing Systems Foundation.
Weston, J., Bordes, A., Chopra, S., Rush, A. M., van Merriënboer, B., Joulin, A. & Mikolov, T. (2015a) Towards AI-complete question answering: A set of prerequisite toy tasks. arXiv preprint 1502.05698. Available at: https://arxiv.org/pdf/1502.05698.pdf.

Building on prior knowledge without building it in

  • Steven S. Hansen (a1), Andrew K. Lampinen (a1), Gaurav Suri (a2) and James L. McClelland (a1)

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed