Skip to main content Accessibility help
×
Home

Being explicit about the implicit: inference generating techniques in visual narrative

  • NEIL COHN (a1)

Abstract

Inference has long been acknowledged as a key aspect of comprehending narratives of all kinds, be they verbal discourse or visual narratives like comics and films. While both theoretical and empirical evidence points towards such inference generation in sequential images, most of these approaches remain at a fairly broad level. Few approaches have detailed the specific cues and constructions used to signal such inferences in the first place. This paper thereby outlines several specific entrenched constructions that motivate a reader to generate inference. These techniques include connections motivated by the morphology of visual affixes like speech balloons and thought bubbles, the omission of certain narrative categories, and the substitution of narrative categories for certain classes of panels. These mechanisms all invoke specific combinatorial structures (morphology, narrative) that mismatch with the elicited semantics, and can be generalized by a set of shared descriptive features. By detailing specific constructions, this paper aims to push the study of inference in visual narratives to be explicit about when and why meaning is ‘filled in’ by a reader, while drawing connections to inference generation in other modalities.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Being explicit about the implicit: inference generating techniques in visual narrative
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Being explicit about the implicit: inference generating techniques in visual narrative
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Being explicit about the implicit: inference generating techniques in visual narrative
      Available formats
      ×

Copyright

This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.

Corresponding author

Address for correspondence: Neil Cohn, Tilburg University, Department of Communication and Cognition, PO Box 90153, 5000 LE Tilburg, The Netherlands. e-mail: neilcohn@visuallanguagelab.com

Footnotes

Hide All
*

Kaitlin Pederson and Ryan Taylor contributed to the featural analysis. Renske van Enschot provided advice on inverted U-curves. Joost Schilperoord and Joe Magliano offered useful feedback on earlier drafts.

Footnotes

References

Hide All
Aragonés, S. (2002). Actions Speak. Dark Horse Comics.
Asmus, J., & S. Lieber, S. (2015). Quantum and Woody Must Die #1. Valiant Comics.
Capullo, G. (1997). The Creech #1. Image Comics.
Costa, B., & Parks, J. (2017). Rickety Skitch and the Gelatinous Goo. Knopf.
Duggan, G., Posehn, B., & Koblish, S. (2012). Deadpool #13. Marvel Comics.
Godek, T. (2006). One Night. Originally posted on 20 March 2006 <http://www.yellowlight.scratchspace.net/comics/onenight/onenight.html>.
Godwin, D., & Cham, J. (2014). Might as well face it, you’re addicted to love. Scientific American Mind online. Originally posted 14 February 2014. Online <https://dwaynegodwin.tumblr.com/image/84481552842>.
Madureira, J. (1998). Battle Chasers #3. Image Comics.
Moore, A., & Gibbons, D. (1987). Watchmen. DC Comics.
Perez, R., & Coughler, R. (2003). Butternutsquash. Originally posted 28 May 2003 <www.butternutsquash.net>.
Savage, D. (2016). Chickenowls. Savage Chickens. Originally posted on 30 May 2016 <www.savagechickens.com>.
Amini, F., Riche, N. H., Lee, B., Hurter, C., & Irani, P. (2015). Understanding data videos: looking at narrative visualization through the cinematography lens. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 14591468). New York: ACM.
Barnes, S. (2017). Studies in the efficacy of motion graphics: the impact of narrative structure on exposition. Digital Journalism, 5(10), 121.
Berlyne, D. E. (1971). Aesthetics and psychobiology. New York: Appleton-Century-Crofts.
Branigan, E. (1992). Narrative comprehension and film. London: Routledge.
Chomsky, N. (1981). Lectures on government and binding. Dordrecht: Foris.
Clark, H. H. (1977). Bridging. In Johnson-Laird, P. N. & Wason, P. C. (Eds.), Thinking: readings in cognitive science (pp. 243263). Cambridge: Cambridge University Press.
Cohn, N. (2010). Extra! Extra! Semantics in comics! The conceptual structure of Chicago Tribune advertisements. Journal of Pragmatics, 42(11), 31383146.
Cohn, N. (2013a). Beyond speech balloons and thought bubbles: the integration of text and image. Semiotica, 2013(197), 3563.
Cohn, N. (2013b). The visual language of comics: introduction to the structure and cognition of sequential images. London: Bloomsbury.
Cohn, N. (2013c). Visual narrative structure. Cognitive Science, 37(3), 413452.
Cohn, N. (2014). You’re a good structure, Charlie Brown: the distribution of narrative categories in comic strips. Cognitive Science, 38(7), 13171359.
Cohn, N. (2015). Narrative conjunction’s junction function: the interface of narrative grammar and semantics in sequential images. Journal of Pragmatics, 88, 105132.
Cohn, N. (2016a). From visual narrative grammar to filmic narrative grammar: the narrative structure of static and moving images. In Wildfeuer, J. & Bateman, J. A. (Eds.), Film text analysis: new perspectives on the analysis of filmic meaning . (pp. 94117). London: Routledge.
Cohn, N. (2016b). A multimodal parallel architecture: a cognitive framework for multimodal interactions. Cognition, 146, 304323.
Cohn, N. (2018). Combinatorial morphology in visual languages. In Booij, G. (Ed.), The construction of words: advances in construction morphology (pp. 175199). London: Springer.
Cohn, N. (in press). Structural complexity in visual narratives: theory, brains, and cross-cultural diversity. In Grishakova, M. & Poulaki, M. (Eds.), Narrative complexity and media: experiential and cognitive interfaces (pp. 174–99). Lincoln, NE: University of Nebraska Press.
Cohn, N., & Kutas, M. (2015). Getting a cue before getting a clue: event-related potentials to inference in visual narrative comprehension. Neuropsychologia, 77, 267278.
Cohn, N., & Kutas, M. (2017). What’s your neural function, visual narrative conjunction? Grammar, meaning, and fluency in sequential image processing. Cognitive Research: Principles and Implications, 2(27), 113.
Cohn, N., Murthy, B., & Foulsham, T. (2016). Meaning above the head: combinatorial constraints on the visual vocabulary of comics. Journal of Cognitive Psychology, 28(5), 559574.
Cohn, N., Paczynski, M., & Kutas, M. (2017). Not so secret agents: event-related potentials to semantic roles in visual event comprehension. Brain and Cognition, 119, 19.
Cohn, N., Pederson, K., & Taylor, R. (2017). A picture is worth more words over time: multimodality and narrative structure across eight decades of American superhero comics. Multimodal Communication, 6(1), 1937.
Cohn, N., & Wittenberg, E. (2015). Action starring narratives and events: structure and inference in visual narrative comprehension. Journal of Cognitive Psychology, 27(7), 812828.
Culicover, P. W., & Jackendoff, R. (2005). Simpler syntax. Oxford: Oxford University Press.
Fauconnier, G., & Turner, M. (1998). Conceptual integration networks. Cognitive Science, 22(2), 133187.
Forceville, C. (2011). Pictorial runes in Tintin and the Picaros. Journal of Pragmatics, 43(3), 875890.
Forceville, C. (2016). Conceptual metaphor theory, blending theory, and other cognitivist perspectives on comics. In Cohn, N. (Ed.), The visual narrative reader (pp. 89114). London: Bloomsbury.
Forceville, C., Veale, T., & Feyaerts, K. (2010). Balloonics: the visuals of balloons in comics. In Goggin, J. & Hassler-Forest, D. (Eds.), The rise and reason of comics and graphic literature: critical essays on the form (pp. 5673). Jefferson, NC: McFarland & Company.
Fortunato, J. (2014). The gaze and the Spielberg face: Steven Spielberg’s application of Lacan’s mirror stage and audience response. Visual Communication Quarterly, 21(1), 4053.
Foulsham, T., Wybrow, D., & Cohn, N. (2016). Reading without words: eye movements in the comprehension of comic strips. Applied Cognitive Psychology, 30, 566579.
Gavaler, C. (2017). Superhero comics. London: Bloomsbury.
Gavaler, C., & Beavers, L. A. (2018). Clarifying closure. Journal of Graphic Novels and Comics, 1–30. Retrieved from <https://doi.org/10.1080/21504857.2018.1540441>.
Goldberg, A. E. (1995). Constructions: a Construction Grammar approach to argument structure. Chicago, IL: University of Chicago Press.
Goldberg, A. E., & Jackendoff, R. (2004). The English resultative as a family of constructions. Language, 80(3), 532568.
Graesser, A. C., Singer, M., & Trabasso, T. (1994). Constructing inferences during narrative text comprehension. Psychological Review, 101(3), 371395.
Grice, H. P. (1967). William James lectures: logic and conversation. In Cole, P. & Morgan, J. L. (Eds.), 1975: Syntax and Semantics 3 (pp. 4158). New York: Academic Press.
Groensteen, T. (2007). The system of comics (trans. Beaty, B. & Nguyen, N.). Jackson, MS: University of Mississippi Press.
Guynes, S. A. (2014). Four-color sound: a Peircean semiotics of comic book onomatopoeia. Public Journal of Semiotics, 6(1), 5872.
Herman, D. (2009). Basic elements of narrative. Chichester: Wiley-Blackwell.
Hutson, J. P., Magliano, J., & Loschky, L. C. (2018). Understanding moment-to-moment processing of visual narratives. Cognitive Science, 42(8), 29993033.
Jackendoff, R. (1997). The architecture of the language faculty. Cambridge, MA: MIT Press.
Jackendoff, R. (2002). Foundations of language: brain, meaning, grammar, evolution. Oxford: Oxford University Press.
Jackendoff, R. (2007). Language, consciousness, culture: essays on mental structure (Jean Nicod lectures). Cambridge, MA: MIT Press.
Joue, G., Boven, L., Willmes, K., Evola, V., Demenescu, L. R., Hassemer, J., … Habel, U. (2018). Handling or being the concept: an fMRI study on metonymy representations in coverbal gestures. Neuropsychologia, 109, 232244.
Kosara, R. (2017). An argument structure for data stories. Paper presented at the Proceedings of the Eurographics Conference on Visualization (EuroVis) 2017, Barcelona, Spain (pp. 3135). doi:10.2312/eurovisshort.20171129
Kowalewski, H. (2018). Heart is for love: cognitive salience and visual metonymies in comics. The Comics Grid: Journal of Comics Scholarship, 8, doi:http://doi.org/10.16995/cg.117
Kraft, R. N., Cantor, P., & Gottdiener, C. (1991). The coherence of visual narratives. Communication Research, 18(5), 601616.
Kukkonen, K. (2008). Beyond language: metaphor and metonymy in comics storytelling. English Language Notes, 46(2), 8998.
Kuperberg, G. R., Paczynski, M., & Ditman, T. (2011). Establishing causal coherence across sentences: an ERP study. Journal of Cognitive Neuroscience, 23(5), 12301246.
Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago, IL: University of Chicago Press.
Lasher, M. D. (1981). The cognitive representation of an event involving human motion. Cognitive Psychology, 13(3), 391406.
Loschky, L. C., Hutson, J. P., Smith, M. E., Smith, T. J., & Magliano, J. (2018). Viewing static visual narratives through the lens of the Scene Perception and Event Comprehension Theory (SPECT). In Dunst, A., Laubrock, J., & Wildfeuer, J. (Eds.), Empirical comics research: digital, multimodal, and cognitive methods (pp. 217238). London: Routledge.
Magliano, J. P., Dijkstra, K., & Zwaan, R. A. (1996). Generating predictive inferences while viewing a movie. Discourse Processes, 22, 199224.
Magliano, J. P., Kopp, K., Higgs, K., & Rapp, D. N. (2017). Filling in the gaps: memory implications for inferring missing content in graphic narratives. Discourse Processes, 54(8), 569582.
Magliano, J. P., Larson, A. M., Higgs, K., & Loschky, L. C. (2015). The relative roles of visuospatial and linguistic working memory systems in generating inferences during visual narrative comprehension. Memory & Cognition, 44(2), 207219.
Magliano, J. P., Taylor, H., & Kim, H.-J. J. (2005). When goals collide: monitoring the goals of multiple characters. Memory and Cognition, 33(8), 13571367.
Manfredi, M., Cohn, N., & Kutas, M. (2017). When a hit sounds like a kiss: an electrophysiological exploration of semantic processing in visual narrative. Brain and Language, 169, 2838.
McCloud, S. (1993). Understanding comics: the invisible art. New York: Harper Collins.
McKoon, G., & Ratcliff, R. (1986). Inferences about predictable events. Journal of Experimental Psychology: Learning, Memory, and Cognition, 12(1), 8291.
McNamara, D. S., & Magliano, J. (2009). Toward a comprehensive model of comprehension. Psychology of Learning and Motivation, 51, 297384.
Metusalem, R., Kutas, M., Urbach, T. P., Hare, M., McRae, K., & Elman, J. L. (2012). Generalized event knowledge activation during online sentence comprehension. Journal of Memory and Language, 66(4), 545567.
Postema, B. (2013). Narrative structure in comics: making sense of fragments. Rochester: RIT Press.
Pratha, N. K., Avunjian, N., & Cohn, N. (2016). Pow, punch, pika, and chu: the structure of sound effects in genres of American comics and Japanese manga. Multimodal Communication, 5(2), 93109.
Pustejovsky, J. (1991). The syntax of event structure. Cognition, 41, 4781.
Saraceni, M. (2000). Language beyond language: comics as verbo-visual texts . Unpublished doctoral dissertation, University of Nottingham.
Saraceni, M. (2001). Relatedness: aspects of textual connectivity in comics. In Baetens, J. (Ed.), The graphic novel (pp. 167179). Leuven: Leuven University Press.
Singer, M., Halldorson, M., Lear, J. C., & Andrusiak, P. (1992). Validation of causal bridging inferences in discourse understanding. Journal of Memory and Language, 31(4), 507524.
Stainbrook, E. J. (2003). Reading comics: a theoretical analysis of textuality and discourse in the comics medium. Unpublished doctoral dissertation, Indiana University of Pennsylvania.
Strickland, B., & Keil, F. (2011). Event completion: event based inferences distort memory in a matter of seconds. Cognition, 121(3), 409415.
Trabasso, T. (2005). Goal plans of action and inferences during comprehension of narratives. Discourse Processes, 39(2), 129164.
Yus, F. (2008). Inferring from comics: a multi-stage account. Quaderns de Filologia. Estudis de Comunicacio, 3, 223249.
Zacks, J. M., & Tversky, B. (2001). Event structure in perception and conception. Psychological Bulletin, 127(1), 321.
Zwaan, R. A. (2004). The immersed experiencer: toward an embodied theory of language comprehension. In Ross, B. H. (Ed.), The psychology of learning and motivation (Vol. 44, pp. 3562). New York: Academic Press.
Zwaan, R. A., & Radvansky, G. A. (1998). Situation models in language comprehension and memory. Psychological Bulletin, 123(2), 162185.
Zwaan, R. A., & van Oostendorp, H. (1993). Do readers construct spatial representations in naturalistic story comprehension? Discourse Processes, 16, 125143.

Keywords

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed