I would like to discuss an important theory born out of cognitive science that, by chance, offers surprisingly good news for visual communication design. This theory is called, dual-coding theory.
Picture that you are driving down the highway, traffic is heavy, and you are maneuvering around large trucks as you pass through slower traffic. As you glance in the rear-view mirror, you notice an aggressive driver coming up behind you at high speed, headlights flashing, and you quickly move aside. The traffic is active – everything is in motion as you speed ahead. Luckily, you are not alone. The car stereo is playing your favourite music. Every time, the chorus of the song plays, you shout out “Same as it ever was – same as it ever was”, and just as you begin to sing the next verse, watch out – slow down quickly – you are boxed in and coming up fast on a slow truck, blindly merging into your lane. You hit the brakes as you sing the next line!
Something is peculiar about this scene – and that is the way your brain is processing information. Although you are almost overloaded with visual sensations, and the traffic is consuming your attention, your brain is processing linguistic input– a song – simultaneously, with no effort. In fact, these two streams of mental processing aren’t even connected.
Dual coding theory assumes that humans have two main information processing systems, one for language, such as spoken, written, mathematical or computer languages, and one for non-verbal signs, such as visual experiences, photographs, illustrations, graphics and maps[1].
A representation of the brain’s system of dual coding is shown in figure 1, showing the nonverbal processing system employing imagens, and the verbal processing system employing logogens.
Figure 1
(adapted from Paivio, 1987, p. 67)
According to Allan Paivio, dual-coding theory’s creator:
Human cognition is unique in that it has become specialized for dealing simultaneously with language and with nonverbal objects and events. Moreover, the language system is peculiar in that it deals directly with linguistic input and output (in the form of speech or writing) while at the same time serving a symbolic function to nonverbal objects, events and behaviours. Any representational theory must accommodate this dual functionality[2].
Paivio’s point here, is that, although our brains have one system to process objects, images and events, and one for language – language processing can also refer to objects, images, and events, and this, in turn, further enriches our mental experiences. Paivio claims that “any representational theory must accommodate this dual functionality”. So visual communications design theory should accommodate this, right? Well, has it?
Although our example of driving through busy traffic suggested that we can easily process language while simultaneously processing images and events, imagine what happens when we simultaneously process language (in the form of text) and images that are expertly designed to interact with each other, and consider dual-coding theory. According to the findings of cognitive scientists, instructional learning is greatly improved, as well as memory. Isn’t this precisely what visual communication designers want to accomplish?
If both the visual and linguistic systems comprehend and process thoughts, and both systems interact in learning and memory, then dual coding may increase the validity of multimodal* visual communication in supporting visual memory, information transfer and learning[3].
So, it seems that dual-coding theory validates a visual communication designer’s approach to the transfer of information, provided that designers choreograph their use of images and textual elements in a strategic way, conceived to function within the brain’s dual-coding mechanism. But before visual communication designers improve communication through the pictorial and linguistic interplay within the brain, we must adopt the necessary concepts from psychology and cognitive science as conceptual foundations of visual communication design. Then we must research their behaviour, and learn to make them work in practice.
By managing the brain’s dual-coding, visual communication designers can increase their impact. And who would be better suited to choreograph the dance of image and text to achieve higher meaning, than visual communication designers?
*Multimodal communications: Communicating through different means simultaneously, such as by using numbers, words, graphics and pictures together (as in a map).
[1] Paivio, A. (1986). Mental representations: A dual coding approach. New York, NY: Oxford University Press.
[2] Paivio, A. (1986). Mental representations: A dual coding approach, p. 53. New York, NY: Oxford University Press.
[3] Sternberg, R. J. (2006). Cognitive psychology (4th ed.), p. 261. Belmont, CA: Thomson Wadsworth.
Share this
Posted in theory. Leave a comment.
Hi David.
Are your references listed somewhere, or are they there and I have missed them?
Thanks for the question Sue. I am currently trying to establish a balance between academic rigour and a casual writing approach. In most cases where I think citations are useful, I have included them in the text, although sometimes I have simply mentioned the source casually in the narrative (such as Charles Osgood in post 2, which is actually Schramm, 1954). I am very conscious that this medium is more casual than a research paper, while I feel it is also important to mention genuine sources of ideas.
So that leaves me with 2 questions. 1) What do you think? In your opinion, should I keep it casual or write it like a paper? Or possibly list references in APA at the end of each post?
And 2) Is there anything specific that you are interested in that I can clarify? I would be happy to.
Again, thanks.