Skilful Minds

At the end of February I co-presented at STLUX’11 with Dave Gray. Our presentation was called Exploring the Usefulness of Chartjunk. The collaboration behind the presentation started as an exchange between the two of us on Twitter regarding whether the whole concept of Chartjunk is a myth. Over a series of conversations about recent research on the relative importance of visual embellishment in how people remember and understand data, I suggested to Dave that we develop a presentation around the topic. Dave agreed and suggested that we also build the presentation in a manner that engaged the audience to share their thinking about the issues involved. 

Dave and I designed the presentation as a simulated debate between the pre-eminent critic of Chartjunk (actually the design theorist who formulated the concept) Edward Tufte and Nigel Holmes, an illustrator and the former Graphics Director at Time Magazine. Homes is well known for his use of visual embellishment in designing graphics that tell stories about data relationships. We designed the presentation around four graphic displays, two by Tufte and two by Holmes. We discussed the graphics and then asked members of the audience to consider each graphic on four dimensions.

1. ease of understanding what it is about
2. ease of understanding the categories and values displayed
3. ease of seeing the basic trend
4. ease of determining whether it conveys a message

I designed a simple survey that allowed us to gather data on those four dimensions using an implicit five point scale, eliciting participation and dialogue with the audience at the same time.  The graphic below provides a view of the survey’s instructions. The participants seemed overall to enjoy the approach and the evaluations confirmed the impression.

 We drew the four dimensions from research done in 2010 by members of the Interaction Lab at the University of Saskatchewan. The Interaction Lab researchers designed an experimental study to test two basic questions: “first, whether visual embellishments do in fact cause comprehension problems; and second, whether the embellishments may provide additional information that is valuable for the reader.” I’m not going to detail the methodology used, however the researchers asked four questions to participants in the research as they reviewed graphics by Holmes with a great deal of visual embellishment or the same graphics after applying the data-ink ratio used by Tufte.

Q1–Subject: ‘What is the chart is about?‘ ‘Tell me about the basic subject of the chart.’
Q2–Values: ‘What are the displayed categories and values?‘ ‘Tell me how the chart is organized and any relevant values.’
Q3–Trend: ‘What is the basic trend of the graph?‘ ‘Tell me whether the chart shows any changes and describe these changes.’ (Note that this question was not relevant for pie charts.)
Q4–Value Message: ‘Is the author trying to communicate some message through the chart?‘ ‘Is the author trying to get across a specific point or is he or she merely presenting objective information?’

We reported on the major findings of the research team to the audience as follows:

1. There was no significant difference between plain and image charts for interactive interpretation accuracy (i.e., when the charts were visible).
2. There was also no significant difference in recall accuracy after a five-minute gap.
3. After a long-term gap (2-3 weeks), recall of both the chart topic and the details (categories and trend) was significantly better for Holmes charts.
4. Participants saw value messages in the Holmes charts significantly more often than in the plain charts.
5. Participants found the Holmes charts more attractive, most enjoyed them, and found that they were easiest and fastest to remember.

 At the end of the presentation, after we covered the research study findings, we then asked the participants to list as many of the graphics from the four discussed earlier and to rate each along the four dimensions. As we broke up the session a few participants asked if we could share the findings from the participative survey.

I agreed to post the results and I am now getting around to it.

Read the rest of this entry »