Archive for December 9, 2010

I talked about this in class, but I want to provide a written explanation of how to structure the Results section of your reports.

The results section is the most important one. You spent a lot of time and effort collecting data, and now is the time to analyze and present it. The results section shows off your work. Use and present all the data you collected, don’t keep it secret!

The results section should progress from broad to more and more specific: The first part should present results across tasks, and the second part, results for each task. Then, include in the Appendix the data for each user. So we move from an aggregate of data to individual data points.

Overall Results Across Tasks

This sub-section presents data that enables comparisons across tasks. Compare the tasks on each metric, and show averages across tasks. This is broad-level data that sets expectations for what’s to come: Which was the easiest task? Which was the most difficult? How does expected difficulty compare to actual difficulty across tasks? And so on. Comparisons are best illustrated with bar graphs.

Results for Each Task

This sub-section presents the metrics for each one of the tasks, and enables comparisons across individual users. This is where we begin to have access to individual-level data. Which participant completed the task fastest? Which one took the most time?

Within the sub-sub-section for each task, present the quantitative and qualitative data for all the metrics you collected, and discuss anything you know from observations that might explain the results. Include quotations that illustrate the main points you extracted from the qualitative data.

This blog post has some charts that show metrics per task, and then overall metrics at the bottom. Take a look and note the difference.


The Appendix presents the results at an even more granular level. Present the results for each participant. So, your appendix will have 5 sections, one for each participant. Start with the demographics (but withdraw information that may compromise the participant’s anonymity), then move on and present the participant’s results for the pre-session questionnaire, metrics for each task (the qualitative data can be a summary with 1-2 quotes, not a full transcription), and the post-session questionnaire.

If you follow this progression, you give your reader different views of the data, starting with a broader picture and moving on to individual data points.

The usability report templates I pointed you to, chapter 8 in the Tullis & Albert book, and the class presentations give you options for presenting this information. See also sample reports by Tullis that give you more ideas about what how to present information.

Bonus link: Nielsen’s famous article Why You Only Need to Test with Five Users. You can cite it to back up the number of participants you tested with.