How my dislike for collating research notes led me to Google Forms
I’ve been doing a lot of user experience research lately — at least one usability test or foundational study each month. And to tell the truth, I have a love-hate relationship with research.
I love that our clients value their customers enough to put them at the center of their product life cycles. I love the impact that iterative user research has on clients’ products, teams, and company cultures.
But, the pleasure of putting users at the center of the design process is spoiled somewhat by the unnecessary frustration and menial work involved in organizing the data and sussing out its relevance. I dread dealing with session notes. I think anyone who has moderated user research can identify–at the end of a study, synthesizing notes, sifting through the details, tallying task success counts, and pulling quotes is time consuming and tedious. I’ve tried taking notes on paper — it’s easy to pull out themes because of my handwritten stars or circles around important notes — but paging through handwritten notes in moderator workbooks for 10+ sessions is mind numbing. I’ve tried typing notes in a doc, but am left with pages of uniformly typed notes that make it difficult to pull out findings related to specific tasks or themes at a glance.
About 3 months ago I had a breakthrough while creating a Google Form for a participant screening survey. When the respondent clicked Submit on the form, each field would be written to a column in a Google Sheet. Wait!?! What if I could set up a research moderator script where each test scenario is a form, and within each form, the tasks, moderator questions, and pass/fail indicators were questions? Each form could link to a Google Sheet…. And voila! The data would be collated for me.
I gave it a try. In the first study, I made a few mistakes.
My first form used structured response choices, for example:
I immediately found that most participants didn’t choose one task initiation method, and the checkboxes made it difficult for me to later analyze the order in which they tried the various methods. It was more useful to provide a text entry field, so I could quickly type the task initiation attempts in the order in which the participant tried them.
I tried putting more than one test scenario in a form. But if I needed to vary the order or skip a scenario, it was difficult to jump around within the form. Creating separate forms for each test scenario proved to be more flexible.
I didn’t include any general free-form entry fields at the end of each task. Observers of the tests liked using the form, but if the discussion meandered, observers were unsure where to record their notes on those un-scripted conversations. I remedied this by adding an “Additional notes” question at the end of each scenario’s form.
After trial and error, I’ve evolved my Google Forms to be a time-saving approach to note-taking, not only during research sessions but also with analysis afterward.
- The first form in each session is a moderator checklist, where I can click checkboxes after I complete each pre-session requirement; for example, reviewing the NDA, explaining participants’ rights, explaining talk-aloud protocol, and starting the recording.
- Each test scenario or discussion topic is a separate Google Form, which is linked to a separate tab in one Google Sheet. The result is a single Google Sheet, with a separate tab for each scenario/topic.
- Each question or topic becomes a column in the sheet, making it easy to scan columns and tally task successes, as well as common themes across participants, and quotes related to a topic.
- I send observers links to the forms, so if they are inclined to take notes during a session, their notes will automatically feed into the Google Sheet along with mine.
Example Google Form linked to Google Sheet
Using this system, I have been able to shave time off of the data processing period. Days, in fact. Not only does this make my job more enjoyable; it results in faster turnaround time for our clients on the analysis and reports after a study. The improved efficiency, in turn, lowers project costs as well.
Thanks to new methods and technologies, it’s easier than ever to include your users in product design. Let us know if we can help.