Simple evaluation tool - the card sort

One of my favorite evaluation tools in the Card Sort. To understand visitors' perspective on something, hand them a stack of cards, each with a 1-2 word description of the topic you have in mind. Ask visitors to pull out the words that best and least describe the topic (limit them to 3-4; many people will want to pull out 8 or 9 cards). Then follow up and ask them why they chose the cards they did.

When you analyze the data, it's interested to add them the numbers for what cards were chosen the most, and which cards were ignored the most. Add to that some really great qualitative information about why people made their selections. Often, you find that people have different interpretations of the words than you, as the researcher, had.

Here's an example. Let's say you want to know how people perceive the library's current collection of fiction. Put together a list of 10-15 words that could possibly describe the collection - and don't be afraid to include some negative words (good selection, new materials, lots of options, worn out, not relevant to me, etc). The words you choose are important. Keep them simple, so people can process them quickly. But be specific and even a bit daring - that will bring out interesting comments and discussion. Above all, make sure they are relevant to what you want to know about. Test your words with a few people from the organization. Then try the list out on a few patrons before going live with the study.

As a rule of thumb, when you go live, ask as many people to do the card sort as it takes until you feel like answers are getting redundant. If you must have a number, I would recommend 30 people as a minimum.

To analyze results, tally up the total of "best" and "worst" hits each word got. Then compare what people said as their reasons why they chose those words. What patterns or themes do you see? One note: the more accurately you transcribe people's responses to why they chose words, the better your qualitative analysis will be. Resist the temptation to summarize people's statements when collecting the data. Try to write down what they say as close to word-for-word as possible. It's tough, but worth it. You don't want to add your layer of interpretation until all the data is collected.