Card Sorting in 2018: Explanation, Example, and Online Tools

In a nutshell, Information Architecture is about grouping and labeling the information. How to do it? When in trouble, use the card sorting.

Let’s apply it to real life task: sorting music moods for our music project, Fugue.

Music Moods?

Right. Preparing our music library, we had to classify the music moods into several categories. I felt like the Thayer Mood Model is a good start.

It looks great, but will our usability testing confirm it? Thayer Mood Classification
Image credits: Michael Nuzzolo, Bhar, 2014.

Questions to Answer

Will our usability testing confirm this model?
Do people intuitively classify our music genres into the same categories?
Will our usability testing improve it?

Data

The list of tags that our music producers brought:

Action
Aggressive
Angry
Beautiful
Calm, etc.

Card Sorting: The Method

We’ll use card sorting. In short, we put each music genre on a separate card. We ask people to sort these cards into the piles in some logical way and name each pile.

Then, after the testing is ready and the participants went away, you sit on the floor late in the night. The goal is to figure out something before the cleaners throw this garbage away in the morning.

The original way of conducting this research involves the actual, physical paper cards. The method is so old that the literature refers to the paper version of this test.

Paper card sorting. Image from the book by Donna Spencer. I didn’t read it, but I’ve read several other books by Rosenfeld Media, and they were all great.

I did it the old paper way, and let me tell you something: it sucks. It requires an extreme amount of manual work even with a few participants. Let’s use the software.

Choosing Software

Card sorting software is not the kind of flashy software that goes ICO or receives funding from YCombinator. Beautiful, contemporary design, responsive pages, and fun copy — that’s not something you’ll find here. There are few services, and they look quite old and dusty.

I’ve used Optimal Workshop before. It was ok, but they rose the prices around 10x. And let me put it this way: it didn’t get 10x better.

Some of the alternatives are too ugly to be used by a service for designers like us, so I chose Usabilititest. Right, the name is misspelled. I wonder if they misspelled it when registering a domain, and decided it’s not worth spending another 8 dollars for the right spelled one.

Pros and Cons of Usabilititest

Usabilititest turned to be a good choice.

It has a relatively OK design. In the world of user interfaces, it’s not a beauty like Scarlett Johansen, but neither Woody Allen.

Usabilititest looks ok. It’s not the cream of the cream, but you won’t be ashamed either.

My biggest cons are related to the interpretation of the results. For example, the page doesn’t use the full width of the screen for a multi-column table.

A usability tool with lousy usability: the table width is fixed, so 50% of the vertical space is wasted. You have to scroll horizontally this multi-column table within a tiny iframe.

Probably the worst of all, it mishandles the custom group titles. If one user created a group called “Intense” and another one created the group called “Intense,” this software creates two columns instead of one.

Two different columns. I’d prefer if the software joins them into a single one.

We’ve created two card sorting tests:

Moods of music (Calm, Romantic)
Usages of music (Internet, TV, etc.)

Creating a test is straightforward. Mostly, you add the cards and call it a day.

Cards

The software also suggests defining some groups. It makes sense, taking into account the software does little to join the groups, even equally named ones. I’m not even telling about clustering the groups that are called differently.

In the end, you’ll get a link to the public page. You’re ready to send it to the participants.

Recruiting Participants

We have the wonderful community at Icons8. They saved us on many occasions. We’ve sent a mailing asking for help and received around 500 responses for each of the tests in less than 24 hours.

The mailing we’ve sent to our participants (the whole message). Many thanks to those who participated!

Another way of recruiting users is Upwork. I used to hire the users there. Just explain the task beforehand and pay the person’s usual hourly rate.

Results

As mentioned above, the reports are not usable. Therefore, let’s:

Export them to CSV
Import to Google Spreadsheets
Group similar columns.

Card sorting results (Google Spreadsheets table). Green are the categories we offered users; yellow is the categories that users suggested. Please note that greens have much higher numbers than the yellow ones; that’s ok.

The final step: let us group the moods that scored some significant votes into the categories. Ta-dam!

Here we go with the final sort of the moods. Sic! Some of the moods fall into multiple categories. That’s fine. Different people look for the same things in different places.

And the final result is on our website:

The final result on our website.

Meet Fugue, ladies and gentlemen.

About the author: Ivan Braun, UX designer, founder of Icons8
Title image: Moose stock photos

Check Fugue, huge collection of free music for videos and games, learn about the myths on usability testing and read a case study on Airbnb redesign for user goals