You may have noticed I love writing about things we all know we should do or work on. Today, we’ll be discussing play testing!
We all know that we should test our games because every blog, podcast, YouTube channel, and forum out there reminds us of this very fact. With so many of us starting fresh in the industry (no prior time developing games) and lacking access to experienced play testers, we often leave a lot of useful information on the table. Mr. Stegmaier gives a perfect example of what I’m talking about in his post, “There is No Perfect Pickle.”
“If you’re like most people, your answer to the question at the beginning of this entry (“If you’re a coffee drinker, do you prefer a dark, rich, hearty roast?”) was yes. However, in truth, the vast majority of people prefer milk, weak, slightly sweet coffee.”
The simple fact is that we don’t know what we don’t know. For most of us, we just don’t know how to break down our experience with a game in a useful way. You may be working with very smart and analytical people, but unless they’ve tried before they don’t know how to tell you that this resource collecting mechanic is overpowered. Worst yet, they could be friends or family who are worried about hurting your feelings. My circle knows I want them to be completely honest, but they don’t always know the difference between helpful criticisms and hurtful opinions – so they sometimes don’t say anything.
Therefore, the question becomes how to get the useful information out of these helpers who don’t know how to give feedback. Ask the right questions! We all learn how to do this, eventually. We make a major change to a game driving mechanic, so we ask a lot of questions about it. Or we make a deck building game, so we take notes and ask questions to ensure players felt like they had options in the choices they made and routes they took. But I’m here to help you figure this out without breaking 4 of your own games and testing 3 – 5 from other people.
Check out this awesome feedback form from Unpub‘s website. It is a free, recommended asset they offer to indie developers. As Mr. Stegmaier said in his “Don’t Copy and Paste,” lesson, don’t just copy this form. The information you gain by dropping this in front of players may be better than asking them, “Did you like it?” But, as I mentioned here, better just means less bad. You know your game. You’ve already played the heck out of it and know some of the holes and weaknesses. So, talk to the testers about those problems. Now, starting a conversation about your game and the tester’s experience will generate good feedback. However, guiding the conversation with smart, directed questions will garner the constructive criticisms you truly want.
The easiest method I’ve found is to bring up an aspect of the game and ask a specific question. For instance: “Back when you guys were building the map, did that seem overly complicated? Do you think it slowed the game down unnecessarily?” Or, “I noticed you went straight for the Bonus cards but didn’t use your character’s abilities, much. Did you not like the character?” Then, give each person time to respond but don’t demand that they comment. In the last question the player’s reasoning could be that they felt the character was too under-powered or they just preferred the random chaos of the Bonus cards. Don’t forget to give them room to occasionally change the subject and / or circle back to a topic from earlier – sometimes it takes a few minutes to figure out how to articulate a feeling. When you get them thinking about specifics and then give them room to converse with you, you’ll both be surprised by the insightful critiques you can come away with.
The biggest key here to continued help is to accept everything as if it is golden advice. Are you going to actually use all of the advice and criticisms? NO. They don’t need to know that, though. Your testers DO need to know that you appreciate their time and thoughts. So, don’t correct, cringe, or chide. Just smile, take notes, and thank them.
Thanks for sticking around for my ninth lesson and letting me talk at you. How do you overcome the generic, “I liked it”? What questions do you find the most helpful to ask your testers?
*Edit: I recently found this extremely helpful article addressing much the same information. I hope it helps: 10 Insightful Playtest Questions.
Thanks for linking to my blog, Derik. I really like your point about asking specific questions to playtesters. I’m curious, do you think those questions are more effective if they’re open-ended or if they lead to a yes or no answer. For example:
“Back when you guys were building the map, did that seem overly complicated?” (this question allows them to answer yes or no, but they’re also likely to say more)
vs
“Back when you guys were building the map, what were your favorite and least favorite aspects of that element of the game?” (more-open ended, but you might get silence if they have no thoughts about that subject)
I’m always looking for better ways to talk to playtesters, so I’m curious about your opinion.
Thanks for checking in, Jamey. I do appreciate it 😀
It has been my experience (with analyzing games and working “the real world”) when dealing with people who aren’t used to processing their recent experience in such a critical way, asking such an open ended question as your 2nd example will receive the silence you predicted.
I’ve done it too. When you don’t have enough exposure to know if your experience could have been better, you typically don’t have much to say. After the conversation gets rolling, testers listening to your questions and feedback from others have a better understanding of the kind of information you are looking for, and are more inclined to give the criticisms and feedback you need without coaxing.
Granted, this only applies when you are able to carry on a conversation with the tester. Although, for surveys you could similarly encourage descriptive analysis by listing specific examples of helpful adjectives while leaving chances for open comments.
Derik: I like that approach–it makes a lot of sense. Get people talking with easier questions and then start asking more open-ended questions.
During the blind playtesting process for Tuscany, one group did something I thought was really cool. After playing the game, one of the group members put their iPhone on the table, opened a voice recorder app, and then started a conversation about how the game went. At first, no one said much, but after about 30 seconds, everyone was talking about various aspects of the game. It was a very open flow of information, and the conversation evolved quite a bit and generated some neat ideas. They later sent the recording to me. It was much more helpful than if one person had typed up what people thought or if people filled out a survey.
That is an outstanding idea. I’ve wondered about that myself a few times. “How do you get good feedback from a remote group of testers.” Apparently, just running a recording on someone’s phone fits the bill 🙂 Thanks for the input, Jamey.