On Friday, April 25th, the Informative Gamers group at the University of Michigan hosted their first research jam. I’m the alum advisor, so naturally I had a few things to say…
JJ Jang, the group’s founder, has been in close contact with me since the group’s inception. We both thought it would be super fun and super helpful if we could start doing these “research jams” — quick-and-dirty usability studies on a design problem that I give — to get students some out of class experience with a researcher who’s both an alum and working in games.
The session started with an intro about me — how I had gotten to where I am today, what we were going to do in the jam, as well as going through a brief list of questions the students had asked me beforehand. Ultimately we wanted to give students a chance to experience how it felt to observe a brand new player go through a game that requires a lot of mental effort.
In school, you’re always taught the traditional things — thinkaloud, do X, do Y (in order), etc. We had students break into pairs, taking turns being researcher/participant. I know it was for the students, but I learned some stuff too. Here we go:
1. Teaching someone how to be a user researcher in 2 hours is nigh impossible.
It would have been a much better experience if everyone had taken at least one class on methods/research in either the psych or SI departments. A couple of students were essentially forcing their will on participants, which is um… yeah. Not right.
I thought my instructions were clear but unfortunately a couple students chose not to listen. That said, MOST of the students there were fantastic. After 2 games of TOME, they were able to provide me some findings (and even recommendations!) that I have been seeing in various studies I’ve conducted in the past several weeks. My hats off to them — good eyes and I appreciate you putting on your critical thinking caps and joining us last night. I definitely had a lot of fun. Please join us again 🙂
2. Zero tolerance for immaturity.
If you’re at the jam, my assumption is that you want to learn, network with me, participate, or some combination of the three. There were a few people whose names I don’t know (but would definitely talk to privately if I knew) who just weren’t “present”. I don’t wanna sound like a tool, but c’mon… I took 2 hours out of my stupidly busy work day to help you learn! To have fun with you! To allow you to pick my brain! Network with someone in the industry!
All these things… and you know what’s funny? Only 1 person sent me any kind of message afterwards. My hats off to you, good sir… you know who you are 😉 (LinkedIn request)
Please show some respect next time. If you don’t want to take the research jam seriously, then leave; I don’t want you there. You’re affecting other folks who want to learn (trust me, I witnessed it).
3. Future research problems.
So for this one, I gave everyone in the group a TOME beta key (the game’s closed to the public). No one had seen it… which was interesting. But also not ideal. The researcher should typically have a great grasp of what to look for and have some experience in the product.
The study was meant to dig at the new player experience. There’s currently no tutorial in the live game and I wanted to see how well players would react to the game by only have a brief MOBA explanation that I delivered before they started. The most obvious thing is that the person who acted as the participant second had a distinct advantage — they already got a chance to see the game and understand a few things they otherwise wouldn’t have. It wasn’t a true first player experience (obviously).
This deviated from our original plan of having the second player play a different KIXEYE game. Not sure why there was an audible called in the nth hour, but so it goes (I haven’t followed up with JJ about it either). It’s cool. This was a good test run.
In the future, it would be awesome to have everyone bring a laptop with their most favorite PC game installed. Even mobile devices could work, since we’re not really recording or anything, just observing and taking notes. Same thing as last time — pair up, take turns being researcher/participant — except this time folks will be handing off a game they have extensive experience in. Hopefully this will lead to more interesting results and will cause students to really have to restrain themselves in telling a new player what to do (big UX research no no). We’ll see.
4. Nothing is perfect.
“What happens in the real world, though?” That’s what I wanted to know when I was a student. While I don’t know exactly what drove specific students to attend, I hope some of them were hoping for this real world insight as well. I don’t feel like I did a good job at that. Maybe I did, maybe I didn’t. I, personally, need both good and bad feedback in order to be a good human being.
In the real world, all participants are different and all researchers are different. One’s “style” needs to adjust properly in order to get the maximum amount of rich qualitative data out of the participant without tainted their answers by leading them on or inserting your own biases. There isn’t always one way to do something. There isn’t just “going by the book”, like you do in school.
This is hard to do, let alone teach in a few hours. I had a great time, and hope we can make the next session run more smoothly and only attract people who want to get something out of it.