Time: 7 months
Methods: exploratory research (semi-structured interviews and observations), data synthesis (affinity diagramming), storyboarding, prototyping
The challenges answered with Endeavor were:
- How might we engage struggling readers with instructional software on a subject that no longer motivates them?
- How might we leverage software for students in a way that allows their teachers to personalize instruction?
Endeavor is a high-fidelity prototype that our team created for Houghton Mifflin Harcourt, one of the largest U.S. publishers of print and digital educational materials. Our team of five CMU master’s students was contracted to investigate issues that students and teachers had with a personalized reading product called System 44 (named after the 44 sounds in English), and devise a solution.
Endeavor is an interactive narrative activity for struggling readers. Students read passages and choose their path through an adventure, promoting a sense of autonomy and personalization. Comprehension and decoding assessments are embedded within the narrative and are designed to quickly identify the types and degrees of errors students make.
As our team’s Research Lead, my responsibilities were to guide two stages: exploratory research and iterative design research, with an eye towards both usability and learnability. As such, some of my duties were to delegate research tasks in a way that maximized my team members’ strengths, and plan and carry out focused research, including literature reviews, interviews, observations and user tests. With the data in hand, our team worked together to synthesize it and use it to make design.
After meeting with our client and getting acquainted with the software, we contacted local teachers using the software and traveled to their schools.
- 7 Class sessions observed
- 16 Students interviewed
- 5 Teachers interviewed
We used affinity diagrams to draw out themes from our notes. Tip: when affinity diagramming, it may be a good idea not to talk to each other, and to make it a norm that people can move notes around at any time, to minimize the amount of influence one person’s ideas have.
Using our interview data, we made personas, to represent “classes” of students and teachers.
Choosing An Idea
With input from our client, we decided to iteratively design an interactive storytelling activity.
We brainstormed several ideas. They stemmed from insights, which informed the personas as well. Here are some:
We created several different prototypes of Endeavor over the following weeks, testing them with students and getting feedback from teachers and our client.
Our first prototype was built with Twine, an open-source tool for telling interactive, nonlinear stories. It was a perfect piece of technology for our goals. Essentially, we built a choose-your-own-adventure story, where students read text, and then are asked to choose a direction for the story to go. At certain points, there would be a challenge to do an activity similar to one on System 44.
The Danger of Chocolate-covered Broccoli
We got feedback from the initial user tests, saying that the challenges (like the one above) felt separate from the rest of the story. They were warning us against what is sometimes called “Chocolate Covered Broccoli: boring educational activities (the broccoli) that you have to do in order to play fun games (the chocolate).”
- We decided to lay out the plot of a story with several “scenes” where there would be challenges, instead of writing a story first and fitting the challenges in after.
- Brainstorm different ways that the students’ correct/incorrect answers would affect the story. For example, in our story, the students hide their gold from a pirate. If they fail to unlock a combination lock with the appropriate word, they lose gold.
- Leverage #2 to make the story a game-based assessment. By carefully creating questions and answer choices, the software can “tell” exactly where the student is struggling — word comprehension, or specific letter-sound pairs.
- Take it all the way — if we want to convince students that this is a real game, then failing challenges needs to result in a game over. This was controversial among the team. We decided to make it so the only way students get a game over is if students failed a challenge that was so easy, they shouldn’t get it wrong unless they were guessing or knew none of the sounds in the word.
We hope that our research and prototype design and findings help make the next version of System 44 a better overall learning experience, and we appreciate the opportunity to learn skills that we will use in our careers.