Time: 7 months

Team: Chenxin Wang, Julia Ridley, Lucy Yang, Zach Mineroff

Methods: exploratory research (semi-structured interviews and observations), data synthesis (affinity diagramming), storyboarding, prototyping 

Tools: HTML, CSS, Javascript, Twine

 

Try Endeavor!

 

 

Summary

The challenges answered with Endeavor were:

  1. How might we engage struggling readers with instructional software on a subject that no longer motivates them?
  2. How might we leverage software for students in a way that allows their teachers to personalize instruction?

Endeavor is a high-fidelity prototype that our team created for Houghton Mifflin Harcourt, one of the largest U.S. publishers of print and digital educational materials. Our team of five CMU master’s students was contracted to investigate issues that students and teachers had with a personalized reading product called System 44 (named after the 44 sounds in English), and devise a solution.

Endeavor is an interactive narrative activity for struggling readers. Students read passages and choose their path through an adventure, promoting a sense of autonomy and personalization. Comprehension and decoding assessments are embedded within the narrative and are designed to quickly identify the types and degrees of errors students make.

My role

As our team’s Research Lead, my responsibilities were to guide two stages: exploratory research and iterative design research, with an eye towards both usability and learnability.  As such, some of my duties were to delegate research tasks in a way that maximized my team members’ strengths, and plan and carry out focused research, including literature reviews, interviews, observations and user tests.  With the data in hand, our team worked together to synthesize it and use it to make design.

Initial Research

After meeting with our client and getting acquainted with the software, we contacted local teachers using the software and traveled to their schools.

Initial research:

  • 7 Class sessions observed
  • 16 Students interviewed
  • 5 Teachers interviewed

interview Baxe

We used affinity diagrams to draw out themes from our notes.  Tip: when affinity diagramming, it may be a good idea not to talk to each other, and to make it a norm that people can move notes around at any time, to minimize the amount of influence one person’s ideas have.

affinitydiagram_teacher

Using our interview data, we made personas, to represent “classes” of students and teachers.

Screen Shot 2018-09-14 at 12.44.18 AM

Screen Shot 2018-09-14 at 12.44.35 AM

Screen Shot 2018-09-14 at 12.44.45 AM

Choosing An Idea

With input from our client, we decided to iteratively design an interactive storytelling activity.

We brainstormed several ideas.  They stemmed from insights, which informed the personas as well.  Here are some:

Screen Shot 2018-09-14 at 1.33.34 AM

Design Process

We created several different prototypes of Endeavor over the following weeks, testing them with students and getting feedback from teachers and our client.

Our first prototype was built with Twine, an open-source tool for telling interactive, nonlinear stories.  It was a perfect piece of technology for our goals.  Essentially, we built a choose-your-own-adventure story, where students read text, and then are asked to choose a direction for the story to go.  At certain points, there would be a challenge to do an activity similar to one on System 44.

Screen Shot 2018-09-14 at 2.03.55 AM.png

We found out from our users that, overwhelmingly, they liked the autonomy of a choose-your-own-adventure story.

Screen Shot 2018-09-14 at 2.04.05 AM

A challenge: students click/tap the robot and hear a word. Then they select the word they think they hear.

The Danger of Chocolate-covered Broccoli

We got feedback from the initial user tests, saying that the challenges (like the one above) felt separate from the rest of the story.  They were warning us against what is sometimes called “Chocolate Covered Broccoli: boring educational activities (the broccoli) that you have to do in order to play fun games (the chocolate).”

edu-choc-broc-460x360

 

Our Response

  1. We decided to lay out the plot of a story with several “scenes” where there would be challenges, instead of writing a story first and fitting the challenges in after.
  2. Brainstorm different ways that the students’ correct/incorrect answers would affect the story.  For example, in our story, the students hide their gold from a pirate.  If they fail to unlock a combination lock with the appropriate word, they lose gold.
  3. Leverage #2 to make the story a game-based assessment.  By carefully creating questions and answer choices, the software can “tell” exactly where the student is struggling — word comprehension, or specific letter-sound pairs.
  4. Take it all the way — if we want to convince students that this is a real game, then failing challenges needs to result in a game over.  This was controversial among the team.  We decided to make it so the only way students get a game over is if students failed a challenge that was so easy, they shouldn’t get it wrong unless they were guessing or knew none of the sounds in the word.
Screen Shot 2018-09-14 at 2.44.08 AM

See #2

Final Prototype

Key features:

Screen Shot 2018-09-14 at 3.01.02 AM.png

A home screen, where students can pick a story at their reading levels.

 

Screen Shot 2018-09-14 at 3.02.27 AM.png

A typical screen of the interactive story.  Choices are in orange.  A visual representation of the “chapters” of the story adds anticipation.  A “Read It To Me” button provides scaffolding, which is critical for struggling readers.

 

Screen Shot 2018-09-14 at 3.02.39 AM.png

A challenge.  Students listen to a word and then try to select its written form.

 

Screen Shot 2018-09-14 at 3.02.55 AM.png

If the student gets the previous challenge incorrect, then they see the consequence — their robot companion becomes more dazed.

 

Screen Shot 2018-09-14 at 3.03.03 AM

A results screen that shows students how they did on challenges, and what items they collected in the story.

 

Concluding Thoughts

We hope that our research and prototype design and findings help make the next version of System 44 a better overall learning experience, and we appreciate the opportunity to learn skills that we will use in our careers.