Scaling Shapes

Context

This semester-long project was completed during my Summer 2020 internship with the Museum of Science, Boston. I was challenged with taking "Scaling Shapes," an existing in-person exhibit, and providing advice on how to adapt it into a virtual version that follows the Productive Struggle framework. The Museum of Science defines Productive Struggle as "learning challenges that are exhilarating, challenging, intensive, and fruitful."

Scaling Shapes is an exhibit within the Math Moves! exhibit collection. Math Moves! is a collection of embodied math learning exhibits. In each of these exhibits, visitors learn about math concepts through physical interaction with tangible objects. In the Scaling Shapes exhibit, visitors enlarge simple objects by doubling them in three dimensions. This activity teaches the math concepts of scaling, proportionality, and similarity.

Overall, I found that the “Invite” and “Disrupt” areas appeared to be present in Scaling Shapes, but the “Support” area did not align with the Framework. In the in-person Scaling Shapes exhibit, no feedback is given to the visitor. The visitor is not told if they have successfully doubled a shape or not. In addition, no dynamic hints or scaffolding are given. Due to these findings, I initially decided to focus on the “Support” area of the Framework when working on my updated, virtual version of Scaling Shapes.

Virtual Prototype

After conducting my initial research using video data, I realized that more research and testing needed to be done. Since the MOS was closed to the public, I was unable to conduct testing using the in-person exhibit. Therefore, it was necessary to create a virtual prototype of the Scaling Shapes exhibit. This prototype needed to be workable enough that visitors could engage with the activity over Zoom. Since I was working without a software developer, I needed to find a pre-existing virtual block building platform. After researching available options, I decided upon the Toy Theater building site. This site allows users to build shapes with cubes. Users are able to place and delete blocks on a grid, as well as rotate the grid to view their shape from different angles—thus, the site allows visitors to complete the Scaling Shapes activity virtually.

I also created a virtual version of the signage for Scaling Shapes. The virtual signage uses the same wording as the in-person Scaling Shapes activity. I also created a video tutorial which explains to users how to use the buttons on the building site.

Testing

After creating the virtual prototype of the Scaling Shapes exhibit, I prepared for a round of virtual testing. The purpose of this testing was to assess the following evaluation questions: 1. What works well in virtual Scaling Shapes? 2. What would participants like to see work differently in virtual Scaling Shapes? and 3. To what extent are participants experiencing Productive Struggle in virtual Scaling Shapes, and what does this look like?

During testing, participants were asked to open the building site in a window on one half of their screen and open the signage page in a window on the other half of their screen. Thus, participants would see the following during testing:

Participants engaged with the activity for approximately 10 minutes using a think-aloud protocol while I took notes, probed, and offered feedback during the observation. Participants then completed a survey, followed by a brief interview. Testing sessions were recorded using Zoom.

Testing Results

Overall, I found that participants enjoyed the activity and found it usable. However, there was low evidence of Productive Struggle. To better elicit Productive Struggle, there is room for improvement in the “Invite,” “Disrupt,” and “Support” areas of the Designing for Productive Struggle Framework. Possible improvements are discussed in the “Design Recommendations” section.

In terms of level of challenge, participants felt they could tell if they had doubled a shape correctly. In addition, all participants reported that the activity felt “easy, but satisfying to keep trying." However, observation data told a different story in terms of level of challenge: 4 of 6 participants incorrectly doubled at least one shape. In other words, 4 of 6 participants thought they had doubled a shape correctly, but actually had not. Because the activity itself does not tell participants if their attempted double is correct or not, some participants walked away from the activity thinking they completed it successfully, when actually they had been unsuccessful. So, participants who felt the activity was easy may not have realized the level of challenge of the activity. Since they did not know when they were wrong, the activity was actually more challenging than they thought.

Testing participants self-reported low levels of disequilibrium, moderate levels of persistence, and high levels of productivity and satisfaction, as shown in the graph below. Thus, results showed that the Scaling Shapes activity is supporting the positive emotions, but is not supporting disequilibrium as much as we would hope for in a Productive Struggle activity. Testing showed that participants did not always know when they were wrong. Thus, participants did not self-report disequilibrium emotions, since they were unaware of the disequilibrium. I believe that if participants are told when they are wrong during the activity, they will report higher levels of disequilibrium.

Design Recommendations

The virtual Scaling Shapes activity has room for improvement in the “Invite,” “Disrupt,” and “Support” areas of the Designing for Productive Struggle Framework.

Invite

• The original prototype involved three different screens (the Toy Theater building site, the signage, and the tutorial video). In addition, the Toy Theater building site was not designed for this activity, and therefore includes some buttons that are confusing to participants and are irrelevant to the Scaling Shapes activity. To minimize these barriers to entry, the activity could be redesigned so that everything is on one screen, and the buttons are more intuitive.

• The original prototype included a one-minute-long tutorial video, showing how to use the buttons in the activity. Results from testing showed that not all participants wanted to watch a video prior to starting the activity. However, participants who watched the video found it helpful. To improve this, the tutorial video could be replaced by an interactive tutorial. This would be similar to the type of tutorial included in a video game. The tutorial would prompt the visitor to use the different buttons. By making the tutorial interactive, this could also serve as a preview of the activity for the visitor.

• The signage for the in-person Scaling Shapes activity shows pictures of Russian nesting dolls and other scaled objects. In the virtual version, these real-world examples could be expanded. Video or other multimedia could be included to show visually what is meant by “doubling”—for example, depictions of model cars or trains could be used. Including a multisensory explanation of scaling would help visitors to more easily understand the objective of the activity. In addition, this would help visitors to see how the activity relates to their everyday lives.

Disrupt

• Testing results showed that although participants felt it was easy to tell if they had doubled a shape correctly, participants had not always correctly doubled a shape when they thought they had. The original virtual prototype did not give the visitor feedback about their doubled version. Participants were not always aware of the level of challenge in the activity, because they sometimes thought they were correct when they were incorrect. In an updated version of virtual Scaling Shapes, the activity should tell the visitor if they have successfully doubled or not. If a visitor thinks they are correct, but then the activity tells them they are wrong, this may introduce disequilibrium for the visitor. Since testing results showed low levels of disequilibrium, this recommendation may increase these levels of disequilibrium, and thus may elicit Productive Struggle for more visitors.

• One way to leverage uncertainty is to force decision making. Forced decision-making could be introduced into the Scaling Shapes activity. If the visitor has to click a button that says “I think I’m done,” they are forced to commit to their doubled version. If the visitor is then told they are incorrect, this may heighten their feelings of disequilibrium. If the visitor is told they are correct, this can serve as a final reward.

• Another way to design for the “Disrupt” area of the Framework is to allow visitors to create their own block to then double. Visitors could be given the option to build a shape, and then create a doubled version of their shape. This would allow visitors to double more challenging shapes if they are not finding the activity challenging enough. In addition, some participants during testing expressed that they would like more freedom to build in the activity, so this recommendation would help satisfy this desire. Participants could also be given the option to create a shape to challenge a friend with. This would introduce more social and competitive aspects into the activity, further enhancing the “Disrupt” area.

Support

• The virtual Scaling Shapes activity has room for improvement in the “Support” area of the Productive Struggle Framework. The original prototype did not include any hints or scaffolding as part of the activity. Some verbal hints were used during testing, and these types of hints could be incorporated into the activity itself. For example, hints such as, “It looks like you have doubled the height, can you double the length and width next?” were stated verbally to participants during testing. One way to incorporate these textual hints into the activity is to allow visitors to ask for a hint related to a specific dimension. For example, they could check to see if they have the correct height for their shape. If their height is correct, the activity would tell them so, which would be a “mini-win” for the visitor. If their height is incorrect, the activity could offer a hint related to this dimension.

• Another way to design for the “Support” area of the Framework is to draw more attention to the number of blocks. The original prototype asks visitors “How does the number of blocks in the model compare with your scaled up version?” However, during testing only 2 of 6 participants attempted to answer this question. It is possible that some visitors do not wish to count the number of blocks in their doubled version, as these doubled shapes are made of a large number of blocks. To help support visitors in answering this question, the activity could count the number of blocks for them. As visitors are building, a table could show the number of blocks in the model and in their doubled shape. This would help visitors to focus on the important part of the question (comparing the two numbers), and help avoid visitors using up too much of their “struggle budget” on counting the blocks.

What I Learned

• From this project, I learned about how to conduct formal testing via an iterative approach. I developed a testing protocol and conducted a preliminary round of testing with colleagues. I then refined my protocol, testing with children, and analyzed my results.

• From this project, I practiced taking an existing, imperfect design and updating it to better fit the Productive Struggle framework. Being able to update and improve on an existing design is a valuable skill that I will take with me moving forward.

What I Did Well

• My design recommendations for an updated, virtual version Scaling Shapes will expand access to Productive Struggle for a larger variety of learners. In the current in-person Scaling Shapes exhibit, Productive Struggle happens for some visitors, even though Scaling Shapes was not designed to support Productive Struggle. My design recommendations include strategies that are known to encourage disequilibrium, persistence, and productivity, and therefore my recommendations will help more learners to experience Productive Struggle in this activity.

• My design recommendations will make the virtual Scaling Shapes activity accessible to more learners. My design makes the activity easier to use for learners of varying abilities. In addition, by creating a virtual version of this exhibit, more learners will be able to engage with this activity during the COVID-19 pandemic.

Next Steps

• After completing this project, I shared my results from testing during a Webinar series about Designing for Productive Struggle. By sharing my experience and my results, I helped other museum professionals to understand what it is like to design an activity to better elicit Productive Struggle.

• I am excited to stay in touch with the Museum of Science as they move forward with developing the virtual version of Scaling Shapes. I know my research, testing, and design recommendations will be extremely useful to the Museum of Science as they move forward with updating this activity.