Goblins is serious about growth.

In this QED study, it doesn't take much usage to see a notable effect. Students averaging 2 or more problems a week achieved +0.237 standard deviations (SD) in NWEA MAP's conditional growth index (CGI) score, a nationally normed reference for how much a student grew above what they were expected to grow. This is roughly equivalent in the research literature to the difference between an average teacher and an excellent teacher. In other words, using Goblins was like taking your average teachers and turning them into your best. Another way to think about this is additional learning time, where 0.237 SD equates to about 3 added months of class time.

The Maryland researchers also found “a strong positive relationship between Goblins usage level and math achievement, where the strongest effects are observed among the highest-use groups.” Students who solved 10–19 problems a week grew 0.40 SD, and those solving 20 or more problems per week (about 30 minutes) grew 0.69 SD, or about 6.9 added months of learning.

This is still a low lift for teachers. It asks for less time than standard math platforms typically do, which makes Goblins a more effective use of screen time than alternatives.

For context, this effect is much larger than the effect sizes typically found for supplemental and intervention math products. To compare products on the same basis, the chart below assumes about 30 minutes of weekly use. For Goblins, that is roughly 20 or more problems per week. CGI is how we measure growth, but to compare against other products, we use a different measure, MAP's RIT score:

Students who need math support the most have historically been the ones most poorly served by technology. Researchers call this “The 5% Problem”: most studies report results that do not include the 95% of students who did not meet the treatment threshold. Whereas roughly 51% of Goblins students reached the same level of growth featured by Khan Academy in its landmark study. That is about 10 times as many students. As the researchers put it, “Adoption of Goblins substantially exceeded that of other computer-assisted learning (CAL) tools, including Khan Academy, Zearn, and IXL.”

Maryland also found something surprising: “Students with an IEP, Hispanic students, and English language learners appear to benefit much more from Goblins than their counterparts”, showing about 1.2, 0.8, and 1.9 additional months of growth, respectively. This is unusual too. As a “possible explanation”, the researchers noted how “Goblins’ multimodal features”, like how Goblins lets students speak and draw, which we know help students build confidence, may have helped “lower barriers to engaging with the platform for these students.”