Nils

Decomposing standards is step 1 of curriculum architecture. As you correctly point out, step two is filling this scaffolding with content. In fact, one of our most important views is that knowledge of content and facts is vastly underrated. We curate, generate, and hand-craft the content on Timeback. It's probably what we spend 90% of our time on. Good content is really important.

@nilslang Hey Nils- (Alpha/GT parent) Is there a list of the content knowledge by “grade level” that alpha is ensuring the kids know? I’m familiar with Core Knowledge (coreknowledge.org) But what is the objective function tha alpha is focused on? (Outside of math)

Schools buy teacher materials from 5 different publishers. Hundreds of teachers then apply their individual training and experience to classroom delivery. When a student fails a standardized test, how can anyone tell what went wrong? The solution is vertical integration!

@griswold 🫡 noted

@nilslang Great article. I really enjoyed the breakdown of the process and thinking. I'd love to read more on the decomposition and telemetry/analysis processes in the future.

@DrLauraReames Why fight us if they can join us?

@nilslang Very exciting stuff! This is revolutionary. But, I am willing to bet the traditional education train and EdTech won't go down without a fight!

@teachthemx3 We just started tagging all test items at Alpha with the standards they test. Based on this, we can assign more targeted hole filling when a student fails a test.

@nilslang Not only that, but we as teachers can't see what standards were missed by each student on the STAAR test. All we see is a score.

@kalio_late @GEVS94 @thejfloor Can't dm

The most capable people I know quit their jobs to build companies. What happens when you put 10 of them at dinner with one of the world's most iconic VCs? We're about to find out! a16z's @GEVS94 is coming to Zurich to meet great founders. @thejfloor gives space to the best builders in the city. World-class builders need world-class partners. Naturally, we're making it happen. This Thursday. a16z x J Floor Founders Dinner. In Zurich. Building in Zurich? Lmk in a DM.

"Real world skills" are just knowledge in disguise. By making K-12 (general knowledge) 10x more efficient, we unlock time for the development of specific knowledge, which is orders of magnitude more valuable. This is how Alpha High students outpace experienced professionals.

Definitely agree about pop-up quizzes. Students don't like being fooled into thinking what is a game vs what is a learning app. Stealth assessment can make it hard to separate "good at the game" from "actually learned the financial concept" though. If a kid makes the right in-game choice because the UI makes it obvious, or because they pattern-matched from previous levels, that's not the same as understanding the concept and being able to generalize it. To have a closed measurement loop, you'd probably wanna have some kind of final assessment that every user has to pass.

When I first started designing Aurum, I realized something: kids can smell an educational game from a mile away. The moment a pop-up quiz interrupts gameplay, you've lost them. So we threw out the quizzes. We're using Stealth Assessment - proving mastery entirely through the choices they make in-game. No tests. Just flow. 🎮

@gitamar @MIT_CSAIL @CuriousMindsHub wish I had more time to read :)

@MIT_CSAIL @CuriousMindsHub @nilslang

A guide to remembering everything you read: bit.ly/4rxUHIr v/@CuriousMindsHub

Happy Monday! If you are questioning what you're currently doing in the face of the AI revolution, or any social or political upheaval, let me remind you: It's Time to Teach!

@sarah_cone @PlaycademyEdu Did you hear this boss @PSkinnerTech @jliemandt ?

@PlaycademyEdu Oh, amazing! It’s by far the best Alpha School app.

Thanks for the shoutout Sarah! Math Raiders is our Direct Instruction math fluency app used at Alpha School. Students love it so much we're having to block access to get them to still use other apps. Even better: learning efficiency is on-par with non-gamified apps!

We agree with Carl that reading comprehension is downstream of vocabulary knowledge. That's why we've been working with him and Sarah Cottinghatt on a vocabulary app that has better pedagogy and is also 10x more fun than the braindead owl. Coming soon to Alpha School!

@julianboolean_ I think both approaches are valid, but the first one is what they're describing in the article: tca2.education.illinois.edu/docs/libraries… I agree this is the most interesting aspect of my article and I'm gonna dive more deeply into this soon.

@nilslang I'm very curious about the intrinsic integration gamification - is the idea to make a game where game actions themselves are somehow practice in the skills the app is trying to teach? (a la say using Factorio to teach operations research) or is the game a narrative wrapper around the same actions one would do in a standard classroom?

still one of my all time favourite 'they really said that?' quotes "Do you think that there’s a conflict between gamification and engagement — the things that you’re historically successful at — and education? Yes, there is. How do you manage that conflict? Very easily. Always go with engagement." - Duolingo CEO Luis von Ahn theverge.com/24267841/luis-…

@randomor Thanks! You're right this opens up a lot of approaches to further optimize in the future

@nilslang I see. Sounds like it can scale if you have more schools and cohorts. Or if you can somehow attribute the time spent per app per student to their standardized learning outcome. The possibilities… Best of luck!

@BarryPCotter @C_Hendrick @_MathAcademy_ @MentavaInc We do spend a lot of time here

Someone at Alpha School spends a lot of time on Twitter; @C_Hendrick, @_MathAcademy_ and @MentavaInc all work with them. The only one of those I found surprising was @MentavaInc ; other apps are at worst slightly lower quality and vastly cheaper.

One future of education is getting the necessary learning done in the minimum of time. Nothing in the article would terrible @mpershan’s thesis that Alpha School sees learning school skills as a necessity to be done in the minimum of time to the highest standards.

@randomor We test more frequently than once a semester and employ a variety of tests. We usually run dedicated pilots for new apps and do pre- and post-tests.

Great series! I’m curious what kind of “standard tests with integration checks” you are using? How frequently these are used? As one would expect most standard tests are summative and not done until end of semester. How does timeback attribute the delta in learning to different edtech tools in each pilot cycle?

Amazing insights! The next generation of Timeback-native apps launches this summer and will hopefully fix a lot of the issues you're describing. We have this role of a process engineer connecting devs with learning scientists, and I'm one of them. Would yoh be interested in talking to me and @PSkinnerTech ?

Full disclosure – my kid goes to Alpha and has used Timeback. Timeback is a good mechanism for filtering out ineffective apps. If you've landed on a top 1% of apps, that's progress, but top 1% in a mediocre field is still not optimal. I've worked in EdTech and video games, run an alt-school similar to Acton Academy, and I've known most apps were ineffective. I also understand the bureaucratic structures that promote mediocrity. I can spot a badly worded or confusing question a mile away, as I oversaw production and edited math scripts (algebra and calculus) for 200+ multimedia lessons. My kid is doing well at Alpha and on the standardized tests despite the 1% apps, not because of them. He's developed his own independent workflow that allows him to understand the concepts where the apps fall short. It makes me wonder how many other students are creating their own interventions. While Timeback has become a great app evaluation tool (I wish I had it when I ran my K–8), it still hasn't solved the much more difficult problem of app creation. My kid doesn’t like the AI-generated questions from Timeback. What has had the bigger impact on his performance and motivation is the in-person experience at Alpha, the accountability structures, habit-building, and community. You are correct in outlining the misaligned incentives in EdTech companies, but it goes deeper than that. Many EdTech companies are primarily focused on capturing public money from school districts through large sales forces rather than on learning innovation. That’s the surface level. On the production level, it’s usually a convoluted process. Learning apps require a convergence of far more disciplines than traditional software development, and they’re often run by an ineffective Producer or Project Manager (who essentially just tracks progress), when what’s really needed is a Process Engineer who can coordinate and communicate across all the disparate disciplines like a watchmaker. You are correct that there’s a “lack of understanding of the science of learning.” The “Senior Curriculum Architects” gave me blank stares whenever I pointed out lessons that broke almost every rule of good learning design. The reason for this is that “Created by Educators, for Educators” became a selling point, and people with degrees in education were positioned as the “subject matter experts.” The problem is that classroom teachers aren’t trained in learning science, and they aren’t trained in media production, two necessary requirements for creating effective learning media. To receive federal funding, NCLB required interventions to be grounded in “scientifically based research,” but the first exposure anyone at the company had to actual learning research was when I sent out a letter outlining how to apply it. The second iteration of the flagship product was based on my design. It wasn’t perfect, but I moved it in a better direction. “Scientifically based” is a tagline, but government bureaucrats or school districts don’t have the wherewithal to verify the claim. Production pressures are another reason for the mediocrity. Creating large amounts of content on a budget with a deadline contributed to teachers haphazardly “throwing things over the wall” and letting the next person deal with it. If you were unfortunate enough to work with me, you got the scripts back with notes. At some point, I would just rewrite them myself to save time. Media literacy is another reason for mediocrity. Beyond EdTech, I’ve worked in feature films, comics, video games, and children’s books, and I’ve taught at the college level. These are all communication vehicles, but each has distinct characteristics and constraints that must be understood to be effective. Multimedia learning apps are no different. The problem was that the subject-matter experts had backgrounds in classroom settings or textbook writing, skills that don’t transfer one-to-one into media creation. Simple mistakes were common, like placing a key term at the end of a sentence so it appeared on screen too late instead of introducing it early so it persisted visually while being explained. The writing itself was usually not appropriate for the medium. In an attempt to get instructors to write more naturally, I had them present their lessons on video, hoping this would consolidate their thinking and lead to more concise scripts. What I discovered was that the higher the degree, the worse the communicator. Lower-level teachers tended to visualize the student and anticipate confusion points, so they added helpful asides and context. AI will eventually flatten many of these problems, but it will still need the guidance of practitioners who understand the difference between trivial facts, true learning, and deep understanding. I know I haven't mentioned the specifics of creating better apps, but this is already long, and it varies by subject. So if anyone is interested, I can share some of my theories.

@GabrielValles Yes please, would love to read more.

I’m glad you clarified. I was planning to respond to your last article and to this line: “It will be built on the principles of learning science, not engagement maximization.” Engagement is a result of good teaching; it isn’t opposed to learning. I fought this battle within the halls of ed-tech nearly 20 years ago. I didn’t win, but as you can see in this article I wrote, I was actively trying to implement the science. When I get a chance, I'll give a longer reply on some other things to consider. linkedin.com/pulse/applying…

Part Two in my series on how we at Alpha School are turning learning science into an engineering discipline. This one dives deep on the basic formula of learning: learning outcomes = engagement * learning efficiency Hoping to "engage" the skeptics in an interesting discussion!