Drew Bailey

@paulnovosad Maybe the middle-left person thought you were asking which p values get PUBLISHED when X and Y are random noise.

Did a fun in-class exercise yesterday. Take a regression Y = b0 + b1 * X, where both X and Y are random noise. Do this 10,000 times. I asked the students to draw the resulting histogram of p-values for b1.

I think this paper is really important. Favoring unbiased estimators can actually increase bias when coupled with selective reporting, and we have lots of reasons to think that economists are doing plenty of this.

Test score gains fade. But long-term outcomes like graduation and earnings often persist. So what does that mean for judging school success? @drewhalbailey joins The Education Gadfly Show to discuss the fadeout effect. Listen: fordhaminstitute.org/national/resou…

@CEDR_US I buy it! As an addendum to your first explanation, we think there's probably also nonzero transfer from test score value added to other kinds of value added measured later.

Effects of teachers on students through different pathways (though test and nontest VA not highly correlated) AND the limits of tests to fully capture learning. Indeed, we discussed this here: caldercenter.org/publications/h… 2/n

Loved this convo about the potential reasons for fade out and how to reconcile test fade out, e.g., of teacher effects, with better later life outcomes associated with interventions/having better teachers. I think it's got to be some combo of ... 1/n fordhaminstitute.org/national/resou…

@educationgadfly Thanks so much for the fun conversation, and to my excellent co-authors @tw_watts and @EmmaRoseHart.

📢 #EdWorkingPapers: @tw_watts, @EmmaRoseHart, & @drewhalbailey analyze 87 RCTs to assess persistence of educational intervention effects & find that fadeout is common. Intervention characteristics explain a small share of differences in persistence. 📄 bit.ly/4aGrXY4

Why do educational intervention impacts fade? Isn't catch-up a good thing? Are sleeper effects real? Does fadeout mean failure? @drewhalbailey, @tw_watts, and I address these questions & more in an EdNext piece & 4 new working papers! educationnext.org/why-do-most-ed…

@AmandaKMontoya @siminevazire No problem. I don’t know what it was, but probably one of these! pubmed.ncbi.nlm.nih.gov/29847549/ academic.oup.com/aje/article/19…

@drewhalbailey @siminevazire I know this seems insane, but I bookmarked this 6 years ago, and came back to it. Do you remember what the first link is, because it doesn't seem to work anymore :-( RIP Twitter

Does anyone know of research looking at people’s perceptions of various causal (& non-causal) verbs/phrases? Eg, how do people perceive “impacts” or “is a risk factor for” etc., in terms of whether they imply a causal effect? Thanks for any tips or retweets!

@evavivalt @sdellavi Very cool and important work; congratulations.

🚨 New working paper! How well do people predict the results of studies? @sdellavi and I leverage data from the first 100 studies to have been posted on the SSPP, containing 1,482 key questions, on which over 50,000 forecasts were placed. Some surprising results below.... 🧵👇

@krichard1212 Very nice of you, thanks.

Good stuff by @drewhalbailey et al. Everyone who wants to make sense of cognitive itnerventions/causality/fadeout effects should read his and his collegaues' papers. annualreviews.org/content/journa…

@JoeJ45665 @AndrewRAConway That's two tweets in a row in which you've been recklessly wrong. I'm afraid you and I are through, Anonymous Joe.

@drewhalbailey @AndrewRAConway Their fact check provided exactly zero facts, just opinions. What is a fact is that we rank 34th globally in math on the 2022 PISA scores. The argument is that we invented the internet and put men on the moon but there’s no evidence we were 1st in math. en.wikipedia.org/wiki/Programme…

A clear and compelling read on IES. I hope policymakers pay attention to this. There is a very strong bipartisan case to be made for continuing to fund the development, evaluation, and syntheses of evaluations of educational programs and policies.

@JoeJ45665 @AndrewRAConway Hadn’t heard that claim before about changes in US performance over time! Not even close to true though. Here’s a nice summary of the US’s performance on international assessments going back to the 1960s. usatoday.com/story/news/fac…

@drewhalbailey @AndrewRAConway When the DOEd was founded the US was ranked number one in education. We’re now 16th in science and 34th in math- out of 81 countries- below Malta and Vietnam. Whatever they’re doing isn’t working. We’re going backwards. Get rid of it.

One of my favorite IES resources are the practice guides. There are 5 guides with the research about teaching math. Given the two stop work orders I received last night (to stop work on 2 upcoming IES STEM practice guides), I'd download these now. buff.ly/4hyHeeI

Our new review, with @tw_watts, @EmmaRoseHart, and April Yu, summarizing some recent work on "Learning about Development from Interventions" is published open access at Annual Review of Developmental Psychology. Comments, questions, insults welcome. annualreviews.org/content/journa…

@FlowTap1 I’m a grinch relative to the median ed policy researcher (long term effects are often overestimated), but a merry elf relative to many of my individual differences colleagues (they are often positive and frequently large enough to pay for themselves!).

The latest compelling, but depressing, research on the fade-out of education intervention effects from @drewhalbailey & company. Drew, can't you give us something to get us into the holiday spirit? edworkingpapers.com/ai24-1069

@MatthewBJane Without reading the paper, maybe they're not doing casewise deletion, so the pretest sample is a subset of the posttest/difference score sample.

Last year, Killingsworth, Kahneman, and Mellers published a paper reporting that, for a group of unhappy people, money does not improve happiness. @dingding_peng, @sewenz, and I wrote separate critical comments of it, which were published today.

@BrentWRoberts Yours was even better!

This is just to say I don't believe your diff-in-diff Forgive me the effects are way too large plus forking paths And so many other things happened during that period