CognitiveMetrics

Dashboard links on CognitiveMetrics can be shared. Take a tour of what the score report for the CORE looks like: cognitivemetrics.com/dashboard/shar…

Psychometrically, a test that strongly measures g is functioning as an intelligence test, even if it was marketed as an achievement/admissions test. The SAT/ACT are pretty g-loaded; we calculated the g-loading for the SAT and ACT at roughly .77-.80, which is in the same range as, or higher than, some tests people readily call IQ tests (such as Raven’s Advanced Progressive Matrices). Also, the ability to store, retain, and retrieve information is itself part of cognitive ability, crystallized knowledge. Long-term memory, working memory, vocabulary acquisition, and accumulated knowledge are all relevant to intelligence and professional IQ tests (such as Weschler and Stanford-Binet) include subtests that test learned information: like vocabulary, arithmetic, general knowledge, and similarities.

@CognitiveMetric The SAT/ACT are not IQ tests. They are retention tests. I scored high on them because I have good long term memory. All the questions are based on what you learned in HS.

Are the SAT and ACT still good IQ tests? We did a survey of high-ability people who were administered an IQ test (CORE) and asked for their modern SAT/ACT scores and found the following: 1. It is much easier to hit the 1600 ceiling on the modern SAT compared to the pre-1994 version. - After fitting a second-degree polynomial regression, we found that scores begin to plateau at approximately 130 FSIQ. Quantile regression estimates the ceiling at around 138 IQ. Visual inspection of the scatter plot also suggests that 1600 scorers begin appearing around the 120-125 FSIQ range and above. - Contrast this with older SAT forms, whose ceilings were far higher, roughly 166 FSIQ. For example, College Board data from the 1984 National College-Bound Seniors cohort showed that only 5 out of 964,739 students earned a perfect 1600. 2. The modern SAT seems to be a slightly worse measure of general intelligence. - The modern SAT was slightly less correlated with IQ compared to older forms, but still a strong proxy (r = ~0.70 with FSIQ after correction for range restriction). Older forms correlated with FSIQ a bit higher, with Frey and Detterman (2004) finding a 0.82 correlation with the ASVAB g-score. - Take this with a bit more caution, considering the sample size is smaller and range restricted. Some caveats though: - The SAT scores are voluntarily self-reported (although the FSIQ scores are not) - The sample size is lower than we'd like and we are still collecting people.

We're not treating the quadratic as the true SAT-IQ function or extrapolating it outside the data. It was just a descriptive way to illustrate the ceiling effects. Since SAT's ceiling is at 1600, it necessarily loses discriminating power near the top. A 130 IQ person and a 160 IQ person can both max it not because they have the same IQ.

Lazy modeling choice. As you can see their model predicts that people with a super high IQ will start to have a lower SAT score vs someone with a moderately high IQ - obviously ridiculous. Not to mention if you extrapolate this outside their range you end up with impossible/ subzero scores on either end. SAT scores are not just a number, they are essentially N bernoulli trials with some latent skill's influence. Using polynomial regression completely ignores the structure of the problem and pretends this is just some random numerical data. Note, that's not to say that a more reasonable model would lead to a different conclusion.

@doctor_WASP Not yet, but something we aim to do in the future for sure.

@CognitiveMetric Any data on modern GRE ?

@MaxDiffusionRL Yes, since we suffer from ceiling effects at SAT 1600 with this high ability sample. Piece-wise regression also supports this.

@CognitiveMetric Shouldn't anova of the correlation show substantial decorrelation at the extreme end

We also found a correlation of 0.66 between the modern SAT-V and SAT-M in our sample, which is directly in line with what was observed by College Board on the old SAT.

Standardized tests like the SAT and ACT help identify disadvantaged students who might otherwise not meet admissions criteria. They improve equity by providing a common metric that partially offsets grading variability and school quality differences. The 2020 UC report: senate.universityofcalifornia.edu/_files/underre…

The faculty review at the University of California found that the SAT and ACT were useful predictors of academic success and were not biased against any social group: “Test scores are predictive for all demographic groups and disciplines, even after controlling for HSGPA. In fact, test scores are better predictors of success for students who are Underrepresented Minority students (URMs), who are first-generation, or whose families are low-income: that is, test scores explain more of the variance in UGPA and completion rates for students in these groups.” Ultimately, the faculty committee recommended against going test-optional.

@CL0CK_WORK @charlesmurray Would you be able to link me towards Emil and Seb's ceiling calculations? Would love to compare ours with theirs.

@CognitiveMetric @charlesmurray Yeah, I think Emil and Seb basically put the a perfect 1600/36 at around 135 - 138. (Really low compared to the Old SAT) Many other papers suggest the same ceiling. A good thing though, is that the predictive validity has not changed.

The title of the article ought to have been "How the College Board Corrupted a Perfectly Good Test of Academic Aptitude." For the sad thing is that the SAT was still an excellent test through the 1980s. There had been drift over time--ETS's internal studies found that from 1963 to 1973, the SAT got easier on Verbal by 8-13 points and on Math by 10-17 points. But it wasn't until after the recentering in the mid-1990s that the test was made much less demanding. BTW, Dick Herrnstein and I published a long but by far the most thorough and accurate analysis (sorry to brag, but it's true) of the meaning of the long decline of SAT scores from 1963 to the early 1980s. It appeared in The Public Interest. You can download it, no paywall, at nationalaffairs.com/storage/app/up…

@CL0CK_WORK @charlesmurray Yeah, I am less confident on our correlation, considering it's a restricted sample as well as lower n. However, the far lower ceiling of the modern SAT seems harder to dispute.

@CognitiveMetric @charlesmurray There is this research paper taken in 2020, that shows a correlation of 0.88 with the ASVAB. Feel free to check it out. digitalcommons.georgefox.edu/edd/188/

Image from IGN: ign.org Iodine Global Network (IGN, formerly ICCIDD) is a nonprofit organization focused on reducing iodine deficiency by advocating salt iodization programs at the national and global levels. Referenced study: pubmed.ncbi.nlm.nih.gov/18676011/ Note: There is no evidence that providing iodine to people who already have enough of the nutrient will raise intelligence. In fact, too much iodine in a person’s diet can cause health problems, though these are less severe than the problems arising from an iodine deficiency.

The leading cause of preventable mental impairment in the world is iodine deficiency. The World Health Organization estimates that nearly 50 million people suffer some degree of mental impairment due to iodine deficiency. The good news is that iodine deficiency is inexpensive to cure, costing about 2 to 5 cents per person per year. Eliminating iodine deficiency worldwide is an ongoing project, largely through the introduction of iodized salt (see Iodine Global Network). Iodine supplementation during pregnancy or infancy can raise average IQ by 12 points in severely deficient populations. Ultimately, treating iodine deficiency is one of the most cost-effective ways to raise intelligence and contribute to economic and social development (Zimmermann et al., 2008).

@ctfpilled The AI scoring from 0-2 is based on whether specific concepts were mentioned rather than how it’s communicated. And CORE CO is primarily a measure of VCI, so it would be better to compare it to your VCI scores rather than FSIQ. You may have a verbal tilt in your profile.

@CognitiveMetric Does the AI grader perceive more formal/eloquent writing styles to be attributed to a higher intelligence? I know for sure I'm not any higher than 120 FSIQ broadly but I got a 99th percentile score on CORE comprehension and I think it's because I wrote pretty formally there.

Little-known fact: Some IQ tests include questions about real-life common sense. One Wechsler subtest, Comprehension (CO), presents questions that assess practical judgement, understanding of social norms and proverbs, and the reasoning behind everyday rules. For our Full-scale IQ test (the CORE), we created our own Comprehension subtest. Rather than being read aloud by a proctor, examinees read the questions on screen and type their responses freely. The 0-, 1-, or 2-point partial scoring has been retained; however, for efficiency, an AI scorer has been implemented that compares user responses against a comprehensive rubric defining an acceptable answer and multiple common example responses for each point threshold. Rigorous manual testing and the high internal reliability of this subtest (0.8975) indicate that the AI scoring system is functioning effectively. Moreover, we have found CORE CO's g-loading to be 0.656 (corrected for SLODR), while CO's g-loading on the WAIS-V is 0.66.

The abilities you described are measured very well on the Analytical section of the old GRE (maybe the Verbal section as well). We have found that it is a great IQ test, showing a g-loading of .85 and correlating highly with other professional IQ tests. We host a high-ceiling hybrid form of the old GRE (1980s) for free on our site. Using data from the ETS, we rescaled the 200-800 scale into IQ. You might be interested: cognitivemetrics.com/test/GRE

@aaron_elli47770 Janitor at MIT.

@CognitiveMetric i got 159 on Wechsler but i feel dumb. CM what career should i choose: janitor moderator crypto trader Internet troll

If you are interested in taking CORE Comprehension, check it out here: cognitivemetrics.com/test/CORE/CO

@SpencrGreenberg What test did you use to measure IQ? Does it correlate highly with well-validated, professional tests? You might be interested in using the many validated IQ tests we host. We could compare/validate your test(s) with ours.

Who has higher IQs: people who believe they can solve society's problems, or people who think they can't? In our study, those who said they can't, maybe because they grasp how hard those problems are! More surprises in my latest YouTube video, which appears to be going viral!⬇️

Your conclusion does not follow, you're conflating norming with validity. A high ability sample would be a concern if we were deriving population norms from it (which we weren't). The old GRE to IQ conversions weren't derived from this sample, they were derived from ETS population data. The convergent validity analysis is asking whether VISA behaves like an established verbal ability measure. Range restriction generally attenuates correlations, yet VISA and GRE-V still correlate around .80 uncorrected.

@CognitiveMetric @cremieuxrecueil Got it, the Y-axis is IQ computed from GRE percentiles, not raw scores. In that case, the data is even more ridiculous. You have 1 sample below 100, but many between 145 and 160. Even if this is real data, the insane amount of sampling bias renders any conclusion invalid.

Neat! Someone just validated that the GRE looks to be an IQ test: its verbal subscale correlates at r = 0.84 with another verbal intelligence test given to adults.