Jonathan Paulson

@lymanstoneky Why can’t the bureaucracy operate faster?

The lowest-controversy use case for euthanasia is "Death is certain in the next few weeks, options exhausted, spare the person weeks of grueling agony." This requires euthanasia programs to process all paperwork, requests, and safeguarding strategies within hours or days of application. If euthanasia is rare, this may be possible. In practice, when bureaucracies have effectively 3-5 day turnaround times and growing demand, implementing safeguards of any kind is effectively impossible. There is an intrinsic incompatibility between "euthanasia only for the clear cases" and "good safeguards for abuses," because the clear cases are operating on an extremely compressed timeline. In the long run, it simply cannot be done.

@conor64 The vaccine rollout (not maximizing lives saved) and the way anti-racism protests were encouraged (despite the lockdowns) seemed like they let leftist ideology override scientific expertise.

A question for everyone: survey data suggests that by the end of the Covid-19 emergency trust in public health institutions had decreased significantly. If you are among the people who reacted that way, why specifically? I'm hoping for long, diverse, individualized answers.

@RichardMCNgo Also HPMOR ends with a 12 year old in charge of magical Britain

One striking illustration of this mindset comes from rationalist fiction, which often ends with the hero gaining total power to design a new world order. Four examples (with many spoilers!):

It’s helpful to think of rationalists as High Modernists specifically about the future. No human was smart enough to successfully plan an economy or a society. But if we hypothesize an AI intelligent enough to do so, we can hold on to many of the same technocratic intuitions.

@Noahpinion Neither has been replaced?

Everyone talking about AI job displacement was always so focused on the working class. They thought only the highest-IQ people would still be valuable. But it turned out mathematicians got replaced long before truckers did.

@sharintaint @LinkofSunshine givewell.org has held up well for many years afaik

@LinkofSunshine Also... has any effective altruism campaign been proven to maintain efficacy? I remember a few years ago people were glazing MacKenzie Scott like she was the new Gates but now everyone sees her as a joke. Same with "just give people money" declining massively in popularity.

@AlejandroZarUrd @prfsanjeevarora We will end up revealing more things that are true if the proofs are done in a way that they can be built on. That will help both humans and AIs.

@prfsanjeevarora Why should we care about humans and elegance? Math isn't there for us, it's a system that exists outside of humanity I do math because I want to reveal things that are true, I do not care about our silly social conventions becsuse Math is alien to our reality

At a recent meeting I heard similar comments/worries from a leader in the Lean community. I suggest that people designing AI tools here make them helpful to humans. We need a loss function that captures elegance and human utility.

@MattBruenig @simonw Lawyers don’t read this stuff before sending it out? “You can have the AI test the code” seems similar to “you can have the AI check the citations”, so I don’t think programmers have it easier.

Not so @simonw. If you have a database that contains all of the relevant law, you can deterministically check each citation and quote against that database.

@Barney55575756 @ChrisCroy @postjawline @goblinodds Religious: wastes a precious life Utilitarian: makes friends and family sad

@ChrisCroy @postjawline @goblinodds How is suicide “morally wrong” what does that even mean?

its extremely funny to me that there's really only ONE single issue that Americans overwhelmingly view as "immoral"

@lymanstoneky I hope this isn’t true! Doesn’t it get a lot easier once they can do things for themselves?

You may have been told that the toddler stage is the hardest stage. You were told lies.

@blowitouturazz @mattyglesias For one he supported en.wikipedia.org/wiki/2020_Cali… which tried to bring back affirmative action in California.

@mattyglesias What part of his actual record is “very left” exactly?

Gavin Newsom is yet another case of an establishment politician who leftists don't like but who's actual record and image is very left. Obama was the opposite — moderate on most things, but *liked* by progressive activists because he'd opposed Iraq.

@Noahpinion We already have some RSI; models probably improve coding productivity. Probably AI will slowly increase %. And models are not *that* much faster than humans. Not clear to me there’s a discontinuity here; if AI is 95% maybe humans aren’t the bottleneck anymore.

In other words, an AI "FOOM" is almost certainly coming soon, where AI suddenly gets insanely better, seemingly almost overnight.

I've been playing around with toy models of recursively self-improving AI, and the one robust conclusion is that whether or not the improvements eventually plateau, the initial phase of RSI will be a very sudden, significant FOOM in capabilities.

@FlareIsGone Accelerant is very similar to catalyst.

In my experience, Poison in Slay the Spire 2 feels much worse than before. Losing Catalyst is a big deal, even though some of the newer cards are quite powerful. It also feels quite bad vs 2 of the 3 final bosses. Anyone else feel this way?

@AugmentedPhoto @TheStalwart Yes :) I think we’re already seeing this to some extent; AI looks very productive on fresh projects but then moderately-sized purely-AI projects get too complicated for either AIs or humans to continue working on. OTOH the AIs are getting better quickly.

@PaulsonJonathan @TheStalwart Ok, if that’s true, programmers will be making their lives hell by using AI to do the “how” up front

Why do a lot of software people like a tool that can allow them to expend their mental energy on higher order problems, while writers dislike the tool that can replace their output completely? Truly one of the great mysteries of our time

@AugmentedPhoto @TheStalwart The “how” ends up mattering a lot in software in a different way. Typically many changes are made over a long period of time so it’s important that the code is easy to work with. Whereas with writing usually you pick a version to publish and don’t change it later.

@PaulsonJonathan @TheStalwart Right…the “how” is what the AI automates. In writing, people see the “how” … they see the “code”… in software that is hidden…

@AugmentedPhoto @TheStalwart Partially. A lot of decisions (especially “how?”) are left to the coders. Also I think the fact that there’s a whole other job dedicated to design indicates that design is a very important part of the process.

@PaulsonJonathan @TheStalwart I thought that was the PM’s job

@AugmentedPhoto @TheStalwart “Deciding what you want to say” in code is also the main bottleneck there.

It does exactly the same for writers in theory…allows them to think more about what to write and can automate the writing process. But unlike software, in writing the bottleneck isn’t really the writing itself, it’s deciding what you want to say. Also nobody sees inelegant code, people just care if it works. In writing, style is huge …using AI kills your unique style and ends up taking you more time since you need to mop up after all the bullshit it adds in on its own.

@TheStalwart I don’t think this is a meaningful distinction; you could just as easily claim the other way around (any PM can write features now without developers vs. now writers can focus on making sure the big ideas are good without agonizing over the details)

@mattyglesias Could be that the academic side of college admissions has become generally less weighted. Capping the ceiling is part of that.

@CentristMadness How did skipping work that day help anything? What is the value of being “right” in a way that accomplishes nothing?

Yeah they were right do you not get it yet

@__venki__ Either you’re buying from the company directly, in which case they have more money to spend on stuff, or from other investors who will repurpose that money to do something else useful.

good question from sophia, that i've reworded, for the capitalism appreciators out there: When you purchase $100k of Google shares, what is the actual counterfactual impact of your investment, that justifies the returns you make? more context: it's pretty clear that if you eg: invest $100k in a factory, the capital you put in is doing something valuable. it's necessary for the machines that make stuff, so you're paid for your part in it. even if it's risk-free, and you're not paid for risk, you're paid because your money could be spent on other capital goods elsewhere, producing economic value. but, Google has lots of money. they don't need your capital. they're not even a net issuer of shares annually - they repurchase more than they issue via stock-based compensation. stock-based acquisitions too are much smaller than annual repurchases. what work is your $100k in google doing, or what value is it producing to justify the returns you make? why isn't your $100k in google stock unproductive capital? claude hasn't been able to give me satisfying answers. there's the rentier capitalism answer which is: Certain companies are sufficiently well-capitalized that investors aren't really helping them with their investment at all, but merely purchasing a share of future cash flows. Maybe smaller companies, and certain capital-intensive larger companies indeed do benefit from marginal capital though. But the total stock market is dominated by companies whose productivity is insensitive to marginal capital, and thus, capital-providers do not in fact provide much value, but collect rents for unclear reasons. there's some retrocausal answer: The market should efficiently values companies based on DCF, and this both: a) helps the more capital-intensive companies b) incentivizes earlier stage investors in companies that still have high marginal productivity per $ invested. so, it's fine, you should just purchase things based on DCF, and this will in markety ways cause investment in companies that will eventually be productive. it seems maybe closest to an explanation, but it feels very handwavy? it also feels uncompelling if the majority of capital is spent on purchasing rights to existing cash flow, and all it does is induce a little bit of productive investment