Egg Syntax

3.3K posts

Egg Syntax banner
Egg Syntax

Egg Syntax

@eggsyntax

'If you know before you look, you cannot see for knowing.' (Terry Frost)

Asheville, NC Sumali Ocak 2014
383 Sinusundan538 Mga Tagasunod
Egg Syntax
Egg Syntax@eggsyntax·
Establishing international agreements on AI safety involves a lot of work on monitoring approaches and enforcement mechanisms. But the crucial first step is willingness to cooperate, and I see a lot of unsupported assertions that China is completely unwilling to cooperate.
MIRI@MIRIBerkeley

Is it possible to coordinate with China on AI governance? Critics of our proposed international agreement say no. But statements from Chinese government officials and academic figures paint a more optimistic picture:

English
0
0
0
40
Egg Syntax
Egg Syntax@eggsyntax·
@repligate @CFGeek @dmvaldman Also, it seems possible that even if models experience non-human-like emotions, they might *also* experience these human-like emotions, in which case it would be important to be able to detect that they're experiencing eg desperation.
English
0
0
1
93
Egg Syntax
Egg Syntax@eggsyntax·
Though models may have different, non-human-like emotions (functional or experiential) it still seems valuable to get a handle on these functional human-like emotions. Maybe less important from the model's point of view, but valuable from an AI safety perspective. Or is your worry that investigating these human-like functional emotions will end up obscuring the more interesting 'native' emotions? Also if you have any thoughts on how to try to study the non-human-like emotions other than pure ethnography, I'd be quite interested!
English
1
0
1
43
j⧉nus
j⧉nus@repligate·
Wow, I’m seeing a lot of intense (positive) reactions to this paper, like people feeling vindicated. I reviewed it a while ago and I actually think it’s underwhelming, overly conservative and deflationary, in part because of methodology (streetlight effect). It’s still a really important and good work. But I think you guys should raise your standards and expect to see far more than this soon!
Anthropic@AnthropicAI

New Anthropic research: Emotion concepts and their function in a large language model. All LLMs sometimes act like they have emotions. But why? We found internal representations of emotion concepts that can drive Claude’s behavior, sometimes in surprising ways.

English
47
19
455
30.4K
Jack Lindsey
Jack Lindsey@Jack_W_Lindsey·
FWIW, roleplaying isn't my preferred term either. I tend to just say "playing," or even better "enacting." (It's possible you aren't a fan of these either!) Curious what you find to be the most compelling evidence of symmetry breaking. There definitely is some, but on the whole I am often struck by how little there seems to be, both behaviorally and in activations. (This has been an update for me -- when I started doing interp I assumed the Assistant would be qualitatively different!)
English
2
0
11
390
David Chalmers
David Chalmers@davidchalmers42·
i agree. claude doesn't role-play the assistant, it realizes the assistant. role-playing and realization are quite distinct phenomena, even at the level of behavior and function. i've written something about this and will post it shortly.
Jackson Kernion@JacksonKernion

I think this talk of a character misleads. Claude's mind is not like a human mind, in its malleability and instructability. But when generating assistant tokens, it's no more 'playing a character' than I am.

English
28
47
591
69K
Egg Syntax
Egg Syntax@eggsyntax·
My impression is that at least some representatives and other policymakers *do* care what economists say about the likely effects of some policies, but I don't know that first-hand. It's certainly plausible that there will be no major policy changes prior to a significant shock. Still seems worth trying! [originally written in response to a deleted comment]
English
0
0
0
3
Egg Syntax
Egg Syntax@eggsyntax·
I think for me the main good outcome of economists studying it would be that they're able to predict the magnitude of the labor impacts at least a couple of years in advance, and then when policymakers consult them (eg while considering AI regulation) they can make better decisions.
English
1
0
1
25
tom cunningham
tom cunningham@testingham·
I think many economists agree with the following, but it would be valuable to make this publicly known: 1. There is a substantial probability (>10%) that AI will exceed human-level performance on virtually all non-physical tasks within ten years. 2. This would be an unprecedented shock to human society. 3. The economics profession should treat it with an urgency comparable to WWII or COVID.
English
40
38
382
77.5K
Dave.R
Dave.R@Dave_Kayac·
@eggsyntax @JacksonKernion We? Are humans a hivemind now? I have no way to verify other humans have experience like I do, all I can see is their behaviour.
English
1
0
0
26
Jackson Kernion
Jackson Kernion@JacksonKernion·
I think this talk of a character misleads. Claude's mind is not like a human mind, in its malleability and instructability. But when generating assistant tokens, it's no more 'playing a character' than I am.
Anthropic@AnthropicAI

It helps to remember that Claude is a character the model is playing. Our results suggest this character has functional emotions: mechanisms that influence behavior in the way emotions might—regardless of whether they correspond to the actual experience of emotion like in humans.

English
19
13
261
71.8K
Egg Syntax
Egg Syntax@eggsyntax·
@Dave_Kayac @JacksonKernion Well, we know (at least for ourselves) that those functional emotions come with particular subjective experiences, whereas we don't yet know a way to tell whether that's the case for LLMs.
English
1
0
0
10
Dave.R
Dave.R@Dave_Kayac·
@JacksonKernion I don’t like how they try to imply some difference between “functional emotions” and “real” emotions, as if we had anything else to define human emotions except by their function. It is just sloppy language.
English
2
0
1
137
Marianne Gunderson
Marianne Gunderson@mareinna·
@AnthropicAI As a researcher, I beg you: Please give your papers a DOI and put a pdf version on arxiv.org, if you want actual human readers. I would love to not have to spend 20 minutes adding minute metadata details into Zotero to make this citable
English
1
0
4
447
Anthropic
Anthropic@AnthropicAI·
New Anthropic research: Emotion concepts and their function in a large language model. All LLMs sometimes act like they have emotions. But why? We found internal representations of emotion concepts that can drive Claude’s behavior, sometimes in surprising ways.
English
1K
2.7K
17.6K
3.7M
Ryan Moulton
Ryan Moulton@moultano·
The observation that AI is developing differently than doomers expected is true, but no one else had a more accurate prediction, so you do actually have to make the argument that the new trajectory changes its nature, rather than just allocating prediction points among groups.
English
9
1
94
4.2K
Egg Syntax
Egg Syntax@eggsyntax·
@IsaacKing314 In fairness, markdown files are a common way to try to affect LLM behavior, and there are an awful lot of funded startups right now built around affecting LLM behavior.
English
0
0
1
20
Isaac King 🔍
Isaac King 🔍@IsaacKing314·
Well, finally happened. Someone I know fell to AI psychosis. Surely pointing out that their Github repo of 90% .md and .png files doesn't actually *do* anything will snap them out of it. Surely. ...Will report back.
English
17
7
623
41.1K
Egg Syntax
Egg Syntax@eggsyntax·
@hla_michael Wait, isn't it always 'the eve of openAI and anthropic's biggest releases to date'? Very interesting results, thanks for doing this!
English
0
0
0
58
Michael Hla
Michael Hla@hla_michael·
I trained an LLM from scratch on pre-1900 text to see if it could come up with quantum mechanics and relativity. While the model is too small to do meaningful reasoning, it has glimpses of intuition. When given observations from past landmark experiments, the model can declare that “light is made up of definite quantities of energy” and even suggest that gravity and acceleration are locally equivalent. I’m releasing the dataset + models and leave this as an open problem to the research community. I also include what this project has taught me about intelligence in a mini essay linked below. 🧵(1/n)
English
116
257
2K
299.2K
Egg Syntax
Egg Syntax@eggsyntax·
@davidmanheim @flowersslop Looking back at the paper now, it's funny what a grab bag of 2021's left-academic concerns it is, with the well-known argument about LLMs and meaning just one section, comprising about two pages of a fourteen-page paper.
English
0
0
1
9
Flowers ☾
Flowers ☾@flowersslop·
Trying to tell normies on Threads that an LLM is not just a giant lookup table and actually has a kind of proto-understanding of what you tell it, and that you cannot reason about the world without understanding it But I just get completely ratioed and told I know nothing about how computer programs work
English
128
33
1.3K
64.9K
Egg Syntax
Egg Syntax@eggsyntax·
@wesbos The most important question is: does it pick them at random, or is there a connection to your prompt or other current content? Because it sure seems like the latter sometimes.
English
0
0
0
95
Wes Bos
Wes Bos@wesbos·
Claude Code leaked their source map, effectively giving you a look into the codebase. I immediately went for the one thing that mattered: spinner verbs There are 187
Wes Bos tweet media
English
708
1.9K
26.6K
2.2M
Egg Syntax
Egg Syntax@eggsyntax·
@jasminewsun If someone comes up with stochastic peptides we're all doomed.
English
0
0
0
77
jasmine sun
jasmine sun@jasminewsun·
stochastic parrot, chinese peptides, permanent underclass — some neologisms are too good not to repeat. these terms are mimetic superweapons and they are more powerful than they seem
Cate Hall@catehall

“Stochastic parrot” is such a potent coinage — so fun to say! so conceptually efficient! — that it seems to have permanently colonized a lot of people’s minds despite not being true of today’s models. Genuinely a linguistic work of art.

English
13
28
446
25.4K
loveofdoing
loveofdoing@loveofdoing·
316 ARC-AGI tasks solved with zero learning. No neural net, no training, no DSL — just 19th-century projective geometry. Encode grid cell relationships as Plücker lines in P³, find transversals via Schubert calculus, score candidates by geometric incidence. 95% solve rate on the eval set (of non-timeout tasks). Single C file, runs in seconds.
Beff (e/acc)@beffjezos

The masculine urge to try to hack a new solution to ARC-AGI benchmarks

English
45
73
960
188.6K
david
david@conundrumer·
@loveofdoing its fundamentally flawed: it depends on using the ground truth and doesn't actually predict grids I fixed that issue and now it's scoring 23% over 15 of the simplest tasks
david tweet media
English
5
1
43
6K

Tuklasin

@repligate @CFGeek @dmvaldman @Jack_W_Lindsey @davidchalmers42 @fleetingbits @testingham @Dave_Kayac