Robert Luciani

@elonmusk Mathematicians see the solution immediately. Just invert the ethical valence and the problem becomes trivial. Pick up five passengers on your normal trolley route, or leave them and pick up the philosopher that has been waiting 2000 years for the AI ethics train...

Grok answers correctly

Provisional ERA5 global temperature for 17th November from @CopernicusECMWF was 1.17°C above 1991-2020 - the warmest on record. Our best estimate is that this was the first day when global temperature was more than 2°C above 1850-1900 (or pre-industrial) levels, at 2.06°C.

🌎🌡📈 ΔT +2.06 °C !! This is what uncharted territory looks like. The first time that the global temperature anomaly has broken through +2.0 °C, for a single day in observational history. Much more to come in the months ahead.

Humans. Limited in intelligence, unbounded in hubris.

@ednewtonrex This whole discussion is really about protecting an industry. Speaking as someone that helped establish a genre of music, I can assure you that what is bad for record labels and "pros" does not translate to being bad for art or consumers of art.

I’ve resigned from my role leading the Audio team at Stability AI, because I don’t agree with the company’s opinion that training generative AI models on copyrighted works is ‘fair use’. First off, I want to say that there are lots of people at Stability who are deeply thoughtful about these issues. I’m proud that we were able to launch a state-of-the-art AI music generation product trained on licensed training data, sharing the revenue from the model with rights-holders. I’m grateful to my many colleagues who worked on this with me and who supported our team, and particularly to Emad for giving us the opportunity to build and ship it. I’m thankful for my time at Stability, and in many ways I think they take a more nuanced view on this topic than some of their competitors. But, despite this, I wasn’t able to change the prevailing opinion on fair use at the company. This was made clear when the US Copyright Office recently invited public comments on generative AI and copyright, and Stability was one of many AI companies to respond. Stability’s 23-page submission included this on its opening page: “We believe that Al development is an acceptable, transformative, and socially-beneficial use of existing content that is protected by fair use”. For those unfamiliar with ‘fair use’, this claims that training an AI model on copyrighted works doesn’t infringe the copyright in those works, so it can be done without permission, and without payment. This is a position that is fairly standard across many of the large generative AI companies, and other big tech companies building these models — it’s far from a view that is unique to Stability. But it’s a position I disagree with. I disagree because one of the factors affecting whether the act of copying is fair use, according to Congress, is “the effect of the use upon the potential market for or value of the copyrighted work”. Today’s generative AI models can clearly be used to create works that compete with the copyrighted works they are trained on. So I don’t see how using copyrighted works to train generative AI models of this nature can be considered fair use. But setting aside the fair use argument for a moment — since ‘fair use’ wasn’t designed with generative AI in mind — training generative AI models in this way is, to me, wrong. Companies worth billions of dollars are, without permission, training generative AI models on creators’ works, which are then being used to create new content that in many cases can compete with the original works. I don’t see how this can be acceptable in a society that has set up the economics of the creative arts such that creators rely on copyright. To be clear, I’m a supporter of generative AI. It will have many benefits — that’s why I’ve worked on it for 13 years. But I can only support generative AI that doesn’t exploit creators by training models — which may replace them — on their work without permission. I’m sure I’m not the only person inside these generative AI companies who doesn’t think the claim of ‘fair use’ is fair to creators. I hope others will speak up, either internally or in public, so that companies realise that exploiting creators can’t be the long-term solution in generative AI.

You can tell when engineers with a passion for computer science are working on ML. This is really inspiring. Great job @AdeptAILabs time for me to wrap the C++ code in Julia :) adept.ai/blog/fuyu-8b

@karpathy Seems reasonable to assume that training encodes a directed A->B edge without considering that the english word "is" implies equality symmetry which would require two edges. At inference time, the mesa-optimizer would have to infer that missing edge.

LLM knowledge is a lot more "patchy" than you'd expect. I still don't have great intuition for it. They learn any thing in the specific "direction" of the context window of that occurrence and may not generalize when asked in other directions. It's a weird partial generalization. The "reversal curse" (cool name) is imo a special case of this.

GPT4 is completely unable to write in iambic pentameter without rhyming couplets :)

@sirbayes It's always disappointing when they just present a hypothesis.

Interesting paper. I agree that language models are not models of the world, they are ways of communicating partial knowledge about world states. We learn world models from vision and action, and language builds on top of (and expands) this. arxiv.org/abs/2306.12672

I sometimes play with the constraints of building a big AI cluster out of consumer hardware in case datacenter regulatory oversight got silly, but I am consciously LARPing as a rebel, to contrast with the AI doomers LARPing as imperials.

@bryan_johnson Black tech-wear would suit you well!

I’d ask you what you think, but I really don’t care 🫶🏻

LLMs will be training themselves soon because, just like with PvNP, validating that an answer is good is much easier than producing a good answer.

My deathcore band Bloodstarved released a new single - Darkwater open.spotify.com/track/3gW7HaAV…

Just posted a photo instagram.com/p/CpiihiOt0dN/…

2023 - people start torrenting Neural Network weights like we used to torrent movies and games.

It was actually me who insisted on including such a clause into my record deals. Hail Satan.

Hotter take: ML would have advanced faster if another front-end language had been available and widely adopted instead of Python. One that is interactive yet fast & compilable, multithreaded (no GIL), isn't bloated, doesn't care about white spaces,... E.g. Julia or some Lisp.

Whether an AI can be "factual" is an epistemological question, not a technical question.

Graph neutral networks are still in their infancy. Here's a great paper about capturing "long range" features. arxiv.org/abs/2205.12454

We’re celebrating the 26th anniversary of Final Fantasy VII with specially curated playlists for each main character in the game! 🎵 Please take your time and enjoy listening to them all here: sqex.link/vwep