Gabriel Pickard

@ysegmond Yes

@ysegmond I have been thinking about this for a while. A number of different angles to come at this, for the functional programming part I think I have a decent approach. For the LP-inspired part I'm really not sure yet.

@simonw "I want to sail across the Atlantic from East to West, where should I start?" -- less capable models reliably get confused by the fact that the American East Coast is on the Western side of the Atlantic.

Anyone got a good short go-to prompt for running against a model that instantly gives an indicator of how capable that model is? I'm trying to figure out a neat way of determining if a model feels gpt-3.5-turbo or gpt-4 level with the minimum amount of work (and API token spend)

Stay fine-tuned for more! 🙃

I’m rounding off the PL features, to make it usable, but already looking for input on application ideas. Certainly it could be great for email processing, maybe data analysis and also for game logic. If you have specific thoughts in these or other areas I’d be delighted to hear!

I've been working on a new kind of LLM-based programming language, combining strengths of both LLM and PL: Making controllable LLMs that can handle constraints & algorithmic tasks, in a PL that has world knowledge and takes vague natural language instructions.

@erhartford @jstevh What kind of power are you talking about here?

@werg @jstevh I don't think that is powerful enough to express all the ideas that human language can express

LLMs should speak Universal Networking Language (UNL) or perhaps a new universal language inspired by it. Then, translation to and from user language done by fast, specialized models. Translate all the training data to UNL and pretrain on that.

@erhartford @jstevh Why not just use predicate logic?

@jstevh Of course! Claude Opus suggest UNL is the closest thing, but that it is lacking, it says that should consider to construct a new language that can be written and spoken as well, but that is designed to be easily and accurately translated into any existing human language.

@EspToTheFuture @tmdanis Super funny. The real word is "Torschlusspanik" not "Torschusspanik" --the later literally meaning "panic at having shot a goal" the latter "panic at gates closing". It's hilarious how likely tokenization primed ChatGPT to make it about sports, whereas it's actually about dating.

@tmdanis Asked my polyglot friend who’s pretty decent at German

can my german moots confirm

For years I’ve been interviewing data annotation workers who are the lifeblood of the AI industry. For years I’ve heard the same story: the platforms they work for wield total power, leaving them precarious & vulnerable to exploitation. A horrible example of this just happened 1/

@boisekitty55 Ration flying, ban private jets - make meat more expensive and veg free - ban cruise ships, create a de growth and circular economy - end economic growth and prioritize saving nature - stopping deforestation essential and banning bottom trawling and deep sea mining

@Euclaise_ Is the idea here that the matrix valued states are multiplied over time?

For AI tasks like reasoning and planning, $100M in GPUs can save you an afternoon in the library.

@iwelsh Such an effort can only succeed if it serves the needs of people in the future, as a simple ideological framework. Lots we can't anticipate. I wager that getting right with Nature will be important to people, materially and spiritually.

If you had to create an "enclave" to survive civilization collapse and maintain as much knowledge, culture and tech as possible, starting about now, how would do it?

@jxmnop @SebastienBubeck We use language to transmit non-obvious, low probability features, hence text is very high information. As visual animals we are fine with pleasing images passing by, we constantly see low information stuff. As language decoders we reject being told stuff that does not inform.

@SebastienBubeck image datasets are terabytes The entire text on the internet is < 1 tb

An amazing mystery of machine learning right now is that state-of-the-art vision models are ~2B parameters (8 gigabytes) while our best text models are ~200B parameters (800 gb) why could this be? philosophically, are images inherently less complicated than text? (no right?)