Yoni Nazarathy

My Feb 2025 op-ed on (Australian) ABC Religion & Ethics: Continuing to insult and vilify “Zionists” will not quell Australia’s antisemitism crisis abc.net.au/religion/insul…

@joshdabelstein @FinancialReview Took the words out of my mouth. The 0.4% Jewish voice will certainly not determine who wins today. Yet the fact, that I also, as many Jews feel compelled to vote as a "Jew first", highlights the moral crisis that our country is in.

Something I wrote for @FinancialReview on tomorrow’s election - in print tmrw morning afr.com/politics/for-t…

While I clearly don't agree with @Sarah__Schwartz and the JCA, let it be clear that I DO NOT support any form of personal vilification towards Sarah and others that took part in the @QUT conference. All sides must remain respectful, honest, and responsible!

The book is now published! youtube.com/watch?v=Hetb-d…

A graduating McGill University student decided to spit on the Dean and another faculty member. The crowd cheered her on. If this is the end product of higher education, universities are in need of serious introspection and remediation.

LLMs add a whole lot of value to organizations. Yet there are risks. In this blog post we outline key cybersecurity and organizational risks that come together with LLMs. We also highlight how such risks can be mitigated. accumulationpoint.com/blog/post/risk…

Prompt Engineering Guide reaches 40K⭐️! Surreal moment to be honest. I've been so passionate about democratizing AI research and education since I started out in this space. Working tireless hours together with the open-source community and @dair_ai is the work I am most proud of. I am humbled by how well this project has been received and adopted: - It has been used by ~3 million students, developers, and researchers. - It has 150+ contributors and translated into 13 languages. - It has been adopted and used by big and small companies for enablement and the development of LLM projects and research. - It's been forked by many companies and research labs to guide internal and external documentation. - It has served as a foundation for our professional training and consulting business @dair_ai. - And a whole lot more. The reason I share this is because open-source projects can have a massive impact, unlike what others may tell you. There is a lot of sacrifice and hard work that happens behind the scenes for any open-source project and it has been no different with this one. We are expanding our efforts and welcome contributions and support of all kinds. We are also open to partnerships or collaborations that can help enhance and scale this project further and get it into the hands of more people. I recognize how much more work there is to do. For example, we are fascinated by topics like RAG, function calling, LLM Agents, LLM engineering, LLMOps, so we have ongoing efforts for those too. This is just the beginning. Thanks to our community and the folks who support our projects.

Apparently our book (@SaratMoka @BenoitLiquet) is already on Amazon even though it will only see the printer in several months. Good news is that online version will always be free on our website. amazon.com.au/Mathematical-E…

Foundations of Computer Vision by Antonio Torralba, Phillip Isola, and Bill Freeman available today and published by @mitpress. I haven't read it yet, but almost certain to be worth reading given the pedigree of the authors.

This blog post introduces the concept of prompts and prompt engineering for LLMs. If you're unsure about the distinction between user prompts and system prompts, this is a must-read. accumulationpoint.com/blog/post/it-i…

Looks great and very useful!

RAFT - Retrieval Augmented Fine Tuning 🔥 RAFT offers a method to fine-tune pre-trained LLMs for specific domain RAG settings. Conventional RAG is like an open-book exam, retrieving documents from an index to provide context for answering queries. This makes it more effective than the closed-book exam setting where LLMs rely solely on their pre-training and fine-tuning to respond to prompts, but doesn't allow the LLM to learn the domain beforehand. RAFT from @tianjun_zhang, @shishirpatil_ proposed a fine-tuning approach that enhances LLMs for domain-specific open-book exams, where the model is trained to attend to relevant docs and ignore irrelevant documents. RAFT creates a synthetic dataset where each data sample consists of: 1️⃣ A question 2️⃣ Two docs: An "oracle" document relevant to the question, and a "distractor" documents irrelevant to the question. 3️⃣ An answer generated from the documents 4️⃣ A Chain-of-Thought explanation including excerpts from the relevant documents. The created synthetic dataset is further used for fine-tuning to improve RAG performance. Thanks to @ravithejads, you can now generate RAFT dataset using our brand new RAFTDatasetPack LlamaPack. LlamaPack: github.com/run-llama/llam… Video Tutorial: youtube.com/watch?v=sqPckk… Paper: arxiv.org/abs/2403.10131

A private business LLM can be a valuable asset in several domains including resource query engines, generative engines, or advanced chatbots, each form has specific sub-tasks it can handle. Our new Accumulation Point blog post says more... accumulationpoint.com/blog/post/what…

JUST IN: Databricks introduces DBRX, a new 132B parameter open LLM Here is a summary of the technical details: DBRX outperforms all the established open-source models on common benchmarks like MMLU and GSM8K. DBRX was pretrained on 12T tokens (text and code) and uses a mixture-of-experts (MoE) architecture. Its inference is up to 2x faster than LLaMA2-70B and is about 40% of the size of Grok-1 in terms of both total and active parameter counts. There is also DBRX Instruct which demonstrates good performance on programming and mathematics. While DBRX is trained as a general-purpose LLM, it still surpasses CodeLLaMa-70 Instruct, a model built explicitly for code generation. DBRX was trained up to a 32K tokens context window and the instruction-tuned version outperforms GPT-3.5 Turbo on long-context tasks and RAG benchmarks. It still falls behind GPT-4 Turbo but the performance gap is reducing.

@JiahaoDiao Well done. Congrads!

Modeling gene content across a phylogeny to determine when genes become associated tandfonline.com/doi/full/10.10… The final chapter of my PhD work has been published. It has been such a long time since the first submission.

Our latest post in the LLM in Practice by Accumulation Point series. Explore the potential of open LLMs with Meta's Llama 2 models, Mistral, Google's Gemma, Falcon, Qwen 1.5, Stable LM Zephyr 3B, Microsoft's Phi-2, Cohere's Aya, and X's Grok-1. accumulationpoint.com/blog/post/open…

GPT-4.... also recognizes there is an easier way....

These provide a great introduction to LLMs!

Today we’re introducing SceneScript, a novel method for reconstructing environments and representing the layout of physical spaces from @RealityLabs Research. Details ➡️ bit.ly/3x2cOzh SceneScript is able to directly infer a room’s geometry using end-to-end machine learning and represent it using language. Compared to previous approaches, this results in representations of physical scenes that are compact, complete, interpretable and extensible.

I think AI agentic workflows will drive massive AI progress this year — perhaps even more than the next generation of foundation models. This is an important trend, and I urge everyone who works in AI to pay attention to it. Today, we mostly use LLMs in zero-shot mode, prompting a model to generate final output token by token without revising its work. This is akin to asking someone to compose an essay from start to finish, typing straight through with no backspacing allowed, and expecting a high-quality result. Despite the difficulty, LLMs do amazingly well at this task! With an agentic workflow, however, we can ask the LLM to iterate over a document many times. For example, it might take a sequence of steps such as: - Plan an outline. - Decide what, if any, web searches are needed to gather more information. - Write a first draft. - Read over the first draft to spot unjustified arguments or extraneous information. - Revise the draft taking into account any weaknesses spotted. - And so on. This iterative process is critical for most human writers to write good text. With AI, such an iterative workflow yields much better results than writing in a single pass. Devin’s splashy demo recently received a lot of social media buzz. My team has been closely following the evolution of AI that writes code. We analyzed results from a number of research teams, focusing on an algorithm’s ability to do well on the widely used HumanEval coding benchmark. You can see our findings in the diagram below. GPT-3.5 (zero shot) was 48.1% correct. GPT-4 (zero shot) does better at 67.0%. However, the improvement from GPT-3.5 to GPT-4 is dwarfed by incorporating an iterative agent workflow. Indeed, wrapped in an agent loop, GPT-3.5 achieves up to 95.1%. Open source agent tools and the academic literature on agents are proliferating, making this an exciting time but also a confusing one. To help put this work into perspective, I’d like to share a framework for categorizing design patterns for building agents. My team AI Fund is successfully using these patterns in many applications, and I hope you find them useful. - Reflection: The LLM examines its own work to come up with ways to improve it. - Tool use: The LLM is given tools such as web search, code execution, or any other function to help it gather information, take action, or process data. - Planning: The LLM comes up with, and executes, a multistep plan to achieve a goal (for example, writing an outline for an essay, then doing online research, then writing a draft, and so on). - Multi-agent collaboration: More than one AI agent work together, splitting up tasks and discussing and debating ideas, to come up with better solutions than a single agent would. I’ll elaborate on these design patterns and offer suggested readings for each next week. [Original text: deeplearning.ai/the-batch/issu…]