CubanHodLer🇨🇺

This guy blocked me and I don’t really know why, must be my frozen model’s weights 🥶

Probably the craziest week in Open Source AI (yet): 1. Mistral (in collaboration with Nvidia) dropped Apache 2.0 licensed NeMo 12B LLM, better than L3 8B and Gemma 2 9B. Models are multilingual with 128K context and a highly efficient tokenizer - tekken. 2. Apple released DCLM 7B - truly open source LLM, based on OpenELM, trained on 2.5T tokens with 63.72 MMLU (better than Mistral 7B) 3. HF shared SmolLM - 135M, 360M, & 1.7B Smol LMs capable of running directly in the browser; they beat Qwen 1.5B, Phi 1.5B and more. Trained on just 650B tokens. 4. Groq put out Llama 3 8B & 70B tool use & function calling model checkpoints - achieves 90.76% accuracy on Berkely Function Calling Leaderboard (BFCL). Excels at API usage & structured data manipulation! 5. Salesforce released xLAM 1.35B & 7B Large Action Models along with 60K instruction fine-tuning dataset. The 7B model scores 88.24% on BFCL & 2B 78.94% 6. Deepseek changed the game with v2 chat 0628 - The best open LLM on LYMSYS arena right now - 236B parameter model with 21B active parameters. It also excels at coding (rank #3) and arena hard problems (rank #3) There's a lot more; Arcee (mergekit) released a series of LLMs, each better than the other, and Numina and HF Numina 72B (based on Qwen 2) and Math datasets, Mixbread with embedding models (english + german) and a lot more! It's fun to see so many releases next week with L3 405B (?) and companions; we might see a shift in the Open LLM landscape! See you next week! What else did I miss? 🤗

COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability arxiv.org/abs/2402.08679

that's all the code you need

supervision-0.22.0 is coming out today one of the things we release is Mediapipe integration along with default visualizers for face and body pose keypoints link: github.com/roboflow/super…

My friend and Neo4j CTO @prathle has written an outstanding blog post that summarizes the recent buzz around GraphRAG, what we've learned from a year of helping users build systems with Knowledge Graphs + LLMs and where we believe the space is going. Thread below. 👇🧵

Gradio Multimodal Demo for LLaVA-NeXT-Interleave😍 : semantic-sam.xyzou.net:6123 Models and Datasets are on 🤗 Hub: huggingface.co/collections/lm…

LLaVA-NeXT-Interleave🔥 - Interleave data format unifies different tasks. - New datasets on 🤗Hub: 1️⃣M4-Instruct, high-quality dataset, 1.1M samples from domains: multi-image, video, 3D & single-image 2️⃣LLaVA-Interleave Bench - Set of tasks to evaluate multi-image capabilities

🚀Introducing LLaVA-NeXT Interleave: Now AI can understand and reason with multiple images at once - This opens up multi-image scenarios like multi-frame videos, multi-view 3D, and multiple inter-leaved images. - An all round LMM that can understand videos, images, and 3D More⬇️

Florence-2 fine-tuning YouTube tutorial is finally out! (sorry it took me so long) - running the pre-trained model with different vision tasks - configuring LoRA - training and benchmarking - Florence-2 vs. top vision model link: youtube.com/watch?v=i3KjYg… ↓ key takeaways

🤯DiffIR2VR-Zero: Zero-shot video restoration to high-resolution using pre-trained image restoration diffusion models. - Framework can handle video denoising and up to 8x super-resolution - outperforms trained models in generalizing across diverse datasets and extreme degradations - Compatible with every 2D restoration model

Code dropped: github.com/ubc-vision/3dg… I've been waiting for far too long to try this!

OverlapMamba: Novel Shift State Space Model for LiDAR-based Place Recognition github.com/SCNU-RISLAB/Ov…

MeshVPR: Citywide Visual Place Recognition Using 3D Meshes github.com/gmberton/MeshV…

Weirdly, the ways things are going, only the LLMs will remember what we wrote, imperfectly.

The Internet is rotting. The destruction of MTV news is the latest. Over 25% of the links embedded in New York Tunes articles just seven years ago & 60% of older links, are now broken. It isn’t good that the only people preserving decades of digital data is the Internet Archive.

Nice survey on LLM-based synthetic data generation, curation, and evaluation. If you are working with LLMs, a lot of effort is going into these areas so it's important to get familiar with concepts. This survey is a good starting point.

💪Demo by @AnnioDance: huggingface.co/spaces/vilarin… Model on 🤗 ExVideo: huggingface.co/ECNU-CILab/ExV… Diffutoon: huggingface.co/camenduru/Diff… 🎥Create engaging experiences with Gradio's Video Component. Stream, analyze, and interact with video data seamlessly🌟 Visit: Gradio.dev

🚨Epic News: Demos for ExVideo and Diffutoon are out! 🤯 Play with the advanced parameters for greater control -- this might be the best open model and demo for text2video/image2video out there!

@tradernewsai @emollick The “financial press” manipulates on their own 😂

@emollick I was thinking the other day, and I’m shocked we don’t see manipulation of the financial press with this stuff. A voice-cloned phone call to a WSJ reporter after you buy or short a stock, etc. It feels like there are so many unknown unknowns out there.

Not enough IT security people know how much AI has compromised traditional approaches to security. They are perfect spear phishing machines, solid autonomous hacking systems, and great at voice & video cloning with just a few seconds of audio. And this is just the public stuff.

Florence-2, the new vision foundation model by Microsoft, can now run 100% locally in your browser on WebGPU, thanks to Transformers.js! 🤗🤯 It supports tasks like image captioning, optical character recognition, object detection, and many more! 😍 WOW! Demo (+ source code) 👇