Casper

@abacaj The problem is that most of the Chinese models are inherently not open and free because they use very restrictive licenses.

The LLMs coming out of china performing really well on benchmarks but I don’t hear much on how they actually perform on real tasks or if anyone outside china is actually using them

@ylecun Now we just need to properly open Llama 3. Make it Apache 2.0 or MIT license and you can finally call it truly open source. Follow Mistral, they set a great example.

Open source AI foundation models will wipe out closed and proprietary AI models for the same reason Wikipedia wiped out generalist commercial encyclopedia: crowd-sourced human contributions to open platforms can cater to a high diversity of interests, cultures, and languages.

@aton2006 @teknium @markatgradient @togethercompute There is a gap in knowledge in general about how quantized models work. The reason most quantized models are slower is because of the compute overhead of high batch size + dequantization.

@teknium @markatgradient @togethercompute I don’t understand why this would be the case. Throughput should be a function of number in the batch by number of sockets, latency should be a function of number of sockets for a batch of 1. Quantization should not impact this.

.@togethercompute's API is crazy fast

@WifeyAlpha @ChiefQuant Been trying since summer 2022, maybe one day 🤞

This is incredible competition from AMD. They now beat Nvidia in both speed and price for inference. They just need to catch up with robust software - with time, it looks like they will beat Nvidia in the performance/cost trade-off. community.amd.com/t5/instinct-ac…

@BlancheMinerva Open source is nothing without open and free weights. Last time I checked, all the great Chinese models are gated by custom licenses and applications to even use them. Doesn’t quite meet my definition of open or free.

China has larger open source models than LLaMA as well as larger closed source models than any open source model. This isn't a real policy position, it's just jingoism.

@far__el Someone uploaded it and split the weights huggingface.co/someone13574/m…

@KyeGomezB They forked their own code and improved it, nice. They trained better models too, double nice.

Mistral's team litterally just forked llama changed less than 40 lines of code and raised 120+M. Investors what are you doing 😢 this is horrendous

@lukebelmar Tether is the last one to go. Until then, crypto is a ticking time bomb.

My problems isn’t Bitcoin. My problems is USDT inflating the price of Bitcoin. What happens if Tether gets smacked the same way Binance did? Does BTC hold? I’m skeptical.

@E0M The authors of this post used GPT-4 to evaluate if a certain phrase was in the text. This does not have any resemblance of a real evaluation and certainly not something I would expect OpenAI to be using

We theoretically could provide any context length we wanted, but effectively using it is a whole different story

@SteveMoraco @OpenAI @GregKamradt These posts are only made to farm reactions on Twitter. Everyone should know that using GPT-4 as an evaluator says nothing about another model as GPT-4 is biased towards itself (surprise)

This Claude 2.1 vs. GPT-4-Turbo chart absolutely blows my mind. Insane levels of fidelity from @OpenAI. Credit: @GregKamradt

Can someone make GPT-4 good again?

@TXMCtrades Good luck with this. There is nothing that gets close to Bloomberg or Refinitiv for real-time data. They collaborate with the government that sends it directly to them.

@EMostaque An LLM as good as GPT-4 with MIT license

What is being released tomorrow? Guesses go here 👇

@parth007_96 @woosuk_k They do acknowledge this in the blog from what I’m reading - DS MII is faster in the case of large contexts and small outputs. They also note that they will optimize this part of vLLM with the SplitFuse strategy.

@woosuk_k I think the two blogs are comparing different load distributions. Deepspeed blog used avg prompt size of 2000 and output size of 128 whereas the image in your post has prompt length 500. Would be good to see the reproduction on both settings. Btw long prompts are fairly common

We’ve just released a new blog post comparing vLLM with DeepSpeed-FastGen. While we are happy to see the open-source technology advancements from the DeepSpeed team, we’ve got different results with more extensive performance benchmarks. vLLM is actually faster than DeepSpeed in many common scenarios. Details here: blog.vllm.ai/2023/11/14/not… (written with @zhuohan123, @simon_mo_, @eqhylxx)

@AravSrinivas Unironically, this happened super early with ChatGPT. When security experts who know nothing about search & LLMs start fearmongering about your product - you know that you have made it.

Perplexity user: “Microsoft banned your domain, so can't use it on work devices”.

That's it, switching from OpenAI to @perplexity_ai. It's simply better in my experience. Now with GPT-4 Turbo, the ChatGPT experienced has dropped for my coding tasks and an effective search for context that Perplexity leverages gives me more accurate answers.

2008 Financial Crisis timeline 👇

@EMostaque Any chance of a commercially compatible license?

Got lots of open models to release this month & other announcements. The creation => control => composition pipeline across every modality will evolve rapidly thanks to open models, super exciting. Holodeck & Star Trek future here we come.. Open source ftw 🚀

@osanseviero Where is the code to reproduce these results?

NEFTune - an exciting paper many missed 🚀 With a very simple trick, your supervised fine tunes will get significantly better metrics! Let's learn about it🧑‍🎓 What's the simple trick? During training, add some noise to the output of the embedding layers. That's it! It sounds too simple, but it helps a lot! Fine-tuning Mistral-7B on Guanaco, @_lewtun obtained a 25% boost! 🤯(the boost is dataset dependant, of course) Here is another one from @tomgoldsteincs 👀 It's as simple as doing this in your forward pass if training: return orig_embed(x) + noise else: return orig_embed(x) How to use this?🚀 For trl, it's as easy as adding neftune_noise_alpha to the trainer's init (check the paper to see what is controlled by it) This method seems to work well independently of model sizes and model types. It also works with QLoRA! Paper: arxiv.org/abs/2310.05914 Thanks to @tomgoldsteincs @neeljain1717 et al for the paper, and @younesbelkada @lvwerra and @_lewtun for the quick integration into the trl library