waterloo intern

- 230 training runs - 1,623 GPU hours (67 B200 days) - 76 TB of training data - a 2x faster model Every paper said it can't be done. Quantization Aware Distillation made it possible.

infra work is interesting

@alphatozeta8148 🙈

@waterloo_intern 📸

committing a federal crime made me more money than my entire net worth from working my day job. this is insane.

@aishwarya_08 caught 📸

@waterloo_intern lol what?

to be clear i am a law-abiding citizen, this is in reference to below

@amiruci ``` dominate them so thoroughly that the comparison looks embarassing ``` should be our new logo

@waterloo_intern this is what you made me do

We now have a product specifically created for AI labs and their closed-weight models: we'll take care of not just inference, but auth, rate limits, metering, and billing integrations. We'll take care of providing both shared and dedicated inference, compliance needs, and matching end customers' geo requirements (us, ca, eu, uk, aus, jp, etc). It's called Baseten Frontier Gateway and is already battle-tested by multiple AI labs, like Poolside and their impressive Laguna M.1 agentic coding model.

@modal this is a sick read...hats off to you guys

@philipkiely HAHAHAHAHA

Developing empathy for LLMs by doing benchmark problems by hand.

this was so fun to work on, i hope you find it useful tried @baseten for GPU access?

@edenchan solid to the power of solid squared

Think fast! This is the voice that powers customer support and sales for Starlink

@Millanphilipose @kanjun your own WHAT... flex 🙈

@AliesTaha @kanjun We're using our own datacenter for now

Twitter’s algorithm is optimized for addiction, not for us. We deserve better. We’re releasing Bouncer today so you can take back control of your feed. Describe what you don't want, and Bouncer removes it. It’s free, doesn’t collect your data, and will be open source soon.

@anandcpatelmdms @part_harry_ x.com/Goosewin/statu…

@AliesTaha @part_harry_ Smelly?

we dug into 1-bit bonsai with @part_harry_ the grand canyon of a gap they showed... is just THREE (3) points away from normal PTQ but they already knew that here's the graph (fixed)

@nisten @part_harry_ we used their axis to plot on their chart their benchmarks to get the intelligence scores the x-axis is the weight file size this is what PrismML used

@AliesTaha @part_harry_ The graph compares model sizes not total memory use dumbass. You're comparing total kv of 1bit float16 vs finetuned fp4PTQ /fp8 kv at 4k context benchmark or like... what are you even comparing? x.com/nisten/status/…

@HenkPoley @part_harry_ fair, the point is more that the graph was designed to make 3 points look like a generational leap

@AliesTaha @part_harry_ 3 percentage point better is still quite a bit better. 🤷‍♂️ 73.8 to 76.8, about 11% less errors on these tests. Given that most of these tests have errors, so a perfect score cannot be achieved, probably even a bit better.

@JoshPurtell @part_harry_ more eyes on benchmarks is only a good thing

@AliesTaha @part_harry_ Taking this as permission to publicly sanity test forthcoming Baseten results/research

@oneill_c whoaaaaa