Bhavya Goyal

hahahaahah

@TrelisResearch check dm once

Best Medical Transcription (ASR) Models -- Evaluating and training models for medical transcription is notoriously hard because of the privacy aspects involved in gathering data. There are very few, if any, solid training sets for medical data, or even good datasets on which to evaluate performance. We had to get creative to create tests that are hard for today's leading models namely by: 1. Trelis/multimed-hard - 50 of the hardest rows from the medical-Youtube based MultiMed dataset. 2. Trelis/eka-hard - 50 of the hardest rows from the Indian-accented EKA dataset read out by medical students. 3. Trelis/medical-terms-2025 - a synthetic dataset of terms appearing only in 2025 on FDA, EMA and WHO sources. We tested 16 different proprietary and open source models and the TL;DR is that Google's proprietary model and Eleven Labs' Scribe v2 lead, while Whisper! is the most general high-performing open source model. Timestamps: 0:00 Introduction: three custom benchmarks for speech recognition models 1:09 Top performers revealed: Google ASR, Scribe, and Whisper 2:12 Entity CER metric: measuring error on medical entities only 3:17 Scribe v2 leads MultiMed Hard benchmark with 13.4% error rate 5:22 Generalization as key factor: Whisper, Gemini, Scribe v2 perform well 6:32 Gemini 2.5 Pro tops EKA Hard benchmark for Indian-accented speech 8:30 MultiMed ST model fails on out-of-distribution Indian-accented data 9:31 Synthetic medical terms dataset from FDA/EMA/WHO sources 10:58 Medical terms benchmark targets 2025 data for generalization testing 12:03 Gemini Pro leads; Speechmatics shows weakness in medical domain 14:04 MultiMed dataset filtering criteria: audio length, text characters, caps 16:09 Median CER methodology using three models to avoid bias 18:21 Medical terms dataset creation process using Kokoro TTS 19:29 Best models achieve 10-20% error rates on hardest medical terms 20:36 Datasets available on Hugging Face: search "Trelis medical"

@drummatick Better to get deep into fundamentals I feel

Stop giving too much attention to every AI repo that pops up. Stop thinking "everyone is doing so much, I'm being left behind". Stop thinking "i need to max out my tokens, learning isn't important". Don't let yourself drown in AI psychosis. I will be honest, there so much noise with AI right now. Some people are building incredible things but there's just too much noise. I spent last weekend going through repos that go viral, to see if they're actually special and work. I found hard-coded paths, very little system design only MD files spread here and there hoping for LLM to find them somehow, and disappointing final results. Won't be going into that detail right now but if you're really not sure, next time you see a viral repo on here, go ahead, install it and try it. You maybe amazed but i was 90% of the time disappointed with no only product/repo but how hyped it was.

@freshlimesofa which?

I am descending my gradients using optimizers you've never heard of.

gemini is the shittiest llm I have ever used

What are algebraic neural networks??

@shhrreeyyy Easyyyyy

@baldwin_IVth 1 book that u would recommend ?

Okay so a whole as list of books for stats and ml/dl ive used until now - Statistics >Start with a book called Naked statistics, yes, you read it right, its a wonderful conversational book, explains you stuff in ideas rather a lot of equations. Easy read. Chill. >Hit up the godly ISLR - Intro to stats learning, nothing better, again a very conversational book, a bit heavier on math and code, but a ten on ten recommended if you wanna mog everybpdy else on stats. It also gets you to ML grind in later chapters. ML >Tom Mitchell - theory might not be great, but very very intuitive book, also great, absolute bonkers problem sets. Loved em through and through. >Huda Hart - Heavy driver, use as a reference rather text. >Christopher Bishop - Absolute KING. Its the best pre workout youll ever have. >Godfellow - absolute must have, simply elegant, again a great ref. Also, use wikipedia a lot for stats and ml. Its amazing. Maybe try to avoid YT channels, except the authors of books themselevs, please avoid indian youtubers who prepare you for exams and shit. PLEASE. Also, do all this on a pen and notebook, do the math, work it out, you know, see what happens if you chime in MSE instead of BCE in logistic regression, see how convolutions work at the hessian level.

@jino_rohit Same here it takes me a lot of time to understand stuff

ive been thinking about things that im particularly good at. i see people ship at the speed of light, i see people have an incredible taste for research, i see people able to concentrate for long hours and grind, i see people do extremely well at math. im none of them. but what i am is im a slow learner, i take usually a couple of days before i can say i own this topic. im usually frustrated when i cant understand a certain topic for hours, but im able to come back to it the next day to try again. but once i understand this topic, im able to condense it and this relates with a lot of people. im able to almost immediately able to build on top of it with my mental model. i think this is my edge, its time to bet on them and see how far theyll take them. i wanna go all in on ml systems this term and do amazing things!

@TheGlobalMinima wdym by distributed systems speciifcally?

With the way things are moving, the next few years will converge into > reinforcement learning > distributed systems > kernel programming picking one of these will be the best way to future proof your career

@jino_rohit @dejavucoder @willccbb Easy

@dejavucoder @willccbb 6-8 months and ill be there too, amen

prime intellect continues to poach amazing people

@Priyanshu__2109 That's why I never claim I am a researcher

bro reading transformer paper doesn't make you AI researcher.

@sontyaaaa man u can take loan and stuff bro but dont go to shitty unis in india

@777BHAVYA Acad scholarship tak toh ho jaayega maybe but living ka idk

Got rejected from Mtech Robert Bosch IISc. They shortlisted on CPI basis maybe 😭😭. Kya fayeda itna research experience hone ka

@sontyaaaa germany isnt that expensive ig ?

@777BHAVYA No budget 😭😭

@Resorcinolworks ofcourse

I can see it coming Backend and ML will soon become a joint role. If you only know one, you won’t be able to safeguard it for longer period.

@Resorcinolworks this questions are covered in cs229 and prob/stats coursess

Okay I need to get something straight - ( big post alert , feel free to skip) No matter how many of us try to look down on preparation that only focuses on being the “getting the job”. It also damn true that these questions are actually quite based. It is as if they are designed and asked in such a way that whether you like it or not, you become aware just by knowing stuffs. Truth is that being aware of what is asked in interviews or the mindset needed crosses paths with actually become good at it. Most of the people here have large numbers of follows and reach for the FAFO kind, just projects and eye candy terms. But let’s be honest, you are not 100% know each line does what in your project, unless put years in into it. And you cannot overpower the theoretical part.

@cneuralnetwork Gpu credits

what are some cool things to buy online (<5k inr)

@neural_avb @archiexzzz @adithya_s_k @auto thanks thanks

I see. Do check for common bugs first (like instruction not been included in the training sequences, or your max length been too small and things getting truncated etc) Then I’d look if you’re underfitting and whether larger lora rank helps Then I’d go for augmentation strategies. I’ve been working on some stuff to generate training pairs from basic data you may find some of these ideas useful for augmentation: github.com/avbiswas/text-… 17k rows of data isn’t exactly insane, esp if there’s no prior CPT or base performance in finance domain for the model you are tuning. The final pass after you have a workable SFT model is to do preference optimization style things by generating multiple rollouts and manually/get an AI pick out the better ones.

Question for the folks working on real post training and not on toy sft datasets how do I clean and normalise the data any repo/resources? @archiexzzz @adithya_s_k @neural_avb @auto