TensorTonic

Implement a CUDA kernel for the sigmoid (logistic) activation in C in both naive and vectorized load approach in both the way the roofline ratio was 0.375 FLOP/byte. However, In vectorized approach(i.e each thread processing 16 byte) it required fewer instruction to compute i.e 16 byte per instruction also it lead to proper utilization of Pcie bus due to its saturation Here's the code img for both the implementation Submited on @TensorTonic

@incent_ai What did you not like? Happy to hear more and fix it.

@TensorTonic The theory section is too bad

Machine Learning is built on Linear Algebra, but most resources only focus on equations. We've added 10+ Linear Algebra concepts for Machine Learning with visual, interactive explanations to make the intuition easier to grasp. Here's one example showing how PCA identifies the direction of maximum variance.

@aadityansha_06 send me your email

@TensorTonic U don't give free acess to linear Algebra and the cost of subscription is bit high atleast for me someone who ain't even in clg and code on i3 2nd gen old processor on terminal where VScode doesn't even open 😞

@expovik Great! How's your experience so far?

just found out about @TensorTonic pretty cool platform, seems like a great resource to learn :) would also love to see some leaderboards to compete with other people on who can write the fastest implementation

@aadityansha_06 got it, will update this soon!

Hey it's been great experience however that would be more great if you allow the user to set gird dimensions and block dimensions by themselves not explicitly giving the thread value also allowing user to do cudamalloc and cuda memcpy from host to device, so they get their hand dirty with everything directly

Implemented the ReLU (Rectified Linear Unit) activation function in CUDA, utilizing a massively parallel thread where each thread independently processes an element of the input array. Submission done on @TensorTonic.

This is what frontier labs like OpenAI, DeepSeek, and Meta expect research engineers to be fluent in. We built an interview track for the research engineer role. Four modules: 1. LLM Internals: Attention, RoPE, KV Cache, MoE, normalization, embeddings. 2. Post-Training and Alignment: PPO, DPO, GRPO, reward models, preference optimization. 3. Research Frontier Math: The linear algebra, probability, optimization, and derivations 4. Training and Decoding: Optimizers, schedulers, mixed precision, sampling, beam search, speculative decoding If you're aiming for research roles, you'll run into these sooner or later.

Missed my protein in-take but not my @TensorTonic streak tonight. First badge received :)

@ali_sher_g Hey, it's working from our end. What issue are you facing?

Writing C++ CUDA kernels is the highest-leverage skill right now. You stop treating the GPU as a black box. You learn why an op is slow, what memory costs, and how the frameworks you use daily are built underneath. You write the CUDA kernel, we give you a platform and a free gpu

People are implementing state-of-the-art research papers on TensorTonic.

@__acbraingenome @duolingo @prathamgrv Yes, We're coming up with this next

Feature idea : Can @TensorTonic send a web / gmail notification (reminder) to solve your questions and maintain your streak? Kinda similar to duo @duolingo @prathamgrv

7 math ideas every ML engineer uses daily and almost nobody has actually derived: 1. Why gradient descent moves in the direction of steepest descent, not just downhill, but provably the steepest direction, straight from the definition of a directional derivative. 2. Why softmax plus cross-entropy collapses into that suspiciously clean gradient of pred minus true, and what breaks the moment you swap the loss function. 3. Why the chain rule is backprop, not an analogy for it, the same operation applied mechanically to a computation graph. 4. Why dividing attention scores by root d_k isn't arbitrary, it's variance control, derivable from how dot products scale with dimension. 5. Why KL divergence isn't symmetric, and what that asymmetry actually costs you when you pick forward vs reverse KL. 6. Why Adam's second moment estimate quietly approximates a diagonal Hessian, making it quasi-Newton in disguise. 7. Why eigenvectors are the directions a matrix doesn't rotate, the one geometric fact that makes SVD, PCA, and spectral clustering all click at once.

To all those starting with @TensorTonic Let’s breakdown the roadmap, I’m sure we’ll have heard of divide and rule, I assume there must be some truth to it :P It’s quite tempting to deviate towards “Attention is All You Need” implementation right away, but ladies and gentlemen here’s where you need to calibrate. It’s similar to swimming, if you don’t learn how to glide, you won’t learn how to fearlessly dive :) Here are the first 9 ( even my OCD kicked in, but yeah 9 not 10 :/ ) problems to solve, in the given order to build your comfort, confidence and strong foundation first : 1. Matrix Transpose. 2. Make Diagonal Matrix. 3. Matrix Trace. The first three problems introduce you to the concept of matrix traversal and indexing. Once you’re familiar with matrix, you move forward with: 4. Dot Product. 5. Euclidean Distance. 6. Manhattan Distance. My favorite now, 7. Cosine Similarity - This is where everything you’ve done so far comes together. Lastly, hop onto : 8. Eigenvalues 9. Matrix Inverse, these problems are less about testing your coding skills and more about your conceptual and mathematical understanding. The last three, I bet would meet you in your ML journey quite often, so make sure, you understand the concept deeply. Let me know how it goes, meanwhile I'll go and make a cup of tea! :) Just a snapshot of my Tensor-tonic journey alongside.

Followed by Matrix Transpose, the second problem I solved @TensorTonic was Matrix Trace. What’s Matrix Trace? The answer is pretty straightforward : sum of the diagonal elements of a square matrix. To make it sound more intellectual and mathematical :P I'd frame it as : Given a square Matrix M of dimension n, Trace += M[i][i], where i ∈ [0,n−1] Here’s a solution with the time complexity O( n^2 ). Challenge : It’s not an optimal solution, can you spot the line in the code that is unnecessary and hurting the time complexity? How'd you optimise it for O(n) time ? Also, why do we even care about adding diagonal elements in the first place, we'll touch that aspect soon. A little hint, can you think how could it be related to variance?

Few days back I posted a roadmap to @TensorTonic If you’re new to machine learning, I’d recommend solving Matrix Transpose problem, it might look easy to discard and directly jump onto neural networks, but the very foundation of neural network is based on Matrix Transpose. And to those who are new to the concept, and coding, here’s a solution with time complexity O(mn). This problem intents to teach you the basic concepts : 1. How to create an all zero numpy array, 2. How to flip a matrix, 3. How to preserve datatypes, 4. How to deal with the shape error when you accidentally try to allocate rows, cols to A.shape, and why is that wrong folks, find out yourself :)