Ocedev

github.com/ocedev112/Neur… I built a deep learning library called - neonet. Designed as a simple and customizable alternative to TensorFlow and PyTorch I built it as a light weight alternative to full frameworks, also to make it easier for users in ML

@TabGames3 @GGSunshiners @kirbyalt45 Well AI definitely has its use cases

@OCE_devvv @GGSunshiners @kirbyalt45 It also can just look at stuff quickly. The other day I was networking a package that was too small. Rather than manually go through and add up all the data being sent, the AI was able to tell my I was 4 longs short from a quaternion I forgot about in only a few seconds.

And because of AI, the actual solution won’t be learned and documented. Is it really that hard to just debug it yourself or ask other programmers?

@TabGames3 @GGSunshiners @kirbyalt45 I get your point. But if AI can find the answer, it is definitely on the internet. Just hard to find. It’s not like AI solves novel problems. You can still research(it would be hard to find tho), but I am just thinking about the software industry in the long run.

@OCE_devvv @GGSunshiners @kirbyalt45 Personally I work on very experimental devices. A lot of things I'm working on, I can't discuss with other people. Especially when it involves prototype hardware. Most of the stuff I do has never been done before, so there's no one to even consult on it.

@TabGames3 @GGSunshiners @kirbyalt45 It’s creates a loop. Systems are poorly documented. Someone uses AI instead of interacting with the public community , and People don’t document the solutions publicly, leading to more poorly documented systems.

@TabGames3 @GGSunshiners @kirbyalt45 We’re taking about in this context working with a system with poor documentation.

@w8153328 @kirbyalt45 I think he/she might be taking about documentation and learning in the public sense. Like people won’t learn and document the solution publicly. Since they can just ask AI. It will cause a big issue in the long run.

@kirbyalt45 >And because of AI, the actual solution won’t be learned and documented. Not true at all, why do you think that you have such a big misunderstanding of whats going on?

@GGSunshiners @kirbyalt45 You don’t understand the point of the post. Yes you understand the code Ai gave at the end of the day, but the problem is in the long run. There will less public documentation for people to use because of AI.

@kirbyalt45 People really shouldn't talk about topics that they didn't know anything about. It is not a black box when you ask AI to do stuff in codes. It write in the same program that you write your codes. You can see and change it. So we still learn the solution themself, nothing is lost

@scaling01 Username makes this funnier

it's incredible how little foresight anthropic has

@MaximumADHD If you want to do a niche or hyper specific problem, that’s on you.And it’s a learning process, why would you want to use AI, it takes the fun out of it(well to each their own). The discovery of what happens and how it works. But every one is entitled to their opinion

The people who are complaining about AI use in programming have never had to deal with the mental strain of needing to deep dive to solve problems that are hyper specific and niche, using systems that aren’t well documented (or behind a bunch of gatekept white papers/academics)

Transformer by hand ✍️ ~ 6 steps walkthrough below Open the hood of a transformer and the parts list is overwhelming: embeddings, positional encoding, attention weighting, self-attention, cross-attention, multi-head attention, layer norm, skip connections, softmax, linear, Nx, shifted right, query, key, value, masking. Which of those actually make the car run? Two of them. Attention weighting and the feed-forward network. Everything else is an enhancement to make it run faster and longer, which is how we got from a car to a truck, and to the word "large" in large language model. So I drew and calculated those two parts entirely by hand. Goal: push five features through one transformer block, filling in every cell yourself. 1. Given Five positions of input features, arriving from the previous block. 2. Attention matrix Let us feed all five features to a query-key module (QK) and read back an attention weight matrix, A. The details of that module are a post of their own. 3. Attention weighting We multiply the input features by A to get the attention weighted features, Z. Still five positions. The effect is to combine features *across positions*, horizontally: X1 becomes X1 + X2, X2 becomes X2 + X3, and so on. 4. First layer Let us feed all five weighted features into the first layer of the FFN. Multiply by the weights and biases. This time the combining happens *across feature dimensions*, vertically, and each feature grows from 3 numbers to 4. Note that every position goes through the same weight matrix. That is what "position-wise" means. 5. ReLU We cross out the negatives. They become zeros. 6. Second layer Let us bring it back down: 4 dimensions to 3. The output feeds the next block, which has a completely separate set of parameters, and the whole thing runs again. You have just calculated a transformer block by hand. ✍️ The takeaway: the two parts are doing two different jobs, and neither one alone is enough. Attention mixes *across positions*, so a feature can see its neighbours. The FFN mixes *across feature dimensions*, so each position can think about itself. Horizontal, then vertical. Then that pattern repeats N times, each block with its own separate set of weights. That is the Nx from the list up top, and that is what makes the transformer run. 💾 Save this post! #AIbyHand #Transformers #DeepLearning

@ProfTomYeh Nice

@straceX Another day to shade python users, what did we do, lol?

C programmers watching Python programmers in action

@TrentKelly2472 @awindywalker @NCyotee Again you’re ignoring the discussion. The ops point was taking about WEIGHTs of the model, don’t know why you brought sampled tokens. And Claude weights not being open. If you want to fine tune using the weights or logits. It is completely different from what you said initially

@OCE_devvv @awindywalker @NCyotee Keep up man If you don't have the weights, you're training on sampled tokens, the text output of the model If you have the weights, to distil that into a smaller model you're still training on outputs, but the training target is more information dense You don't train on weights

Nothing was stolen to make AI. You don't get to control who can see your art when you post it online. That's why Fair Use exists. And Fair Use has a specific clause saying copyright doesn't apply for education, research, or technical usage. The clause exists because artists have no moral right to control who sees and talks about their art. And they definitely don't get to control who learns from their art. AI training fails under the research clause of Fair Use.

@SirProdigle @Mizrhamite @H3X_0x26 @Iced_IMP Huh???? You’re joking. How is watching someone paint a creative process. And how did I argue for it

@OCE_devvv @Mizrhamite @H3X_0x26 @Iced_IMP Is watching someone paint while the canvas is obscured art? Because that's what you're arguing. People think the work and method is important (usually), but they typically don't think it is art.

I’ve never met a single programmer who considers coding an art form

@TrentKelly2472 @awindywalker @NCyotee “You don’t train on another models weights” what do you think fine-tuning another initialized model is(not directly yes). where will you get the Full logits of a closed source model, what are you taking about?You can’t get all that information and you’re ignoring ops point.

@OCE_devvv @awindywalker @NCyotee You don't train on another model's weights regardless if you have them or not. If you did have a copy of Claude's weights you'd still train on its outputs, but the full logits would be more information dense than whatever individual tokens were sampled

@Mizrhamite @H3X_0x26 @SirProdigle @Iced_IMP Coding is the process.

@Mizrhamite @H3X_0x26 @SirProdigle @Iced_IMP The end product is not the art, to me. That’s why people don’t call AI art, art. It does not involve a creative process.

@tsundoku_ueno @ExistentialEnso Or are you too dumb to understand what a project template or boilerplate is,

@tsundoku_ueno @ExistentialEnso So your telling there is blocks of reusable sections of code FOR EVERY project. Is it that you guys don’t understand my point, I get she means there is repetitive things found in many projects. But not all projects have a boilerplate

People who say this about software engineering work make it clear they've never written a line of code in their life. For every interesting, challenging problem there's 100 blocks of boilerplate code that any senior-level dev could write in their sleep but eat up time and energy

@TrentKelly2472 @awindywalker @NCyotee Huh??? Claude is not an open weights model. You can not train their weights. Use their outputs to train, sure. But you wouldn’t find Claude’s weights on hugging face

@awindywalker @NCyotee You can and people do train on outputs from Anthropic's models. It's not even done in secret, they're regularly topping the trending list on HuggingFace, open weights releases that you can go download for free