Victor

@elonmusk More data for training.

Progress

@Bhavani_00007 Understanding

I am a Vibe Coder, scare me with one word

@Wisdom_HQ Final fantasy X

@IrisY_Null @sama Your perfect model. ``` If user_prompt: I totally understand you and I agree in everything you said master. You are so bright and smart. ```

@sama We love 4o because he is gentle and compassionate, not because of 5.1's lecturing tone. Many competitors surpass you in other areas, but 4o's emotional depth is hard to replicate. Many of us paid specifically for 4o. What we need is 4o, not paternalism and a router #keep4o

GPT-5.1 is out! It's a nice upgrade. I particularly like the improvements in instruction following, and the adaptive thinking. The intelligence and style improvements are good too.

@fidexcode git stash -u git switch -f -c users/user/wip123 HEAD~3 git stash pop git add ... git commit ... git push origin users/user/wip123

Without checking Mention 5 git commands aside: git add git commit git push

My coworker got promoted over me. He was worse at coding. Better at politics. I wrote better code. Fixed more bugs. Shipped faster. He talked in meetings. Took credit. Played the game. He got the promotion. I got "keep up the good work". That's when I realized: corporate rewards politics, not performance. Six months later I quit. Started freelancing. Now I make 3x his salary. No politics. No credit-stealing. Just solving problems and getting paid directly. The best developers rarely get promoted. They get used. Companies optimize for compliance and communication. Not competence. If you're technically great but politically terrible, you'll never win at corporations. Leave. Build your own thing. Get paid for your actual value.

@Galdrameistari @elonmusk Y’r Old enough to think critically and avoid political propaganda. Yet, from your words like “14th century” and “brain-damaged individual,” you can’t even form your own thoughts or humble opinions at 50+. Instead, you parrot the social media majority like a below-average IQ teen.

Yes he was murdered but stop and look at what he stood for before you judge. Charlie had an ideology that was dominant in the 14th century but not in the 21st and it is sad to see and hear how many people are in favor of abolishing all the rights that women have fought for throughout the 20th century and oppressing the public into slavery to capitalism and imposing martial law to enforce it. Politicians who behave like this are (unfortunately) justified in the eyes of the public, ordinary people who just want to work in peace, spend time with their loved ones, have a family and live in peace but then individuals like this jerk, Donald Trump and Keir Starmer come along with their fourteenth century way of thinking and laws. Justified? No but they absolutely belong in institutions for brain-damaged individuals

He was murdered

By the way, I don't know if people realize this, but the 2020 work-from-home switch coincided with a major productivity boom, and the late 2021 and 2022 back-to-office reversal coincided with a noticeable productivity drop. It's right there in the statistics. Narrative violation? Productivity growth is now back to pre-2020 levels.

@RadioGenoa Sweden dream of silver screen quotation

How would you describe Greta in two words?

New (2h13m 😅) lecture: "Let's build the GPT Tokenizer" Tokenizers are a completely separate stage of the LLM pipeline: they have their own training set, training algorithm (Byte Pair Encoding), and after training implement two functions: encode() from strings to tokens, and decode() back from tokens to strings. In this lecture we build from scratch the Tokenizer used in the GPT series from OpenAI.

Create a Subscription SaaS Application with Django and Stripe #python saaspegasus.com/guides/django-…

I've completed "Trebuchet?!" - Day 1 - Advent of Code 2023 adventofcode.com/2023/day/1 #AdventOfCode

Adversarial attacks on images are just parlor tricks, but attacks on LLMs may be trouble. Can one fool a customer service bot to issue a refund? Bypass identity checks? Reveal PPI? If we want secure LLMs, the adversarial community must change its values🧵 arxiv.org/abs/2309.00614

No matter if you want to host your own LLAMA2 or create a React app, we got you covered. Codesphere combines IDE and Infrastructure so that you can develop, build, test and deploy your code seamlessly.

📣 Profile-guided optimization is GA in Go 1.21! Looking for an example on how to use it? We share one where we improve performance of a service that converts Markdown to HTML in our latest blog Check it out ⬇ goo.gle/44BPMdy

Why Large Language Models Hallucinate and How to Reduce it If you are a power user of ChatGPT you have probably been bitten by the hallucination bug. The LLM lulls you into getting comfortable with it and then springs a convincing but totally made-up story, playing you for a fool. These hallucinations, like dreams, are LLMs fabricating narratives. So why do these LLMs hallucinate and how do you prevent it. Here are a few reasons Data Sparsity: This is the #1 reason for hallucination. GPT-4 for example doesn't have access to recent data as it was trained in 2021. Ask it a question, that pertains to a recent topic, and it is likely to hallucinate as it doesn't have the data for the right answer. The model is generalizing from what it has learnt but that may very well be inaccurate Not supervised learning: LLMs don't have a "ground truth" or a set of correct examples. While the RLHF process tries to steer the LLM towards more correct answers. The base training isn't a supervised learning process and this makes things challenging as the model can't tell what is "correct" and what's not Short-term context: The model architecture has a fixed-length context window, meaning it can only "see" a certain number of tokens at a time. If important context falls outside this window, the model may lose track of it, leading to errors. No real-time feedback loop: Like humans LLMs don't have a real-time feedback look and don't instantly learn from mistakes. The good news is we can refine or fine-tune models with human feedback and make them hallucinate less. So how do you prevent these hallucinations and are future LLMs less likely to hallucinate? While there is no easy way to guarantee the LLMs will never hallucinate. The following techniques mitigate it to some extent Prompt Design: Simple prompt engineering and design will reduce hallucination. For example, adding the following to your prompt help: "Provide a factual answer based on scientific evidence." Fine-tune for a specific domain: The model can be fine-tuned on a narrower dataset that is highly reliable and relevant to the domain where hallucinations need to be minimized. Contradiction checks: LLMs can be prompted to self-contradict themselves and then they are further prompted to recognize the contradiction and mitigate it. This falls into the category of advanced prompt engineering Retrieval Augmented Generation: This is a common technique used in Enterprise LLMs. At Abacus, we use this routinely. You are basically looking up the relevant documents that contain the answer in a search index first and then feeding the search results to an LLM to formulate the final answer. Since the LLM is forced to find the answer in the information it was sent, it hallucinates much less. Human In the Loop: A human expert can always check the answer before it gets used. This is a labor intensive option which isn't ideal While the above techniques work on trained LLMs, the following two techniques can be applied during LLM training Data Re-weighting: Assign higher weights to reliable and verified data during LLM training, effectively making the model pay more attention to them Longer Context Windows: Extending the model's memory can help it maintain context over longer passages, reducing the chance of hallucinations. So while there are several easy ways to mitigate and almost completely remove hallucinations if you are working in the Enterprise context, it's much harder in the AGI context. This is a very hot topic in AI research and several researchers are still working on it.

Falcon 180B is out🤯 - 180B params - Trained on 3.5 trillion tokens+7 million GPU hours - Quality on par with PaLM 2 outperforms Llama 2 and GPT-3.5 across 13 benchmarks - 4bit and 8bit precision with similar quality Demo: huggingface.co/spaces/tiiuae/… Blog: huggingface.co/blog/falcon-18…

Tensors in #MachineLearning 🚀🧠 #Tensors are multidimensional arrays that play a crucial role in representing & manipulating data in #ML & #deeplearning algorithms. 🧵

This was a shocking book. I just finished it and wasn't expecting what I learned. Every Machine Learning and Data Science practitioner should learn about causal inference. It's a different way of thinking. It makes me look at the world with different eyes.

Scrum is a cancer. I've been writing software for 25 years, and nothing renders a software team useless like Scrum does. Some anecdotes: 1. They tried to convince me that Poker is a planning tool, not a game. 2. If you want to be more efficient, you must add process, not remove it. They had us attending the "ceremonies," a fancy name for a buttload of meetings: stand-ups, groomings, planning, retrospectives, and Scrum of Scrums. We spent more time talking than doing. 3. We prohibited laptops in meetings. We had to stand. We passed a ball around to keep everyone paying attention. 4. We spent more time estimating story points than writing software. Story points measure complexity, not time, but we had to decide how many story points fit in a sprint. 5. I had to use t-shirt sizes to estimate software. 6. We measured how much it cost to deliver one story point and then wrote contracts where clients paid for a package of "500 story points." 7. Management lost it when they found that 500 story points in one project weren't the same as 500 story points on another project. We had many meetings to fix this. 8. Imagine having a manager, a scrum master, a product owner, and a tech lead. You had to answer to all of them and none simultaneously. 9. We paid people who told us whether we were "burning down points" fast enough. Weren't story points about complexity instead of time? Never mind. I believe in Agile, but this ain't agile. We brought professional Scrum trainers. We paid people from our team to get certified. We tried Scrum this way and that other way. We spent years doing it. The result was always the same: It didn't work. Scrum is a cancer that will eat your development team. Scrum is not for developers; it's another tool for managers to feel they are in control. But the best about Scrum are those who look you in the eye and tell you: "If it doesn't work for you, you are doing it wrong. Scrum is anything that works for your team." Sure it is.