Yongsheng Wu

Hyperspell (@hyperspell) gives AI agents memory. It connects to Slack, Gmail, Notion, and Drive, so agents get context and remember what they learn. Congrats @conor_ai and @maebert on the launch! ycombinator.com/launches/Ocs-h…

Most benchmarks test agents on short-horizon reasoning, not real-world, long-running tasks with stateful environments. To solve this, @agenthublabs is launching Benchmarks: one-click testing for your agents on realistic, economically valuable tasks, starting with E-Commerce and CRM automations. No infra setup needed. Try it for free at agenthublabs.com.

People have asked about our culture and recent employee communications. Cognition has an extreme performance culture, and we’re upfront about this in hiring so there are no surprises later. We routinely are at the office through the weekend and do some of our best work late into the night. Many of us literally live where we work. We know that people who joined Windsurf didn’t expect to join Cognition and while we’re proud of how we work, we understand it’s not for everyone. We gave our team the opportunity to decide. We offered for all employees who joined via acquisition to opt into our culture with full clarity on what entails. We know that we will lose some strong talent in doing this, but we truly believe the level of intensity this moment demands from us is unprecedented. While not everyone is looking for a culture like ours, everyone deserves respect and appreciation for their work. Regardless of their decision, we accelerated and cashed out all four years of equity for everyone from Windsurf, even for the 85% of employees who hadn’t hit their one year cliff. And for those who opt out, we’re providing an additional nine months of pay on top of this.

You are staying up all night fixing AI slop, while @GitStart PRs are getting merged to production! Here is how🧵

I don't wanna say "I told you so", but I told you so. Quote: "Ilya Sutskever, co-founder of AI labs Safe Superintelligence (SSI) and OpenAI, told Reuters recently that results from scaling up pre-training - the phase of training an AI model that uses a vast amount of unlabeled data to understand language patterns and structures - have plateaued." ... @yannlecun/post/DCTeagdN_th?xmt=AQGzwugSx2clYnCPrawJ8Ait-sjxVVyyDLUThj1y7YGM5w" target="_blank" rel="nofollow noopener">threads.net/@yannlecun/pos…

I am excited to finally share that we have shipped over a billion lines of code by @GitStart, with under 1.7 average code review cycles to get it merged. Here is how we did it and why we are raising more capital on our initial bet ⬇️

Whether you’re looking to build a remote-first team or considering hybrid, there’s a lot to learn from those who’ve done it successfully. 👉 Check out the Tech Teams Today podcast for weekly episodes with insights from top engineering leaders. Shoutout to guests @yswu at @granica_ai, @pedrotabio at Vibrant, and Chris Coffey at @collegevine #RemoteWork #HybridTeams #DistributedTeams #EngineeringLeadership #FutureOfWork #RemoteFirst #EngineeringCulture

Max Levchin on how to disagree

Join Revelo CEO @lachdec and @yswu, VP of Engineering @granica_ai as they explore the essentials of AI and machine learning. Now available on all major podcast platforms. 🎧 Tune in now -> youtube.com/watch?v=Tho-I-… #AI #MachineLearning #TechPodcast #DataScience #EngineeringLeadership

In 2018, Sam Altman simultaneously managed YC(a $30B+ Startup Accelerator) , invested >$400M in startups on the side, and built the world’s best team of frontier AI researchers with Elon for Open AI. His answer to ‘How do you get so much done?’ is powerful. A thread 🧵

Generative #AI is rapidly becoming table-stakes tech. But what does it take to implement it across #engineering and product? Granica VP of Engineering @YsWu joins other top engineering leaders to discuss the issue. Watch now in this @Allstacks roundtable. bit.ly/3KkHkrs

Crowdstrike Analysis: It was a NULL pointer from the memory unsafe C++ language. Since I am a professional C++ programmer, let me decode this stack trace dump for you.

I’m more bullish than I have ever been about crypto. I have been building @Circle for over 11 years, and at no time have I been more optimistic than right now. I also believe that the overwhelming majority of people have an extremely narrow and limited understanding of what’s unfolding. And that’s super bullish too. This post explains why I am so optimistic. My perspective here draws on closely watching internet technology adoption life cycles over the past ~35 years. We’ve seen an unrelenting march of open networks, open protocols and open software, with layer upon layer of infrastructure on the internet that deepens its utility for society and the economy. Each successive wave has transformed major industries, improving utility for people, inverting or transforming unit economics, and opening up radical new possibilities. The collective contribution of open IP to this ongoing internet revolution actually appears to be accelerating, and crypto seems like it's on the cusp of catapulting society and the economy forward in tremendously powerful new ways. 11+ years ago when Sean and I were thinking about this space, it was totally apparent that crypto represented the next logical layer of infrastructure for the internet. Internet infrastructure had enabled frictionless, nearly free movement of data and seamless ability to connect and deploy software and hardware on a global network, and it was clearly struggling with its own success and weight. The internet lacked a layer for trust, and without that it was capped in terms of the utility it could provide to the world. There was no way to have fully trusted data, transactions or compute, which led to deepening dependencies on hyper centralized entities (corporate and government). At the same time, the role of the internet in society was proliferating, and the ability of the internet to play a larger and larger role in how society and the economy were organized was apparent. And it was right at this moment that Bitcoin burst onto the scene, and a ton of incredibly sharp technologists began to think more deeply about how the fundamentals of crypto could be expanded to provide a more generalized internet infrastructure that could be foundational to society and the economy. Digital tokens, issued on public blockchains, intermediated by smart contracts could unleash a trusted environment on a global scale that would be the foundation of how nearly all of the building blocks of society and the economy could become internet-native. This is what drew me into this space; I could see clearly then that this would unfold, that these new decentralized internet computers would achieve scale, and that it would ultimately usher in a wave of change that far exceeds the kinds of changes we’ve seen from the internet of information and communications. In 2013, these ideas were considered insane. Any affiliation with Bitcoin or crypto was viewed as highly fringe, probably illegal, and for most technologists, a largely uninteresting technology development. Back then, the technology was extremely limited, slow, expensive, complex to operate. Fiduciary institutions – banks, accounting and audit firms, insurance companies, regulators – were extremely hostile and deeply terrified to be associated with anything in this space. The primary focus of the media was on darknet markets, the silk road, and the winklevoss twins BTC purchases. But if you actually paid attention to what mostly young and highly creative builders were thinking about and doing, you could see clearly that the bigger vision was going to unfold, though over what exact time-scale it was not clear. For those of us who have been building and working in this space since 2012 (and many from even earlier!), it’s totally and utterly extraordinary where we are at now. And, as I like to say often, we are truly only in the very early stages of the adoption of crypto in the world, which makes me insanely bullish today, given how far we’ve come in the past decade. How far have we come? A non-exhaustive list of accomplishments and technical progress. Public blockchain infrastructure has evolved into its 3rd generation, providing global scale network computers that can handle large scale applications with trusted data, transactions and compute. There is a massive, thriving and growing competitive and innovative community of dozens of major blockchain network ecosystems, all around the world, that are constantly improving and innovating in the fundamental technology of these networks including in data availability, compute, security, privacy, transaction throughput, and so much more. We are at the early broadband stage of blockchain networks. Guess what comes next? We are seeing breakthroughs in security, privacy and scalability based on ZK tech, and now FHE. We can see a world where crypto computing becomes the basis for most significant and important applications. There are literally tens of thousands of startups all around the world building on top of this infrastructure. Digital assets have become an accepted part of the emerging global financial system, with virtually every major government in the world setting clear rules for how digital assets can be issued, used and traded. Bitcoin itself has become one of the largest and most important alternative investment assets on the planet. The biggest asset management firms in the world are offering products and services built on blockchain technology and offering investments into the underlying digital commodities. Crypto has become a global political issue, as its importance to national competitiveness becomes clear. Governments around the world are racing to compete with one another to figure this out and ensure that innovation in this space is both responsible but also fostered. We’re seeing product UX that unlocks consumer-scale usage in ways we’ve not seen before, giving a clearer view of how this will unfold for billions of users in the coming years. Most of the world's largest payments companies are actively using this technology and exploring how to expand their usage as the benefits of public chains and stablecoins become apparent to everyone. Stablecoins have exploded in scale and use, crypto clearest killer app, unleashing digital dollars in the world, bringing more people into the future onchain economy, and starting to fulfill the promise of banking the unbanked, lower the costs of remittances, and unlocking more seamless cross-border commerce. Stablecoins are becoming a legally defined and accepted form of digital money in nearly every major jurisdiction in the world. By the end of 2025, stablecoins will be “legal electronic money” almost everywhere, which sets them up to become a larger and larger portion of the $100T+ market for electronic money. The infrastructure to build, deploy and operate blockchain apps has advanced massively, with enterprise-grade products and services to help use these networks, with custody infrastructure that scales for end-user controlled self-custody to infrastructure that the world’s largest banks and asset managers can depend on. Developer tooling, SDKs, and knowledge are proliferating at an accelerating pace. More and more people are becoming “blockchain capable”. Massive consumer scale companies are bringing online apps that connect to public chains and use digital tokens for a wide array of use cases. National governments are investing in blockchain infrastructure, ecosystem development and passing laws to provide incentives to companies to build in their regions. We’re seeing more and more exciting uses of the tech gaining traction every week, from payments to social to gaming to ticketing to enterprise use-cases. I could go on and on and on, but the scale of all of this right now is truly astounding compared to where we were a decade ago. Like prior waves of open internet infrastructure, this wave is growing, and is coming on stronger every day and every week. And, like I said earlier, we’re still in the VERY EARLY STAGES in the adoption of crypto. That’s insanely bullish. What does this look like when digital tokens are a widely understood and legally used form of incentives, governance, and record-keeping all around the world? What does it look like when a larger and larger portion of finance and commerce constructs are executed and intermediated by smart contracts on public chain infrastructure? What does it look like when 4th generation blockchain networks support billions of users and millions of applications? What does it look like when onchain organizations are legally defined and explode and compete for organizing labor and capital and consistently outperform legacy multi-national corporations? What does it look like when political bodies – cities, states, nations, and new network states – adopt onchain governance and improve how democratic values are expressed in the age of the internet? What does it look like when 10% of global economic money is stablecoins, and when credit intermediation moves from fractional reserve lending to onchain credit markets built from the ground up on safer, digital cash instruments (e.g. stables), and opens up credit and debt to the long tail of supply and demand in the same way that Amazon did for commerce and AdWords did for advertising? All of this is achievable over the next 10+ years. The time goes by fast, but when you zoom out and look at what has been accomplished and how that sets us up for the future, it’s hard not to be insanely optimistic right now. JA

Day 0 of @OpenAI with @thefriley ♥️

📽️ New 4 hour (lol) video lecture on YouTube: "Let’s reproduce GPT-2 (124M)" youtu.be/l8pRSuU81PU The video ended up so long because it is... comprehensive: we start with empty file and end up with a GPT-2 (124M) model: - first we build the GPT-2 network - then we optimize it to train very fast - then we set up the training run optimization and hyperparameters by referencing GPT-2 and GPT-3 papers - then we bring up model evaluation, and - then cross our fingers and go to sleep. In the morning we look through the results and enjoy amusing model generations. Our "overnight" run even gets very close to the GPT-3 (124M) model. This video builds on the Zero To Hero series and at times references previous videos. You could also see this video as building my nanoGPT repo, which by the end is about 90% similar. Github. The associated GitHub repo contains the full commit history so you can step through all of the code changes in the video, step by step. github.com/karpathy/build… Chapters. On a high level Section 1 is building up the network, a lot of this might be review. Section 2 is making the training fast. Section 3 is setting up the run. Section 4 is the results. In more detail: 00:00:00 intro: Let’s reproduce GPT-2 (124M) 00:03:39 exploring the GPT-2 (124M) OpenAI checkpoint 00:13:47 SECTION 1: implementing the GPT-2 nn.Module 00:28:08 loading the huggingface/GPT-2 parameters 00:31:00 implementing the forward pass to get logits 00:33:31 sampling init, prefix tokens, tokenization 00:37:02 sampling loop 00:41:47 sample, auto-detect the device 00:45:50 let’s train: data batches (B,T) → logits (B,T,C) 00:52:53 cross entropy loss 00:56:42 optimization loop: overfit a single batch 01:02:00 data loader lite 01:06:14 parameter sharing wte and lm_head 01:13:47 model initialization: std 0.02, residual init 01:22:18 SECTION 2: Let’s make it fast. GPUs, mixed precision, 1000ms 01:28:14 Tensor Cores, timing the code, TF32 precision, 333ms 01:39:38 float16, gradient scalers, bfloat16, 300ms 01:48:15 torch.compile, Python overhead, kernel fusion, 130ms 02:00:18 flash attention, 96ms 02:06:54 nice/ugly numbers. vocab size 50257 → 50304, 93ms 02:14:55 SECTION 3: hyperpamaters, AdamW, gradient clipping 02:21:06 learning rate scheduler: warmup + cosine decay 02:26:21 batch size schedule, weight decay, FusedAdamW, 90ms 02:34:09 gradient accumulation 02:46:52 distributed data parallel (DDP) 03:10:21 datasets used in GPT-2, GPT-3, FineWeb (EDU) 03:23:10 validation data split, validation loss, sampling revive 03:28:23 evaluation: HellaSwag, starting the run 03:43:05 SECTION 4: results in the morning! GPT-2, GPT-3 repro 03:56:21 shoutout to llm.c, equivalent but faster code in raw C/CUDA 03:59:39 summary, phew, build-nanogpt github repo

Lessons on Eng Productivity from 200 Engineering leaders #JimGrundner @allstacks @yswu @granica_ai (via #PlatoElevate @platohq)

.@PlatoHQ's #ElevateConference 2024 will challenge your views on what you know about engineering. Today, our VP of #Engineering @YsWu is one of 200 engineering leaders sharing lessons on productivity at a workshop hosted by @Allstacks! Join us! ⬇️ bit.ly/3RfNcWR

We're delighted to announce Maria Zhang has joined GitStart's board. As former VP at Google and seasoned industry veteran, Maria brings invaluable experience to our mission, and we're excited about the future with her onboard! gitstart.com/blog/maria-boa…

Bestie Q&A: "If you were 25 years old, where would you live to maximize your financial returns?" @chamath's answer: "I think that this is the most obvious answer in the world... which is the Bay Area." Specifically, south of SF in the peninsula. Why? "... there's literally nothing to do there." "If you're trying to do something really important, it takes enormous dedication, almost to the point of having no other priorities." "And that is much easier to do in a place where there are no distractions." "This is the place where you go to get work done." "(Startups) are these very ambitious ideas that can very quickly die if people don't breathe life into it." "And I think you're much more likely to do that when you have nothing else to do except to give your life to that idea." Do you agree?

Almost all of the federal funding to universities goes to STEM and biomedical research. The amount going to social science is a rounding error, and to humanities (including social criticism), it's essentially zero. STEM research in universities is the root of *all* the technologies your companies are based on. Do you really want to kill the golden goose? Note: HHS funding mostly goes to medical research and life science through NIH. Cutting those funds wouldn't just kill the golden goose. It would kill people.