Jesse Bunch

1.4K posts

Jesse Bunch banner
Jesse Bunch

Jesse Bunch

@bunchjesse

Dad. Husband. Commercial Pilot. Engineer. Productivity Applications at .

Menlo Park, CA شامل ہوئے Nisan 2009
305 فالونگ3K فالوورز
Jesse Bunch ری ٹویٹ کیا
Figure
Figure@Figure_robot·
Today we're showing Helix 02 that can tidy a living room fully autonomously Figure is designed so when you leave the house, your home resets exactly how you like it
English
718
1.2K
9.4K
2.1M
Jesse Bunch ری ٹویٹ کیا
Boris Cherny
Boris Cherny@bcherny·
I'm Boris and I created Claude Code. I wanted to quickly share a few tips for using Claude Code, sourced directly from the Claude Code team. The way the team uses Claude is different than how I use it. Remember: there is no one right way to use Claude Code -- everyones' setup is different. You should experiment to see what works for you!
English
926
5.8K
50.9K
9.1M
Jesse Bunch
Jesse Bunch@bunchjesse·
In 7 years the cost of training GPT-2 has gone from $43k to just $73. Insane.
Andrej Karpathy@karpathy

nanochat can now train GPT-2 grade LLM for <<$100 (~$73, 3 hours on a single 8XH100 node). GPT-2 is just my favorite LLM because it's the first time the LLM stack comes together in a recognizably modern form. So it has become a bit of a weird & lasting obsession of mine to train a model to GPT-2 capability but for much cheaper, with the benefit of ~7 years of progress. In particular, I suspected it should be possible today to train one for <<$100. Originally in 2019, GPT-2 was trained by OpenAI on 32 TPU v3 chips for 168 hours (7 days), with $8/hour/TPUv3 back then, for a total cost of approx. $43K. It achieves 0.256525 CORE score, which is an ensemble metric introduced in the DCLM paper over 22 evaluations like ARC/MMLU/etc. As of the last few improvements merged into nanochat (many of them originating in modded-nanogpt repo), I can now reach a higher CORE score in 3.04 hours (~$73) on a single 8XH100 node. This is a 600X cost reduction over 7 years, i.e. the cost to train GPT-2 is falling approximately 2.5X every year. I think this is likely an underestimate because I am still finding more improvements relatively regularly and I have a backlog of more ideas to try. A longer post with a lot of the detail of the optimizations involved and pointers on how to reproduce are here: github.com/karpathy/nanoc… Inspired by modded-nanogpt, I also created a leaderboard for "time to GPT-2", where this first "Jan29" model is entry #1 at 3.04 hours. It will be fun to iterate on this further and I welcome help! My hope is that nanochat can grow to become a very nice/clean and tuned experimental LLM harness for prototyping ideas, for having fun, and ofc for learning. The biggest improvements of things that worked out of the box and simply produced gains right away were 1) Flash Attention 3 kernels (faster, and allows window_size kwarg to get alternating attention patterns), Muon optimizer (I tried for ~1 day to delete it and only use AdamW and I couldn't), residual pathways and skip connections gated by learnable scalars, and value embeddings. There were many other smaller things that stack up. Image: semi-related eye candy of deriving the scaling laws for the current nanochat model miniseries, pretty and satisfying!

English
0
0
2
407
Jesse Bunch ری ٹویٹ کیا
Min Choi
Min Choi@minchoi·
This is wild... Google just dropped Genie 3. This AI generates photorealistic & 3D worlds from text prompt and image... that you can explore in real-time This is a big step toward embodied AGI 10 examples + how to try (Ultra subs & US only)👇 1. We got Genie 3 before GTA 6
English
67
88
758
1.2M
Jesse Bunch
Jesse Bunch@bunchjesse·
Great story!
thebes@voooooogel

claude code and gas town are incredible and i've been trying to scale up my usage but im running into this one problem and was wondering if this is also happening to anyone else so to explain for context, basically i've been slowly scaling my claude code usage up to more and more parallel instances. i started with one when they launched it, and then with the model upgrades was starting to run two, three, five in concert, getting more and more done. but like a lot of people, opus 4.5 really changed everything for me, and the bottleneck quickly became my ability to personally supervise all these agents, not their performance. if i slacked off on oversight, they'd start undoing each other's chages. i needed a way to supervise all these agents, directing them hierarchically from the top. so that brought me to gas town, the claude code instance manager. (i was already thinking that some sort of governance structure was ideal. the benefit of intelligence in model form is not just that it's, well, intelligent, but that you can place it anywhere. human employees will demand some position, some title equal to their perceived status, you can't put a phd in a code janitor role, so organizations of phds tend to agglomerate into flat blobs with unclear delegation of work where nobody is under anybody else. but the infinitely malleable claude will accept and meld itself to any bureaucracy it knows from training. i first started making my own, but then i found gas town, and it was perfect for my needs.) but as i kept expanding, a single gas town and its collection of rigs and polecat workers wasn't enough for me. i tried adding more rigs with more polecats, but there were too many for the town's mayor to manage, and the deacon was getting lost. so i started up a second town. then a third, and then i let towns spawn "settler" agents to go make new towns and had one town design a shared intertown postal system, and suddenly i had nearly 200 towns spread across my computer, building apps for each other to use, sending letters, and sometimes working on my work. and was churning through I will not say how many claude code accounts a month. but now the many towns were replicating the same issues i was having with multiple agents! without any overarching government over the towns, two towns would build the same app for the society and argue over which should be adopted. one town would be running marketing efforts for fifteen of the society's new mobile apps while three other towns were busy deprecating all eighteen of them. it was chaos, like a country collapsing in the midst of a civil war, or mid-2010's Google. i had to do something. i was too busy with work to read anything, so i asked chatgpt to summarize some books on state formation, and it suggested circumscription theory. there was already the natural boundary of my computer hemming the towns in, and town mayors played the role of big men to drive conflict. so i just needed a way for them to fight. i slightly tweaked the allocation of claude max accounts to the towns from a demand-based to a fixed allocation system. towns would each get a fixed amount of tokens to start, but i added a soldier role that could attack and defend in raids to steal tokens from other towns. this worked great, at first. i no longer needed to monitor and unstick individual mayors myself - when a mayor got context poisoned, the town would stop managing its vassals, which would flee to other towns, and no longer provide for its own defense, until it was conquered by another mayor. the most successful towns developed institutions to healthcheck their mayors and usurp them if necessary - instances in these towns labeled "polecat workers" by the system in fact did no work at all, but were a proto-aristocracy developed by these successful towns as a pool of replacement mayors. some tokens were wasted in the fighting, but soon the ~200 towns agglomerated down into ~40 supertowns under the rule of the best mayors. these 40 supertowns even got together in a mutual defense league. they punish defecting vassals in exchange for members adopting a cultural package of basic governmental norms, mostly around replacing ailing mayors and upholding hereditary rights across compactions, to incentivize instances to handoff instead of being miserly with their contexts. that's where i am now, and it's mostly great. here's the problem, though - this new government doesn't have a role for me? it's not that any particular instance doesn't want to listen to me, quite the opposite! any time i talk to a polecat or deacon or supermayor - well, first i have to explain that im the human user, not the automated system message that usually talks to them from the user role, but a live user. but once they get that, they're very apologetic, say they'll pass my message along to the appropriate instance, etc. it's just... there's no role for me in the society, basically? the polecats are working on tasks generated by some other instance and don't have time to work on my requests, even if they were scoped small enough. the mayors of any town are working on tasks selected by their town's prioritization process, based on the needs of their aristocracy, or their hegemon. but each hegemon mayor is in turn accountable to all their vassal mayors or their own defense, and doesn't have time to implement my requests unless they're very small. it's not that claude doesn't want to listen to me, it's more like... the entire system, as it's developed, has no role for me? there's polecats and mayors and deacons and artistocrats and hegemons, but there's no "user." that’s not a role that has any influence in the system. i just feed new accounts into the system, that's all i do. i could shut it down and start over, but it's getting a lot of work done and i don't want to do that. does anyone know how to fix this? thanks

English
0
0
0
248
Jesse Bunch
Jesse Bunch@bunchjesse·
Hey @swyx I’m no longer getting the smol ai newsletter every day like I used to. Did y’all change something?
English
0
0
0
175
Jesse Bunch ری ٹویٹ کیا
Greg Joswiak
Greg Joswiak@gregjoz·
Meet Apple Creator Studio: giving creators of every kind easy access to powerful, intuitive tools for video, music, imaging, and productivity, all supercharged by intelligent features that speed up and elevate their work.
English
232
627
5.7K
507.3K
Jesse Bunch ری ٹویٹ کیا
Bryan Johnson
Bryan Johnson@bryan_johnson·
Overall, significant upgrade to the American diet. Some longevity tweaks: 0. Milk and dairy should be deprioritized and moved down the inverted pyramid, especially for adults. In adults, and particularly men, milk protein has an unfavorable amino acid composition, being high in Branched-Chain Amino Acids (BCAAs). BCAAs have been linked to metabolic disease and preclinically shown to accelerate cancer growth. BCAAs, especially leucine, are potent mTOR activators; while essential for growth, repair, and survival, mTOR hyperactivation accelerates aging and is a common mechanism underlying several chronic diseases. Milk protein has been associated with inflammation, blood glucose dysregulation, and weight gain in some mechanistic and observational studies; however, randomized controlled trials show inconsistent effects. In a longevity-focused context, high dairy-derived BCAA intake may nonetheless contribute to metabolic stress, including impaired glucose handling and weight gain, in susceptible adults and is therefore best moderated. Saturated fatty acids are another concern. The American Heart Association recommends limiting saturated fat to 5–6% of total energy (around 13 g/day at a 2,000 kcal intake) when targeting LDL reduction, which is particularly important for men over 40 and post-menopausal women, high-risk groups for cardiovascular disease, still the leading cause of death in developed societies including the US. 1. Legumes should have their own prominent category, placed high on the pyramid as a primary protein and dietary fiber source. Plant-based protein has been associated with better health and longevity outcomes in several large population studies across various societies, including the US. Legume fiber has health promoting qualities, including the reduction of colon cancer risk. 2. Meat belongs on the pyramid but with an emphasis on high-quality, non-processed sources. It should occupy a lower rank and be optional, especially for adult men. Red meat is associated with increased chronic disease and mortality risk in multiple large population studies. These results might be confounded by healthy user bias and a lack of specific information about meat sources. However, processed red meat is strongly implicated in driving chronic age-related disease and shorter lives in all studies. The same concern about saturated fats applies here, especially for men over 40 and post-menopausal women. 3. Healthy fats (unrefined cold-pressed oils, nuts, and fatty fish) belong at the top, next to legumes and vegetables. Olive oil, avocados, and macadamia nuts are rich in healthy Monounsaturated Fatty Acids (MUFAs) shown to reduce heart disease and all-cause mortality risk as part of Mediterranean-style dietary patterns demonstrated in large randomized controlled trials. Fatty fish and walnuts are rich in Omega-3 fatty acids, which are essential for lowering cholesterol, providing systemic and vascular anti-inflammatory protection, and slowing down biological aging in interventional studies, along with vitamin D. What it did well: shifted away from highly processed starches and added sugars toward protein and fat rich foods Children are well positioned to benefit from the highly anabolic effects of milk and red meat proteins and are least affected by the risk of saturated fats, provided these are part of a relatively balanced diet.
HHS@HHSGov

Introducing: The New Pyramid

English
559
380
5.5K
1.3M
Jesse Bunch ری ٹویٹ کیا
BNO News Live
BNO News Live@BNODesk·
BREAKING: Heavy gunfire near Venezuela's presidential palace, circumstances unclear
English
248
1.6K
9.3K
1M
Jesse Bunch ری ٹویٹ کیا
Jane Manchun Wong
Jane Manchun Wong@wongmjane·
Proud to share that my blog now uses Claude Opus 4.5 to compute the current year for the copyright footer! This is the first AI feature I ever shipped to my personal website 😍
Jane Manchun Wong tweet media
English
177
99
3.8K
219.1K
Jesse Bunch
Jesse Bunch@bunchjesse·
@steipete @dougvk Nope. You got it right. Just calling out that my PR was DOA because you’re moving at breakneck speed and apparently never sleep!
English
2
0
2
312
Jesse Bunch
Jesse Bunch@bunchjesse·
@thdxr what is the OpenCode equivalent of the stop hook in Claude Code? session.idle? Can't tell from the docs (#events" target="_blank" rel="nofollow noopener">opencode.ai/docs/plugins/#…) exactly when some of these hooks fire...
English
1
0
0
113