Justin Allport

@patrickrh16 @usembassytokyo Check out Tyrell Corp.. I hear they're hiring. Full remote.

@usembassytokyo I used to work for them after my contract ended with Umbrella Corp. I thought Umbrella had a bad health insurance plan. I just started at ExoGeni and I'm optimistic. A lot of travel involved.

Deputy Chief of Mission Aaron Snipe recently met with Shoji Yutani, CEO of Weyland-Yutani Corporation, to discuss greater 🇺🇸 🇯🇵 coordination in deep-space exploration. With companies like Weyland-Yutani considering new large-scale terraforming and atmosphere-processing projects on distant planets, ties between government and private industry have never been stronger. #weylandyutani #LV426

@kevin2kelly I prefer to start in the city, soon after head out. Then returning to the city it actually feels a bit like home. Tokyo for the second time was magic, after visiting the countryside.

@kevin2kelly Agree in part, but that's a bit like running from a bright room outside and expecting to see the stars. The faraway place is usually more subtle, and a bit of time can help you to see it with good eyes. On day 1 you'll miss it.

Here in brief is the method I’ve honed to optimize a two-week vacation: When you arrive in a new country, immediately proceed to the farthest, most remote, most distant place you intend to reach during the trip. If there is a small village, remote spa, a friend’s farm, or a wild place you plan on seeing on the trip, go there immediately. Do not stop near the airport. Do not rest overnight in the arrival city. Do not pause to acclimate. If at all possible proceed by plane, bus, jeep, car directly to the furthest point without interruption. Make it an overnight journey if you have to. Then once you reach your furthest point, unpack, explore, and work your way slowly back to the big city, wherever your international departure airport is. In other words you make a laser-straight rush for the end, and then meander back. Laser out, meander back. This method is somewhat contrary to many people’s first instincts, which are to immediately get acclimated to the culture in the landing city before proceeding to the hinterlands. The thinking is: get a sense of what’s going on, stock up, size up the joint. Then slowly work up to the more challenging, more remote areas. That’s reasonable, but not optimal because most big cities around the world are more similar than different. All big cities these days feel same-same on first arrival. In Laser-Back travel what happens is that you are immediately thrown into Very Different Otherness, the maximum difference that you will get on this trip. You go from your home to extreme differences so fast it is almost like the dissolve effect in a slide show. Bam! Your eyes are wide open. You are on your toes. All ears. And there at the end of the road (but your beginning), your inevitable mistakes are usually cheaper, easier to recover from, and more fun. You take it slower, no matter what country you are in. Then you use the allotted time to head back to the airport city, at whatever pace is your pace. But, when you arrive in the city after a week or so traveling in this strangeness, and maybe without many of the luxuries you are used to, you suddenly see the city the same way the other folks around you do. After eight days in less fancy digs, the bright lights, and smooth shopping streets, and late-night eateries dazzle you, and you embrace the city with warmth and eagerness. It all seems so … civilized and ingenious. It’s brilliant! The hustle and bustle are less annoying and almost welcomed. And the attractions you notice are the small details that natives appreciate. You see the city more like a native and less like a jaded tourist in a look-alike urban mall. You leave having enjoyed both the remote and the adjacent, the old and new, the slow and the fast, the small and the big. We’ve also learned that this intensity works best if we aim for 12 days away from home. That means 10 days for in-country experience, plus a travel day (or two) on each end. We’ve found from doing this many times, with many travelers of all ages and interests, 14 days on the ground is two days too many. There seems to be a natural lull at about 10 days of intense kinetic travel. People start to tune out a bit. So we cut it there and use the other days to come and go and soften the transitions. On the other hand 8 days feels like the momentum is cut short. So 10 days of intensity, and 12 days in a country is what we aim for. Laser-back travel is not foolproof, nor always possible, but on average it tends to work better than the other ways I’ve tried. #KKtraveltips

@lauriewired NP hard-scifi

Time Dilation kind of makes the whole “datacenters in space” idea more fun. Technically…something like a GPS Block III CPU runs an extra ~7,000 clock cycles per day compared to the same machine on earth. Extend this to the extreme, and you get the whole subfield of CS+physics called relativistic hypercompuation. There’s some (fun?) papers that allow you to solve the halting problem by placing yourself dangerously close to a black hole…while your computer safely computes for ~infinite-ish amounts of time. One of the better papers on this field appears to be: "Relativistic computers and the Turing barrier" (Németi & Dávid 2006) (sadly, the maximum speedup just escaping earths gravity well is something like 1 x 10 ^ (-10), so yeah the blackhole thing is kinda necessary)

@GamewithDave Space Quest 3 - change my mind.

For anyone who used a computer between 1990 & 2005… what’s the one game you still think about?

@RobPalumbo We've had lots of founders come out of @swiftmedical! Love seeing it!

Canadian founders: what are the best Canadian companies for future founders to work at these days? e.g., Cohere, Ada, WS, etc. Where else?

@torybruno @blueorigin Beauty. Simpler than it looks. I studied the Apollo hatch design in some detail years ago. What are some of the main differences here?

Ok. As was correctly guessed it's the hatch mechanism. Designed and built by my awesome @blueorigin HBR Robotic's team. Here's a pic. Keeps the air in. Will absolutely NOT open when it's not supposed to. Absolutely WILL open when it is, after splash down. Allows the crew OR the recovery team to open it. LAUGHS in the face of insane heat, skull rattling vibration, and the violent aero-loads of reentry... Let me know if you'd another one.

@Cappy_Nate Nice idea, very needed. ^Helping machines see, understand, and heal us, right here in Canada!

Started an Actual Canadian Builders group chat. -Hardware -Software -Space -Defence -Web3 -Health -AI -Energy -Manufacturing You name it. But it's a Canadian only zone. 🇨🇦 If you want in, drop your name in the comments and ping anyone who should be there! LFG! 💪🇨🇦🏗️🚀

Realizing that Houston and Artemis need a "shared visual canvas" just as much as AIs do. @DanielleFong

Inspiring watching the astronauts fly behind the moon, describing part of it as "looking like a large healing wound". See! Wound care is a universal problem! @swiftmedical

NEWS: Massive budget cuts for US science proposed again by Trump administration "It's an extinction-level event for science". The US government is proposing massive cuts to almost every branch of science, from NASA to the National Institutes of Health. NSF would completely eliminate the social, economic and behavioral sciences directorate. This would decimate the world's leading scientific system. nature.com/articles/d4158…

@karpathy You should collab with @DanielleFong, tbh.

LLM Knowledge Bases Something I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest. In this way, a large fraction of my recent token throughput is going less into manipulating code, and more into manipulating knowledge (stored as markdown and images). The latest LLMs are quite good at it. So: Data ingest: I index source documents (articles, papers, repos, datasets, images, etc.) into a raw/ directory, then I use an LLM to incrementally "compile" a wiki, which is just a collection of .md files in a directory structure. The wiki includes summaries of all the data in raw/, backlinks, and then it categorizes data into concepts, writes articles for them, and links them all. To convert web articles into .md files I like to use the Obsidian Web Clipper extension, and then I also use a hotkey to download all the related images to local so that my LLM can easily reference them. IDE: I use Obsidian as the IDE "frontend" where I can view the raw data, the the compiled wiki, and the derived visualizations. Important to note that the LLM writes and maintains all of the data of the wiki, I rarely touch it directly. I've played with a few Obsidian plugins to render and view data in other ways (e.g. Marp for slides). Q&A: Where things get interesting is that once your wiki is big enough (e.g. mine on some recent research is ~100 articles and ~400K words), you can ask your LLM agent all kinds of complex questions against the wiki, and it will go off, research the answers, etc. I thought I had to reach for fancy RAG, but the LLM has been pretty good about auto-maintaining index files and brief summaries of all the documents and it reads all the important related data fairly easily at this ~small scale. Output: Instead of getting answers in text/terminal, I like to have it render markdown files for me, or slide shows (Marp format), or matplotlib images, all of which I then view again in Obsidian. You can imagine many other visual output formats depending on the query. Often, I end up "filing" the outputs back into the wiki to enhance it for further queries. So my own explorations and queries always "add up" in the knowledge base. Linting: I've run some LLM "health checks" over the wiki to e.g. find inconsistent data, impute missing data (with web searchers), find interesting connections for new article candidates, etc., to incrementally clean up the wiki and enhance its overall data integrity. The LLMs are quite good at suggesting further questions to ask and look into. Extra tools: I find myself developing additional tools to process the data, e.g. I vibe coded a small and naive search engine over the wiki, which I both use directly (in a web ui), but more often I want to hand it off to an LLM via CLI as a tool for larger queries. Further explorations: As the repo grows, the natural desire is to also think about synthetic data generation + finetuning to have your LLM "know" the data in its weights instead of just context windows. TLDR: raw data from a given number of sources is collected, then compiled by an LLM into a .md wiki, then operated on by various CLIs by the LLM to do Q&A and to incrementally enhance the wiki, and all of it viewable in Obsidian. You rarely ever write or edit the wiki manually, it's the domain of the LLM. I think there is room here for an incredible new product instead of a hacky collection of scripts.

@DanielleFong @mrginden F'ing go

@mrginden interface I have it emit to a html page or OBSIDIAN. nowadays I have a zoomable 175+ fps infinite canvas WASM/RUST stack I can write images to, so i have it write out to that

i had claude code opus 4.6 + gemini 3 pro image (nano banana 2 pro) imagine a "The Sims" interface except it's the White House and there are agentic ghosts of all the previous presidents there, to like haunt around Trump ghosts of presidents day past style here are some stills

@tobi @sandeeptodi Government loves to specify esoteric workflow and output, rather than adapt process to existing and bulletproof software. Often the same for large orgs as well.

Share a bit. I don’t want to belittle the category of software. People did amazing work in the world of payroll software (agile manifest etc). But at some point a csv is created and it has a row per payee and it happens twice a week. Shopify pays out millions of businesses and moves billions a day. And we took 20m of financing ever before we went public (and had it all still in the bank when we did). The scale of nonsense that’s happening with government bespoke software is just unexplainable without fraudulent intent. But it also tracks with everything else you hear about government efficiency (minus maybe military). It’s not that it costs 10b to make a payroll software that is the problem. It’s that it costs this much for anything that the government tries to do itself. The only conclusion is that the government needs to do a lot less.

I built enterprise scale payroll. I’m supposed to know a thing or two about how payroll evolves, scalability (100s of 1000s of paystubs), accuracy, compliance. And the cost of building and maintaining these systems. This is shocking by just about any yardstick I can think of.👇

Loving the Matic vacuum @mehul! Dog starting to shed in the warmer weather tho.. any chance of ordering new vacuum bags to Canada? 😂

RIP Sora, you gave us the greatest ai video of all time

The transistor, Unix, nylon, Teflon and the laser all have one thing in common: They were a result of the golden age of corporate R&D. In 1985 IBM had 400,000 employees but only 8 called "Wild Ducks." They could break all the rules, pull people off other projects, get budget on demand, and reported directly to the CEO. Bell Labs alone produced 11 Nobel laureates and 28,000 patents. Its budget came from American phone bills. Fortune 500 companies won 41% of America's top innovation awards in the 1970s. By 2006, that number dropped to 6%. Here's what killed American R&D: 1. The hostile takeover wave of the 1980s pushed executives toward short-term results 2. The AT&T breakup gutted Bell Labs from 26,000 to 19,000 3. Venture capital gave the best researchers a better deal than staying inside a corporation 4. Offshoring broke the feedback loop between making things and understanding them 5. Jack Welch turned GE from an industrial research company into a financial engineering shop and donated RCA's research lab to a nonprofit 6. The 2017 tax law penalized R&D spending so aggressively that some companies faced 4x higher tax bills for doing more research Today the U.S. spends nearly $1 trillion a year on R&D, but two-thirds of it goes to incremental product improvement. The labs that built modern America are gone. I'm reverse-engineering what made them work. And what a modern skunkworks looks like.

@hamen Carmack bio tweet without a rocket arc is 🤔.

Imagine you're John Carmack you're 22 years old and you just wrote a 3D engine in assembly that runs at 35fps on a 486 Doom drops. Quake drops. Half the planet is playing your code. you're the reason GPUs exist. you're the reason your friend Jensen has a yacht today. then in 2009, you sell id Software. people call it betrayal. you call it "they made an offer I couldn't refuse." VR obsession. Oculus. Meta buys it for $2B. you're CTO. but Meta thinks you're a liability. your demos are "too intense." your emails are "too long." your focus on frame timing is "slowing us down." 2022. they push you out. not fired officially. just "restructured." the media writes "end of an era." some crypto bro calls you "washed up." silicon valley moves on. but you don't. you don't write a book. you don't start a podcast. you don't collect speaking fees. you go completely quiet. you take the money. you buy a warehouse in Texas. you hire 10 engineers. and you start coding. not games. not VR. AGI. two years. radio silence. no tweets. no conference talks. while everyone's debating ChatGPT, you're debugging CUDA kernels at 3AM, testing world models. then in 2025, Keen Technologies pivots hard. you're not "exploring" anymore. you're building it. here's what people get wrong: everyone calls it a comeback. a redemption arc. "revenge on Meta." it's none of that. you're a 54-year-old engineer who still codes 12 hours a day because you genuinely can't stop. most CTOs would have bought an island. most legends would have written memoirs. you just kept typing. the most dangerous person in any codebase is the one who goes quiet and never stops shipping commits. karma doesn't need to be real. but obsession is. welcome back, Carmack.

@AndrewMayne I still haven't seen a good spreadsheet on this. And certainly if regulatory is the thing they're trying to bypass, what about at-sea?

Firstly: I'm a huge SpaceX fan. I went to the first Falcon 9 launch and the first Super Heavy. I own more SpaceX gear than anything else. Secondly: I'm a very big space economics geek. I literally wrote a book on space economics (tldr: tourism is a terrible business model but R&D and materials is exciting.) Thirdly: I helped design experiments that ran on the International Space Station. I invested in the first company to run an LLM in space (Besxar). But seriously... - An NVIDIA H100 is on the order of ~$25k+. - Running one 24/7 for a year is ~6 MWh, which is roughly ~$850–$1,000 in Abilene-area electricity (call it ~$900). - Swapping a GPU on Earth is minutes of labor. But the plan is to launch the GPU + batteries + solar + radiator + comms + shielding into LEO...only to have the next-gen GPU show up ~12 months later with another big perf/watt jump? What problem is “put it in space” actually solving other than grabbing headlines?

@mtavitschlegel @ID_AA_Carmack Super interesting work. Is this essentially equivalent to separation of objective state abstractions in the same way that GNC loops close around relevant dynamics operating over relevant time horizons and at relevant frequency/bandwidth?

Hey! Thanks a lot for presenting and discussing our paper. This was a very cool surprise for us this morning! I just wanted to add a few clarifications/additional info regarding the points you raised: - Re SSM vs TF: For the “Ant” experiments, we used SSMs for their efficiency, as the sequences are significantly longer in these environments than in the discrete grid-world. - Re 2D/3D Ant: To be clear, the ant is moving in a 3D world, with low-level actions being torque forces applied to joints of the 3D model. The agent is equipped with a LiDAR-style sensor, so besides its proprioceptive inputs (state of its joints), it only gets the distances to other objects and walls relative to its current position. It does not receive its world as a one-hot encoding. - Re Option Discovery: Just to clarify, the entire option discovery process is completely unsupervised. That is, neither the switch/termination times of segments in a sequence are revealed, nor which type of segment the current timestep corresponds to. But of course - learning the options from expert demonstrations, even if unsupervised, is simpler than hierarchical RL from scratch.

I like and bookmark so many interesting sounding papers here, and don’t get back to most of them. Time to start making a dent. I’m going to try to at least skim one of the papers in my bookmarks each weekday for the rest of the month. #PaperADay 2025: Emergent temporal abstractions in autoregressive models enable hierarchical reinforcement learning (Google) I like their statement of the hierarchical goal problem  as “how long does it take a twitching hand to win a game of chess?”  @RichardSSutton is fond of the “options” framework in RL, but we don’t have a clear method to learn them from scratch. Their Ant environment is designed to require two levels of planning: the standard mujoco Ant locomotion work to be able to move at all, and routing decisions to get to the colored squares in the correct order, which will happen hundreds of frames apart. Basically, this takes a pre-trained sequence predicting model that predicts what separately trained expert models (manually steered) do, and inserts a metacontroller midway through it, which can tweak the residual values to perform high level “steering”, and can be RL’d at high level switch points to much greater performance than the base pre-trained model. A key claim here is that learning to predict actions in a supervised next-token manner from lots of existing expert examples, even if you don’t know the goals, results in inferring useful higher level goals. This sounds plausible, but their experiment makes it rather easy for the model: the expert RL models that generated the training data were explicitly given one of four goals in each segment, and the option learning model just classifies the sequences into one of four categories. This is a vastly simpler problem than free form option discovery. A State Space Model is used for the more complex Ant environments, while a transformer is used for the simpler grid world environments. I didn’t see an explanation for the change. The internal “walls” are more like “poison tiles”, since they don’t block movement like the map edges, they just kill the ant when its center passes into them. The 3D renderings (with shadow errors that hurt my gamedev eyes) are somewhat misleading, since it is really a 2D world that the agent gets to fully observe in a low dimensional one-hot format. It doesn’t do any kind of partially observed or pixel based sensing. Everything is done with massively parallel environments, avoiding the harder online learning challenges. The success rates still aren’t great after a million episodes. I would like to see this applied to Atari, basically doing GATO with less capable experts or lower episode quantities, then trying to identify free form options that can be usefully used to RL to higher performance.