JetsonHacks

There are now sample JetsonHacks Newsletters on the website! They'll usually be an issue or so behind. To get the latest and greatest delivered directly to your inbox, sign up for the Newsletter! jetsonhacks.com/newsletters/

Add private web search to your local AI Agent. We'll untangle tool calling, MCP, and Skills while we're at it. Available now to members (natch), 8:15am PST May 14 to everyone: youtu.be/zTs9SJw7-mU

Qwen 3.6 on the Jetson AGX Thor and AGX Orin. It's interesting how much difference the inference server makes: youtu.be/PdHnioaSqTo

@xm_build @NVIDIARobotics Not sure I understand the question. It's a non-deterministic LLM, not a real time system.

@NVIDIARobotics @Jetsonhacks gemma 4 sounds promising, but how does it handle real-time latency for applications needing under 50ms response?

Running LLMs on edge devices? This is a must-watch from @Jetsonhacks. 👀 Watch as he unlocks 10x performance with Gemma 4 on NVIDIA Jetson Orin Nano by understanding what's really required for edge AI—model size, quantization, and memory optimization done right. 🎥 nvda.ws/4tHwznU

Let's put those little Jetson rascals to work! Gemma 4 on Jetson. youtu.be/EGnA_kqu3is

@KerseyFabs @GooseworxMusic @glitch_prod It's good to be king!

Now I know which bedroom is mine thanks to my daughter. @GooseworxMusic @glitch_prod #tadc

Follow-up to my Jetson LLM benchmarks - here's how I'm actually deploying this for DAWN. Two Jetson boards, two models, two use cases: Iron Man Helmet - Jetson Orin 16GB @ 28W (custom power profile): - Gemma 3 4B IT with vision - 13.9 tok/s generation, 158ms warm TTFT - 2.5 GB model + 0.85 GB vision projector - Plenty of room for Whisper ASR + Piper TTS + DAWN on 16GB - 89.7% on our instruction-following test - sudo ./install.sh -P A3 (this A3 specifies the model and config) Home Server - Jetson AGX Orin 64GB @ MAXN (60W): - Qwen3.5 35B-A3B MoE with vision - 29.6 tok/s generation, 1.3s warm TTFT - 19.9 GB model + 0.86 GB vision projector - 128K context with zero speed penalty (more on that in the next post) - ~29 GB free for concurrent local image generation (and other processes) - 94.8% on our instruction-following test - sudo ./install.sh -P F Biggest gotcha I found: power mode. The AGX Orin 64GB defaults to 30W. Our 27B model benchmarked at 2.1 tok/s and we almost wrote it off. Switched to MAXN - 3.4x speedup across the board. The 4B went from 10.4 to 35.5 tok/s. Always run 'nvpmodel -q' before benchmarking on Jetson. The DAWN llama.cpp install script auto-detects your Jetson model and memory, recommends the best preset, downloads model files if needed, and configures the systemd service. 9 presets from 4B to 35B. Upgrading the helmet from the Qwen3 4B I had before: gained vision capability while only dropping from 15.2 to 13.9 tok/s. Quality actually went up (from 84.8% to 89.7% on a more accurate test with a better quantization). github.com/The-OASIS-Proj…

Run Gemma 4 Locally on AGX Orin — Private NemoClaw for Itself and Other ... youtu.be/z-D0KLt5S6g?si… via @YouTube

Spent part of the day benchmarking vision-capable LLMs on NVIDIA Jetson for DAWN, my open-source in-helmet and in-home AI assistant. 8 models, two platforms, some interesting results. Hardware: Jetson AGX Orin 64GB Developer Kit at MAXN (60W). Ampere GPU, 2048 CUDA cores, ~204 GB/s memory bandwidth. All models Q4_K_M via llama.cpp, bartowski quantizations. MoE models are the bringing new life to edge inference. (The percentages below are accuracy on a custom test I developed for my assistants.) 4B class: - Gemma 3 4B: 89.7%, 36.3 tok/s, vision - Qwen3 4B: 94.8%, 35.1 tok/s, no vision - Qwen3.5 4B: 90.5%, 28.4 tok/s, vision MoE class: - Qwen3.5 35B-A3B (3B active): 94.8%, 29.6 tok/s, vision - Gemma 4 26B-A4B (4B active): 94.8%, 32.2 tok/s, vision* Dense large: - Gemma 3 12B: 89.7%, 16.1 tok/s, vision - Qwen3.5 27B: 91.4%, 7.2 tok/s, vision - Gemma 4 31B: 100%, 6.8 tok/s, vision* The Qwen3.5 35B-A3B MoE hits 29.6 tok/s WITH vision on a single Jetson board. 35B model quality at near-4B speed. Only 3B parameters active per token, 256 experts, hybrid SSM+Transformer architecture. Voice-viable on edge hardware. *Gemma 4 models have a thinking content leak in llama.cpp as of this week — the 26B-A4B is faster and ties on quality with my recommended Qwen, while the 31B dense scored a perfect 100% but is too slow for voice. Once the thinking issue is fixed upstream, the 26B-A4B becomes the top pick. Quality tested with DAWN's FRIDAY persona and JSON tool-calling test suite (13 tests covering command format, persona, music, getters, search, weather, calendar, email, clarification, and conversational). Everything is open source: github.com/The-OASIS-Proj…

Here's an article on some of the little hurdles that need to be changed to get NemoClaw running on the Jetson Orins: jetsonhacks.com/2026/03/25/nem…

Getting NemoClaw up and running on Jetson Orins is a little bit of a challenge. A Jetson Orin Nano is the smallest spec machine NemoClaw runs on. Coincidentally, here's a video on how to do exactly that: youtu.be/g2fzaLKNRLs

@HaroldSinnott @domingonarvaez1 I think people miss the part about that when an agent does a task for you, you take responsibility for that task. Lawyers won't be suing your agent. That'll bite hard in highly regulated environments, especially privacy. And it's your reputation too.

@domingonarvaez1 Thanks, @domingonarvaez1 . Great breakdown by @Jetsonhacks , really highlights how privacy and security are becoming core to deploying agentic AI at the edge, not an afterthought. 🦞 #NVIDIAGTC

First Look: NemoClaw on Jetson with a Local LLM youtu.be/7HQSFgP6vOE?si… via @YouTube

@monadical @ConnectTechInc And to think, they only want money for it in exchange!

@Jetsonhacks @ConnectTechInc Oooh, want one!

When you are going from development to production, there are a lot of decisions to make. @ConnectTechInc offers the Gauntlet board for Jetson Thor to help with the transition, along with engineering services: youtu.be/ndhiJm2QOwQ

That's the relatively easy part. What about data retention? There are all sorts of rules depending on what type of business you're in. And it may not even be in your jurisdiction. For example, the EU has a very interesting position on any type of data that flows there. Not to talk about actually sensitive data.

hmmm…not sure. Illinois law might be interpreted as the ai being a bonafide agent—or alter ego of the owner. A new area of law of uncertain outcomes. They have already ruled that consulting an ai is not privileged—and further—that your own attorneys converstions with an AI destroys all privilege. and then…reddit.com/r/cars/comment… 😂😂

@Jetsonhacks i looked at your new “first principals” “agentic” Repository on Github…great! especially timely given Canonical roll-out of new ubuntu with agentic ai support and native support for Cuda and ROCm. worth watching the rollout vid if you haven’t seen it yet😉

It's much more nuanced that I thought it would be when I first started thinking about it. The basic question is "How do you interface probabilistic systems with deterministic ones?". It's beyond dangerous, but so are a lot of other things. The one thing that this hype cycle has done is given a lot of creedence to how people think their systems *should* run (revealing the opportunity). To me, more interesting than "privacy" concerns is responsibility and reputation concerns. We've heard a lot of "I hired 5 agents to run my new company". That doesn't mean much, because the owner is still responsible for all the rules and laws that govern business. The lawyers ain't going to be suing the agents! When the agent sends an email, even though it may be using the agents email address, that doesn't mean it doesn't count against the owner. And we've all sent emails that we later regret. Now the agent can send those automatically! All this is even before you bring in bad faith actors.

@Jetsonhacks I think people with some background look at the agent phenomenon as kind of a gimmick… a rather dangerous one at that…Mark Shuttleworth said even he got burned a little. but its grabbed the public imagination…and just proves that use cases can come from the strangest places😂

Building walking robots that actually walk is challenging, even if you have Jetsons on your side: youtu.be/67S9-MrqWJg?si…

We're thrilled to announce our Falcon Vehicle System has won Best In Show at Embedded World this year. Falcon won in the AI & Machine Learning category, confirming what we've believed all along: that smarter vehicle systems start with smarter foundations. connecttech.com/falcon-vehicle…