JP Hwang

Hi there, my name is JP. I’m a developer educator. I create: - courses datacamp.com/courses/end-to… (@datacamp), linkedin.com/learning/vecto… (@linkedin learning) - educational diagrams (jphwang.com/gallery/) and projects (github.com/databyjp/weeke…) - blogs & videos. Let’s connect ❤️

The most important tools an agent has are the search tools to build its own context. Here are the 6 principles I follow to build one: 1. Building the right tools following the “low floor, high ceiling” principle 2. Adding descriptions to the metadata, so the agent can find the right index to search 3. Prompting: Making sure the agent calls the right tool by careful tool naming, writing good tool descriptions, adding reasoning parameters, reinforcing instructions in the agent’s system prompt, and forcing tool usage. 4. Number and complexity of parameters: Making sure the agent generates the right parameters by writing good parameter definitions, thinking about the number and complexity of parameters 5. Optimizing the tool responses for token efficiency and context relevance 6. Error handling: Enabling self-correction through proper error handling

The result is that MTEB score drops less than 2(!) points. For anyone building any vector stores, like RAG pipelines or agentic workflows at scale, that's a trade-off worth knowing about. More on this (and Matryoshka truncation) in my latest video: youtu.be/2xBgwHW-h8M?si…

The typical catch is retrieval quality degradation due to precision loss. But Jina's team solved this in their v5 embedding model training with GOR (Generalized Orthogonal Regularization). GOR spreads embedding values more uniformly, making the compression nearly lossless.

Your embeddings might be 32× larger than it needs to be! Binary quantization compresses each embedding dimension from a (32-bit) float to a single bit. Positive values become 1, negative become 0. That takes a 4,096-byte vector down to... (checks notes) 128 bytes!

Just saying

4B-class embedding quality from models 1/6th or 1/16th the size? ✅ with Distillation + LoRA task adapters ➡️ strong retrieval & multilingual performance, with Matryoshka +BQ Check out our @elastic_devs video on @JinaAI_ v5 models at 677M or 239M params youtu.be/2xBgwHW-h8M?si…

The result: Jina v5-small matches the 4B teacher on retrieval benchmarks at just 677M parameters. I cover exactly how this works, and what the Jina team found didn't work in my latest video. Link: youtu.be/2xBgwHW-h8M?si…

Distillation powers Jina AI's new v5 embedding models. • The student trains to minimise the gap to the teacher embeddings • A projection layer bridges the output dimensions difference • Repeat until the student has absorbed the teacher's knowledge

How do you get 4-billion-parameter quality out of a model that's 6× smaller? That's knowledge distillation: • A large "teacher" model (Qwen3-Embedding-4B) and a smaller "student" model (i.e. Jina-v5-text-small or nano) process the same text pairs

obligatory GTC keynote tweet when you make the top slides: building HNSW graphs with up to 12x the throughput and 7x faster merges on elasticsearch and NVIDIA cuVS (a GPU-accelerated library for vector search) powered by CAGRA (graph-based ANN algorithm built to run natively on GPUs) that still works with CPUs for search PS: yeah, this is a lot of acronyms

@_jphwang 😂

Choose your own adventure, but every ending is more work.

My first video for @elastic_devs is an exciting one - we've just launched @elastic Agent Skills! Try it out and your agent would already be way smarter at using Elastic than dummies like me. 😆

At this point, I'm very unlikely to pay for the nonsense on X, but I *would* pay for a verified-human only social network.

I'm deeply uncomfortable with Claude Code / Cursor CLI spinning up new processes, and killing them. Is this a footgun? Does this ever go wrong for anyone? Like, have you had your agents kill the wrong process with the wrong PID?

principle of floors and ceilings for agents: how to deal with ambiguous and predictable questions. and how not to over-optimize for one over the other because you'll miss out on the full space of tasks

🤔 Quite misleading take from @aarondotdev Context: - Devs w/ AI spent ~30% of total time (30mins) on setup - Productivity will compound subsequently The paper is literally about the tradeoffs of coding assistants & how to intentionally mitigate them.

With the 0.5B Qwen model & the 237M Jina embedding model, **you can even build one for < 1 billion parameters.** I'm thinking of doing a video about this - would you be interested? (Give us a like or comment if yes)

In fact, I was playing with the new Qwen 3.5 vision language models and the Jina AI v5 embedding models. These models will get you a pretty good multimodal RAG pipeline with fewer than 5B parameters!

Each $200 Claude Code subscription might be worth... -2400% net profit on revenue!? Either: a) inference needs to get a lot cheaper, or b) we're in for a ton of pricing shock / enshittification in the near future. Local models are looking pretty enticing about now. 🧵