Johannes Rudolph

My job as a Rust Engineering Lead at a startup became victim of restructuring and I'm looking for a new role. I enjoy working on infra software, Open Source, with Rust, Scala, or whatever is needed. I'm also available contractually to support previous libs like Pekko Http.

@hadilq @debasishg If you need fast append/insert/delete, usually the solution is not "just an array" but clever pre-allocation schemes and a mixture between linear and linked data to amortize the cost of updates while still keeping most of the linear memory benefits for sequential access.

@debasishg Wait what?! Did I understand it correctly? If you use LinkedList, with O(1) add/remove, in your LRU implementation, instead of array, with O(n) insert/delete, the array one will be faster because of CPU architecture!?

Linux recently improved their page fault handling replacing Linked Lists and Red Black trees with a new data structure that offers better cache friendliness. This talk has some pointers to why linked lists fail on modern CPUs and what it takes to make a cache friendly data structure. The Maple Tree FTW .. youtu.be/TEHRMzZ01nE?si…

@debasishg @hadilq Another downside is that the reference chain in a linked list adds a dependency chain between elements in the list that prevents potential OOO benefits and SIMD optimizations.

With a linked list every time u traverse to the next item in the list, u r jumping to a totally random location in the memory. And potentially it's a cache miss. With earlier processors, processor speeds were roughly comparable to that of a memory access. So there was no substantial diff between accessing the next element of an array and that of a linked list. But this diff is huge with modern processors using L1, L2 or even L3 caches. So a cache miss is much costlier today. With arrays, elements are placed in contiguous locations and hence cache hit ratio is much more compared to linked lists where elements are placed in random memory locations.

@forked_franz I see. I looked into uprobes again for github.com/jvm-profiling-… but JIT code / code in anon mappings is still not supported. It seems that you can use mem eXecution watchpoints to trace function entry but something like return probes still seems quite elusive.

@virtualvoid But the problem is that will observe just the java side , and with some overhead too :/. And will be blind to native frames and the rest of the kernel stacks, clearly...(2/2)

Anyone aware of #function-graph-tracer" target="_blank" rel="nofollow noopener">kernel.org/doc/html/v4.18… but in some Java profiler?

Almost 50 years ago mastermind Loriot created this visualization of what happens during a rolling upgrade if you implemented sharding naively (w/o consistent hashing) dailymotion.com/video/x2x2dhm

@Matsluni Thanks, Matthias!

If you have the chance to hire Johannes, do so. Quickly. He is a very talented engineer in many fields.

In other news, I won another prize at the Vesuvius Challenge for vesuvius-gui, an interactive browser for 3D volumes (like the vesuvius scrolls) written in Rust with egui. Thanks a lot to challenge team! scrollprize.substack.com/p/may-progress…

@lukasz_bialy @WojciechM_dev Great stuff, maybe I should retry llama2.scala on Scala Native as well since it was also hampered e.g. by ByteBuffer issues.

It turned out to be a bumpy road but @WojciechM_dev was there to lend a helpful hand and fix all the issues and perf problems I encountered. The journey finished yesterday in the evening and in the end I can only say: we are so back! Read more about the whole thing here: github.com/lbialy/1brc/tr… Beside @WojciechM_dev I'd like to thanks @velvetbaldmime and @armanbilge for inspiring me to look into this!

In these last hours of #1brc challenge I want to share with you my small exploration not of how fast I can make JVM go BRRRRR but of whether Scala Native is now a real, usable contender in the space of functional langs compiled to native binaries. The biggest issue, of course, was whether I can parallelise the load. Scala Native did not support multithreading for the longest time but now that has changed! In mid-January I implemented a relatively (no Panama and Unsafe, no SWAR) fast solution inspired by work of @sampullara, Yavuz Tas and @royvanrijn. Then, I cooked the binary using scala-cli and benchmarked it against the fastest JVM in the tournament. #scala 1/*

Thank you, @scrollprize, for awarding me a prize in the Vesuvius Challenge for my Open Source contributions cataloguing released segments and running OS ink detection models on them. It has been a fun journey and great collaboration! blog.virtual-void.net/2023/12/11/ves…

Amazing contributions to the @scrollprize @spacegaier, @virtualvoid and others and congrats for the prizes won! scrollprize.substack.com/p/many-open-so…

@ernerfeldt @scrollprize Thanks for egui :) For the first time in 15 years, I have been enjoying working on a gui after avoiding GUIs at all cost because of the lack of good options...

A short report on trying out different algorithms for selecting top-p elements from a distribution (aka Nucleus Sampling): blog.virtual-void.net/2023/08/29/cal…

@hetzner The whole point of cloud volumes is to be able to view storage detached from compute. Optimally, storage can be easily migrated to new compute in cases of problems. Quite a bummer if storage cannot be relocated in cases like this... @Hetzner_Online

Mmh, @hetzner cloud volume hangs while unmounting since one hour. No more actions on the server are possible, after shutting the node down other mounted volumes are also blocked, server cannot be restarted. Big share of cluster storage is unavailable => extended downtime.

Also, the day, when you first run a code generation model on your engine asking for suggestions about how to improve algorithms of the underlying engine 🤯 (the suggestions are mostly weird and broken, admittedly, but had some ideas to follow up) gist.github.com/jrudolph/fb764…

llama2.scala now has quantization support, AVX2 kernels, multithreading and can load some GGML models directly. github.com/jrudolph/llama…

@RealNeilC @karpathy's llama2.c is interesting because it removes all abstraction and breaks down inference into a single file of C (which can be seen as a lingua franca of computation because data structures map closely to actual memory layout and operations to CPU instructions)

@RealNeilC Python's speed is mostly irrelevant since ML means instructing the GPU which matrix op to do next. In big models, each multiplication takes so long that it dwarfs the overhead of the language. If you are looking at state-of-the-art inference on CPU look at llama.cpp.

Apache Pekko Http 1.0.0 has been released, see #pekko-http" target="_blank" rel="nofollow noopener">pekko.apache.org/download.html#… and pekko.apache.org/docs/pekko-htt… for more details. Apache Pekko Http also includes Scala 3.3 support, the result of an ongoing community effort that spanned years!