LakeSail

Spark rebuilt in Rust — no JVM, 8x faster, 94% less cost.

Deep dive: lakesail.com/blog/sessions/

Sessions bring secure execution for humans and agents alike.

Production data sits in a private cluster in your own cloud, no public address. Reaching it from outside has usually meant putting yourself on the network first: a VPN, a bastion, something. So we shipped Sessions: with a scoped, time-boxed token, you can reach the cluster from anywhere through one hardened gate, while the cluster stays private and your data never leaves your account. The same token works from your laptop or a coding agent. So you can give an agent real compute on real production data, scoped to one workload and time-boxed, without ever granting it standing access. Check it out here: lakesail.com/blog/sessions/

In 2026, you no longer have to be stuck tuning the JVM. Run your Spark workloads on a Rust-native runtime for 94% less hardware cost and zero garbage collection. pip install pysail

Keep your Spark code, just plug in a Rust-native runtime underneath and say goodbye to the JVM forever. That's Sail in a nutshell.

We built Sail so you don't have to keep paying the JVM tax.

We dive deeper into how Sail compares to Spark on this front in our blog: lakesail.com/blog/sail-and-…

Slow Python is simply unacceptable for AI workloads. Agents on a compute engine need native speeds, and Sail was built from the ground up in Rust with exactly that in mind.

If you are at the @Snowflake Summit today or SF at large, come join us tonight for a panel on the future of Data=>AI at Yes SF! luma.com/8tvd2xla?tk=Lt… Your truly will join the illustrious engineering and devrel leaders Mark Hoerth of Datastrato / Apache Gravitino, Andrew Madson of the newly combined juggernaut of Fivetran / dbt Labs, and Josh Wo of LiveRamp. I will represent LakeSail with our founder and CEO Shehab Amin keeping me honest from the bench! Come learn all about our the real Rust AI future and the Semantic Layer going much beyond OSI metrics.

Check out Sail on GitHub: github.com/lakehq/sail

Spark accelerators like Photon speed up portions of execution, but are still tied to the JVM. Sail, written in Rust, removes the JVM entirely. The result is drastically better performance with no JVM tuning required. The same familiar Spark API on a Rust-native runtime. That’s Sail.

Read our full breakdown how Sail compares to Spark accelerators here → lakesail.com/blog/sail-and-…

Why fully rebuild Spark in Rust instead of just accelerating it? Spark accelerators speed up parts of execution, but they still inherit Spark's JVM control plane, memory model, shuffle path, and Python serialization costs. You're still tuning a JVM. With Sail, we took a clean-slate approach: a fully Rust-native runtime, with no JVM, no heap tuning, and no GC pauses. Same Spark interface with an entirely new runtime underneath.

We wrote a book, with Codex, delving deep into @lakesail architecture. Sail is a modern data ecosystem with deep roots in @ApacheArrow, @ApacheDataFusio, and rebuilding the whole @ApacheSpark ecosystem in @rustlang. One of the key advantages of an AI-native stack is extensibility. We are convening the community to build Sail extensions. A proposal is on the table in github.com/lakehq/sail. In order to extend the engine as profoundly efficient as Sail, you need to operate at several levels — physical, logical plans, loading and linking, and performing at the top both in a single node and in cluster mode. The book is written as an exploration of the codebase, of the overall Sail architecture, its use of Arrow and DataFusion, its implementation of SparkConnect protocol, and everything else pertinent to the extensions. github.com/alexy/sail-rus…

Check out Sail on GitHub: github.com/lakehq/sail

What if Spark was rewritten in Rust? Well, that's Sail. Besides all the advantages you get from a Rust-native engine, the best part is your Spark code still works as is.

github.com/lakehq/sail Or pip install pysail

You can also switch to Sail for even bigger gains. Companies moving from Spark to Sail are cutting test suite runtimes by 12 to 15x on average. At Adyen, wall-clock time dropped from 423s on JVM Spark to 32s on Sail, a 92.4% reduction and 13.2x speedup. Another large European insurance company reduced its test suite from 1 hour to 5 minutes. The best part: there is no rewrite required. Your existing Spark code works as is.

Check out Sail on GitHub: github.com/lakehq/sail

Most of what people call "Spark tuning" is really just JVM tuning. Memory fractions, GC flags, heap sizing. Those knobs exist because the JVM was largely built for app servers and Spark naturally inherited that runtime's shape. So we rebuilt the engine in Rust. Same Spark Connect protocol. Same SQL and DataFrame API. But no JVM, no GC, no knobs to set.