Ballista A modern distributed compute platform

This Week in Ballista #10

Welcome to “This Week in Ballista”, a weekly newsletter that summarizes activity in the Ballista Distributed Compute project.

Ballista is a modern distributed compute platform powered by Apache Arrow and primarily implemented in Rust, but designed to provide first-class support for other programming languages, including Python, C++, and Java.

Ballista has been donated to the Apache Arrow project

We missed a few editions of this newsletter while we worked through the process of donating the project to Apache Arrow.

The good news is that we are now through that process and the Ballista Rust codebase has now been donated to Apache Arrow via this pull request.

We can now resume development by creating pull requests against the Apache Arrow repo and an initial list of issues has been created here.

DataFusion and Ballista to move to new repository soon

DataFusion and Ballista are tightly coupled and there is an opportunity now to have a common scheduler that can scale queries seamlessly across cores in DataFusion and across nodes in Ballista.

DataFusion and Ballista have different release cycle requirements than the core Arrow crate and for this reason there is a proposal to move these components out of the Apache Arrow repo and into a new top-level DataFusion repository. The project will continue to operate under the governance of the Apache Arrow PMC and discussions will happen on the Apache Arrow mailing lists.

Community

Follow the @BallistaCompute Twitter account to receive notifications when new editions of “This Week in Ballista” are published.

Join the ASF Slack Channel to chat with the core contributors in #arrow-rust.