Contribution of Velox4J

[zhztheplayer]

Hi folks,

Firstly thank @pedroerp for the recent discussion on this.

The thread is a formal discussion about the contribution of Velox4J to the Velox community. The project is for providing Velox's Java bindings through JNI and Velox's JSON plan / expression protocol.

I started the project one month ago, it's still in the earlier stage but has already begun to take the shape as a usable Java database acceleration library along with the development iterations. In the Apache Gluten community, we are running a PoC to use Velox4J to accelerate apache Flink's streaming processing, the solution works smoothly and the code is made even simple.

Every Java developer can try Velox4J out following the instructions from here.

Also there were some efforts in the Velox community on the similar topic before, for example, this PR, so I am glad to hear thoughts from previous contributors. (cc @hn5092)

And given the Velox4J project is still mainly by my personal effort so far, some users and myself hope to see if the project can be contributed to Velox community so can be benefited by community power, certainty will be gained as well. So I have been exploring this in the past few days.

Besides, assuming the project can be accepted by Velox, one important thing is how the code will be organized. For this I'm thinking about the opportunity to have an individual GitHub project for Velox4J, but still have it in the Velox community - but not sure about the difficulty on this because Velox is currently under facebookincubator which seems to be a big umbrella for a bunch of Meta projects. So this will be a question for now.

Thanks in advance, look forward to having more insights from the community here.

[pedroerp]

Hi @zhztheplayer thanks for starting this project and being willing to fold it under the Velox umbrella. Indeed, bindings to other languages like Java and Python are thing we would like to provide as part of the library.

Functionally, I feel like having the bindings as part of the same repo makes things more convenient. We could make it an optional component, similar to what we do for PyVelox. We have velox/py, we could have velox/java. At this stage, we frequently make API breaking changes, so having the bindings as the same repo means we could fix them as part of the same PR. Assuming we go that route, you would naturally maintain the credits as original authors, and could become the maintainers of that code if you would like.

I wonder what other things (cc: @majetideepak @assignUser @kgpai @mbasmanova @xiaoxmeng @Yuhta @zhli1142015 @rui-mo @jinchengchenghh )

[zhztheplayer]

Thank you for the insights.

I feel like having the bindings as part of the same repo makes things more convenient.

Putting the Java bindings into the same repo is technically feasible but could bring some further complexities that we will need to manage. Like once a C++ PR that involves changes to the JSON plan protocol is opened, Java UTs may fail because corresponding changes to Java code may be required. This complexity also applies to other changes that require for Java code changes as well.

Assuming we go that route, you would naturally maintain the credits as original authors, and could become the maintainers of that code if you would like.

Thanks a lot. I would understand and appreciate that. But it's equally important to make sure the development model is built right for both Velox core lib and the Java bindings. Indeed based on your perspective I can assume new contributors of the Java bindings could be more encouraged if their contributions can be all counted to the original Velox repository, this could a bonus.

Maybe we can try using the same repository for a while, and then adopt further solutions if it doesn't work? Given if it is the first option here. For example, once the repo is getting unmanageable, we can try :

1. Pin the required Velox code to a fixed commit / version in Velox4J, then to update this version periodically (just like the current Velox4J is doing)
2. Or create separate repository for the Java bindings if 1 doesn't help

By the way, Apache Arrow recently moved arrow/java to an individual repository arrow-java. See their discussion. @assignUser I noticed you were involved in that discussion as well so if you like to share your view on the same topic about Velox? Thanks.

Agreed but I think it should live in top level dir java/ as it's standalone and there is no technical need to put the code under velox/.

Yes agree on this if using the same repo.

[assignUser]

By the way, Apache Arrow recently moved arrow/java to an individual repository arrow-java

Yes that's true though Arrow is of course in a very different positon being more mature with less frequent breaking changes in the api (even though we release major versions quarterly). When things change very frequently, as they do here, it can be more convenient to couple CI for cpp and bindings so necessary changes to bindings can be adressed in the PRs making the cpp changes.

On the other hand we have to keep in mind that merging to the velox repo is the biggest bottleneck, due to the path through the mono repo, keeping the bindings in a separate repo would make that much easier and would allow you and others direct write access.

[zhztheplayer]

@assignUser

Thanks. And yes, I think having separate repos is the ideal model as long as it's viable for Meta.

For the mono-repo approach, I found that it might help a lot if Velox can have all the C++ code isolated in subdirectory velox, so the Java bindings could still pin to a fixed commit of Velox C++ with a cross-module dependency manner, so will unnecessarily manage recursive dependency relationship which makes things super strange.

Is that possible for Velox? Say, can we just move velox/CMakeLists.txt to velox/velox somehow? This way C++ code will seem more isolated so velox/java can depends on velox/velox which is considered the core C++ module.

[zhztheplayer]

On the other hand we have to keep in mind that merging to the velox repo is the biggest bottleneck

Yes I am worried if the community has enough Java / JNI reviewers ...

[assignUser]

Is that possible for Velox? Say, can we just move velox/CMakeLists.txt to velox/velox somehow?

As I just found out with my pyvelox changes that has to be cleared to not cause issues with metas internal mapping, so I have no idea^^

[jinchengchenghh]

I have a question for the maintenance, I assuming it will integrate as velox/java, do we need to also update the java implement when the PlanNode in C++ side is modified?

Looks like the implement depends on Arrow Batch, Arrow to Velox is an expensive cost, do you have any proposal for the conversion?

In Gluten, now we use substrait for the multiple backend compatibility, gluten-flink only supports velox backend, so we can use this Java binding, how do you think about it?

[zhztheplayer]

I have a question for the maintenance, I assuming it will integrate as velox/java, do we need to also update the java implement when the PlanNode in C++ side is modified?

Yes Java changes could be needed. It's one of my concerns for monorepo.

Looks like the implement depends on Arrow Batch, Arrow to Velox is an expensive cost, do you have any proposal for the conversion?

My opinion on this is to reduce the cost of interchange between Arrow and Velox first. This is important for Velox to be in the ecosystem.

In Gluten, now we use substrait for the multiple backend compatibility, gluten-flink only supports velox backend, so we can use this Java binding, how do you think about it?

Yes this is what contributors like @shuai-xu is doing for Apache Gluten now.

[majetideepak]

Arrow to Velox is an expensive cost

@jinchengchenghh Can you open an issue for this?

[jinchengchenghh]

But I don't see any optimization we can do for it. In my test, the expense is nearly to column to row.
The expense contains RowVector to ArrowArray in c++ side and ArrowArray to Spark ColumnarBatch in java Side.

[hn5092]

Sorry, I haven’t been following up lately as I recently changed jobs.
In my opinion, the core requirement for velox4j is to ensure that performance is not degraded compared to using Velox directly — ideally within 1–3%.
I suggest referring to the approach I used before: directly manipulating Velox pointers without relying on Arrow.

[zhztheplayer]

I suggest referring to the approach I used before: directly manipulating Velox pointers without relying on Arrow.

An arrow-compatible API doesn't forbid one from using / implementing any Velox-native data manipulation functionalities in Java at all. Having Arrow support is basically a must-to-have feature and we don't have to do much about it in Java given Velox already has Arrow ABI integration. This is not a black-or-white debate. Arrow support is just a feature for Velox4J as well as for Velox.

[hn5092]

If everyone is moving toward native execution, eventually all operators will need to run natively. In that case, it essentially becomes a matter of defining a unified in-memory format.
In my previous engine design, I unified Spark’s memory layout directly onto the Velox layer — essentially replacing Spark’s off-heap vectors with Velox vectors.
This brought huge performance benefits, and we were even able to reuse all of Spark’s existing file formats.
The overall architecture also became much simpler as a result.

[zhztheplayer]

I didn't intend to debate on Arrow vs. Velox format in Java. Like I mentioned, we can have both at the same time. However the features we add to Java API should be evaluated with the question whether it's worth to put the maintenance resource on it.

why not just operate directly on raw_data pointers, and use something like Spark’s off-heap unsafe access to read the data?

From Spark's usage, there's no such need for a Java Velox format API. We already have native implementations to convert between Spark rows and Velox vectors. We don't access Velox raw data directly from Java side. Even Arrow, it's not often used except for some specialised cases like with PySpark.

But Arrow support is required anyway because it's used by the whole ecosystem. We can keep away from Arrow if building a database on top of Velox from scratch. But we are not doing that.

[hn5092]

In my understanding, velox4j should simply mean Velox to Java. The path Velox → Arrow → Java already exists, but it can't really be called velox4j — it's just one possible route that happens to work for that purpose. However, I think the true velox4j should refer directly to the connection from Velox to Java. Introducing Arrow into this chain makes things confusing, because no one would typically jump in halfway and use Arrow unless what they actually need is velox4arrow.

[hn5092]

and import arrow is not just a arrow code , it's contain to many dependency and to many problem

[zhztheplayer]

Velox4J only reuses Velox's Arrow ABI code with only hundreds lines of additions for Java code. Velox-to-Arrow / Arrow-to-Velox (referred to as v2a / a2v) can be avoided while a whole query plan tree is sent to Velox to process. User will only need v2a from the output end of the pipeline.

[hn5092]

this sdk goal is the best performance or just to expose to user to use velox, if only use velox, i don't think care the impl, otherwis i'm worry about the performance, we must make sure the performance only lose below 3% when you convert to arrow