javacpp-pytorch meeting message

[mullerhai]

Hi @saudet @sbrunk , I'd like to share with you some communication, demonstrations, and the final intention to cooperate with Huawei engineers today.

First, I showed them the process of model development and training in a regular IDE. Then, I demonstrated the effect of PyTorch on Scala3 using a Jupyter notebook with the Scala kernel based on amond.sh. After that, I showed them the recently completed Gloo distributed training results. Finally, I showed them the code for my PyTorch on Scala3 lessons and the content of the Chinese book. They were very satisfied and surprised, and I believe this exceeded their expectations. They mentioned that in previous collaborations, no one had ever provided such a rich range of solutions.

After that, we discussed the terms of cooperation with Huawei:

1. I told them that Bytedeco is a non-profit open-source organization, but it needs sponsors to help developers do a better job, as everyone's energy is limited. Therefore, if Huawei can cooperate with us, they should sponsor our organization. They said it was no problem. All budgets will be tallied after the Chinese Lunar New Year in 2026, probably around the end of February 2026. They said a sponsorship of $1,000 per month is completely fine. If the cooperation goes well, the sponsorship fee could be increased further each year. The cooperation will be based on the actual start date, and if the sponsorship is delayed, a portion will be retroactively paid. In the future, the sponsorship名义 (nomination) might be from Huawei Corporation or one of its subsidiaries. Their sponsorship funds will be transferred from the department's finance to Bytedeco's account.
2. They asked about the open-source licenses supported by javacpp-pytorch and storch. I am not very familiar with this, but I think they should be able to understand if they look at the license themselves. However, my personal thought is that if Huawei is willing, they could additionally pay patent fees for the javacpp-pytorch code to Bytedeco. In the future, Huawei expects javacpp-pytorch and PyTorch on Scala3 to be pre-installed on their cloud computing servers and to be supported by their GPU chip drivers, CANN. They need Bytedeco to port CANN to a javacpp-cann glue layer and maintain it for at least one to two years. There are also some smaller frameworks under CANN that need to be ported as well. They hope Bytedeco can iterate and update along with version updates. Additionally, they mentioned that besides CANN, there are some commonly used toolkits in C++ and Python, and if they are written in C++, they hope Bytedeco can assist in porting them to Java versions.
3. In the future, they might provide us with one or two Euler Linux system cloud hosts with Huawei GPUs for us to conduct experiments and validation tests.
4. Their customers are increasingly having multi-modal demands, including text, images, video, audio, radar data, point cloud data, GPS spatial geographic information, time-series information, and other multi-modal data. They hope Bytedeco can support multi-modal scenarios, such as having javacpp versions of C++ libraries like OpenCV, FFmpeg, Librosa, Open3D, and Supervision, which might help them win over customers.
5. Currently, domestic customers are all using large models, and they are using LangChain, RAG, VectorDB, LangIndex, etc. They hope javacpp can also support their large model applications in the future, enabling implementation in Java. I told them that the langchain4j framework is quite good.
6. For future contract signing, they think a contract will be signed annually, and they believe the cooperation should be long-term, of course, this will also be accompanied by adjustments in their business. But Huawei's GPUs will not go out of business.
7. They believe that javacpp's Gloo distributed training will have applications for them. Gloo should indeed be able to drive their CANN and Huawei GPUs.
8. Their customers have big data training and inference tasks that require javacpp-pytorch to support the Parameter Server. I told them this should be the content of the libtorch RPC module. Their customers are quite dependent on this now, so it is necessary for us to implement the libtorch RPC module. They also hope that the GPU performance monitoring module (profiler) will also be supported.
9. They asked about the modules in javacpp-pytorch for model quantization, mixed precision, and flash-attention kv_cache. Because I am not very proficient in this area, I could not answer. I will research it, and if libtorch has these, we can try to port them.

In summary, I think Huawei is sincere about cooperating, and their needs are something that others cannot solve. Bytedeco will be their best choice. We need to use our advantages to make them rely on us. Huawei has a good reputation in terms of sponsorship; it is a responsible large company, and cooperating with them will also promote the development of the Bytedeco ecosystem.

In addition, in the future, I will also contact other Chinese GPU chip manufacturers, including Cambricon, Enflame, and Moore Threads. They may also follow Huawei's lead in creating Java versions of PyTorch for GPU adaptation in the future. I will establish contact with them and persuade them to cooperate and sponsor Bytedeco.

At this stage, I think our users are not just Huawei; Huawei is just a custom client. We have many more ordinary users who use NVIDIA CUDA GPUs. In mid-December, there will be a Scala3 meetup in Beijing, and I will be giving a speech on PyTorch on Scala3. I have already received inquiries from some readers asking if our underlying javacpp-pytorch supports ProcessGroupNccl, as this will determine whether they will actually use javacpp-pytorch in their work. Their company's computing power servers are all Ubuntu 22 or 24, with NVIDIA A100, H20, GTX-6000 GPUs. They tested that Gloo's performance is only about 35% of NCCL's speed. I said that it should not be a problem in the future, and we will consider supporting ProcessGroupNccl as soon as possible.

In conclusion, I personally feel that in the future, more and more people will recognize javacpp-pytorch. The scenarios we support are richer, and people are willing to use it.

[saudet]

Thanks for the detailed report! It all sounds alright, with a large enough team of maybe 10 good engineers I'm confident we can get all that running in a couple of years, but it needs to be clear that they can't expect all of that to get done in any reasonable time frame with only $1000 per month. I'm OK with starting working on small tasks and see how things go after a few months, for example.

Also, they need to understand that, for example, PyTorch needs Python to optimize models. That's not something that the C++ API can do. So we still need Python for many tasks.

As for licensing, JavaCPP itself and everything written as part of Bytedeco is as per this file, which should be fine:
https://github.com/bytedeco/javacpp-presets/blob/master/LICENSE.txt
However, that doesn't cover the dependencies such as PyTorch itself. That's something users need to be responsible of.

[mullerhai]

Thanks for the detailed report! It all sounds alright, with a large enough team of maybe 10 good engineers I'm confident we can get all that running in a couple of years, but it needs to be clear that they can't expect all of that to get done in any reasonable time frame with only $1000 per month. I'm OK with starting working on small tasks and see how things go after a few months, for example.

Also, they need to understand that, for example, PyTorch needs Python to optimize models. That's not something that the C++ API can do. So we still need Python for many tasks.

As for licensing, JavaCPP itself and everything written as part of Bytedeco is as per this file, which should be fine: https://github.com/bytedeco/javacpp-presets/blob/master/LICENSE.txt However, that doesn't cover the dependencies such as PyTorch itself. That's something users need to be responsible of.

hi @saudet ，thanks your reply。
Next time I communicate with Huawei, I will continue to share your ideas with them. I will make sure they manage their expectations; javacpp is not a universal solution, and we will definitely implement anything that can be done well.

First, I believe Huawei is a trustworthy company, and their promises should be deliverable. Of course, we shouldn't put all our eggs in one basket. I think we can also treat our collaboration with Huawei as an advertisement for the effectiveness of javacpp. We can use Huawei's reputation to enhance Bytedeco's professionalism and build our name. Let more internet and technology companies know that we have received recognition from Huawei, a leading telecommunications giant in China and the world. Let this collaboration create a ripple effect. We can participate in more meetups and explain our technology and products, and other companies will come to us.

As Bytedeco gains the trust and patronage of more companies, they will become more dependent on us. In the future, the donations and revenue we receive will surely exceed what we would get from Huawei alone. javacpp is indeed a core technology that greatly simplifies development. For example, languages like Rust, Go, and Moonbit all want to create language bindings for PyTorch, but they don't have javacpp technology.

In the future, I believe we can also develop Scala3Cpp or KotlinCpp, which would greatly promote our ecosystem and help us capture a larger market share in the AI era. If Bytedeco needs to commercialize in the future, we will have the accumulated technology, upstream and downstream relationships, and a base of clients from the early stages.

@sbrunk pytorch on scala3 will gain more powerful tech