Will we see an official multi-arch docker image release on Docker Hub?

Will the official docker hub repo https://hub.docker.com/r/mozilla/syncserver ever get updated with multi-arch images (armhf, armv8 etc)?

[jrconlin]

Hmm... not really sure I can answer that, but let me offer my opinion.

syncstorage-rs is the next generation build of the sync back end. We're building this for a few reasons, some of those are definitely more "fix our problems". One of the bigger ones is that we need to use something that keeps data better than what the original sync server was doing.

Running sync server on a raspberry pi for yourself, you'd be unaware of a lot of those problems. Running sync servers for hundreds of millions of users on a limited budget, and you become deeply aware of those issues.

A bunch of those are the driving reasons for why we're rebuilding in Rust and using a cloud based database. We're still absolutely keeping support for local MySQL for a number of reasons.

The nice thing is that rust offers multi platform support itself, so it should be possible to run much of the code on whatever you prefer.

So, what does all that mean?

1. If there's a docker repo made, more than likely it's going to be more for our needs than anyone else's. (We're a small team (3 people) working on a big project, It's not that we don't love everyone, it's just that sometimes all we can offer is a smile and a wave as we run by.)
2. That shouldn't mean that you can't build your own. It's a bit of lifting to figure to customize the Dockerfile or the docker-compose.yaml file, but it's do-able. Plus, you really want to use your own master secret instead of the test one we have in there right now.
3. If we can make additional docker images, it'll probably not be a long term thing because, again, not really critical to what we need and we're resource strapped. It might mean that the specialized versions get orphaned because it was a critical fix and the person rolling out the deploy forgot to update the other images.

Does that help answer your question?

[sethidden]

I had to build the images natively on an ARM device (Raspberry Pi 4B). It'd be much easier for maintainers here if docker buildx worked, since you'd be able to build for ~8 (well, actually 4, because the rust base image supports only 4 architectures) architectures for free in your regular CI. Sadly it didn't work for me - it failed on the python requirements download step if building for arm64. More exotic architectures like riscv64, 386 etc. failed even earlier (which is understandable, since the rust base image is not built for riscv64, but 386 failure is a mystery - since it's supported)

EDIT: links removed because they no longer work

[zygimantus]

@sethidden , your shared links are not accessible anymore.

[nekromantik]

@sethidden were you able to get it running on Pi 4b?
if so do you have any instructions or docker file?
I got K3s running on my Pi4

[snoweuph]

So if I understand here, the current codebase and project setup don't make it to easy to develop for all of the simultaneously, and there are not enough human and funding resources to give this any Priority.

@ Maintainers, would this become anywhere viable with enough community funding, or is the major problem, the limited amount of people?

[jrconlin]

Pretty much a combination of things.

1. There aren't a lot of folk paid to work on this, and we split our time between this project and a few others. In essence, our top priority is to make sure it works for us, since we have to keep the service alive and cost effective. (Like a lot of cloud things, running this service isn't cheap.) Frankly, I would be thrilled if more folk ran their own servers because it would reduce the overall costs for us, but that's a fairly tall ask for a huge portion of our user base.

2. We don't have a lot of outside folk pitching in. Put frankly, since we don't have the bandwidth to work on making this happen, we kinda rely on the larger community to help. @Eragonfr has been doing the lion's share of work on getting the MySQL interface working, and frankly, she's doing as a labor of love and is pretty strapped herself. (I mean, she's a hero for picking that up, but that doesn't really pay rent.) MySQL support is also pretty high up on the list of "things you need for a stand-alone server", and I'd probably put that ahead of any docker builds.

If you're really interested in seeing this up and working, there are a lot of older issues for self hosting phase 1 and self hosting phase 2 that could take looking into.

All that said, the Sync API hasn't changed in quite some time. We moved to rust and spanner because, frankly, it was far more cost saving than running python and mysql (less VMs required, less database failures, smaller monthly bills, etc). If some other brave soul wants to take the old python 2.7 syncserver, rewrite it in modern 3.12 python and roll that out as a solution, have at. (I mean, let us know so that we can remember to tell y'all if there's some weird, breaking change we have to introduce, but otherwise, it's part of the fun of open source.)

And yeah, Times change in the past 5 years and we've de-prioritized support for MySQL. We no longer consider it necessary (but I tried to design things so that you could plug whatever database you wanted into sync).