Rust Support

[alsuren]

Moving here because I accidentally posted to votchallenge/toolkit#67 .

Original Description:

Some background: I was looking for a benchmark dataset for evaluating a rust-based implementation of the MOSSE tracker. I found the OTB-2015 dataset, which was really hard to use, and then I found the VOT datasets, which are really easy to use 😃.

I threw together a simple implementation of the trax protocol, and used it to evaluate the tracker's performance. My implementation only supports RGB images with square target regions, and doesn't support filenames with spaces in, etc. but it was good enough to run against the 2020 dataset, to get us started.

The code for the protocol bit currently lives here: https://github.com/jjhbw/mosse-tracker/pull/6/files#diff-760f763636fdf5a7b407ee67396404031d573d2e203d67025b17a939173b7870 and there is an example usage in main.rs. It's currently "demo quality" code.

Do you think that it would be worth me cleaning up the code and publishing it for others to use?

If so, should I do it as a PR against this repo, or host it under the https://github.com/rust-cv namespace?

If you would prefer to keep all implementations uniform (binding into the C library) then that's also fine. If that is the case, I'll leave my code where it is until someone does the rust trax implementation properly, and then port the mosse-tracker benchmark code across.

Response from @lukacu :

Hi, thank you for your work, this is really exciting. It would be better to move this discussion to trax project issue tracker. PRs should also be made towards that repo and not the toolkit.

The protocol is pretty stable and no immediate updates are planned (although extension to multiple targets may be interesting in the future). So a pure implementation would be ok, that is the point of a protocol anyway. But from maintenance standpoint I would prefer C bindings, I do not know Rust well enough to maintain a separate implementation. Is C-Rust integration difficult?

I will admit that I've never tried. In theory I'd expect it to be a bit harder than C++, and a bit easier than Python? I'm not really sure.

If you wanted to wrap this library in Rust, you'd probably also want to wrap the cmake build system, so that user of your library can just cargo add trax-sys or whatever, and have it fetch/build the C source code for you. Bundling up the C source code, and driving CMake from Rust's "build.rs" doesn't sound too tricky though.

Whenever you cross a language barrier, you will always get an impedance mismatch that makes things like error handling and debugging a bit uglier. I expect you will have noticed that when writing the python client.

It's worth noting that my implementation of the protocol is very half-arsed. I'm not sure how easy it would be to make a half-arsed wrapper around a C library. I guess you could just wrap the types and functions that you care about first, and it would come out to the same amount of work for a PoC?

The other question that's worth asking is "do people care about Rust enough or is it just going to be a maintenance burden with no users?". The rust computer vision ecosystem is very immature at the moment. In terms of object tracking, there is a MOSSE implementation (which is currently less capable than the dlib one), and you could probably cobble something together using AKAZE features. There isn't really anything state-of-the-art. If your benchmark suddenly had first class support for Rust, would people suddenly start trying to build trackers in Rust? I suspect that the answer is "no".

Maybe the best thing would be to leave this issue open for a year and see if it draws any interest. If not, we can close it.

[lukacu]

In terms of usefulness I would say that research-wise adoption of Rust is not very likely, however, in the industry it makes sense. Therefore I would like to have support one way or another so that production-level trackers may also be evaluated.

Like you have said, I will keep the issue open, at least somebody will find your code and use it as it is. For the reference implementation I would opt for rust wrapper to keep the redundancy low. You can also make your standalone, unofficial half working implementation and we will gladly link it as "community effort" for now.