A simple test with OrienterNet

[basbaba]

Hi sarlin:

I make a small testing sample, followed maploc/data/mapillary/prepare.py
left all the key parameters as default:

crop_size_meters: 64
z_max: 32
x_max: 32
pixel_per_meter: 2

It's a single-test, I took 4 photos at the same place 1 by 1 as the following map indicates:

This is really where I stand and bearing. Then use OrienterNet to get the results(in the same order):

The red arrow is a dummy position/direction, because beside a high building the accuration of gps is really bad, so I manually filled a random data in image_infors/xxx.json.

I uploaded my testing files as ON_test and shared you with the access, sorry for personal privacy I don't want to make it public.
Just type:
python test_single.py
If you have time please have a try.

The result is not ideal. I tried so many modification on parameters but failed,such as:

1. enlarge crop_size_meters
2. Increase/decrease z_max/x_max/ppm

I think, maybe the problem is the depth estimation, it's easy to inference a relative depth but not absolute depth from a mono photo, for example, in the first photo there is a building about 200 meters away, but that appeared in photo and be figured out in BEV, this is already far beyond the range of z_max(32 meters), so it's hard to let the BEV to match the map in real size. I had tried to increase z_max but my GPU only has 40GB ram, after adjust z_max to 36, it crashed OOM.
I know the current BEV-Map registering is by a FFT optimized template matching, the rotation can be 64/256, is it possible to add a scale mechanism just like rotation in the matching process(of cause the resolution should be down sampled to fit the GPU RAM size).

Thanx!

BTW, why don't you use SuperGlue in this project?

[basbaba]

And I had already used PerspectiveFiled to check these photos, such as:

I think they are in good shape, so left them untouched.

[basbaba]

I tried to dump out depth_prob from

OrienterNet/maploc/models/bev_projection.py

Line 44 in 09f8fa3

depth_prob = torch.softmax(depth_scores, dim=1)

each column:

1. photo
2. depth_prob in Polar coordination
3. depth_prob in Cartesian coordination
4. bev