AR for distance detection, etc?

[gardar]

Would it be possible (and would it make sense) to add AR to frigate for surface detection, which could then be used for things like distance/depth? In a similar fashion to android's arcore technology.

Example use cases would be for only detecting persons that are within distance.
I also wonder if it would make sense to use the AR distance detection to mask everything that's out of distance and save the processing power for objects that are within distance. But perhaps the distance detection has the same or even more processing penalty as the object detection.

[NickM-27]

The most difficult part is this AR on phones is done using multiple sensors, cameras, sometimes LiDAR, etc and wouldn't be available through a single camera feed.

Would need additional sensors to provide this information and a way to account for differences in latency between camera stream and the sensor.

[gardar]

Yeah that makes sense, I surely wouldn't expect that it would work on all cameras but I do wonder if there are some cameras that have hardware components that could be used for this, with multiple lenses, and perhaps IR could help? Perhaps it's possible to do software processing in some sense even though hardware processing would always be better.
With that being said, apparently arcore can use just a single camera https://9to5google.com/2019/12/09/google-arcore-depth/

Now putting all that aside and assuming that we wouldn't have to worry about the hardware, do you think it would make sense to add this or is the benefit minimal for the work that's required?

[NickM-27]

With that being said, apparently arcore can use just a single camera https://9to5google.com/2019/12/09/google-arcore-depth/

Oh cool, didn't know they got that working

Now putting all that aside and assuming that we wouldn't have to worry about the hardware, do you think it would make sense to add this or is the benefit minimal for the work that's required?

I don't personally know enough to say for sure, but I don't think it'll be the next thing to work on even if it does make sense. Using object sizes along with zones would do this somewhat already (setting min and max sizes for different zones). Better models (Frigate+) will help with that as well.

There's also general talks of better tracking in general in a future release which would help as well as helping other things as well.

Personally, I think it might be a stretch goal but to me that can have a better idea once these other improvements have been implemented.