Change in postcode behaviour between 3.4.2 and 4.0.1

[eeit232]

Hi,

I recently updated some servers from Nominatim 3.4.2 to 4.0.1 and have noticed a change in behaviour for GB postcode lookups.
The bounding box returned in 4.0.1 is much larger than in 3.4.2, and seems inappropriate for a 'street-level' postcode.

Servers have britain-and-ireland-latest.osm.pbf and external GB postcode data (noting different format of postcode data between these versions) loaded at start and not updated.

Examples of postcode searches and the results:

Nominatim 3.4.2
https://www.nominatim.org/data/gb_postcode_data.sql.gz
curl "localhost/search?q=bs16+1ej&format=json&limit=1"
[{"place_id":9481853,"licence":"Data © OpenStreetMap contributors, ODbL 1.0. https://osm.org/copyright","boundingbox":["51.501853114444","51.501953114444","-2.5419370268355","-2.5418370268355"],"lat":"51.5019031144444","lon":"-2.54188702683548","display_name":"Bristol Business Park, Stoke Gifford, Harry Stoke, South Gloucestershire, West of England, England, BS16 1EJ, United Kingdom","class":"place","type":"postcode","importance":0.325}]

Nominatim 4.0.1
https://nominatim.org/data/gb_postcodes.csv.gz
curl "localhost/search?q=bs16+1ej&format=json&limit=1"
[{"place_id":11167378,"licence":"Data © OpenStreetMap contributors, ODbL 1.0. https://osm.org/copyright","boundingbox":["51.34188595","51.66188595","-2.7028520781917","-2.3828520781917"],"lat":"51.50188595","lon":"-2.5428520781916957","display_name":"Bristol Business Park, Stoke Park and Cheswick, Warmley, Bradley Stoke, South Gloucestershire, West of England, England, BS16 1EJ, United Kingdom","class":"place","type":"postcode","importance":0.325}]

Is this simply a function of the difference between gb_postcode_data.sql.gz (which contains a geometry element) and gb_postcodes.csv.gz (which only contains lat/lon data), or is there something more fundamental with how postcodes are treated in the newer version causing this?

Seems like a regression as bounding box in the earlier version was much more suitable for a GB postcode, which is usually a single or few houses. Version 4.0.1 seems more appropriate for a city not a postcode, for example a London postcode has a bounding box which covers the whole of Greater London?

If this is just a data issue, is there a way to include bounding box info in the gb_postcodes.csv format?

[mtmail]

4.0.1 was released two years ago (3.4.1 five years ago, the .2 was a patch release). Even if we can provide an answer it will be unclear if it's already outdated and we wouldn't make any changes to the old version.

This is the output of 4.5.0
https://nominatim.openstreetmap.org/ui/search.html?q=bs16+1ej
https://nominatim.openstreetmap.org/search.php?q=bs16+1ej&format=jsonv2&limit=1

Are bounding boxes for postcodes reliable? Royalmail only provides center points with roughly 100m accuracy. (Their paid version apparently 2m but that's not open data).

[eeit232]

Hi mtmail,

Many thanks for your speedy response, and appreciate that I'm working with some very old versions here. I would much rather be running 4.5.0 but unfortunately there are issues within my organisation that prevent that, lets just call it legacy thinking ;-)

Running the query against 4.5.0 returns the same bounding box (except for precision), so the behaviour is still present. Understand that even if this was changed in a current build you wouldn't entertain backporting, for me it's more about being able to explain the change in behaviour to customers, and to explore if there are any options for changing to the previous behaviour.

I decoded some of the old geometry fields from the gb_postcode_data.sql.gz and can see they are just points too, so no bounding boxes in either dataset. Also location_postcode table has geometry field which is just a point, and no bounding box info. This suggests it's not the import process but the query response adding the bounding box.

Postcodes search ranks seem to be between 21 and 30, so would have an assumed extent of <1km, should the bounding box not reflect that?

[mtmail]

I can confirm only center points of postcodes were imported. It used to be its own table, later location_postcodes (country, postcode, lat, lon). The bounding box gets calculated based on the place type (here: postcode) bbox_from_result() just before the result is send back to the user.

Nominatim/src/nominatim_api/v1/classtypes.py

Line 4 in d856788

#

In the last years Nominatim was rewritten to a new programming language (PHP -> Python) so huge amounts of code got moved to different files. Rollout was in multiple steps. Nominatim 3.5 was all PHP, 4.5 is all Python. In the PHP code there was a similar method ClassTypes::getDefRadius.

The algorithm isn't very sophisticated. It's applied around the world, not just GB. So the question is if it should be configurable or different for countries with more precise postcodes (GB, CA, Ireland).

[lonvia]

Somebody else already noted recently that the bounding boxes are a bit on the large side. I need to look into this. We have the search rank that should ensure they get a smaller box in the UK than, say, in Germany.

Note that this will only be fixed for upcoming releases though. You'll need to do on your own backporting to 4.0, I'm afraid.

[eeit232]

Thanks for your response. Will look to address locally by overriding leaflet default behaviour for postcodes as a temporary fix.