-
-
Notifications
You must be signed in to change notification settings - Fork 499
Description
Subject: NullPointerException in ThreddsHarvester.extractMetadataFromOpendapDDX (4.2.2 & 4.4.4)
Description of the problem:
When attempting to harvest a THREDDS catalog using the "THREDDS" harvester type in GeoNetwork, a recurring NullPointerException
occurs in the extractMetadataFromOpendapDDX
method. This error prevents the successful harvesting of dataset metadata.
Affected versions:
We have consistently replicated this behavior in GeoNetwork 4.2.2 and 4.4.4.
Steps to reproduce:
- Deploy a GeoNetwork instance (in our case, using Docker Compose, version 4.2.2).
- Log in to the GeoNetwork administration interface.
- Create a new harvester of type "THREDDS".
- Configure the THREDDS catalog URL:
http://172.16.132.22:8004/thredds/catalog/auv/glider/sl3u1064-scb_sl3u1064/L1/2025/catalog.xml
- Ensure that the custom XSLT
ThreddsCatalog-to-ISO19119_ISO19139.xsl
is correctly located at/var/lib/jetty/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19139/convert/
and referenced inschema-conversions.xml
for the conversionfrom="thredds"
toiso19139
. - Execute the harvester.
Expected behavior:
The harvester should process the THREDDS catalog, apply the XSLT transformation, and import the ISO 19139 metadata without errors.
Observed behavior (with DEBUG logs):
The harvester starts, reads the main THREDDS catalog, but when attempting to process individual datasets and access the OPeNDAP DDX service, it throws a NullPointerException
. The detailed logs indicate that GeoNetwork successfully accesses the DDX XML, but the error occurs during its internal processing of this DDX. A second NullPointerException
in Xml.transform
is observed, which appears to be a consequence of the first.
Relevant log snippet:
2025-07-16T11:13:01,262 INFO [geonetwork.harvester] - Starting harvesting of gn422_threddsmiguel
2025-07-16T11:13:01,282 INFO [geonetwork.harvester] - Started harvesting from node : gn422_threddsmiguel (ThreddsHarvester)
2025-07-16T11:13:01,284 INFO [geonetwork.harvester] - Retrieving remote metadata information for : gn422_threddsmiguel
catalog valid = true
access auv/glider/sl3u1064-scb_sl3u1064/L1/2025/dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20_data_rt.nc valid = true
dataset dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20_data_rt.nc valid = true
access auv/glider/sl3u1064-scb_sl3u1064/L1/2025/dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20.nc valid = true
dataset dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20.nc valid = true
access auv/glider/sl3u1064-scb_sl3u1064/L1/2025/dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09_data_rt.nc valid = true
dataset dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09_data_rt.nc valid = true
access auv/glider/sl3u1064-scb_sl3u1064/L1/2025/dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09.nc valid = true
dataset dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09.nc valid = true
dataset 2025 valid = true
2025-07-16T11:13:01,376 INFO [geonetwork.harvester] - Catalog read from http://172.16.132.22:8004//thredds/catalog/auv/glider/sl3u1064-scb_sl3u1064/L1/2025/catalog.xml is
2025-07-16T11:13:01,376 INFO [geonetwork.harvester] - Crawling the datasets in the catalog....
2025-07-16T11:13:01,377 INFO [geonetwork.harvester] - Crawling through 2025
2025-07-16T11:13:01,377 INFO [geonetwork.harvester] - Trying to find ThreddsMetadata for dataset 2025
2025-07-16T11:13:01,377 INFO [geonetwork.harvester] - Crawling through dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20_data_rt.nc
2025-07-16T11:13:01,377 INFO [geonetwork.harvester] - Processing dataset: dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20_data_rt.nc with URL: http://172.16.132.22:8004//thredds/catalog/auv/glider/sl3u1064-scb_sl3u1064/L1/2025/catalog.xml?dataset=auv%2Fglider%2Fsl3u1064-scb_sl3u1064%2FL1%2F2025%2Fdep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20_data_rt.nc
2025-07-16T11:13:01,492 ERROR [geonetwork.harvester] - Thrown Exception java.lang.NullPointerException during dataset processing
java.lang.NullPointerException
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.extractMetadataFromOpendapDDX(Harvester.java:1364)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.getMetadata(Harvester.java:864)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.examineThreddsDataset(Harvester.java:604)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:462)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:458)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvestCatalog(Harvester.java:411)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvest(Harvester.java:284)
at org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester.doHarvest(ThreddsHarvester.java:76)
at org.fao.geonet.kernel.harvest.harvester.AbstractHarvesterHarvestWithIndexProcessor.process(AbstractHarvester.java:602)atorg.fao.geonet.kernel.harvest.harvester.AbstractHarvester.harvest(AbstractHarvester.java:673)atorg.fao.geonet.kernel.harvest.harvester.HarvesterJob.execute(HarvesterJob.java:69)atorg.quartz.core.JobRunShell.run(JobRunShell.java:202)atorg.quartz.simpl.SimpleThreadPoolWorkerThread.run(SimpleThreadPool.java:573)
2025-07-16T11:13:01,493 INFO [geonetwork.harvester] - Processing service: http://172.16.132.22:8004/thredds/iso/ for dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20_data_rt.nc
2025-07-16T11:13:01,493 INFO [geonetwork.harvester] - Crawling through dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20.nc
2025-07-16T11:13:01,494 INFO [geonetwork.harvester] - Processing dataset: dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20.nc with URL: http://172.16.132.22:8004//thredds/catalog/auv/glider/sl3u1064-scb_sl3u1064/L1/2025/catalog.xml?dataset=auv%2Fglider%2Fsl3u1064-scb_sl3u1064%2FL1%2F2025%2Fdep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20.nc
2025-07-16T11:13:01,546 ERROR [geonetwork.harvester] - Thrown Exception java.lang.NullPointerException during dataset processing
java.lang.NullPointerException
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.extractMetadataFromOpendapDDX(Harvester.java:1364)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.getMetadata(Harvester.java:864)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.examineThreddsDataset(Harvester.java:604)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:462)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:458)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvestCatalog(Harvester.java:411)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvest(Harvester.java:284)
at org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester.doHarvest(ThreddsHarvester.java:76)
at org.fao.geonet.kernel.harvest.harvester.AbstractHarvesterHarvestWithIndexProcessor.process(AbstractHarvester.java:602)atorg.fao.geonet.kernel.harvest.harvester.AbstractHarvester.harvest(AbstractHarvester.java:673)atorg.fao.geonet.kernel.harvest.harvester.HarvesterJob.execute(HarvesterJob.java:69)atorg.quartz.core.JobRunShell.run(JobRunShell.java:202)atorg.quartz.simpl.SimpleThreadPoolWorkerThread.run(SimpleThreadPool.java:573)
2025-07-16T11:13:01,547 INFO [geonetwork.harvester] - Processing service: http://172.16.132.22:8004/thredds/iso/ for dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20.nc
2025-07-16T11:13:01,547 INFO [geonetwork.harvester] - Crawling through dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09_data_rt.nc
2025-07-16T11:13:01,547 INFO [geonetwork.harvester] - Processing dataset: dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09_data_rt.nc with URL: http://172.16.132.22:8004//thredds/catalog/auv/glider/sl3u1064-scb_sl3u1064/L1/2025/catalog.xml?dataset=auv%2Fglider%2Fsl3u1064-scb_sl3u1064%2FL1%2F2025%2Fdep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09_data_rt.nc
2025-07-16T11:13:01,606 ERROR [geonetwork.harvester] - Thrown Exception java.lang.NullPointerException during dataset processing
java.lang.NullPointerException
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.extractMetadataFromOpendapDDX(Harvester.java:1364)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.getMetadata(Harvester.java:864)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.examineThreddsDataset(Harvester.java:604)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:462)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:458)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvestCatalog(Harvester.java:411)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvest(Harvester.java:284)
at org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester.doHarvest(ThreddsHarvester.java:76)
at org.fao.geonet.kernel.harvest.harvester.AbstractHarvesterHarvestWithIndexProcessor.process(AbstractHarvester.java:602)atorg.fao.geonet.kernel.harvest.harvester.AbstractHarvester.harvest(AbstractHarvester.java:673)atorg.fao.geonet.kernel.harvest.harvester.HarvesterJob.execute(HarvesterJob.java:69)atorg.quartz.core.JobRunShell.run(JobRunShell.java:202)atorg.quartz.simpl.SimpleThreadPoolWorkerThread.run(SimpleThreadPool.java:573)
2025-07-16T11:13:01,606 INFO [geonetwork.harvester] - Processing service: http://172.16.132.22:8004/thredds/iso/ for dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09_data_rt.nc
2025-07-16T11:13:01,606 INFO [geonetwork.harvester] - Crawling through dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09.nc
2025-07-16T11:13:01,607 INFO [geonetwork.harvester] - Processing dataset: dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09.nc with URL: http://172.16.132.22:8004//thredds/catalog/auv/glider/sl3u1064-scb_sl3u1064/L1/2025/catalog.xml?dataset=auv%2Fglider%2Fsl3u1064-scb_sl3u1064%2FL1%2F2025%2Fdep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09.nc
2025-07-16T11:13:01,663 ERROR [geonetwork.harvester] - Thrown Exception java.lang.NullPointerException during dataset processing
java.lang.NullPointerException
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.extractMetadataFromOpendapDDX(Harvester.java:1364)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.getMetadata(Harvester.java:864)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.examineThreddsDataset(Harvester.java:604)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:462)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:458)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvestCatalog(Harvester.java:411)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvest(Harvester.java:284)
at org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester.doHarvest(ThreddsHarvester.java:76)
at org.fao.geonet.kernel.harvest.harvester.AbstractHarvesterHarvestWithIndexProcessor.process(AbstractHarvester.java:602)atorg.fao.geonet.kernel.harvest.harvester.AbstractHarvester.harvest(AbstractHarvester.java:673)atorg.fao.geonet.kernel.harvest.harvester.HarvesterJob.execute(HarvesterJob.java:69)atorg.quartz.core.JobRunShell.run(JobRunShell.java:202)atorg.quartz.simpl.SimpleThreadPoolWorkerThread.run(SimpleThreadPool.java:573)
2025-07-16T11:13:01,664 INFO [geonetwork.harvester] - Processing service: http://172.16.132.22:8004/thredds/iso/ for dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09.nc
2025-07-16T11:13:01,665 INFO [geonetwork.harvester] - Adding dataset metadata...
2025-07-16T11:13:01,665 WARN [geonetwork.harvester] - Raised exception while harvesting from : gn422_threddsmiguel (ThreddsHarvester)
2025-07-16T11:13:01,665 WARN [geonetwork.harvester] - (C) Class : NullPointerException
2025-07-16T11:13:01,666 WARN [geonetwork.harvester] - (C) Message : null
2025-07-16T11:13:01,666 ERROR [geonetwork.harvester] - null
java.lang.NullPointerException: null
at org.fao.geonet.utils.Xml.transform(Xml.java:493) ~[gn-common-4.2.2-0.jar:?]
at org.fao.geonet.utils.Xml.transform(Xml.java:400) ~[gn-common-4.2.2-0.jar:?]
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.createDatasetMetadata(Harvester.java:1545) ~[gn-harvesters-4.2.2-0.jar:?]
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvestCatalog(Harvester.java:420) ~[gn-harvesters-4.2.2-0.jar:?]
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvest(Harvester.java:284) ~[gn-harvesters-4.2.2-0.jar:?]
at org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester.doHarvest(ThreddsHarvester.java:76) ~[gn-harvesters-4.2.2-0.jar:?]
at org.fao.geonet.kernel.harvest.harvester.AbstractHarvesterHarvestWithIndexProcessor.process(AbstractHarvester.java:602)
[˜gn−harvesters−4.2.2−0.jar:?]atorg.fao.geonet.kernel.harvest.harvester.AbstractHarvester.harvest(AbstractHarvester.java:673)
[˜gn−harvesters−4.2.2−0.jar:?]atorg.fao.geonet.kernel.harvest.harvester.HarvesterJob.execute(HarvesterJob.java:69)
[˜quartz−2.3.2.jar:?]atorg.quartz.core.JobRunShell.run(JobRunShell.java:202)
[˜ quartz−2.3.2.jar:?]atorg.quartz.simpl.SimpleThreadPoolWorkerThread.run(SimpleThreadPool.java:573) ~[quartz-2.3.2.jar:?]