Skip to content

[BUG] NullPointerException in ThreddsHarvester.extractMetadataFromOpendapDDX (4.2.2 & 4.4.4) #8938

@gdauden

Description

@gdauden

Subject: NullPointerException in ThreddsHarvester.extractMetadataFromOpendapDDX (4.2.2 & 4.4.4)

Description of the problem:
When attempting to harvest a THREDDS catalog using the "THREDDS" harvester type in GeoNetwork, a recurring NullPointerException occurs in the extractMetadataFromOpendapDDX method. This error prevents the successful harvesting of dataset metadata.

Affected versions:
We have consistently replicated this behavior in GeoNetwork 4.2.2 and 4.4.4.

Steps to reproduce:

  1. Deploy a GeoNetwork instance (in our case, using Docker Compose, version 4.2.2).
  2. Log in to the GeoNetwork administration interface.
  3. Create a new harvester of type "THREDDS".
  4. Configure the THREDDS catalog URL: http://172.16.132.22:8004/thredds/catalog/auv/glider/sl3u1064-scb_sl3u1064/L1/2025/catalog.xml
  5. Ensure that the custom XSLT ThreddsCatalog-to-ISO19119_ISO19139.xsl is correctly located at /var/lib/jetty/webapps/geonetwork/WEB-INF/data/config/schema_plugins/iso19139/convert/ and referenced in schema-conversions.xml for the conversion from="thredds" to iso19139.
  6. Execute the harvester.

Expected behavior:
The harvester should process the THREDDS catalog, apply the XSLT transformation, and import the ISO 19139 metadata without errors.

Observed behavior (with DEBUG logs):
The harvester starts, reads the main THREDDS catalog, but when attempting to process individual datasets and access the OPeNDAP DDX service, it throws a NullPointerException. The detailed logs indicate that GeoNetwork successfully accesses the DDX XML, but the error occurs during its internal processing of this DDX. A second NullPointerException in Xml.transform is observed, which appears to be a consequence of the first.

Relevant log snippet:

2025-07-16T11:13:01,262 INFO  [geonetwork.harvester] - Starting harvesting of gn422_threddsmiguel
2025-07-16T11:13:01,282 INFO  [geonetwork.harvester] - Started harvesting from node : gn422_threddsmiguel (ThreddsHarvester)
2025-07-16T11:13:01,284 INFO  [geonetwork.harvester] - Retrieving remote metadata information for : gn422_threddsmiguel
catalog valid = true
access auv/glider/sl3u1064-scb_sl3u1064/L1/2025/dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20_data_rt.nc valid = true
dataset dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20_data_rt.nc valid = true
access auv/glider/sl3u1064-scb_sl3u1064/L1/2025/dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20.nc valid = true
dataset dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20.nc valid = true
access auv/glider/sl3u1064-scb_sl3u1064/L1/2025/dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09_data_rt.nc valid = true
dataset dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09_data_rt.nc valid = true
access auv/glider/sl3u1064-scb_sl3u1064/L1/2025/dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09.nc valid = true
dataset dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09.nc valid = true
dataset 2025 valid = true
2025-07-16T11:13:01,376 INFO  [geonetwork.harvester] - Catalog read from http://172.16.132.22:8004//thredds/catalog/auv/glider/sl3u1064-scb_sl3u1064/L1/2025/catalog.xml is

2025-07-16T11:13:01,376 INFO  [geonetwork.harvester] - Crawling the datasets in the catalog....
2025-07-16T11:13:01,377 INFO  [geonetwork.harvester] - Crawling through 2025
2025-07-16T11:13:01,377 INFO  [geonetwork.harvester] - Trying to find ThreddsMetadata for dataset 2025
2025-07-16T11:13:01,377 INFO  [geonetwork.harvester] - Crawling through dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20_data_rt.nc
2025-07-16T11:13:01,377 INFO  [geonetwork.harvester] - Processing dataset: dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20_data_rt.nc with URL: http://172.16.132.22:8004//thredds/catalog/auv/glider/sl3u1064-scb_sl3u1064/L1/2025/catalog.xml?dataset=auv%2Fglider%2Fsl3u1064-scb_sl3u1064%2FL1%2F2025%2Fdep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20_data_rt.nc
2025-07-16T11:13:01,492 ERROR [geonetwork.harvester] - Thrown Exception java.lang.NullPointerException during dataset processing
java.lang.NullPointerException
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.extractMetadataFromOpendapDDX(Harvester.java:1364)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.getMetadata(Harvester.java:864)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.examineThreddsDataset(Harvester.java:604)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:462)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:458)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvestCatalog(Harvester.java:411)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvest(Harvester.java:284)
at org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester.doHarvest(ThreddsHarvester.java:76)
at org.fao.geonet.kernel.harvest.harvester.AbstractHarvesterHarvestWithIndexProcessor.process(AbstractHarvester.java:602)atorg.fao.geonet.kernel.harvest.harvester.AbstractHarvester.harvest(AbstractHarvester.java:673)atorg.fao.geonet.kernel.harvest.harvester.HarvesterJob.execute(HarvesterJob.java:69)atorg.quartz.core.JobRunShell.run(JobRunShell.java:202)atorg.quartz.simpl.SimpleThreadPoolWorkerThread.run(SimpleThreadPool.java:573)
2025-07-16T11:13:01,493 INFO  [geonetwork.harvester] - Processing service: http://172.16.132.22:8004/thredds/iso/ for dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20_data_rt.nc
2025-07-16T11:13:01,493 INFO  [geonetwork.harvester] - Crawling through dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20.nc
2025-07-16T11:13:01,494 INFO  [geonetwork.harvester] - Processing dataset: dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20.nc with URL: http://172.16.132.22:8004//thredds/catalog/auv/glider/sl3u1064-scb_sl3u1064/L1/2025/catalog.xml?dataset=auv%2Fglider%2Fsl3u1064-scb_sl3u1064%2FL1%2F2025%2Fdep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20.nc
2025-07-16T11:13:01,546 ERROR [geonetwork.harvester] - Thrown Exception java.lang.NullPointerException during dataset processing
java.lang.NullPointerException
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.extractMetadataFromOpendapDDX(Harvester.java:1364)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.getMetadata(Harvester.java:864)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.examineThreddsDataset(Harvester.java:604)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:462)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:458)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvestCatalog(Harvester.java:411)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvest(Harvester.java:284)
at org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester.doHarvest(ThreddsHarvester.java:76)
at org.fao.geonet.kernel.harvest.harvester.AbstractHarvesterHarvestWithIndexProcessor.process(AbstractHarvester.java:602)atorg.fao.geonet.kernel.harvest.harvester.AbstractHarvester.harvest(AbstractHarvester.java:673)atorg.fao.geonet.kernel.harvest.harvester.HarvesterJob.execute(HarvesterJob.java:69)atorg.quartz.core.JobRunShell.run(JobRunShell.java:202)atorg.quartz.simpl.SimpleThreadPoolWorkerThread.run(SimpleThreadPool.java:573)
2025-07-16T11:13:01,547 INFO  [geonetwork.harvester] - Processing service: http://172.16.132.22:8004/thredds/iso/ for dep0010_sl3u1064_scb-sl3u1064_L1_2025-05-20.nc
2025-07-16T11:13:01,547 INFO  [geonetwork.harvester] - Crawling through dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09_data_rt.nc
2025-07-16T11:13:01,547 INFO  [geonetwork.harvester] - Processing dataset: dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09_data_rt.nc with URL: http://172.16.132.22:8004//thredds/catalog/auv/glider/sl3u1064-scb_sl3u1064/L1/2025/catalog.xml?dataset=auv%2Fglider%2Fsl3u1064-scb_sl3u1064%2FL1%2F2025%2Fdep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09_data_rt.nc
2025-07-16T11:13:01,606 ERROR [geonetwork.harvester] - Thrown Exception java.lang.NullPointerException during dataset processing
java.lang.NullPointerException
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.extractMetadataFromOpendapDDX(Harvester.java:1364)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.getMetadata(Harvester.java:864)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.examineThreddsDataset(Harvester.java:604)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:462)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:458)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvestCatalog(Harvester.java:411)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvest(Harvester.java:284)
at org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester.doHarvest(ThreddsHarvester.java:76)
at org.fao.geonet.kernel.harvest.harvester.AbstractHarvesterHarvestWithIndexProcessor.process(AbstractHarvester.java:602)atorg.fao.geonet.kernel.harvest.harvester.AbstractHarvester.harvest(AbstractHarvester.java:673)atorg.fao.geonet.kernel.harvest.harvester.HarvesterJob.execute(HarvesterJob.java:69)atorg.quartz.core.JobRunShell.run(JobRunShell.java:202)atorg.quartz.simpl.SimpleThreadPoolWorkerThread.run(SimpleThreadPool.java:573)
2025-07-16T11:13:01,606 INFO  [geonetwork.harvester] - Processing service: http://172.16.132.22:8004/thredds/iso/ for dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09_data_rt.nc
2025-07-16T11:13:01,606 INFO  [geonetwork.harvester] - Crawling through dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09.nc
2025-07-16T11:13:01,607 INFO  [geonetwork.harvester] - Processing dataset: dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09.nc with URL: http://172.16.132.22:8004//thredds/catalog/auv/glider/sl3u1064-scb_sl3u1064/L1/2025/catalog.xml?dataset=auv%2Fglider%2Fsl3u1064-scb_sl3u1064%2FL1%2F2025%2Fdep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09.nc
2025-07-16T11:13:01,663 ERROR [geonetwork.harvester] - Thrown Exception java.lang.NullPointerException during dataset processing
java.lang.NullPointerException
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.extractMetadataFromOpendapDDX(Harvester.java:1364)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.getMetadata(Harvester.java:864)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.examineThreddsDataset(Harvester.java:604)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:462)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.crawlDatasets(Harvester.java:458)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvestCatalog(Harvester.java:411)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvest(Harvester.java:284)
at org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester.doHarvest(ThreddsHarvester.java:76)
at org.fao.geonet.kernel.harvest.harvester.AbstractHarvesterHarvestWithIndexProcessor.process(AbstractHarvester.java:602)atorg.fao.geonet.kernel.harvest.harvester.AbstractHarvester.harvest(AbstractHarvester.java:673)atorg.fao.geonet.kernel.harvest.harvester.HarvesterJob.execute(HarvesterJob.java:69)atorg.quartz.core.JobRunShell.run(JobRunShell.java:202)atorg.quartz.simpl.SimpleThreadPoolWorkerThread.run(SimpleThreadPool.java:573)
2025-07-16T11:13:01,664 INFO  [geonetwork.harvester] - Processing service: http://172.16.132.22:8004/thredds/iso/ for dep0009_sl3u1064_scb-sl3u1064_L1_2025-04-09.nc
2025-07-16T11:13:01,665 INFO  [geonetwork.harvester] - Adding dataset metadata...
2025-07-16T11:13:01,665 WARN  [geonetwork.harvester] - Raised exception while harvesting from : gn422_threddsmiguel (ThreddsHarvester)
2025-07-16T11:13:01,665 WARN  [geonetwork.harvester] -  (C) Class    : NullPointerException
2025-07-16T11:13:01,666 WARN  [geonetwork.harvester] -  (C) Message : null
2025-07-16T11:13:01,666 ERROR [geonetwork.harvester] - null
java.lang.NullPointerException: null
at org.fao.geonet.utils.Xml.transform(Xml.java:493) ~[gn-common-4.2.2-0.jar:?]
at org.fao.geonet.utils.Xml.transform(Xml.java:400) ~[gn-common-4.2.2-0.jar:?]
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.createDatasetMetadata(Harvester.java:1545) ~[gn-harvesters-4.2.2-0.jar:?]
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvestCatalog(Harvester.java:420) ~[gn-harvesters-4.2.2-0.jar:?]
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvest(Harvester.java:284) ~[gn-harvesters-4.2.2-0.jar:?]
at org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester.doHarvest(ThreddsHarvester.java:76) ~[gn-harvesters-4.2.2-0.jar:?]
at org.fao.geonet.kernel.harvest.harvester.AbstractHarvesterHarvestWithIndexProcessor.process(AbstractHarvester.java:602) 
[˜gn−harvesters−4.2.2−0.jar:?]atorg.fao.geonet.kernel.harvest.harvester.AbstractHarvester.harvest(AbstractHarvester.java:673) 
[˜gn−harvesters−4.2.2−0.jar:?]atorg.fao.geonet.kernel.harvest.harvester.HarvesterJob.execute(HarvesterJob.java:69) 
[˜quartz−2.3.2.jar:?]atorg.quartz.core.JobRunShell.run(JobRunShell.java:202) 
[˜ quartz−2.3.2.jar:?]atorg.quartz.simpl.SimpleThreadPoolWorkerThread.run(SimpleThreadPool.java:573) ~[quartz-2.3.2.jar:?]

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions