Subtemplates fixes #8893

sebr72 · 2025-06-25T14:38:28Z

This PR fixes the following issues:

Ensure the service context is not 'corrupted', to avoid loss of ui-language while processing subtemplates
On subtemplate's update, trigger indexation for every md referencing it
Perform validation on MD with substituted subtemplates (xsd and schematron)

Checklist

sebr72 · 2025-07-02T12:48:55Z

core/src/main/resources/config-spring-geonetwork.xml

@@ -55,7 +55,7 @@
        lazy-init="true"/>

  <bean id="ThesaurusManager" class="org.fao.geonet.kernel.ThesaurusManager" lazy-init="true">
-    <property name="thesaurusCacheMaxSize" value="\${thesaurus.cache.maxsize}"/>
+    <property name="thesaurusCacheMaxSize" value="${thesaurus.cache.maxsize}"/>


@fxprunayre Hello. We couldn't work out the meaning of the "\" you introduced. We had to remove it to make our tests pass. Could you please have a look and confirm this is OK with you ?

Not sure but all properties loaded from config.properties (cf. https://github.com/geonetwork/core-geonetwork/blob/main/web/src/main/webResources/WEB-INF/config.properties#L1) are escaped (eg. https://github.com/geonetwork/core-geonetwork/blob/main/web/src/main/webResources/WEB-INF/config-spring-geonetwork.xml#L76-L77) if we don't want them to be filtered by maven filtered resource ?
Can be that thesaurus.cache.maxsize is missing somewhere when running test?

Could be simplier for first approach to keep the escaping \: with escaping \ test runs green from bash with mvn but fails with ide:
Error creating bean with name 'org.fao.geonet.kernel.datamanager.base.BaseMetadataManager#0': Unsatisfied dependency expressed through field 'thesaurusManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ThesaurusManager' defined in URL [file:/home/cmangeat/sources/core-geonetwork/core/target/classes/config-spring-geonetwork.xml]: Initialization of bean failed; nested exception is org.springframework.beans.TypeMismatchException: Failed to convert property value of type 'java.lang.String' to required type 'int' for property 'thesaurusCacheMaxSize'; nested exception is java.lang.NumberFormatException: For input string: "\400000" ...

@fxprunayre thank you for the explaination.

@fxprunayre @cmangeat Indeed. Thanks for the explanations :-)

Locally, I also get bad substitution message like NumberFormatException: For input string: "\400000" ... in Intellij from time to time but if building from command line, then it is solved (Intellij probably refresh some files). Not sure if it is an Intellij build issue or mis configuration

@fxprunayre @cmangeat @josegar74 Based on your feedback, I reverted this change. I don't mark it as resolved to let you confirm

josegar74 · 2025-07-02T15:42:36Z

common/src/main/java/org/fao/geonet/exceptions/GNException.java

@@ -0,0 +1,34 @@
+//=============================================================================
+//===	Copyright (C) 2001-2023 Food and Agriculture Organization of the


Suggested change

//=== Copyright (C) 2001-2023 Food and Agriculture Organization of the

//=== Copyright (C) 2001-2025 Food and Agriculture Organization of the

@josegar74 Done. I don't mark it as resolved to let you confirm.

josegar74 · 2025-07-02T15:43:25Z

core/src/test/java/org/fao/geonet/kernel/LocalXLinksUpdateDeleteTest.java

@@ -1,9 +1,15 @@
 package org.fao.geonet.kernel;


Please add the file header.

@josegar74 Done. I don't mark it as resolved to let you confirm.

josegar74 · 2025-07-02T15:44:43Z

services/src/test/java/org/fao/geonet/api/processing/ValidateApiTest.java

@@ -0,0 +1,114 @@
+package org.fao.geonet.api.processing;


Please add file header.

@josegar74 Done. I don't mark it as resolved to let you confirm.

josegar74 · 2025-07-02T15:48:24Z

core/src/main/java/org/fao/geonet/kernel/datamanager/base/BaseMetadataManager.java

+        List<Hit> hits = (List<Hit>) response.hits().hits();
+        hits.stream().map(Hit::id).forEach(consumerUuid -> {
+            try {
+                String consumerId = this.metadataUtils.getMetadataId(consumerUuid);


Have you check to retrieve the metadata id from the index? That should avoid querying the database.

@josegar74 Indeed it was possible. Thanks. I don't mark it as resolved to let you confirm.

josegar74 · 2025-07-02T15:51:50Z

core/src/main/java/org/fao/geonet/kernel/SpringLocalServiceInvoker.java

+            if (srvContext != null) {
+                srvContext.setAsThreadLocal();
+            }


Can you explain this code? It can be good to add some comment to clarify it, thanks.

@josegar74 Done. I don't mark it as resolved to let you confirm.

josegar74 · 2025-07-02T15:53:12Z

core/src/main/java/org/fao/geonet/kernel/datamanager/base/BaseMetadataManager.java

+        String query = String.format("xlink:*%s*", subTemplate.getUuid());
+        SearchResponse response = this.searchManager.query(query, null, 0, maxMdsReferencingSubTemplate);
+        if (response.hits().total().value() > maxMdsReferencingSubTemplate) {
+            throw new GNException("Not implemented");


It can be relevant to log the details and the exception message can be more clear than Not implemented.

@josegar74 Done. I don't mark it as resolved to let you confirm.

josegar74 · 2025-07-02T15:55:16Z

common/src/main/java/org/fao/geonet/exceptions/GNException.java

+//==============================================================================
+package org.fao.geonet.exceptions;
+
+public class GNException extends RuntimeException {


This exception seems only used when the number of metadata referencing the subtemplates exceeds a number. Maybe can be given a clear name.

@josegar74 Done. I don't mark it as resolved to let you confirm.

…nguage while processing subtemplates

rely on metadataIndexer.batchIndexInThreadPool for async indexing test threadPoolIndexation starts for expected md throw exception when updating a subtemplate too resource hungry (too many md to index, i.e. more than 10000)

…atron)

sebr72 force-pushed the subtemplates-fixes branch from 8f6a389 to d85b6ce Compare June 25, 2025 16:07

cmangeat force-pushed the subtemplates-fixes branch 4 times, most recently from 4577f96 to 0307b20 Compare July 2, 2025 11:44

sebr72 force-pushed the subtemplates-fixes branch from 0307b20 to 398c814 Compare July 2, 2025 11:53

sebr72 marked this pull request as ready for review July 2, 2025 12:46

sebr72 commented Jul 2, 2025

View reviewed changes

josegar74 reviewed Jul 2, 2025

View reviewed changes

sebr72 force-pushed the subtemplates-fixes branch 4 times, most recently from f230412 to 438a82b Compare July 7, 2025 11:59

sebr72 added 4 commits July 7, 2025 14:03

Ensure the service context is not 'corrupted', to avoid loss of ui-la…

0bd6db4

…nguage while processing subtemplates

Perform validation on MD with substituted subtemplates (xsd and schem…

ae12824

…atron)

Re-enable test (stop hiding failures inside try catch)

2b3c10a

sebr72 force-pushed the subtemplates-fixes branch from 438a82b to 2b3c10a Compare July 7, 2025 12:03

		@@ -0,0 +1,34 @@
		//=============================================================================
		//=== Copyright (C) 2001-2023 Food and Agriculture Organization of the

	//=== Copyright (C) 2001-2023 Food and Agriculture Organization of the
	//=== Copyright (C) 2001-2025 Food and Agriculture Organization of the

Uh oh!

Subtemplates fixes #8893

Are you sure you want to change the base?

Subtemplates fixes #8893

Uh oh!

Conversation

sebr72 commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Uh oh!

sebr72 Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cmangeat Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sebr72 Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sebr72 Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sebr72 Jul 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sebr72 commented Jun 25, 2025 •

edited

Loading

sebr72 Jul 2, 2025 •

edited

Loading

cmangeat Jul 2, 2025 •

edited

Loading

sebr72 Jul 7, 2025 •

edited

Loading

sebr72 Jul 7, 2025 •

edited

Loading

sebr72 Jul 7, 2025 •

edited

Loading