Adding a suggester in CreatingOrUpdatingIndex hangs

[jrunestone]

When using the CreatingOrUpdatingIndex event to add a suggester to the ExternalIndex index ExamineX tries to create the index as per the logs but then does nothing more, backoffice UI is "loading" indefinitely, no index is created in Azure. In fact, the index (if already existing) is removed but then never re-created.

The only thing I get in the debug logs is seemingly this:
2025-01-20T08:05:33.892896060+01:00 [08:05:33 DBG] Creating new index (ExternalIndex)

I'm using the free tier (both ExamineX and Azure AI Search) if that might have anything to do with it.

To Reproduce
Having this in the event CreatingOrUpdatingIndex: args.AzureSearchIndexDefinition.Suggesters.Add(new SearchSuggester("mysuggester", "nodeName")); Basically what this guide instructs: https://shazwazza.com/post/configuring-a-suggester-with-examinex-and-azure-ai-search/

I can add the suggester via REST directly but the same thing happens when rebuilding in Umbraco.

I tried both with the index already created and with no index created.

Versions

• ExamineX 6.0.10
• Umbraco 13.5.1
• Examine version

[Shazwazza]

Hi @jrunestone,

Thanks for posting this question. The Free tier will not have any limits on customizing the index with event handlers, etc...

When you don't have any customizations, etc... does ExamineX behave the way you expect and index rebuilding, etc... works?

[jrunestone]

It depends, mostly no problems what so ever. I've had the occasional hiccup when the rebuild failed because missing fields but I always assumed it's because I do so much testing. Some times ExamineX doesn't update the contents in the index when I publish changes to a single page.

But most of the time it works fine. This issue is particularly deterministic, it's like an on-off switch. Now it works, now it doesn't. With the event handler there but empty there's no problem either. I'm also using the option mapping to change the default analyzer which works fine.

I'm not getting any error logs in Azure either but I'm not sure the diagnostic setting even captures this stuff..

[Shazwazza]

Some times ExamineX doesn't update the contents in the index when I publish changes to a single page.

This shouldn't occur, but ExamineX just does what Umbraco tells it to since it is just an implementation of the Examine abstractions. If you have more info on this or have steps to reproduce please file a separate issue as this is the first I've heard of this.

I'll try to whip up another example of the suggestor code from the blog post with the versions you are referring to to see if I can replicate the problem.

[jrunestone]

OK I will try to narrow down the case where a publish action doesn't update the index. I understand that it's probably an Umbraco issue. I'm load balanced which has proven to cause some index building weirdness in Umbraco in the past..

I'll also try to create a brand new Umbraco installation and see if I can reproduce the suggester issue there.

[jrunestone]

I have some results here from a vanilla Umbraco 13.5.2 site.

Azure is giving a bad request due to the suggester I'm trying to create for the "nodeName" field has an invalid field not yet in the index. Seems like a catch 22 :) This occurs both when creating a new index and when rebuilding an existing from Umbraco. I have the code for the suggester in the CreatingOrUpdatingIndex event as previously stated.

Maybe there's a better place in the pipeline to put this? A composer/component gives me the same error.

2025-01-28T13:06:20.239782931+01:00 [13:06:20 INF] Rebuilding index 'ExternalIndex'
2025-01-28T13:06:20.240548815+01:00 [13:06:20 INF] Starting async background thread for rebuilding index ExternalIndex.
2025-01-28T13:06:20.283271481+01:00 [13:06:20 DBG] Rebuilding existing index (ExternalIndex)
2025-01-28T13:06:20.658898027+01:00 [13:06:20 ERR] An error occurred creating the index
2025-01-28T13:06:20.658898027+01:00 Azure.RequestFailedException: The
 request is invalid. Details: definition : One or more fields in suggester 'mysuggester' are not defined as fields in the index. Please remove them from the suggester definition. Fields: nodeName.
2025-01-28T13:06:20.658898027+01:00 Status: 400 (Bad Request)
2025-01-28T13:06:20.658898027+01:00 ErrorCode: InvalidRequestParameter
2025-01-28T13:06:20.658898027+01:00 
2025-01-28T13:06:20.658898027+01:00 Content:
2025-01-28T13:06:20.658898027+01:00 {"error":{"code":"InvalidRequestParameter","message":"The request is invalid. Details: definition : One or more fields in suggester 'mysuggester' are not defined as fields in the index. Please remove them from the suggester definition. Fields: nodeName.","details":[{"code":"InvalidSuggesterField","message":"One or more fields in suggester 'mysuggester' are not defined as fields in the index. Please remove them from the suggester definition. Fields: nodeName. Parameters: definition"}]}}
2025-01-28T13:06:20.658898027+01:00 
2025-01-28T13:06:20.658898027+01:00 Headers:
2025-01-28T13:06:20.658898027+01:00 Cache-Control: no-cache,no-store
2025-01-28T13:06:20.658898027+01:00 Pragma: no-cache
2025-01-28T13:06:20.658898027+01:00 Server: Microsoft-IIS/10.0
2025-01-28T13:06:20.658898027+01:00 client-request-id: 0fc50e63-14d6-4d43-a760-2c6e40abf4c6
2025-01-28T13:06:20.658898027+01:00 x-ms-client-request-id: 0fc50e63-14d6-4d43-a760-2c6e40abf4c6
2025-01-28T13:06:20.658898027+01:00 request-id: 0fc50e63-14d6-4d43-a760-2c6e40abf4c6
2025-01-28T13:06:20.658915655+01:00 elapsed-time: 73
2025-01-28T13:06:20.658915655+01:00 Strict-Transport-Security: REDACTED
2025-01-28T13:06:20.658915655+01:00 Date: Tue, 28 Jan 2025 12:06:20 GMT
2025-01-28T13:06:20.658915655+01:00 Content-Length: 490
2025-01-28T13:06:20.658915655+01:00 Content-Type: application/json; charset=utf-8
2025-01-28T13:06:20.658915655+01:00 Content-Language: REDACTED
2025-01-28T13:06:20.658915655+01:00 Expires: -1
2025-01-28T13:06:20.658915655+01:00 
2025-01-28T13:06:20.658915655+01:00    at Azure.Search.Documents.IndexesRestClient.CreateOrUpdate(String indexName, SearchIndex index, Nullable`1 allowIndexDowntime, String ifMatch, String ifNoneMatch, CancellationToken cancellationToken)
2025-01-28T13:06:20.658915655+01:00    at Azure.Search.Documents.Indexes.SearchIndexClient.CreateOrUpdateIndex(SearchIndex index, Boolean allowIndexDowntime, Boolean onlyIfUnchanged, CancellationToken cancellationToken)
2025-01-28T13:06:20.658915655+01:00    at ExamineX.AzureSearch.AzureSearchIndex.A(Boolean, SearchIndex, SearchIndexer, SearchIndex& , SearchIndexer& )

[Shazwazza]

@jrunestone Apologies for the delay here - it is certainly on our radar and can try to get to this tomorrow/monday but FYI I'm away for the remaining of next week in case you don't hear back.

[Shazwazza]

Hi @jrunestone,

I have this working and we even have unit tests for this, but the underlying issue is due to the analyzer used for suggester fields.

By default in Umbraco, it will use a specialized default analyzer which differs based on index:

• InternalIndex = whitespace, lowercase, ascii-folding
• ExternalIndex = lucene standard analyzer

If trying to run this with the Internal index, an error will occur. This will also be logged so I'm not sure why you aren't seeing these logs:

Azure.RequestFailedException
  HResult=0x80131500
  Message=The request is invalid. Details: definition : Field 'Country' in suggester 'sg' uses an unsupported analyzers configuration. Suggesters are only supported with the default analyzer and language analyzers in this version of the API.
Status: 400 (Bad Request)
ErrorCode: InvalidRequestParameter

This is because the suggester fields are not compatible with the default analyzer for the index. I'm not entirely sure which analyzers are fully supported for the Azure Search suggester, but for sure the Standard Lucene analyzer is.

To enable this for an index that doesn't have the Standard Lucene analyzer as the default (i.e. Internal), you can do this:

services.PostConfigure<AzureSearchIndexOptions>("InternalIndex", options =>
{
    // create/assign custom value types factory
    options.IndexValueTypesFactory = new Dictionary<string, IAzureSearchFieldValueTypeFactory>(StringComparer.OrdinalIgnoreCase)
    {
        // Add the standard lucene analyzer as a field type called "standard"
        ["standard"] = new AzureSearchFieldValueTypeFactory(s =>
            new AzureSearchFieldValueType(s, SearchFieldDataType.String, LexicalAnalyzerName.StandardLucene.ToString()))
    };

    // Assign this field type to the fields used in the suggester:
    options.FieldDefinitions.AddOrUpdate(new FieldDefinition("City", "standard"));
    options.FieldDefinitions.AddOrUpdate(new FieldDefinition("Country", "standard"));
});

Alternatively, if you are also using Examine/Lucene in development but ExamineX/AzureSearch for prod, you can just configure the Examine/Lucene index field definitions/fields and ExamineX will automatically map these over.

Let me know if you get this working and then I can close this.

We will start adding these types of full code examples for specific use cases to the docs to make it easier for folks to get up and running.

[jrunestone]

Thank you for the info, however I've already changed the analyzer on the ExternalIndex to be sv.Microsoft which should be supported:

I can't remember whether I ever tried to create the suggester with the Standard Lucene analyzer so I will get around to test that again on monday.