Add .NET 10 target, #1219

[paulirwin]

• You've read the Contributor Guide and Code of Conduct.
• You've included unit or integration tests for your change, where applicable.
• You've included inline docs for your change, where applicable.
• There's an open issue for the PR that you are making. If you'd like to propose a change, please open an issue to discuss the change or find an existing issue.

Add .NET 10 target

Fixes #1219

[paulirwin]

Note that this PR reverts the hardcoded .NET 9 SDK since a new version is out that resolves that issue.

[NightOwl888]

Looking better. However, I found some things to address.

The analyzer is now warning about some extension methods in StreamExtensions and it is correct that these are not as robust as they should be, since they could return fewer bytes than requested. See: https://chatgpt.com/share/6954024f-aab8-8005-a3e9-66d3acf90968. It would probably be best to put this into a separate PR because the only thing about net10.0 this relates to is the more aggressive analyzer.

Severity	Code	Description	Project	File	Line	Suppression State	Details
Warning (active)	CA2022	A call to 'Stream.Read' may return fewer bytes than requested, resulting in unreliable code if the return value is not checked.	Lucene.Net (net10.0)	F:\Projects\lucenenet\src\Lucene.Net\Support\IO\StreamExtensions.cs	210		
Warning (active)	CA2022	A call to 'Stream.Read' may return fewer bytes than requested, resulting in unreliable code if the return value is not checked.	Lucene.Net (net10.0)	F:\Projects\lucenenet\src\Lucene.Net\Support\IO\StreamExtensions.cs	235		
Warning (active)	CA2022	A call to 'Stream.Read' may return fewer bytes than requested, resulting in unreliable code if the return value is not checked.	Lucene.Net (net10.0)	F:\Projects\lucenenet\src\Lucene.Net\Support\IO\StreamExtensions.cs	263		

There are also some warnings on the Replicator tests about WebHostBuilder being deprecated. It would probably be best to put this into a separate PR, also.

Severity	Code	Description	Project	File	Line	Suppression State	Details
Warning (active)	ASPDEPR004	'WebHostBuilder' is obsolete: 'WebHostBuilder is deprecated in favor of HostBuilder and WebApplicationBuilder. For more information, visit https://aka.ms/aspnet/deprecate/004.'	Lucene.Net.Tests.Replicator	F:\Projects\lucenenet\src\Lucene.Net.Tests.Replicator\ReplicatorTestCase.cs	110		
Warning (active)	ASPDEPR004	'WebHostBuilder' is obsolete: 'WebHostBuilder is deprecated in favor of HostBuilder and WebApplicationBuilder. For more information, visit https://aka.ms/aspnet/deprecate/004.'	Lucene.Net.Tests.Replicator	F:\Projects\lucenenet\src\Lucene.Net.Tests.Replicator\ReplicatorTestCase.cs	88		

I ran the tests and there was a new failure on net10.0 that appears to be related to thread safety. See: https://dev.azure.com/shad0962/Experiments/_build/results?buildId=2704&view=ms.vss-test-web.build-test-results-tab&runId=1066178&resultId=100155&paneView=debug.

Error message
Lucene.Net.Util.LuceneSystemException : i.cfs
Data:
OriginalMessage: Lucene.Net.Util.LuceneSystemException: i.cfs
---> System.IO.DirectoryNotFoundException: i.cfs
at Lucene.Net.Store.CompoundFileDirectory..ctor(Directory directory, String fileName, IOContext context, Boolean openForWrite) in //src/Lucene.Net/Store/CompoundFileDirectory.cs:line 107
at Lucene.Net.Index.SegmentReader.ReadFieldInfos(SegmentCommitInfo info) in //src/Lucene.Net/Index/SegmentReader.cs:line 224
at Lucene.Net.Index.SegmentReader..ctor(SegmentCommitInfo si, Int32 termInfosIndexDivisor, IOContext context) in //src/Lucene.Net/Index/SegmentReader.cs:line 89
at Lucene.Net.Index.StandardDirectoryReader.Open(Directory directory, SegmentInfos infos, IList1 oldReaders, Int32 termInfosIndexDivisor) in /_/src/Lucene.Net/Index/StandardDirectoryReader.cs:line 213 --- End of stack trace from previous location ---    at Lucene.Net.Util.IOUtils.ReThrow(Exception th) in /_/src/Lucene.Net/Util/IOUtils.cs:line 642    at Lucene.Net.Index.StandardDirectoryReader.Open(Directory directory, SegmentInfos infos, IList1 oldReaders, Int32 termInfosIndexDivisor) in //src/Lucene.Net/Index/StandardDirectoryReader.cs:line 289
at Lucene.Net.Index.StandardDirectoryReader.DoOpenFromCommit(IndexCommit commit) in //src/Lucene.Net/Index/StandardDirectoryReader.cs:line 402
at Lucene.Net.Index.StandardDirectoryReader.DoOpenNoWriter(IndexCommit commit) in //src/Lucene.Net/Index/StandardDirectoryReader.cs:line 397
at Lucene.Net.Index.StandardDirectoryReader.DoOpenIfChanged(IndexCommit commit) in //src/Lucene.Net/Index/StandardDirectoryReader.cs:line 335
at Lucene.Net.Index.StandardDirectoryReader.DoOpenIfChanged() in //src/Lucene.Net/Index/StandardDirectoryReader.cs:line 320
at Lucene.Net.Index.DirectoryReader.OpenIfChanged(DirectoryReader oldReader) in //src/Lucene.Net/Index/DirectoryReader.cs:line 174
at Lucene.Net.Index.TestIndexWriterCommit.ThreadAnonymousClass.Run() in //src/Lucene.Net.Tests/Index/TestIndexWriterCommit.cs:line 423
--- End of inner exception stack trace ---
at Lucene.Net.Index.TestIndexWriterCommit.ThreadAnonymousClass.Run() in //src/Lucene.Net.Tests/Index/TestIndexWriterCommit.cs:line 436
at J2N.Threading.ThreadJob.SafeRun(ThreadStart start)
----> System.IO.DirectoryNotFoundException : _i.cfs
(Test: Lucene.Net.Index.TestIndexWriterCommit.TestCommitThreadSafety)


To reproduce this test result:


Option 1:


Apply the following assembly-level attributes:


[assembly: Lucene.Net.Util.RandomSeed("0x22d417d4bdf30090:0x1bc763c1beb7863c")]
[assembly: NUnit.Framework.SetCulture("it")]


Option 2:


Use the following .runsettings file:


<RunSettings>
  <TestRunParameters>
    <Parameter name="tests:seed" value="0x22d417d4bdf30090:0x1bc763c1beb7863c" />
    <Parameter name="tests:culture" value="it" />
  </TestRunParameters>
</RunSettings>
Option 3:


Create the following lucene.testsettings.json file somewhere between the test assembly and the root of your drive:


{
"tests": {
"seed": "0x22d417d4bdf30090:0x1bc763c1beb7863c",
"culture": "it"
}
}


Fixture Test Values

Random Seed:           0x22d417d4bdf30090:0x1bc763c1beb7863c
Culture:               it
Time Zone:             (UTC+02:00) Jerusalem
Default Codec:         Lucene3x (PreFlexRWCodec)
Default Similarity:    DefaultSimilarity


System Properties

Nightly:               False
Weekly:                False
Slow:                  True
Awaits Fix:            False
Directory:             random
Verbose:               False
Random Multiplier:     1



Stack trace
[at Lucene.Net.Index.TestIndexWriterCommit.ThreadAnonymousClass.Run() in /_/src/Lucene.Net.Tests/Index/TestIndexWriterCommit.cs:line 436](https://dev.azure.com/shad0962/Experiments/_git/4882ad91-3a7d-453c-bd92-8068ed8f2711?path=%2F_%2Fsrc%2FLucene.Net.Tests%2FIndex%2FTestIndexWriterCommit.cs&version=GBissue%2F1219&_a=contents&line=436&lineEnd=437&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
at J2N.Threading.ThreadJob.SafeRun(ThreadStart start)
--- End of stack trace from previous location ---
at J2N.Threading.ThreadJob.RethrowFirstException()
at J2N.Threading.ThreadJob.Join()
[at Lucene.Net.Index.TestIndexWriterCommit.TestCommitThreadSafety() in /_/src/Lucene.Net.Tests/Index/TestIndexWriterCommit.cs:line 378](https://dev.azure.com/shad0962/Experiments/_git/4882ad91-3a7d-453c-bd92-8068ed8f2711?path=%2F_%2Fsrc%2FLucene.Net.Tests%2FIndex%2FTestIndexWriterCommit.cs&version=GBissue%2F1219&_a=contents&line=378&lineEnd=379&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
at System.RuntimeMethodHandle.InvokeMethod(ObjectHandleOnStack target, Void** arguments, ObjectHandleOnStack sig, BOOL isConstructor, ObjectHandleOnStack result)
at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
--DirectoryNotFoundException
[at Lucene.Net.Store.CompoundFileDirectory..ctor(Directory directory, String fileName, IOContext context, Boolean openForWrite) in /_/src/Lucene.Net/Store/CompoundFileDirectory.cs:line 107](https://dev.azure.com/shad0962/Experiments/_git/4882ad91-3a7d-453c-bd92-8068ed8f2711?path=%2F_%2Fsrc%2FLucene.Net%2FStore%2FCompoundFileDirectory.cs&version=GBissue%2F1219&_a=contents&line=107&lineEnd=108&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
[at Lucene.Net.Index.SegmentReader.ReadFieldInfos(SegmentCommitInfo info) in /_/src/Lucene.Net/Index/SegmentReader.cs:line 224](https://dev.azure.com/shad0962/Experiments/_git/4882ad91-3a7d-453c-bd92-8068ed8f2711?path=%2F_%2Fsrc%2FLucene.Net%2FIndex%2FSegmentReader.cs&version=GBissue%2F1219&_a=contents&line=224&lineEnd=225&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
[at Lucene.Net.Index.SegmentReader..ctor(SegmentCommitInfo si, Int32 termInfosIndexDivisor, IOContext context) in /_/src/Lucene.Net/Index/SegmentReader.cs:line 89](https://dev.azure.com/shad0962/Experiments/_git/4882ad91-3a7d-453c-bd92-8068ed8f2711?path=%2F_%2Fsrc%2FLucene.Net%2FIndex%2FSegmentReader.cs&version=GBissue%2F1219&_a=contents&line=89&lineEnd=90&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
[at Lucene.Net.Index.StandardDirectoryReader.Open(Directory directory, SegmentInfos infos, IList`1 oldReaders, Int32 termInfosIndexDivisor) in /_/src/Lucene.Net/Index/StandardDirectoryReader.cs:line 213](https://dev.azure.com/shad0962/Experiments/_git/4882ad91-3a7d-453c-bd92-8068ed8f2711?path=%2F_%2Fsrc%2FLucene.Net%2FIndex%2FStandardDirectoryReader.cs&version=GBissue%2F1219&_a=contents&line=213&lineEnd=214&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
--- End of stack trace from previous location ---
[at Lucene.Net.Util.IOUtils.ReThrow(Exception th) in /_/src/Lucene.Net/Util/IOUtils.cs:line 642](https://dev.azure.com/shad0962/Experiments/_git/4882ad91-3a7d-453c-bd92-8068ed8f2711?path=%2F_%2Fsrc%2FLucene.Net%2FUtil%2FIOUtils.cs&version=GBissue%2F1219&_a=contents&line=642&lineEnd=643&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
[at Lucene.Net.Index.StandardDirectoryReader.Open(Directory directory, SegmentInfos infos, IList`1 oldReaders, Int32 termInfosIndexDivisor) in /_/src/Lucene.Net/Index/StandardDirectoryReader.cs:line 289](https://dev.azure.com/shad0962/Experiments/_git/4882ad91-3a7d-453c-bd92-8068ed8f2711?path=%2F_%2Fsrc%2FLucene.Net%2FIndex%2FStandardDirectoryReader.cs&version=GBissue%2F1219&_a=contents&line=289&lineEnd=290&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
[at Lucene.Net.Index.StandardDirectoryReader.DoOpenFromCommit(IndexCommit commit) in /_/src/Lucene.Net/Index/StandardDirectoryReader.cs:line 402](https://dev.azure.com/shad0962/Experiments/_git/4882ad91-3a7d-453c-bd92-8068ed8f2711?path=%2F_%2Fsrc%2FLucene.Net%2FIndex%2FStandardDirectoryReader.cs&version=GBissue%2F1219&_a=contents&line=402&lineEnd=403&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
[at Lucene.Net.Index.StandardDirectoryReader.DoOpenNoWriter(IndexCommit commit) in /_/src/Lucene.Net/Index/StandardDirectoryReader.cs:line 397](https://dev.azure.com/shad0962/Experiments/_git/4882ad91-3a7d-453c-bd92-8068ed8f2711?path=%2F_%2Fsrc%2FLucene.Net%2FIndex%2FStandardDirectoryReader.cs&version=GBissue%2F1219&_a=contents&line=397&lineEnd=398&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
[at Lucene.Net.Index.StandardDirectoryReader.DoOpenIfChanged(IndexCommit commit) in /_/src/Lucene.Net/Index/StandardDirectoryReader.cs:line 335](https://dev.azure.com/shad0962/Experiments/_git/4882ad91-3a7d-453c-bd92-8068ed8f2711?path=%2F_%2Fsrc%2FLucene.Net%2FIndex%2FStandardDirectoryReader.cs&version=GBissue%2F1219&_a=contents&line=335&lineEnd=336&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
[at Lucene.Net.Index.StandardDirectoryReader.DoOpenIfChanged() in /_/src/Lucene.Net/Index/StandardDirectoryReader.cs:line 320](https://dev.azure.com/shad0962/Experiments/_git/4882ad91-3a7d-453c-bd92-8068ed8f2711?path=%2F_%2Fsrc%2FLucene.Net%2FIndex%2FStandardDirectoryReader.cs&version=GBissue%2F1219&_a=contents&line=320&lineEnd=321&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
[at Lucene.Net.Index.DirectoryReader.OpenIfChanged(DirectoryReader oldReader) in /_/src/Lucene.Net/Index/DirectoryReader.cs:line 174](https://dev.azure.com/shad0962/Experiments/_git/4882ad91-3a7d-453c-bd92-8068ed8f2711?path=%2F_%2Fsrc%2FLucene.Net%2FIndex%2FDirectoryReader.cs&version=GBissue%2F1219&_a=contents&line=174&lineEnd=175&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
[at Lucene.Net.Index.TestIndexWriterCommit.ThreadAnonymousClass.Run() in /_/src/Lucene.Net.Tests/Index/TestIndexWriterCommit.cs:line 423](https://dev.azure.com/shad0962/Experiments/_git/4882ad91-3a7d-453c-bd92-8068ed8f2711?path=%2F_%2Fsrc%2FLucene.Net.Tests%2FIndex%2FTestIndexWriterCommit.cs&version=GBissue%2F1219&_a=contents&line=423&lineEnd=424&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)

[NightOwl888]

I have made all of the updates to support Visual Studio 2022 (limited support for contributors, excluding net10.0, netstandard2.0, and all .NET Framework versions from the build when building inside VS 2022).

I also fixed an issue with how the scripts use the PrintTargetFrameworks target. Note that this target is what requires projects to identify themselves with <IsTestProject>true</IsTestProject> and why it is conditional in some cases. This target tells the scripts which target frameworks are supported for a specific project file after all MSBuild conditions are run. However, it was failing because the output of MSBuild changed to include more information. The fix was to add a key identifier for the "field" and use a Regex to parse it rather than relying on a position in the output.

NOTE: This fix will need to be copied to all of our other projects because they all use the same approach to determine which target frameworks to run tests on.

I am currently testing the local build with those changes and will report back if there are any issues with it, but it is limited to a function that has already been tested in the Generate-TestWorkflows.ps1 script, so it is not likely that it will fail.

The two test failures are real issues. One of them is similar to the issue I reported earlier - something having to do with CompoundFileDiretory thread safety on .NET 10 (I saw it happen on macOS, also). The other failure is the ongoing issue we are having with Antlr4BuildTasks, which should be fixed after the new version is released and we can drop the dependency on Antlr4.CodeGenerator as well as the following probing for the .jar file within that package.

  <PropertyGroup>
    <Antlr4CodeGeneratorVersion>$([System.IO.Path]::GetFileName('$(PkgAntlr4_CodeGenerator)'))</Antlr4CodeGeneratorVersion>
    <!-- Required for Antlr4BuildTasks to find the antlr4 .jar file. When missing,
        it causes an error on the SonarCloud workflow. -->
    <Antlr4ToolPath>$(PkgAntlr4_CodeGenerator)\tools\antlr4-csharp-$(Antlr4CodeGeneratorVersion)-complete.jar</Antlr4ToolPath>
  </PropertyGroup>

[paulirwin]

Minor issue with a comment, that I'll fix

[paulirwin]

There already was a separate issue for the Antlr flaky tests, #1228. I've logged one for the directory one, #1233. Those should not hold up this PR. Thanks for contributing the VS2022 support. I hope all contributors can get on VS2026 or Rider making this change obsolete as soon as possible. It will be important to test locally against .NET 10.

[NightOwl888]

The directory issue wasn't a single test, that was just an example on how it can be reproduced. I saw at least 2 tests fail. I suspect this is a systemic concurrency problem that will frequently fail on the net10.0 target, so ideally, we wouldn't merge this until after it is addressed.

Since this will get in the way of normal development, I suggest do one of the following on the master branch:

1. Set the default target framework for tests to net9.0
2. Continually run the tests, record all of the ones that fail, and mark them with the [AwaitsFix] attribute
3. Revert this PR

We can then undo the change once the issue has been addressed.

[paulirwin]

The PR does not need to be reverted. If this is an issue in .NET 10, it would surface to users anyways, so better to get it merged so more people can test it.

I am not convinced this is a systemic concurrency problem. I ran the test locally tens of thousands of times on macOS and Ubuntu under .NET 10 and could not reproduce, even tweaking the parameters like the number of threads and wait time. If it were systemic, it would be reproducible at least once every few thousand tries.

My current working theory is that .NET 10 is so fast that we're now dealing with microsecond race conditions that we weren't before, and as noted in #1233, the test does not test real-world conditions. It also might be true that my machine is too fast for this to surface, and it only occurs in the significantly slower Actions containers. Even if someone were trying to create and destroy IndexWriters without reusing them that fast under extremely heavy small-write concurrent load with a RAMDirectory in the real-world, they probably would have their system resilient enough to retry on such an exception, and the chance of it happening twice is very small.

But, I am in favor of #2 above. We should keep testing for this and mark awaits fix if needed. I created PR #1234 to catch the file not found exception, but if someone can demonstrate that there is an actual real-world concurrency issue here with a different solution, I can abandon that PR.