[6.0] Test failure in System.Formats.Asn1 ReadVeryLongOid_WithAppContext: could not load file or assembly in NET48

[carlossanlop]

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=734748
Build error leg or test failing: System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier.ReadVeryLongOid_WithAppContext

Error Message

{
  "ErrorMessage" : "System.IO.FileLoadException: Could not load file or assembly 'System.Formats.Asn1",
  "BuildRetry" : false,
  "ExcludeConsoleLog" : false
}

• PR: Merging internal commits for release/6.0 #104624
• Queue: Build windows x86 Release Libraries_NET48
• Job result: https://dev.azure.com/dnceng-public/public/_build/results?buildId=734748&view=logs&j=53220604-e0d7-54f7-2c45-66449e2152b0&t=a5a9ae28-5fa3-5329-2929-bf8173943473&l=54
• Log file: https://helixre107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-runtime-refs-pull-104624-merge-e34e8dcdb99a41739a/System.Formats.Asn1.Tests/1/console.35a6665f.log?helixlogtype=result
• Output:

C:\h\w\B561097C\w\B5950A12\e>xunit.console.exe System.Formats.Asn1.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing  
  Discovering: System.Formats.Asn1.Tests (app domain = on [no shadow copy], method display = ClassAndMethod, method display options = None)
  Discovered:  System.Formats.Asn1.Tests (found 530 test cases)
  Starting:    System.Formats.Asn1.Tests (parallel test collections = on, max threads = 2)

Unhandled Exception: System.IO.FileLoadException: Could not load file or assembly 'System.Formats.Asn1, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
   at System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier.<>c.<ReadVeryLongOid_WithAppContext>b__9_0()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.DotNet.RemoteExecutor.Program.Main(String[] args)
    System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier.ReadVeryLongOid_WithAppContext [FAIL]
      Microsoft.DotNet.RemoteExecutor.RemoteExecutionException : Remote process failed with an unhandled exception.
      Stack Trace:
        
        Child exception:
          System.IO.FileLoadException: Could not load file or assembly 'System.Formats.Asn1, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
        File name: 'System.Formats.Asn1, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'
           at System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier.<>c.<ReadVeryLongOid_WithAppContext>b__9_0()
        
        WRN: Assembly binding logging is turned OFF.
        To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
        Note: There is some performance penalty associated with assembly bind failure logging.
        To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].
        
        
        Child process:
          System.Formats.Asn1.Tests, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier+<>c Void <ReadVeryLongOid_WithAppContext>b__9_0()
        
        

Unhandled Exception: System.IO.FileLoadException: Could not load file or assembly 'System.Formats.Asn1, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
   at System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier.<>c.<ReadVeryLongOidArc_WithAppContext>b__10_0()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.DotNet.RemoteExecutor.Program.Main(String[] args)
    System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier.ReadVeryLongOidArc_WithAppContext [FAIL]
      Microsoft.DotNet.RemoteExecutor.RemoteExecutionException : Remote process failed with an unhandled exception.
      Stack Trace:
        
        Child exception:
          System.IO.FileLoadException: Could not load file or assembly 'System.Formats.Asn1, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
        File name: 'System.Formats.Asn1, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'
           at System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier.<>c.<ReadVeryLongOidArc_WithAppContext>b__10_0()
        
        WRN: Assembly binding logging is turned OFF.
        To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
        Note: There is some performance penalty associated with assembly bind failure logging.
        To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].
        
        
        Child process:
          System.Formats.Asn1.Tests, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51 System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier+<>c Void <ReadVeryLongOidArc_WithAppContext>b__10_0()
        
        
  Finished:    System.Formats.Asn1.Tests
=== TEST EXECUTION SUMMARY ===
   System.Formats.Asn1.Tests  Total: 3423, Errors: 0, Failed: 2, Skipped: 0, Time: 6.249s

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=734748
Error message validated: [System.IO.FileLoadException: Could not load file or assembly 'System.Formats.Asn1]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 7/9/2024 10:09:27 PM UTC

Report

Build	Definition	Test	Pull Request
863852	dotnet/runtime	System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier.ReadVeryLongOid_WithAppContext	#109620
863860	dotnet/runtime	System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier.ReadVeryLongOid_WithAppContext
863848	dotnet/runtime	System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier.ReadVeryLongOid_WithAppContext
2575696	dotnet-runtime	System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier.ReadVeryLongOid_WithAppContext	#44696
860323	dotnet/runtime	System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier.ReadVeryLongOid_WithAppContext	#109517
860319	dotnet/runtime	System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier.ReadVeryLongOid_WithAppContext	#109516
853522	dotnet/runtime	System.Formats.Asn1.Tests.Reader.ReadObjectIdentifier.ReadVeryLongOid_WithAppContext	#109238

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	7

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-formats-asn1, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

[carlossanlop]

@dotnet/area-system-formats-asn1 I see this happening often in servicing. It's now going to be carried into the August Release.

PR: #104925
Job: https://dev.azure.com/dnceng-public/public/_build/results?buildId=742030&view=logs&j=885df564-30f3-5dad-0bdc-caf444244bdf&t=71aa97be-149e-50ab-5d1c-74d5540a091f
Log: https://helixre107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-runtime-refs-pull-104925-merge-0583664e3f1e4ca5b4/System.Formats.Asn1.Tests/1/console.e3050cac.log?helixlogtype=result

Is this a concerning error?

[adamsitnik]

It seems that it's a new test that has been recently backported to 6.0 in #104624 and 8.0 in #104623 (cc @bartonjs who has most likely authored it)

The failure does not seem to be a product issue, more like a testing problem? The code executed by the remote executor fails to load the assembly, because there is a manifest mismatch.

System.IO.FileLoadException: Could not load file or assembly 'System.Formats.Asn1, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)

@carlossanlop @ericstj what could have caused such an exception?

My only idea for now it to disable it for Full Framework (because it seems to be the only affected configuration?).

[ericstj]

Remote executor needs to be run with the same app.config as the test assembly. Since it's not getting the test assembly's app.config it's missing bindingRedirects.

I see remoteExecutor passes the deps and runtimeconfig for .NETCore processes (solving a similar problem). I bet it's never been plumbed for .NETFramework and folks don't notice except in servicing.

There's no way to pass the config on the commandline for .NETFramework. There might be an environment variable to set, barring that you could copy the config file. cc @ViktorHofer

[ViktorHofer]

We have this code piece which should already cover the app.config scenario on .NET Framework: https://github.com/dotnet/arcade/blob/fc2f7ce8372a55725aab7b48c25bad7327a9769d/src/Microsoft.DotNet.RemoteExecutor/src/build/Microsoft.DotNet.RemoteExecutor.targets#L8-L33

Note that there is no plumbing required on .NET Framework as we directly execute the RemoteExecutor.exe executable and that by default already picks up the .config file with the same name.

[bartonjs]

If it's being noise, I'm fine with turning the test off; and that's the only change I feel empowered to make.

If infra wants to take a stab at making it work, that's cool with me. If not, off it goes.

[ericstj]

Adjusting the milestone since noone will look at 6.0.0. This is still failing regularly in servicing builds.

[jeffhandley]

Moving this out of 9.0.0, but when we do disable the test we should backport that to the servicing branches for 6.0, 8.0, and 9.0.