Test with [SkipOnTargetFramework] and [ConditionalFact] grouped as 'No Traits' in Visual Studio Test Explorer

[weifenluo]

I have the following test:

[SkipOnTargetFramework(TargetFrameworkMonikers.NetFramework)]
[ConditionalFact(typeof(RemoteExecutor), nameof(RemoteExecutor.IsSupported))]
public static void TrimDbEntityType()
{
    ...
}

which uses RemoteExecutor to test against AppContext switch.

The test project is multi-targeting .Net Framework 462 and .Net 8.0.

I have following assembly level traits in AssemblyInfo.cs of the test project:

using Xunit;

#if AOT
[assembly: AssemblyTrait("test.aot", "")]
#else
[assembly: AssemblyTrait("test", "")]
#endif

The dependencies in Directory.Build.props if matters:

  <PropertyGroup Condition="'$(IsTestProject)' == 'true'">
    <IsPackable>false</IsPackable>
    <RestoreAdditionalProjectSources>https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json</RestoreAdditionalProjectSources>
    <xUnitVersion>2.9.2</xUnitVersion>
    <MicrosoftDotNetToolsVersion>9.0.0-beta.24421.7</MicrosoftDotNetToolsVersion>
  </PropertyGroup>

  <ItemGroup Condition="'$(IsTestProject)' == 'true'">
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.11.0" />
    <PackageReference Include="Moq" Version="4.10.0" />
    <PackageReference Include="xunit" Version="$(xUnitVersion)" />
    <PackageReference Include="xunit.extensibility.core" Version="$(xUnitVersion)" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.8.2" />
    <PackageReference Include="coverlet.collector" Version="6.0.2" />
    <PackageReference Include="Microsoft.DotNet.RemoteExecutor" Version="$(MicrosoftDotNetToolsVersion)" />
    <PackageReference Include="Microsoft.DotNet.XUnitExtensions" Version="$(MicrosoftDotNetToolsVersion)" />
  </ItemGroup>

The test runs as expected when targeting .Net 8.0. However when targeting .Net Framework 461, the test is displayed under 'No Traits' in Visual Studio Test Explorer and run as failed because RemoteExecutor is not supported by .Net Framework.

Am I doing anything wrong? I'm expecting the test is completely filtered out of Test Explorer.

[weifenluo]

I ended up with giving up using [SkipOnTargetFramework] and [ConditionalFact], by exclude source code files from .csproj:

1. All tests with RemoteExecutor are in separated source code file *.RemoteExecutor.cs;
2. In .csproj add the following:

  <ItemGroup>
    <Compile Remove="**\*.RemoteExecutor.cs" />
  </ItemGroup>

  <ItemGroup>
    <None Include="**\*.RemoteExecutor.cs" />
  </ItemGroup>

  <ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETCoreApp'">
    <Compile Include="**\*.RemoteExecutor.cs" />
  </ItemGroup>

[ViktorHofer]

Yes, for those attributes to work, you need to filter out "failing" test attributes in your .runsettings file: https://github.com/dotnet/maintenance-packages/blob/5aba9a6630d795ce652afca2d15440b0360c50ca/eng/testing/.runsettings#L13-L14

[ViktorHofer]

See #15195

[MichelZ]

Should this also work with ConditionalFact? I can't seem to be able to filter those out in Test Explorer, where the SkipXXX attributes seem to work