Incremental builds - MSBuild target is skipped on deleted input

[zijchen]

Issue Description

Whether a MSBuild target is skipped or not seems to be determined by the timestamps of its inputs, but in the case when an input is deleted (while others remain the same), the target is still skipped regardless.

Steps to Reproduce

Repro here: https://github.com/zijchen/TargetTest

I created a task to zip a folder to output, initial build creates the .zip file, but if I remove any file from the SourceDirectory, the target is skipped even though the .zip file has an outdated set of files.

  <Target Name="ZipOutputPath" AfterTargets="Build" Inputs="@(ZipFiles)" Outputs="$(OutputPath)\output.zip">
    <Message Importance="high" Text="Zipping files @(ZipFiles)" />
    <ZipDirectory
      SourceDirectory="$(MSBuildThisFileDirectory)\Zip"
      DestinationFile="$(OutputPath)\output.zip"
      Overwrite="true" />
  </Target>

Expected Behavior

MSBuild target to be run when input is deleted.

Actual Behavior

MSBuild target is skipped.

Analysis

No response

Versions & Configurations

No response

[MichaeIDietrich]

The issue comes from the fact that you gather the input files using a globbing pattern. So when a file is deleted it will simply not be listed as input file and therefore it cannot affect timestamp comparison. Msbuild simply does not know from its own there was another file in the run before.

The sdk style project also uses globbing to gather .cs files and has the same limitation.
They worked around this by using a cache file that contains a hash over the previously used files so they can detect whether a file was removed or added by the globbing on the next run.

Have a look at the _GenerateCompileDependencyCache target:

msbuild/src/Tasks/Microsoft.Common.CurrentVersion.targets

Lines 3861 to 3884 in f51a5b9

	<Target Name="_GenerateCompileDependencyCache" Condition="'$(DesignTimeBuild)' != 'true' and '$(BuildingProject)' == 'true'" DependsOnTargets="ResolveAssemblyReferences;_GenerateCompileInputs">
	<ItemGroup>
	<CustomAdditionalCompileInputs Include="$(IntermediateOutputPath)$(MSBuildProjectFile).CoreCompileInputs.cache" />
	<CoreCompileCache Include="@(Compile)" />
	<CoreCompileCache Include="@(ReferencePath)" />
	<CoreCompileCache Include="$(DefineConstants)" />
	<CoreCompileCache Include="$(LangVersion)" />
	<CoreCompileCache Include="$(Deterministic)" />
	<CoreCompileCache Include="$(PathMap)" />
	<CoreCompileCache Include="@(_CoreCompileResourceInputs)"/>
	</ItemGroup>
	<Hash
	ItemsToHash="@(CoreCompileCache)"
	IgnoreCase="$([MSBuild]::ValueOrDefault(`$(CoreCompileCacheIgnoreCase)`, `true`))">
	<Output TaskParameter="HashResult" PropertyName="CoreCompileDependencyHash" />
	</Hash>
	<WriteLinesToFile Lines="$(CoreCompileDependencyHash)" File="$(IntermediateOutputPath)$(MSBuildProjectFile).CoreCompileInputs.cache" Overwrite="true" WriteOnlyWhenDifferent="true" />
	<ItemGroup>
	<FileWrites Include="$(IntermediateOutputPath)$(MSBuildProjectFile).CoreCompileInputs.cache" />
	</ItemGroup>
	</Target>

So you will need to do a little bit more work to detect this correctly, I hope this helps.

[AR-May]

Team Triage: This behavior is by design and concerns a known limitation of the MSBuild incrementality model. MSBuild determines whether to skip a target by comparing the timestamps of all inputs against the outputs. If all inputs are older than the outputs, the target is skipped. However, if an input is removed since the previous run, this check still passes, and the target is skipped.

The recommended workaround, as mentioned above, is to maintain a list or hash of the previously used input files and include that as an additional input. This ensures the target is executed when some input is deleted.

We should improve our documentation and explicitly mention that such behavior can happen and above is the recommended pattern for resolving the issue.