ItemDefinitionGroup metadata referencing well-known metadata is passed to tasks unexpanded when the task runs in a TaskHost #14763

Issue Description

Metadata declared in an <ItemDefinitionGroup> whose value references well-known (built-in) metadata — e.g. <OutputName>%(Filename)</OutputName> — is not expanded eagerly. It is expanded lazily, when the metadata is read, by ProjectItemInstance.TaskItem.GetMetadataEscaped().

That late expansion only happens for in-proc task execution. When a task runs in an out-of-proc TaskHost, the item is serialized through a path that copies metadata in bulk and skips the late expansion, so the task receives the literal string %(Filename).

This fails silently: the task gets a syntactically valid but wrong value, which typically ends up embedded in generated file paths (e.g. ...\%(Filename).cs) instead of raising an error.

Only item-definition metadata is affected. Metadata set directly on the item is unaffected, which makes the failure look inconsistent — within the same item, one metadata value expands and another does not.

Steps to Reproduce

Single file, no external assembly required. -mt is used here only because it routes tasks to out-of-proc TaskHosts.

repro.proj:

<Project>

  <UsingTask TaskName="ShowMeta" TaskFactory="RoslynCodeTaskFactory"
             AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.Core.dll">
    <ParameterGroup>
      <Files ParameterType="Microsoft.Build.Framework.ITaskItem[]" Required="true" />
    </ParameterGroup>
    <Task>
      <Code Type="Fragment" Language="cs"><![CDATA[
        foreach (var item in Files)
        {
            Log.LogMessage(MessageImportance.High,
                "ItemSpec={0}  OutputName='{1}'", item.ItemSpec, item.GetMetadata("OutputName"));
        }
      ]]></Code>
    </Task>
  </UsingTask>

  <ItemDefinitionGroup>
    <Thing>
      <OutputName>%(Filename)</OutputName>
    </Thing>
  </ItemDefinitionGroup>

  <ItemGroup>
    <Thing Include="folder\hello.txt" />
  </ItemGroup>

  <Target Name="Build">
    <ShowMeta Files="@(Thing)" />
  </Target>

</Project>

folder\hello.txt does not need to exist.

msbuild repro.proj          # in-proc  -> correct
msbuild repro.proj -mt      # TaskHost -> wrong

Expected Behavior

Both invocations print:

ItemSpec=folder\hello.txt  OutputName='hello'

Actual Behavior

In-proc is correct, but the TaskHost run prints the unexpanded value:

ItemSpec=folder\hello.txt  OutputName='%(Filename)'

Not specific to -mt

-mt is just a convenient trigger. Any out-of-proc TaskHost reproduces it, including an explicit TaskHostFactory on an ordinary compiled task, with no -mt:

public class ShowMeta : Microsoft.Build.Utilities.Task
{
    [Required] public ITaskItem[] Files { get; set; }

    public override bool Execute()
    {
        foreach (ITaskItem item in Files)
        {
            Log.LogMessage(MessageImportance.High, "OutputName='{0}'", item.GetMetadata("OutputName"));
        }
        return true;
    }
}

<UsingTask TaskName="ShowMeta" AssemblyFile="ShowMeta.dll" TaskFactory="TaskHostFactory" />

Same result: OutputName='%(Filename)'. MSBuildRuntime/MSBuildArchitecture mismatches that force a task host behave the same way.

Analysis

%(...) in item definition metadata is stored unexpanded and expanded on read — src/Build/Instance/ProjectItemInstance.cs, TaskItem.GetMetadataEscaped():

escapedValue = GetItemDefinitionMetadataEscaped(metadataName);

if (escapedValue != null && Expander<ProjectProperty, ProjectItem>.ExpressionMayContainExpandableExpressions(escapedValue))
{
    Expander<ProjectPropertyInstance, ProjectItemInstance> expander = new Expander<...>(null, null, new BuiltInMetadataTable(null, this), FileSystems.Default);
    return expander.ExpandIntoStringLeaveEscaped(escapedValue, ExpanderOptions.ExpandBuiltInMetadata, ElementLocation.EmptyLocation);
}

When a task runs out-of-proc, items are converted by TaskParameter.TaskParameterTaskItem, whose constructor copies metadata in bulk (src/Shared/TaskParameter.cs:588):

IDictionary nonGenericEscapedMetadata = copyFromAsITaskItem2.CloneCustomMetadataEscaped();

ProjectItemInstance.TaskItem.CloneCustomMetadataEscaped() (and CloneCustomMetadata()) enumerate MetadataCollection directly, which yields the raw, still-unexpanded item-definition values:

foreach (KeyValuePair<string, string> metadatum in MetadataCollection)
{
    clonedMetadata[metadatum.Key] = metadatum.Value;   // %(Filename) stays literal
}

The literal value is then stored as direct metadata on the deserialized item, so the late expansion can never happen again downstream.

By contrast, CopyMetadataTo → BulkImportMetadata already handles this correctly — it detects potentially expandable item-definition values and re-reads each one through GetMetadataEscaped():

// If we have item definitions with potential expandable expressions, fully evaluate all entries.
if (HasAnyExpandableExpressions())
{
    metadataToImport = metadataToImport
        .Select(metadatum => new KeyValuePair<string, string>(metadatum.Key, GetMetadataEscaped(metadatum.Key)));
}

So the fix looks like giving CloneCustomMetadataEscaped() / CloneCustomMetadata() the same treatment BulkImportMetadata already has. Other consumers that enumerate MetadataCollection raw (e.g. EnumerateMetadata()) may be worth auditing for the same problem.

Versions & Configurations

Reproduced on Windows (.NET Framework MSBuild):

This does not appear to be a recent regression — it looks long-standing in the out-of-proc path. It has become much easier to hit with -mt, which routes essentially every task not marked [MSBuildMultiThreadableTask] to a TaskHost, so builds that previously ran such tasks in-proc now silently get literal %(...) values.