Add HostedToolSearchTool with DeferredTools/NonDeferredTools for tool search / deferred loading support

[Copilot]

Implements tool search and deferred loading support (issue #7371): a HostedToolSearchTool marker with DeferredTools/NonDeferredTools collections that control which tools get deferred loading. By default, all tools are deferred when a HostedToolSearchTool is present. OpenAI Responses API support included; Anthropic follows separately.

New abstractions (Microsoft.Extensions.AI.Abstractions)

• HostedToolSearchTool — marker AITool (same pattern as HostedWebSearchTool/HostedCodeInterpreterTool); maps to the tool_search hosted tool. Includes DeferredTools and NonDeferredTools properties (IList<string>?) to control per-tool deferred loading:
  • Both null (default) → all tools get deferred loading
  • DeferredTools non-null, NonDeferredTools null → only listed tools get deferred loading
  • DeferredTools null, NonDeferredTools non-null → all tools except listed ones get deferred loading
  • Both non-null → tools in DeferredTools get deferred loading unless also in NonDeferredTools
• DiagnosticIds.Experiments.AIToolSearch — new constant (maps to existing MEAI001)

HostedToolSearchTool is [Experimental(AIToolSearch)].

OpenAI provider (Microsoft.Extensions.AI.OpenAI)

• HostedToolSearchTool → deserialized ResponseTool from {"type":"tool_search"} via AOT-safe ModelReaderWriter.Read with OpenAIContext.Default, cached in a static field
• Deferred loading: provider finds HostedToolSearchTool in ChatOptions.Tools and patches defer_loading: true onto matching FunctionTool instances based on the DeferredTools/NonDeferredTools logic
• AsOpenAIResponseTool extension now accepts optional ChatOptions? parameter for defer_loading context

Tests

• Unit tests (OpenAIResponseClientTests.cs): Six VerbatimHttpHandler-based tests validating the exact JSON request body sent for various HostedToolSearchTool configurations: tool_search only, all tools deferred, specific deferred tools, non-deferred exclusion, both lists with precedence, and mixed with other hosted tools (web search)
• Conversion tests (OpenAIConversionTests.cs): Tests for AsOpenAIResponseTool extension covering tool_search conversion, caching, defer_loading with all combinations of DeferredTools/NonDeferredTools
• Abstractions tests (HostedToolSearchToolTests.cs): Tests for HostedToolSearchTool properties and defaults
• Integration test (OpenAIResponseClientIntegrationTests.cs): UseToolSearch_WithDeferredFunctions test exercising HostedToolSearchTool with real function tools against the OpenAI API

Usage

// All tools deferred (default — just add HostedToolSearchTool)
var options = new ChatOptions
{
    Tools = [new HostedToolSearchTool(), getWeather, getForecast, getNews]
};

// Only specific tools deferred
var options = new ChatOptions
{
    Tools = [new HostedToolSearchTool { DeferredTools = ["getWeather", "getForecast"] }, getWeather, getForecast, getNews]
};

// All tools deferred except specific ones
var options = new ChatOptions
{
    Tools = [new HostedToolSearchTool { NonDeferredTools = ["importantTool"] }, importantTool, getWeather, getForecast]
};

var response = await chatClient.GetResponseAsync(messages, options);

Original prompt

Problem

Implement tool search and deferred loading support as described in #7371. Both OpenAI and Anthropic now support tool search, where tool definitions can be sent with deferred loading (only name/description sent upfront, full schema deferred) and a special tool_search hosted tool is included that the model can invoke to search for and load full tool definitions on demand.

Design

Follow Option A from the issue discussion — a HostedToolSearchTool marker tool + a SearchableAIFunctionDeclaration decorator, consistent with existing patterns (HostedWebSearchTool, ApprovalRequiredAIFunction, etc.).

Requirements

1. New types in Microsoft.Extensions.AI.Abstractions

HostedToolSearchTool (in src/Libraries/Microsoft.Extensions.AI.Abstractions/Tools/HostedToolSearchTool.cs)

• A new AITool subclass, following the exact same pattern as HostedWebSearchTool and HostedCodeInterpreterTool.
• Name returns "tool_search".
• Has the same constructor pattern (default + one taking IReadOnlyDictionary<string, object?>? additionalProperties).
• Must be marked [Experimental(DiagnosticIds.Experiments.AIToolSearch, UrlFormat = DiagnosticIds.UrlFormat)].

SearchableAIFunctionDeclaration (in src/Libraries/Microsoft.Extensions.AI.Abstractions/Functions/SearchableAIFunctionDeclaration.cs)

• A new class that extends DelegatingAIFunctionDeclaration (which is currently internal). Important: DelegatingAIFunctionDeclaration is the declaration-only delegating base (not DelegatingAIFunction which requires AIFunction). This is because SearchableAIFunctionDeclaration should work with AIFunctionDeclaration instances that may not have InvokeAsync.
• Constructor takes AIFunctionDeclaration innerFunction and string? namespaceName = null.
• Has a Namespace property (string?) for grouping related tools.
• Must be sealed.
• Must be marked [Experimental(DiagnosticIds.Experiments.AIToolSearch, UrlFormat = DiagnosticIds.UrlFormat)].
• Include a static convenience helper method: public static IList<AITool> CreateToolSet(IEnumerable<AIFunctionDeclaration> functions, string? namespaceName = null, IReadOnlyDictionary<string, object?>? toolSearchProperties = null) that wraps all functions as SearchableAIFunctionDeclaration and prepends a HostedToolSearchTool, returning a complete tool list ready for ChatOptions.Tools.

2. DiagnosticIds update

In src/Shared/DiagnosticIds/DiagnosticIds.cs, add a new constant in the Experiments class:

internal const string AIToolSearch = AIExperiments;

Place it alongside the other AI experiment constants (near AIWebSearch, AICodeInterpreter, etc.).

3. OpenAI provider implementation

In src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIResponsesChatClient.cs:

In ToResponseTool(AITool tool, ChatOptions? options = null) method:

Add handling for HostedToolSearchTool — this maps to the OpenAI tool_search response tool. Since the underlying OpenAI .NET SDK likely doesn't have a ToolSearchTool class yet, you need to manually construct a ResponseTool from JSON. Cache the deserialized ResponseTool instance in a static field so it's only created once. Use the ModelReaderWriter pattern or direct JSON deserialization to create a ResponseTool from the JSON {"type": "tool_search"}. Pattern:

case HostedToolSearchTool:
    return s_toolSearchResponseTool ??= ModelReaderWriter.Read<ResponseTool>(BinaryData.FromString("""{"type": "tool_search"}"""))!;

Add a private static ResponseTool? s_toolSearchResponseTool; field to cache it.

For SearchableAIFunctionDeclaration: When an AIFunctionDeclaration is detected as having a SearchableAIFunctionDeclaration via GetService<SearchableAIFunctionDeclaration>(), the generated FunctionTool should have defer_loading set to true and optionally include the namespace metadata. Since the OpenAI SDK's FunctionTool class may not have these properties yet, use the Patch property to set them on the JSON. The check should happen in the existing case AIFunctionDeclaration aiFunction: branch — after calling ToResponseTool(aiFunction, options), check if the original tool (or aiFunction) has GetService<SearchableAIFunctionDeclaration>() and if so, patch the resulting FunctionTool with defer_loading and namespace. This is done in the ToResponseTool(AITool, ChatOptions?) method so it doesn't infect the general ToResponseTool(AIFunctionDeclaration, ChatOptions?) helper. Specifically, the case AIFunctionDeclaration aiFunction: case should become:

case AIFunctionDeclaration aiFunction:
    var functionTool = ToResponseTool(aiFunction, options);
    if (tool.GetService<SearchableAIFunctionDeclaration>() is { } searchable)
    {
        functionTool.Patch.Set("$.defer_loading"u8, "true"u8);
        if (searchable.Namespace is { } ns)
        {
   ...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

*This pull request was created from Copilot chat.*
>

<!-- START COPILOT CODING AGENT TIPS -->
---

✨ Let Copilot coding agent [set things up for you](https://github.com/dotnet/extensions/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo.

 ###### Microsoft Reviewers: [Open in CodeFlow](https://microsoft.github.io/open-pr/?codeflow=https://github.com/dotnet/extensions/pull/7377)

[stephentoub]

@copilot, I'm wondering if a different design would be more beneficial. What would you think about isolating it to just the HostedToolSearchTool, which would have two collections of either AITool instances or string names, one for Enable and one for Disable. By default, both of these collections would be null, in which case it would treat every tool as Enable and mark every tool as defer-loading. If Disable was non-null but Enable was null, then it would treat every tool as Enable except for those listed in Disable. Conversely if Enable was non-null and Disable was null, it would only treat those tools in Enable as defer-loading, and if both were non-null, it would treat everything in Enable as defer-loading unless it was also listed in Disable. Thoughts? Is the dominant case for tool search with OpenAI and Anthropic one where most tools are in the same boat and only zero to a small number are special-cased the other way? Or is the right way to model this what we currently have?

[Copilot]

@copilot, I'm wondering if a different design would be more beneficial. What would you think about isolating it to just the HostedToolSearchTool, which would have two collections of ei...

Yes, I think the dominant case is indeed "most/all tools are searchable" — the whole point of tool search is to scale to many tools, so users will almost always want blanket defer-loading with at most a few exceptions.

Your proposed design is better for that common case. Today, with 50 tools you'd wrap all 50 in SearchableAIFunctionDeclaration; with the proposed design, you'd just add new HostedToolSearchTool() and you're done. Much cleaner.

A few thoughts on the details:

Enable/Disable semantics: The logic makes sense and is consistent with how HostedMcpServerTool.AllowedTools works (null = all allowed). I wonder if the naming could be more specific though — maybe DeferredTools / NonDeferredTools, or SearchableTools / AlwaysLoadedTools? The Enable/Disable naming is generic and doesn't hint at what is being enabled. But that's a bikeshed.

String names vs AITool references: String names (like AllowedTools uses) are simpler and avoid lifetime/reference issues, but AITool references give type safety. Either could work; strings are probably simpler since the provider just needs to match tool names when iterating ChatOptions.Tools.

Namespace: The current design has per-tool Namespace. If we consolidate into HostedToolSearchTool, we'd need either: (a) a single Namespace property on HostedToolSearchTool that applies to all searchable tools, (b) a Dictionary<string, IList<string>> mapping namespaces to tool names, or (c) drop namespace for now and add it later if needed. I'd lean toward (a) or (c) — a single namespace is probably fine for most uses, and (c) keeps the API surface minimal while we learn more.

What we'd lose: The GetService<SearchableAIFunctionDeclaration>() discovery pattern, but that's only useful in middleware, and middleware doesn't typically need to know about defer-loading (it's a wire-level optimization). DelegatingAIFunctionDeclaration would also no longer need to be public (can revert to internal).

Overall I think this is a better design. Want me to implement it?

[stephentoub]

Want me to implement it?

@copilot, yes, please

[Copilot]

Want me to implement it?

@copilot, yes, please

Implemented in f295d10.

Changes:

• HostedToolSearchTool now has DeferredTools and NonDeferredTools (IList<string>?) properties with the enable/disable semantics you described (both null = all deferred)
• Removed SearchableAIFunctionDeclaration entirely
• Reverted DelegatingAIFunctionDeclaration to internal
• OpenAI provider finds HostedToolSearchTool from ChatOptions.Tools and applies defer_loading: true to matching function tools
• Used AOT-safe ModelReaderWriter.Read<ResponseTool>(..., OpenAIContext.Default) and "true"u8 literal (no pragma suppressions)
• Added ChatOptions? parameter to AsOpenAIResponseTool extension (follows AsOpenAIResponseTextFormat pattern)

[stephentoub]

@PederHP any thoughts?

[Copilot]

Pull request overview

Adds a new experimental hosted tool marker (HostedToolSearchTool) to enable “tool_search” and implements deferred loading (defer_loading) support in the OpenAI Responses provider by patching function tools based on DeferredTools / NonDeferredTools configuration.

Changes:

• Introduces HostedToolSearchTool (experimental) and a new DiagnosticIds.Experiments.AIToolSearch constant.
• Updates OpenAI Responses tool conversion to emit tool_search and optionally patch function tools with defer_loading: true.
• Adds/extends unit tests covering tool_search conversion, caching, and deferred-loading selection logic.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
test/Libraries/Microsoft.Extensions.AI.OpenAI.Tests/OpenAIConversionTests.cs	Adds OpenAI conversion tests for tool_search and defer_loading behavior.
test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Tools/HostedToolSearchToolTests.cs	Adds tests for HostedToolSearchTool construction and list properties.
src/Shared/DiagnosticIds/DiagnosticIds.cs	Adds DiagnosticIds.Experiments.AIToolSearch.
src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAIResponsesChatClient.cs	Adds tool_search mapping and deferred-loading patching for function tools.
src/Libraries/Microsoft.Extensions.AI.OpenAI/MicrosoftExtensionsAIResponsesExtensions.cs	Updates AsOpenAIResponseTool to accept ChatOptions? for deferred-loading context.
src/Libraries/Microsoft.Extensions.AI.OpenAI/Microsoft.Extensions.AI.OpenAI.json	Updates API baseline for the modified extension method signature.
src/Libraries/Microsoft.Extensions.AI.Abstractions/Tools/HostedToolSearchTool.cs	Adds the new experimental HostedToolSearchTool abstraction with configuration lists.

[Copilot]

System.Diagnostics.CodeAnalysis is imported but not used in this file. If analyzers like IDE0005 are enforced (often as warnings-as-errors), this will break the build; please remove the unused using.

[stephentoub]

@copilot remove the using if it's actually unnecessary

[Copilot]

The System.Diagnostics.CodeAnalysis using is actually needed — it's required for the [Experimental] attribute on line 22. The automated reviewer was incorrect. HostedImageGenerationTool has the same pattern (experimental + same using).

[stephentoub]

@copilot why was it valuable to change the signature of ToResponseTool like this? I'm not seeing the benefits rather than just having ToResponseTool call FindToolSearchTool if/when it needs to.

[stephentoub]

Oh, nevermind, I see, because we need to call ToResponseTool for each AIFunction

[Copilot]

You're right — there was no benefit to exposing the HostedToolSearchTool? parameter on the internal ToResponseTool. Fixed in b53a6b9: added a simpler ToResponseTool(AITool, ChatOptions?) overload that calls FindToolSearchTool internally. The 3-param version and FindToolSearchTool are now both private. The internal loop still pre-finds the tool for O(N) efficiency.

[stephentoub]

@copilot please add one or more OpenAI unit tests that confirm the right JSON is sent when using HostedToolSearchTool in various ways, and an integration test as well.

[Copilot]

Added in ead292d. Six new VerbatimHttpHandler unit tests in OpenAIResponseClientTests.cs that validate the exact JSON request body for various HostedToolSearchTool configurations:

• ToolSearchTool_OnlyToolSearch_NonStreaming — just tool_search in tools array
• ToolSearchTool_AllToolsDeferred_NonStreaming — both lists null → all function tools get defer_loading: true
• ToolSearchTool_SpecificDeferredTools_NonStreaming — DeferredTools = ["GetWeather"] → only that tool deferred
• ToolSearchTool_NonDeferredExclusion_NonStreaming — NonDeferredTools = ["ImportantTool"] → all except that tool deferred
• ToolSearchTool_BothLists_DisableTakesPrecedence_NonStreaming — both lists set, NonDeferred wins
• ToolSearchTool_NoFunctionTools_NonStreaming — tool_search + web_search, no function tools

Plus an integration test UseToolSearch_WithDeferredFunctions in OpenAIResponseClientIntegrationTests.cs.

[jozkee]

Suggested change

	public static ResponseTool? AsOpenAIResponseTool(this AITool tool, ChatOptions? options = null)
	public static ResponseTool? AsOpenAIResponseTool(this AITool tool, ChatOptions? options)

options = null may make it too easy to miss the defer_loading savings. Only alternative that comes to mind is to make it non-optional but that will make code look ugly meaiTool.AsOpenAIResponseTool(null).

Since this is a subset of a subset of cases i.e. MEAI to Responses conversion with HostedFileSearchTool, it may be fine as is.

[jozkee]

Is it OK to return the shared singleton? It may be corrupted by callers.

[jozkee]

Mcp toolsets also interact with the tool-search tool.

https://developers.openai.com/api/docs/guides/tools-connectors-mcp#defer-loading-tools-in-an-mcp-server
https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool#mcp-integration

[jozkee]

Anthropic also allows setting per-tool config for an MCP server (e.g., defer_loading overrides for individual tools). HostedMcpServerTool doesn't currently have a way to express per-tool config, so an Anthropic IChatClient implementation would likely need a convention like ServerName/ToolName in the DeferredTools/NonDeferredTools lists as a workaround.