Python: Defining the response_format of a ChatCompletionAgent in a YAML file

[bankaboy]

I have the following class for structuring the response of a ChatCompletionAgent

from semantic_kernel.kernel_pydantic import KernelBaseModel

## define the class for the output structure of the file discovery agent
class FilesList(KernelBaseModel):
    files_list: Annotated[List[str], "a list of paths of datasets"]

I want to create the ChatCompletionAgent using the AgentRegistry.create_from_file() method and the agent should pull the AzurePromptExecutionSettings from the YAML file.

My current YAML file

type: chat_completion_agent
name: FileDiscoveryAgent
template: |
  <message role="system">
    You are an AI agent that searches for files. As the agent, you give the user all the files that are present in the specified folder.
    
    # File search context 
    Folder: {{datasets_directory}}

    Return the paths of all files found to the user.

template_format: semantic-kernel
description: An assistant that gathers the list of files in a given directory.
instructions: You are an AI agent that searches for files. As the agent, you give the user all the files that are present in the specified folder. Return the response in the format that has been given to you.
tools:
  - id: FileDiscoveryPlugin.find_files
    type: function
input_variables:
  - name: datasets_directory
    description: The folder with the datasets.
    is_required: true
output_variable:
  - description: The list of files in the folder
    json_schema: {"files_list": ["path_1", "path_2", "path_3", "path_n"]}
execution_settings:
  default:
    temperature: 0.0
    top_p: 0.0

Plugin

def get_data_filenames(datasets_directory: str, safe_file_exts: tuple[str] = ('.csv', '.xml', '.txt', '.json')) -> List[str]:
    

    data_files_paths: List[str] = [] # store the paths of all the files                                                     

    # start the traversal
    for dirpath, dirs, files in os.walk(datasets_directory):

        for filename in files:
            filepath = os.path.join(dirpath,filename)

            if filepath.endswith(safe_file_exts):
                data_files_paths.append(filepath)

    return data_files_paths


class FileDiscoveryPlugin:
    @kernel_function(description="Find the files in the given dictionary")
    async def find_files(self, datasets_directory: Annotated[str, 'the path of the directory where the datasets are present']) -> Annotated[List[str], 'list of paths of the datasets in that directory']:
        "Find the files in the given dictionary"
        return get_data_filenames(datasets_directory)


kernel = Kernel()

## add the plugin to the kernel
kernel.add_plugin(
    plugin = FileDiscoveryPlugin(),
    plugin_name = "FileDiscoveryPlugin"
)

If I use get_response, I get the result back in free text and not structured to the FilesList class

agent: ChatCompletionAgent = await AgentRegistry.create_from_file(
    yaml_path, service = AzureChatCompletion(), kernel=kernel, plugins = [FileDiscoveryPlugin()],arguments=KernelArguments(datasets_directory="./datasets/")
)

response = await agent.get_response(messages=f"find all the files in ./datasets/")
print(f'response content:{response.message.content}, type: {type(response.message.content)}\n')

The result is the same whether I do

output_variable:
  - description: The list of files in the folder
    json_schema: '{"files_list": ["path_1", "path_2", "path_3", "path_n"]}'

or

output_variable:
  - description: The list of files in the folder
    json_schema: {"files_list": []}

or

output_variable:
  - description: The list of files in the folder
    json_schema: '{"files_list": []}'

This is what I'm trying to achieve (agent initialised in code itself):

# define the settings for the agent
model_settings = AzureChatPromptExecutionSettings()
model_settings.function_choice_behavior = FunctionChoiceBehavior.Required()
model_settings.temperature = 0.0
model_settings.top_p = 0.4
model_settings.response_format = FilesList

# intitiate the agent that will look for data attributes
files_discovery = ChatCompletionAgent(
    service = AzureChatCompletion(),
    name = "FileDiscoveryAgent",
    instructions = """You are an AI agent that searches for files. As the agent, you give the user all the files that are present in the specified folder. Return the response in the format that has been given to you""",
    kernel = kernel,
    arguments = KernelArguments(settings = model_settings),
    # settings = model_settings
    )


query = 'find all the files in ./datasets/'

response = await files_discovery.get_response(messages = query)
print(f'response content:{response.message.content}, type: {type(response.message.content)}\n')
response_json = json.loads(response.message.content)
print(f'response_json:{response_json}, type: {type(response_json)}\n')

I want a way the AgentRegistry.create_from_file method either:

1. Infers the FilesList class declared in the code is what the response_format: FilesList in the Yaml is reffering to
2. It follows the json schema defined for the output variable

[bankaboy]

This is where I am structuring my YAML from Semantic Kernel YAML Schema