Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Support internal connector in register model #212

Open
amitgalitz opened this issue Nov 29, 2023 · 1 comment
Open

[FEATURE] Support internal connector in register model #212

amitgalitz opened this issue Nov 29, 2023 · 1 comment
Labels
backlog Good to have functionality not critical for next release enhancement New feature or request

Comments

@amitgalitz
Copy link
Member

Is your feature request related to a problem?

Right now we have a step called RegisterRemoteModel in our framework that correlates to the register_model API In ml-commons when a remote model is given. Currently we only support creating a remote model with a given connector ID. For example:

POST /_plugins/_ml/models/_register
{
    "name": "openAI-gpt-3.5-turbo",
    "function_name": "remote",
    "model_group_id": "1jriBYsBq7EKuKzZX131",
    "description": "test model",
    "connector_id": "a1eMb4kBJ1eYAeTMAljY"
}

However, ml-commons also supports creating an internal connector inside the model itself through this API. We currently don't support this, example payload:

POST /_plugins/_ml/models/_register
{
    "name": "openAI-GPT-3.5: internal connector",
    "function_name": "remote",
    "model_group_id": "lEFGL4kB4ubqQRzegPo2",
    "description": "test model",
    "connector": {
        "name": "OpenAI Connector",
        "description": "The connector to public OpenAI model service for GPT 3.5",
        "version": 1,
        "protocol": "http",
        "parameters": {
            "endpoint": "api.openai.com",
            "max_tokens": 7,
            "temperature": 0,
            "model": "text-davinci-003"
        },
        "credential": {
            "openAI_key": "..."
        },
        "actions": [
            {
                "action_type": "predict",
                "method": "POST",
                "url": "https://${parameters.endpoint}/v1/completions",
                "headers": {
                    "Authorization": "Bearer ${credential.openAI_key}"
                },
                "request_body": "{ \"model\": \"${parameters.model}\", \"prompt\": \"${parameters.prompt}\", \"max_tokens\": ${parameters.max_tokens}, \"temperature\": ${parameters.temperature} }"
            }
        ]
    }
}

What solution would you like?

There can be two solutions for this, either we create a new step name called register_model_with_internal_connector/register_internal_connector/register_internal_model. Or we change the register_remote_model step to allow for either a connector ID or a connector object by changing the parsing method here: https://github.com/opensearch-project/flow-framework/blob/main/src/main/java/org/opensearch/flowframework/workflow/RegisterRemoteModelStep.java#L109

To satisfy what a connector object is, we can potentially refactor some of the code in the CreateConnectorStep so we don't have duplicates for the connector parsing.

@amitgalitz amitgalitz added enhancement New feature or request untriaged and removed untriaged labels Nov 29, 2023
@dbwiddis
Copy link
Member

This relates to #339 implemented in #340 where I combined the register + deploy steps. This would be really nice to include.

@dbwiddis dbwiddis added backlog Good to have functionality not critical for next release and removed v2.15.0 labels May 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backlog Good to have functionality not critical for next release enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants