Langchain Model Deploy failed when putting input_sample

[prise6]

Hello, when i use something like below to log langchain model

input_example = {"user_message": "foobar"}
with mlflow.start_run():
    model_info = mlflow.langchain.log_model(
        lc_model=lc_model_path,
        artifact_path=artifact_path,
        input_example=input_example,
        signature=mlflow.models.infer_signature(
            model_input=input_example, model_output="output"
        ),
    )

And try to deploy the logged model with default image endpoint (example: azureml://registries/azureml/environments/mlflow-py39-inference/versions/2). There is an error :

 File "/opt/miniconda/envs/userenv/lib/python3.11/site-packages/azureml_inference_server_http/server/user_script.py", line 77, in load_script
   main_module_spec.loader.exec_module(user_module)
 File "<frozen importlib._bootstrap_external>", line 940, in exec_module
 File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
 File "/var/mlflow_resources/mlflow_score_script.py", line 374, in <module>
   input_param, output_param, params_param = get_parameter_type(sample_input, sample_output, sample_params)
                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/var/mlflow_resources/mlflow_score_script.py", line 342, in get_parameter_type
   param_arg[key] = NumpyParameterType(value, enforce_shape=False)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/opt/miniconda/envs/userenv/lib/python3.11/site-packages/inference_schema/parameter_types/numpy_parameter_type.py", line 33, in __init__
   raise Exception("Invalid sample input provided, must provide a sample Numpy array.")
Exception: Invalid sample input provided, must provide a sample Numpy array.

This is because mlflow_score_script.py contains theses lines :

#...
            elif isinstance(sample_input_ex, dict):
                _logger.info("sample input is a dict")
                # TODO keeping this around while _infer_schema doesn't work on dataframe string signatures
                param_arg = {}
                for key, value in sample_input_ex.items():
                    param_arg[key] = NumpyParameterType(value, enforce_shape=False)
                input_param = StandardPythonParameterType(param_arg)
#...

Am i doing something wrong ?

Thank you for you help.

[Abhi7739309]

HELP IN

[Tarun-Chevula]

@prise6 Could you please share the ML model and the input JSON file for the LangChain model you're using? You can find MLmodel as in the below:

https://mlflow.org/docs/latest/llms/langchain/notebooks/langchain-quickstart.html

Just to clarify, it is recommended to use either "dataframe" or "ndarray" as input formats, but you seem to be providing a dictionary.

[prise6]

Here is my old MLmodel file

artifact_path: chain
flavors:
  langchain:
    code: null
    langchain_version: 0.2.17
    streamable: true
  python_function:
    env:
      conda: conda.yaml
      virtualenv: python_env.yaml
    loader_module: mlflow.langchain
    model_code_path: /mnt/azureml/cr/j/f7af03b147c94d348194da9500adfee4/exe/wd/demo_azureml/indus_model.py
    predict_stream_fn: predict_stream
    python_version: 3.11.10
    streamable: true
mlflow_version: 2.18.0
model_size_bytes: 486
model_uuid: 4a5077d1814f45f3aa33a49c709b227d
run_id: d7da29f8-776a-4fa9-9e0d-e61beb96007c
saved_input_example_info:
  artifact_path: input_example.json
  serving_input_path: serving_input_example.json
  type: json_object
signature:
  inputs: '[{"type": "string", "name": "user_message", "required": true}]'
  outputs: '[{"type": "string", "required": true}]'
  params: null
utc_time_created: '2024-11-20 10:26:39.293137'

and input_example.json file :

{
  "user_message": "foobar"
}

So you're saying that my input_example variable must be already a ndarray at the logging step. ok. I'm not sure i will soon try it because i now use witch to promptflow server and deployment instead of mlflow model.

If someone can confirm to resolve the issue, feel free.

[github-actions]

This issue has been marked as stale because it has been inactive for 60 days.

[github-actions]

This issue has been automatically closed due to inactivity.