[Bug]: Not providing auto ID field in upsert request raises an error "Insert missed an field id to collection without set nullable==true or set default_value"

[issamemari]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

Not providing auto ID field in upsert request raises an error "Insert missed an field id to collection without set nullable==true or set default_value"

Expected Behavior

The upsert request should not fail because the ID field is configured to be auto_id

Steps/Code To Reproduce behavior

from pymilvus import CollectionSchema, DataType, FieldSchema
from pymilvus.milvus_client.index import IndexParams

client = MilvusClient(
    uri=<SECRET>,
    token=<SECRET>,
)

dimension = 4
collection_name = "testing_collection"
client.create_collection(
    collection_name,
    schema=CollectionSchema(
        fields=[
            FieldSchema(
                name="id",
                dtype=DataType.INT64,
                is_primary=True,
                auto_id=True,
                nullable=True,
                description="Auto increment ID",
            ),
            FieldSchema(
                name="vector",
                dtype=DataType.FLOAT_VECTOR,
                dim=dimension,
                description="Embedding vector",
            ),
        ],
    ),
)

client.create_index(
    collection_name,
    index_params=IndexParams(
        field_name="vector",
        index_type="IVF_FLAT",
        metric_type="COSINE",
    ),
)

client.load_collection(collection_name)

client.upsert(
    collection_name,
    data=[{"vector": [i, i, i, i]} for i in range(10)],
)

Environment details

- Hardware/Softward conditions (OS, CPU, GPU, Memory): MacOS
- Method of installation (Docker, or from source): pip install
- Milvus version (v0.3.1, or v0.4.0): pymilvus==2.5.3

Anything else?

The issue seems to be coming from this line

pymilvus/pymilvus/client/prepare.py

Line 562 in c5a3c18

if field_info.get("nullable", False) or field_info.get("default_value", None):

Here it should be checked whether the field is auto_id and allow it not to be provided

[issamemari]

I'm proposing a fix in this PR #2602

[yanliang567]

@issamemari This is by design, because upsert() need to know which IDs are going to be updated. As it is auto id, so the return results is with different ids.

/assign @issamemari
/unassign

[issamemari]

@yanliang567 hmm that seems strange to me. I would have expected this to run fine. To upsert is to update or insert. If the ID is not provided it should be assumed that this is an insert request. Otherwise the upsert method cannot be really used to insert.

Thanks anyway, I guess I can work around this by using inserts explicitly then.

[yanliang567]

yes, please use insert() instead in this case.