Virtual Populate foreignField is array in combination with UUIDs

[JustDoItSascha]

Hello,

I have the exact same issue @Namchee had in this other issue. The query which Mongoose performs is correct, it would return the correct documents from the foreign schema. But it seems like it's not getting merged correctly afterwards...

Problem exists with Mongoose 8.12.0

P.S. I found out, it's only happening when using UUIDs instead of ObjectId.

All of the cases here refers to array of IDs in virtualized-schema to ID in reference schema. But, what if I have the opposite case?

Example:

Announcement Schema

const announcementSchema = new Schema({

  title: {

    type: String,

    trim: true,

    unique: true,

    required: true,

    maxlength: 25,

  },



  content: {

    type: String,

    trim: true,

    required: true,

    maxlength: 500,

  },



  validUntil: {

    type: Date,

    required: true,

  },



  important: {

    type: Boolean,

    default: false,

  },



  categories: {

    type: [Schema.Types.ObjectId],

    ref: 'Category',

    default: [],

  },

});

Category Schema

const categorySchema = new Schema({

  name: {

    type: String,

    trim: true,

    required: true,

    index: true,

    maxlength: 50,

  },



  desc: {

    type: String,

    trim: true,

    required: true,

    maxlength: 250,

  },

});



categorySchema.virtual('announcementCount', {

  ref: 'Announcement',

  localField: 'id',

  foreignField: 'categories',

  count: true,

});

I've tried the above solution, but I keep getting undefined when I tried to access announcementCount after populating it.

Originally posted by @Namchee in #4585

[JustDoItSascha]

Ok, I found out, that the error is in this line:

https://github.com/Automattic/mongoose/blob/master/lib/helpers/populate/assignRawDocsToIdStructure.js#L81

The ID is in Binary Format, one need to call toUUID() instead of String() then it would work. But I'm not sure if this is the correct place to do it? Maybe the root cause is already before? Maybe it needs to be something like

sid = typeof id.toUUID === 'function' ? id.toUUID() : String(id); ?

[JustDoItSascha]

Edit: My quick fix is destroying other populations which are not virtual. So it's wrong. It seems like the rawDocs are having string keys.

I could only fix it by changing line 81 of assignRawDocsToldStructure to

sid = id.toString('hex').replace(
        /^(.{8})(.{4})(.{4})(.{4})(.{12})$/,
        '$1-$2-$3-$4-$5'
    )

and line 4715 of model.js to

key = _val.toString('hex').replace(
            /^(.{8})(.{4})(.{4})(.{4})(.{12})$/,
            '$1-$2-$3-$4-$5'
        );

But this will not work with ObjectIds anymore. I know what the error is, but I have no idea how to fix it properly, because I don't know why it's sometimes a Buffer, sometimes a Binary... :(

[JustDoItSascha]

@vkarpov15 Thank you very much :) Will test it when it's released 🙏

[JustDoItSascha]

@vkarpov15 Hey, I tested it now with 8.13.2 and have to say that it's still not working :( If I populate a virtualField while using UUIDs, the field is still null...

[JustDoItSascha]

The issue is, you fixed this code here:

if (id?.constructor?.name === 'Binary' && id.sub_type === 4 && typeof id.toUUID === 'function') {
      // Workaround for gh-15315 because Mongoose UUIDs don't use BSON UUIDs yet.
      sid = String(id.toUUID());
    } else {
      sid = String(id);
    }

But then you do this doc = resultDocs[sid];. But the resultDocs are already wrong, because the keys of this object are still looking like this �9㦪G�����\\\x06\x0F instead of the uuid as string.

Edit:

This line here is wrong:

mongoose/lib/model.js

Line 4715 in 95a03cb

key = String(_val);

And probably this here too:

mongoose/lib/model.js

Line 4692 in 95a03cb

key = String(__val);

If I change them to

key = _val.toString('hex').replace(
              /^(.{8})(.{4})(.{4})(.{4})(.{12})$/,
              '$1-$2-$3-$4-$5'
          );

it's a temporary fix for me, which works. If I copy your fix from assignRawDocsToldStructure it's NOT working, because the function toUUID() is missing and therefore the toUUID() is not called.