Scoped Counters only working with a single reference_field

[jpolvora]

Given any model wich I want to scope by two reference_fields ['fieldA','fieldB'] this would give an error of mongo duplicate key.

const mongoose = require('mongoose'),
    Schema = mongoose.Schema;

const AutoIncrement = require('mongoose-sequence')(mongoose);

const schema = new Schema(
    {
        empresa: { type: Number, required: true },
        pedido: { type: Number, required: true },
        key: Number, //sequence
    });

schema.plugin(AutoIncrement, { id: 'items', inc_field: 'key', reference_fields: ['empresa','pedido'] });
´``

This bug can be reproduced with the latest 4.0.1 version.

[ramiel]

Can you please report exactly the error? Which key in which collection is marked as duplicate?

[ozzyknox]

Hi @jpolvora, did you manage to resolve this problem? I am getting the same behaviour when working with 2 reference fields. Works fine with a single reference field.

[ozzyknox]

Hi @ramiel. I have managed to reproduce my error in your test code. If you change the block of tests on lines 306 to 3045 as follows, you should experience the duplicate key error:

describe('a counter which referes others fields => ', function() {

    before(function() {
        var ComposedSchema = new Schema({
            country: Schema.Types.ObjectId,
            city: String,
            inhabitant: Number
        });
        ComposedSchema.plugin(AutoIncrement, {id: 'inhabitant_counter', inc_field: 'inhabitant', reference_fields: ['city', 'country']});
        this.Composed = mongoose.model('Composed', ComposedSchema);
    });

    it('increment on save', function(done) {
        var t = new this.Composed({country: mongoose.Types.ObjectId('59c380f51207391238e7f3f2'), city: 'Paris'});
        t.save(function(err) {
            if (err) return done(err);
            assert.deepEqual(t.inhabitant, 1);
            done();
        });
    });

    it('saving a document with the same reference increment the counter', function(done) {
        var t = new this.Composed({country: mongoose.Types.ObjectId('59c380f51207391238e7f3f2'), city:'Paris'});
        t.save(function(err) {
            if (err) return done(err);
            assert.deepEqual(t.inhabitant, 2);
            done();
        });
    });

    it('saving with a different reference do not increment the counter', function(done) {
        var t = new this.Composed({country: mongoose.Types.ObjectId('59c380f51207391238e7f3f2'), city:'Carcasonne'});
        t.save(function(err) {
            if (err) return done(err);
            assert.deepEqual(t.inhabitant, 1);
            done();
        });
    });

});

The error stack trace looks like this:

  1) Basic =>
       Global sequences =>
         a counter which referes others fields =>
           saving with a different reference do not increment the counter:
     MongoError: E11000 duplicate key error index: mongoose-sequence-testing.counters.$id_1_reference_value_1 dup key: { : "inhabitant_counter", : ""59c380f51207391238e7f3f2"" }
      at Function.MongoError.create (C:\Users\ozzy\dev\misc\mongoose-sequence\node_modules\mongodb-core\lib\error.js:31:11)
      at C:\Users\ozzy\dev\misc\mongoose-sequence\node_modules\mongodb-core\lib\connection\pool.js:497:72
      at authenticateStragglers (C:\Users\ozzy\dev\misc\mongoose-sequence\node_modules\mongodb-core\lib\connection\pool.js:443:16)
      at Connection.messageHandler (C:\Users\ozzy\dev\misc\mongoose-sequence\node_modules\mongodb-core\lib\connection\pool.js:477:5)
      at Socket.<anonymous> (C:\Users\ozzy\dev\misc\mongoose-sequence\node_modules\mongodb-core\lib\connection\connection.js:321:22)
      at readableAddChunk (_stream_readable.js:176:18)
      at Socket.Readable.push (_stream_readable.js:134:10)
      at TCP.onread (net.js:547:20)

I am able to temporarily fix this by commenting out the index creation, but I suppose this is not ideal.

This is happening because of the way mongoose is indexing the array. It seems to only reference the object id value and not a combination of the object id and the string value.

Are you able to look into this?

[ramiel]

Thank you @ozzyknox for the detailed report. I'll definetely look at this soon

[ozzyknox]

Hi @ramiel. I have run an isolated test in MongDB where I changed the reference_value from being an Array to being an Object. So, in the new test scenario, our document above, which currently looks like this (and fails when inserting the second document):

{
    "_id" : ObjectId("5afe972c4b98754d4d8343d3"),
    "id" : "inhabitant_counter",
    "reference_value" : [ 
        "\"Paris\"", 
        "\"France\""
    ],
    "seq" : 2
}

{
    "_id" : ObjectId("5afe972c4b98754d4d8343d4"),
    "id" : "inhabitant_counter",
    "reference_value" : [ 
        "\"Paris\"", 
        "\"Carcasonne\""
    ],
    "seq" : 1
}

changes to look like this (and succeeds on the insert of the second document):

{
    "_id" : ObjectId("5afe972c4b98754d4d8343d3"),
    "id" : "inhabitant_counter",
    "reference_value" : { 
        "city": "Paris", 
        "country": "France"
    },
    "seq" : 2
}

{
    "_id" : ObjectId("5afe972c4b98754d4d8343d4"),
    "id" : "inhabitant_counter",
    "reference_value" : { 
        "city": "Paris", 
        "country": "Carcasonne"
    },
    "seq" : 1
}

I have tested this scenario with the same unique index applied and it seems to work well. I have also run some tests with a single attrtibute inside the reference_value object and it all works well.

I hope this can help you.

[ramiel]

Yes, thank you. I don't want to modify the array to an object. Not because is a bad solution but because I cannot release it without having a major release. This means I won't solve the problem for who's using the library now. I must solve it using the same index. Nonetheless thank you for discovering this

[ramiel]

I'm discovering that mongo, when you specify an index on an array field, create an index for "each" value (https://docs.mongodb.com/manual/core/index-multikey/). This means that this solution never worked.

[ramiel]

Version 5.0.0 should fix this issue. Please check

[HRITIKARAUT27]

I want to scope by a single reference_field is it possible?