Skip to content

Question about conditional sampling and moco-k during training #5

Open
@andsild

Description

@andsild

Hi,

Thanks for the code and paper :)

I'm trying to understand this line, where the model checks for conditional sampling:

if self.condition:
# conditional ssl
logits = torch.mm(q, k.T) / self.T
labels = torch.arange(logits.shape[0], dtype=torch.long).cuda()
return logits, labels

It seems to me that this skips updating the self.queue which I believe corresponds to the "dictionary" in the MoCo paper (section 3.1)

Am I right? If so, does this mean you effectively train with args.moco-k at the same as the batch size (m = 128)? If so, when you compared against an model without conditional sampling in your paper, did you also set moco-k to 128 or was it the default 65536?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions