About Visualization Code

[viperdosth]

Good Work!
But could you provide the code for visualization of patch embeddings which shown in Fig.4 in your paper.
Thanks a lot.

[caoyunkang]

Thank you for your interest! I apologize for the current state of the visualization code—it’s still somewhat unorganized, and I haven't had the time to properly restructure it. However, I’m attaching the core part of the code below for your reference. I hope this helps to address your issue:

    def visualize_feat_map(self, patch_tokens):
        concat_patch_tokens = torch.cat(patch_tokens, dim=2)
        B, L, C = concat_patch_tokens.shape
        H = int(np.sqrt(L))
        concat_patch_tokens = concat_patch_tokens.squeeze(0).cpu().numpy()

        n_components = 1
        F_reshaped = concat_patch_tokens.reshape(-1, C)
        pca = PCA(n_components=n_components)
        F_pca = pca.fit_transform(F_reshaped)

        scaler = MinMaxScaler()
        F_pca_normalized = scaler.fit_transform(F_pca)

        visualized_image = F_pca_normalized.reshape(H, H, n_components)

        visualized_image = visualized_image * 255
        visualized_image = visualized_image.astype(np.uint8)

        return visualized_image