Difface

[rousseab]

A reworking of the 'Diffusion MACE' architecture.

The main insight is that the node attributes should probably NOT be the values of sigma; sigma is often close to zero, so a linear layer without bias will just send everything to zero.

Here we create "augmented" node attributes by mixing the attributes coming from the one-hot encoded atom types and a MLP (with biases) applied to sigma.

This new architecture can overfit a single example of the score with fixed values of sigma:
From experiments/diffusion_mace_harmonic_data/overfit_diffusion_mace.py
(Different experiments have sigma = 0.001, 0.1, 0.5, and some fiddling with number of MLP layers and channels)

This requires much fewer parameters than the previous monstrosities because the tensorproduct is only on 0e x 0e channels.

Finding the right parameters is very 'fiddly'. There's much exploration to be done to see if this can learn what we want.