Per-Token MSE Loss visualization

[jiangzhengguo]

Hello, thank you for open-sourcing your excellent work! However, I have a small question regarding Figure 6 "Per-Token MSE Loss" in the paper:
On which samples and with how many data points was this figure generated? Why does it show such an obvious pattern, resembling nine "M" shapes?
In my own experiments with 20 samples, the results did not exhibit any clear regularity. Could you please share some details about the visualization of this figure?
Thank you very much!

[mi804]

The loss curve is averaged over about random 5,000 samples from EliGen subsets. The periodic pattern arises because the 81 tokens output by Nexus-Gen correspond exactly to a 9x9 grid. From the results, it appears that tokens located closer to the center in terms of width and height have higher loss values.