Position of cls_tokens

[pfeatherstone]

vit-pytorch/vit_pytorch/vit_1d.py

Line 104 in 5578ac4

x, ps = pack([cls_tokens, x], 'b * d')

Is there a good reason to put the cls_tokens at the beginning of the sequence?
If I want a 1d sequence classifier that is used in streaming mode, then I want to limit, maybe even forbid, right context. In the extreme case of zero right context, then you essentially have causal attention and in that case, it probably makes more sense to put cls_tokens at the end so that it can attend to the whole sequence right