More information about the specific implementation of relative attention bias.

[buaaliyi]

Hi @jiaqizhai

We can find some previous issues,

#148
#36

which mentioned the specific implementation of relative attention bias.
For the exact rab_{p, t}(i, j) setting in this particular codebase, the relative timespan between token i and j is calculated by timestamp[i] and timestamp[j+1] (not timestamp[j]), furthermore, the relative positional gap between token i and j is calculated by N - (j - i) (not the same like "google text-to-text transformers" paper source codebase, which is j - i). So that I am strange about this rab implementation, but what does it advantages for?

Thank you for more information about the implementation details.

[buaaliyi]

In addition, what about the rab for ranking task, the same as the above retrieval task or not？

[jiaqizhai]

Hi,

The rab timestamp setting we used for public datasets (retrieval) is exactly what we discussed in #148.

For the interleaved ranking setting, you can either reuse the same code or just j-i - these two should perform similarly.