[FEAT] Add LoReTTA

[mbaddar1]

This PR is to integrate LORETTA : A Tensor-Train based Low-Rank approach for tuning BERT-family of model
Related issue #2813

[BenjaminBossan]

@mbaddar1 Just asking if you still plan on working on this.

[mbaddar1]

Hi
sorry for being inactive. I just got busy. Is there a deadline or so to be aware of ?

[BenjaminBossan]

Hi, no worries, there is no time pressure. I just wanted to ping you, in case you simply forgot about this. Take all the time you need.

[mbaddar1]

No I have it in my plan. The only scarce resource, is time (as usual :)

[mbaddar1]

Just to keep ticket alive. I am starting working on it starting from 25 Nov.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

[mbaddar1]

comment to keep alive.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

[githubnemo]

not stale

[BenjaminBossan]

As discussed, let's close this for now to avoid the endless stale / not stale cycle. Once work on this picks up again, we can re-open.