Search results

[example] Add example on using Nadir Custom Optimizers with HuggingFace

exampleAdd or change an example of the repositoryAdd or change an example of the repository

Status: Open.

#63 In OptimalFoundation/nadir;

· bhavnicksm opened on Jun 19, 2024

Assert "valid" values for hyperparams in the Config

Status: Open.

#62 In OptimalFoundation/nadir;

· bhavnicksm opened on Jun 15, 2024

Adam optimizer <code>.eval()</code> not working well with ScheduleFree

bugSomething isn't workingSomething isn't working

Status: Open.

#61 In OptimalFoundation/nadir;

· bhavnicksm opened on Jun 15, 2024

Create <code>apply_update()</code> fn to manage post update transformations

Status: Open.

#59 In OptimalFoundation/nadir;

· bhavnicksm opened on Jun 15, 2024

Add SAdam (SoftPlus the denom)

Status: Open.

#58 In OptimalFoundation/nadir;

· bhavnicksm opened on Jun 9, 2024

Add AdamD Optimizer

Status: Open.

#57 In OptimalFoundation/nadir;

· bhavnicksm opened on Jun 9, 2024

Add Gradient Centralization for all the optimizers

enhancementNew feature or requestNew feature or request

Status: Open.

#51 In OptimalFoundation/nadir;

· bhavnicksm opened on Jun 8, 2024

Refactor Optimizer Config API

enhancementNew feature or requestNew feature or request

wontfixThis will not be worked onThis will not be worked on

Status: Open.

#45 In OptimalFoundation/nadir;

· bhavnicksm opened on Jun 1, 2024

Create a template to quickly add and test new optimisers

Status: Open.

#41 In OptimalFoundation/nadir;

· bhavnicksm opened on May 31, 2024

Add Sophia Optimiser

Status: Open.

#40 In OptimalFoundation/nadir;

· bhavnicksm opened on May 30, 2024

GaLore-ify every optimiser

enhancementNew feature or requestNew feature or request

help wantedExtra attention is neededExtra attention is needed

Status: Open.

#39 In OptimalFoundation/nadir;

· bhavnicksm opened on May 30, 2024

Implement Optimizers

enhancementNew feature or requestNew feature or request

help wantedExtra attention is neededExtra attention is needed

Status: Open.

#22 In OptimalFoundation/nadir;

· bhavnicksm opened on Mar 3, 2023