Code

llama: Attempt to add ModernBert#14014

Code

Open

huydt84 wants to merge 26 commits intoggml-org:masterggml-org/llama.cpp:masterfrom

huydt84:huydt/mbhuydt84/llama.cpp:huydt/mbCopy head branch name to clipboard

Commits

Commits on Jun 4, 2025

• llama: attempt to add modern-bert

  huydt-bti

  committed
  045b1ac

  Copy full SHA for 045b1ac

• Merge branch 'master' into huydt/mb

  huydt-bti

  committed
  95f49d9

  Copy full SHA for 95f49d9

• re-format and delete unused implementations

  huydt-bti

  committed
  eab776e

  Copy full SHA for eab776e

• overload set_swa_pattern for modern bert

  huydt-bti

  committed
  7143840

  Copy full SHA for 7143840

• modern-bert doesn't have bias

  huydt-bti

  committed
  6aa1335

  Copy full SHA for 6aa1335

• delete unnecessary files

  huydt-bti

  committed
  9e1179a

  Copy full SHA for 9e1179a

Commits on Jun 5, 2025

• add build_attn_inp_no_cache_iswa with symmetric swa

  huydt-bti

  committed
  fa23480

  Copy full SHA for fa23480

• add modern-bert to llama_model::create_memory

  huydt-bti

  committed
  a72cb3b

  Copy full SHA for a72cb3b

• fix lint

  huydt-bti

  committed
  adea1c9

  Copy full SHA for adea1c9

• access n_swa via hparams

  huydt-bti

  committed
  cfebb6e

  Copy full SHA for cfebb6e

Commits on Jun 6, 2025

• revert changes in convert script

  huydt-bti

  committed
  31e87e4

  Copy full SHA for 31e87e4

• add set_vocab to modernbert convert class

  huydt-bti

  committed
  1004327

  Copy full SHA for 1004327

• Merge branch 'master' into huydt/mb

  huydt-bti

  committed
  81f4797

  Copy full SHA for 81f4797

• parmas-related fix

  huydt-bti

  committed
  03693fa

  Copy full SHA for 03693fa

• handle mask token in modern-bert bpe

  huydt-bti

  committed
  2f5a72f

  Copy full SHA for 2f5a72f

• add modern-bert to pre_type check

  huydt-bti

  committed
  68f399e

  Copy full SHA for 68f399e

• change log warning when no mask token of modern-bert

  huydt-bti

  committed
  ad2a19a

  Copy full SHA for ad2a19a

Commits on Jun 8, 2025

• fix modern-bert swa logic

  huydt-bti

  committed
  c6b84e2

  Copy full SHA for c6b84e2

• fix modern-bert class register

  huydt-bti

  committed
  6751e69

  Copy full SHA for 6751e69

• Merge branch 'master' into huydt/mb

  huydt-bti

  committed
  8b794f9

  Copy full SHA for 8b794f9

Commits on Jun 9, 2025

• handle when no cls_b and cls_out_b

  huydt-bti

  committed
  4d6b804

  Copy full SHA for 4d6b804

• fix unnecessary operations

  huydt-bti

  committed
  5821d6c

  Copy full SHA for 5821d6c

• revert incorrect change

  huydt-bti

  committed
  820cee1

  Copy full SHA for 820cee1

• Merge branch 'master' into huydt/mb

  huydt-bti

  committed
  4fc4bf6

  Copy full SHA for 4fc4bf6

• use build_ffn with LLM_FFN_GEGLU

  huydt-bti

  committed
  16b73d4

  Copy full SHA for 16b73d4

• fix for use of models without n_swa

  huydt-bti

  committed
  333eeed

  Copy full SHA for 333eeed