Code

Granite Four#13550

Code

Draft

gabe-l-hart wants to merge 78 commits intoggml-org:masterggml-org/llama.cpp:masterfrom

gabe-l-hart:GraniteFourgabe-l-hart/llama.cpp:GraniteFourCopy head branch name to clipboard

Commits

Commits on Aug 21, 2024

• llama : initial Mamba-2 support

  compilade

  committed
  1f0fea7

  Copy full SHA for 1f0fea7

• ggml : SIMD ggml_ssm_scan for Mamba-2

  Show description for dceff23
  compilade

  committed
  dceff23

  Copy full SHA for dceff23

• llama : support running Mamba-Codestral-7B-v0.1

  compilade

  committed
  2bfe9de

  Copy full SHA for 2bfe9de

• llama : fix Mamba-2 conv state saving

  Show description for aff9692
  compilade

  committed
  aff9692

  Copy full SHA for aff9692

Commits on Aug 22, 2024

• llama : remove unused variable

  compilade

  committed
  e04910d

  Copy full SHA for e04910d

• llama : add missing break

  compilade

  committed
  fa358e7

  Copy full SHA for fa358e7

• convert_hf : prefer SentencePiece tokenizer for Mamba-2 when present

  Show description for 38913dc
  compilade

  committed
  38913dc

  Copy full SHA for 38913dc

Commits on Sep 18, 2024

• Merge branch 'master' into compilade/mamba2

  compilade

  committed
  0e601ca

  Copy full SHA for 0e601ca

Commits on Oct 1, 2024

• llama : avoid redundant state copy for Mamba 1 and 2

  compilade

  committed
  273e7a4

  Copy full SHA for 273e7a4

• Merge branch 'master' into compilade/mamba2

  compilade

  committed
  7d6cb36

  Copy full SHA for 7d6cb36

Commits on Oct 2, 2024

• metal : attempt to adapt SSM_SCAN for Mamba-2

  compilade

  committed
  2c77d79

  Copy full SHA for 2c77d79

• metal : fix SSM_SCAN pipeline scope

  compilade

  committed
  87b97d0

  Copy full SHA for 87b97d0

• metal : use log and exp instead of log1pf and expf in SSM_SCAN

  compilade

  committed
  03d0e6e

  Copy full SHA for 03d0e6e

• metal : remove unused arguments for SSM_SCAN

  Show description for 7a351ab
  compilade

  committed
  7a351ab

  Copy full SHA for 7a351ab

• metal : add back n_seqs to SSM_SCAN args

  Show description for 8b15bc6
  compilade

  committed
  8b15bc6

  Copy full SHA for 8b15bc6

• metal : fix SSM_SCAN state head offset

  compilade

  committed
  5b8ec2b

  Copy full SHA for 5b8ec2b

Commits on Oct 3, 2024

• metal : fix wrong number of tokens per sequence in SSM_SCAN

  compilade

  committed
  62b09b3

  Copy full SHA for 62b09b3

Commits on Oct 12, 2024

• Merge branch 'master' into compilade/mamba2

  compilade

  committed
  038d958

  Copy full SHA for 038d958

• ggml : remove unused fast broadcast path in GGML_MUL

  Show description for 805512a
  compilade

  committed
  805512a

  Copy full SHA for 805512a

Commits on Nov 1, 2024

• Merge branch 'master' into compilade/mamba2

  compilade

  committed
  7d16e1b

  Copy full SHA for 7d16e1b

Commits on Nov 4, 2024

• ggml : avoid multiply by D in GGML_OP_SSM_SCAN

  Show description for 3bc7103
  compilade

  committed
  3bc7103

  Copy full SHA for 3bc7103

• Merge branch 'master' into compilade/mamba2

  compilade

  committed
  8d8f065

  Copy full SHA for 8d8f065

• convert : fix flake8 lint

  compilade

  committed
  b4e9c59

  Copy full SHA for b4e9c59

Commits on Nov 25, 2024

• Merge branch 'master' into compilade/mamba2

  compilade

  committed
  1ee6c48

  Copy full SHA for 1ee6c48

Commits on Feb 26, 2025

• Merge branch 'master' into compilade/mamba2

  compilade

  committed
  c9ecf62

  Copy full SHA for c9ecf62

Commits on May 1, 2025

• Merge branch 'master' into compilade/mamba2

  compilade

  committed
  35d06fa

  Copy full SHA for 35d06fa

• metal : fix confusion between ; and ,

  compilade

  committed
  cf4f0a4

  Copy full SHA for cf4f0a4

• metal : add missing args for nb references in ssm_scan_f32_group

  compilade

  committed
  6def5cd

  Copy full SHA for 6def5cd

Commits on May 2, 2025

• metal : single-user mamba2 inference works

  compilade

  committed
  791998b

  Copy full SHA for 791998b

• kv-cache : remove const_cast when setting inputs for s_copy

  Show description for 94c3d53
  compilade

  committed
  94c3d53

  Copy full SHA for 94c3d53

• Merge branch 'master' into compilade/mamba2

  compilade

  committed
  929fe85

  Copy full SHA for 929fe85

• convert : avoid AutoConfig for Mamba and Mamba2 hparams

  compilade

  committed
  d55b0d0

  Copy full SHA for d55b0d0

• kv-cache : allow context shift for recurrent models

  compilade

  committed
  e94f393

  Copy full SHA for e94f393

Commits on Jun 9, 2025

• feat: Add llama_model_is_hybrid API call

  Show description for b9e8f94
  gabe-l-hart

  committed
  b9e8f94

  Copy full SHA for b9e8f94

• feat: Add c++ side constants for attention layer indices hparam

  Show description for 897dce5
  gabe-l-hart

  committed
  897dce5

  Copy full SHA for 897dce5

• feat: Add support for distinguishing recurrent vs non-recurrent layers in hparams

  Show description for eb6c979
  gabe-l-hart

  committed
  eb6c979

  Copy full SHA for eb6c979

• feat: Auto-fill hparams.recurrent_layer_arr based on whether the model is recurrent

  Show description for ba362dc
  gabe-l-hart

  committed
  ba362dc

  Copy full SHA for ba362dc

• refactor: rename *_is_hybrid -> *_is_hybrid_recurrent

  Show description for a391645
  gabe-l-hart

  committed
  a391645

  Copy full SHA for a391645

• feat: Add layer filter to recurrent cache

  Show description for e00e525
  gabe-l-hart

  committed
  e00e525

  Copy full SHA for e00e525

• fix: Use per-layer sizing everywhere in kv caches

  Show description for f0cb485
  gabe-l-hart

  committed
  f0cb485

  Copy full SHA for f0cb485

• feat: First pass at llama_kv_cache_hybrid_recurrent

  Show description for 6a822b7
  gabe-l-hart

  committed
  6a822b7

  Copy full SHA for 6a822b7

• feat: Construct hybrid recurrent cache for hybrid recurrent models

  Show description for fe39803
  gabe-l-hart

  committed
  fe39803

  Copy full SHA for fe39803

• fix: Fix wrong bool condition for split equal in hybrid cache

  Show description for 8cb41a9
  gabe-l-hart

  committed
  8cb41a9

  Copy full SHA for 8cb41a9

• fix: Fix shift logic to defer to unified cache

  Show description for 980ae73
  gabe-l-hart

  committed
  980ae73

  Copy full SHA for 980ae73

• feat: Support hybrid recurrent in llama-graph

  Show description for 31be8ae
  gabe-l-hart

  committed
  31be8ae

  Copy full SHA for 31be8ae

• fix: Fix logic for initializing inputs and attn layers for hybrid caches

  Show description for 2e9e969
  gabe-l-hart

  committed
  2e9e969

  Copy full SHA for 2e9e969

• fix: Use @compilade's suggested fix for seq_id indexing with equal splits in unified cache

  Show description for b576579
  gabe-l-hart

  committed
  b576579

  Copy full SHA for b576579

• fix: Update recurrent cache for changes to remove intermediate kv_cache interface

  Show description for c84f607
  gabe-l-hart

  committed
  c84f607

  Copy full SHA for c84f607

• fix: Fix status for init_update sig for recurrent cache state

  Show description for ecaac6b
  gabe-l-hart

  committed
  ecaac6b

  Copy full SHA for ecaac6b

• fix: Add missing padding to n_ctx for hybrid cache construction

  Show description for f69d82c
  gabe-l-hart

  committed
  f69d82c

  Copy full SHA for f69d82c

• fix: Update clear signature for data argument after rebase

  Show description for 60ca3ba
  gabe-l-hart

  committed
  60ca3ba

  Copy full SHA for 60ca3ba

• tests: Add retry logic for all HTTP requests in server tests

  Show description for 39a93b3
  gabe-l-hart

  committed
  39a93b3

  Copy full SHA for 39a93b3

• Merge branch 'compilade/mamba2' into GraniteFour

  Show description for 1534e5a
  gabe-l-hart

  committed
  1534e5a

  Copy full SHA for 1534e5a

• feat: Add conversion for Bamba models

  Show description for 23765a3
  gabe-l-hart

  committed
  23765a3

  Copy full SHA for 23765a3

• feat: Add Granite 4 conversion

  Show description for ae15274
  gabe-l-hart

  committed
  ae15274

  Copy full SHA for ae15274

• feat: Plumb bamba through llama-arch

  Show description for 6115383
  gabe-l-hart

  committed
  6115383

  Copy full SHA for 6115383

• feat: Add bamba to llama_arch_is_hybrid_recurrent

  Show description for 795e8dc
  gabe-l-hart

  committed
  795e8dc

  Copy full SHA for 795e8dc

• feat: Add optional mamba ssm_in bias tensor

  Show description for 6a8a1aa
  gabe-l-hart

  committed
  6a8a1aa

  Copy full SHA for 6a8a1aa

• feat: Add template specialization for get_arr to load a vector<uint32_t> for layer index arr in hparams

  Show description for 02466ca
  gabe-l-hart

  committed
  02466ca

  Copy full SHA for 02466ca

• feat: Use an explicit bool to determine mamaba vs mamba2

  Show description for 3dcf0d4
  gabe-l-hart

  committed
  3dcf0d4

  Copy full SHA for 3dcf0d4

• feat: Isolate mamba(2) and granite attention layer building in static methods

  Show description for 0dae07e
  gabe-l-hart

  committed
  0dae07e

  Copy full SHA for 0dae07e

• fix: Use per-layer sizes in mamba layer builders

  Show description for 5f0e939
  gabe-l-hart

  committed
  5f0e939

  Copy full SHA for 5f0e939

• fix: Use per-layer sizes in granite build_attention_layer

  Show description for 0e6c71a
  gabe-l-hart

  committed
  0e6c71a

  Copy full SHA for 0e6c71a

• feat: First (broken) pass at end-to-end Bamba implementation

  Show description for 0fb198e
  gabe-l-hart

  committed
  0fb198e

  Copy full SHA for 0fb198e

• fix: Only do Granite multipliers if set

  Show description for 43a0d86
  gabe-l-hart

  committed
  43a0d86

  Copy full SHA for 43a0d86

• refactor: Pull granite ffn portion into a static function and reuse in hybrid

  Show description for 04c6ed4
  gabe-l-hart

  committed
  04c6ed4

  Copy full SHA for 04c6ed4

• feat(py): Allow gguf duplicate keys if they match by value and type

  Show description for dfe5c26
  gabe-l-hart

  committed
  dfe5c26

  Copy full SHA for dfe5c26

• refactor(py): Simplify granitemoehybrid conversion to use parents better

  Show description for f3bfc22
  gabe-l-hart

  committed
  f3bfc22

  Copy full SHA for f3bfc22

• feat: Add GRANITE_MOE_HYBRID through llama-arch

  Show description for 1d773bf
  gabe-l-hart

  committed
  1d773bf

  Copy full SHA for 1d773bf

• feat: Support GRANITE_MOE_HYBRID in llama-model

  Show description for 0a5b7ef
  gabe-l-hart

  committed
  0a5b7ef

  Copy full SHA for 0a5b7ef

• style: Fix flake8 errors

  Show description for 7abf92b
  gabe-l-hart

  committed
  7abf92b

  Copy full SHA for 7abf92b

• fix: Fix recurrent cache get after rebase

  Show description for 64d60bd
  gabe-l-hart

  committed
  64d60bd

  Copy full SHA for 64d60bd

• fix: Fix hybrid granite implementation for signature changes in build_mamba*_layer

  Show description for 48ebcec
  gabe-l-hart

  committed
  48ebcec

  Copy full SHA for 48ebcec

• fix: Fix the bad merge of kv-cache-recurrent for mamba2 changes

  Show description for 552a5ce
  gabe-l-hart

  committed
  552a5ce

  Copy full SHA for 552a5ce

• fix: Fix all models for kv state refactors

  Show description for 2834711
  gabe-l-hart

  committed
  2834711

  Copy full SHA for 2834711

• fix: Fix removal of LLM_KV_TOKENIZER_CHAT_TEMPLATE_N (0974ad7a) after rebase

  Show description for b44b5bd
  gabe-l-hart

  committed
  b44b5bd

  Copy full SHA for b44b5bd

• fix: Fix recurrent cache clear signature after rebase

  Show description for 1486804
  gabe-l-hart

  committed
  1486804

  Copy full SHA for 1486804

• fix: Add missing padding to n_ctx for hybrid cache construction

  Show description for 6b8d79b
  gabe-l-hart

  committed
  6b8d79b

  Copy full SHA for 6b8d79b