Skip to content

Actions: alea-institute/kl3m-tokenizers

Actions

All workflows

Actions

Loading...
Loading

Showing runs from all workflows
15 workflow runs
15 workflow runs

Filter by Event

Filter by Status

Filter by Branch

Filter by Actor

updated with pruning args CI #15: Commit c0ce8be pushed by mjbommar
April 24, 2025 20:04 4m 47s main
main
April 24, 2025 20:04 4m 47s
implementing cheesecloth entropy filtering and adding system1/2 speci… CI #14: Commit 16874b0 pushed by mjbommar
April 24, 2025 11:40 4m 54s main
main
April 24, 2025 11:40 4m 54s
updated example with support for local and remote hf datasets CI #13: Commit 83bbf6a pushed by mjbommar
March 23, 2025 17:00 3m 38s main
main
March 23, 2025 17:00 3m 38s
batched decode for config example CI #12: Commit de96823 pushed by mjbommar
March 23, 2025 00:28 3m 31s main
main
March 23, 2025 00:28 3m 31s
example with config file CI #11: Commit bc134f1 pushed by mjbommar
March 23, 2025 00:20 3m 44s main
main
March 23, 2025 00:20 3m 44s
Updated with support for random whitespace/chunk tokenizer research CI #10: Commit 308a35f pushed by mjbommar
March 22, 2025 20:47 3m 19s main
main
March 22, 2025 20:47 3m 19s
cased mlm tokenizer for sake of complete reproduction CI #9: Commit 2e26c66 pushed by mjbommar
January 27, 2025 16:59 1m 10s main
main
January 27, 2025 16:59 1m 10s
fixing char tokenizer example CI #8: Commit d18cc39 pushed by mjbommar
December 31, 2024 15:20 3m 28s main
main
December 31, 2024 15:20 3m 28s
updated char comparison sorting and readme CI #7: Commit 511e9ce pushed by mjbommar
December 31, 2024 15:20 3m 24s main
main
December 31, 2024 15:20 3m 24s
adding character tokenizer training source and tokenizers CI #6: Commit 0d8134f pushed by mjbommar
December 31, 2024 14:33 3m 22s main
main
December 31, 2024 14:33 3m 22s
added mlm version of 128k-uncased CI #5: Commit 02f1740 pushed by mjbommar
November 20, 2024 22:06 48s main
main
November 20, 2024 22:06 48s
updated after hf hub sync CI #4: Commit e772ec2 pushed by mjbommar
November 10, 2024 15:11 53s main
main
November 10, 2024 15:11 53s
adding kl3m 004-128k uncased tokenizer and updated docs CI #3: Commit fd6347c pushed by mjbommar
November 7, 2024 15:00 48s main
main
November 7, 2024 15:00 48s
added hf links in readme CI #2: Commit 58c3212 pushed by mjbommar
October 13, 2024 10:49 47s main
main
October 13, 2024 10:49 47s
initial commit after refactoring from kl3m-embedding-research repo CI #1: Commit 5935119 pushed by mjbommar
October 12, 2024 13:25 54s main
main
October 12, 2024 13:25 54s