Search results

Is there a way to get vocabulary size of an initialized tokenizer?

Status: Open.

#47 In dmitry-brazhenko/SharpToken;

· zhuzihan728 opened on Apr 21, 2025

add support for utf8 input and output

Status: Open.

#46 In dmitry-brazhenko/SharpToken;

· abdulkareemnalband opened on Sep 13, 2024

SharpToken.GptEncoding.GetEncoding(o200k_base) takes minutes

Status: Open.

#45 In dmitry-brazhenko/SharpToken;

· vovanb opened on Jul 25, 2024

Gbt4o support

Status: Open.

#42 In dmitry-brazhenko/SharpToken;

· daniell0gda opened on May 16, 2024

cohere support

Status: Open.

#38 In dmitry-brazhenko/SharpToken;

· omri-suissa-clearmash opened on Apr 17, 2024

Anthropic (claude) support

Status: Open.

#32 In dmitry-brazhenko/SharpToken;

· omri-suissa-clearmash opened on Mar 14, 2024