-
Notifications
You must be signed in to change notification settings - Fork 15
Open
Description
Currently, the Worker
struct is defined as follows:
struct Worker<'a> {
...
}
where 'a
is a lifetime parameter of the Tokenizer
. By this definition, the Worker
can refer to the Tokenizer
automatically for every tokenization.
This definition causes a problem when creating wrappers for other programming languages that use garbage collectors (GC).
The above definition means that the Tokenizer
cannot be removed when the Worker
is alive, but there is no way to impose this constraint on the GC.
To solve this problem, we need to remove the lifetime parameter and give the Worker
struct to the Tokenizer
for every tokenization.
kampersanda
Metadata
Metadata
Assignees
Labels
No labels