Expected duration for Ollama example

[doccosmos]

How long should it take to run the Ollama example? I tried to run it this week and it took over 24 hours just producing lots of lines like:

[ Info: Tokens: 556 in 9.1 seconds

before finally crashing

[atharvas]

Thanks for checking out the package!

Each of that line signifies a query to the underlying LLM. The basic example works locally for me but I have a GPU accelerated backend which makes things faster. Let me reproduce this error over the weekend locally (on a Mac and Windows) and get back to you.

[atharvas]

Looked at this over the weekend and I found a couple of places where the code might be causing slowdowns. Also figured out what could be causing the crash. I've reimplemented a large chunk of the codebase and I'm getting reasonable speedups now (~1e3 programs a second). The PR is not completely finished yet. I want to add some safety rails so this issue doesn't happen and I want to clean up the code / formatting before merging.

I'll keep you updated once I have a stable-r release. Sorry about that!

[atharvas]

@doccosmos Try the ollama example after running ] update LibraryAugmentedSymbolicRegression. It should now display a progress bar with the expected completion time. I'd recommend starting off with the LLM operation probability really low (e.g: p=1e-4) and increasing it depending on resource availability.

using SymbolicRegression: SRLogger
using TensorBoardLogger
import LibraryAugmentedSymbolicRegression: LaSRRegressor, LLMOptions, LLMOperationWeights
import MLJ: machine, fit!, predict, report

# Dataset with 5 features:
X = randn(Float64, 5, 100)

# and one target:
y = 2 * cos.(X[4, :]) + X[1, :] .^ 2 .- 2

# with some noise:
y = y .+ randn(100) .* 1e-3

# Adding a logger so we can see LaSR's progress.
# Run $ pip install tensorboard
# and then $ tensorboard --logdir .
logger = SRLogger(TBLogger("logs/lasr_runs"); log_interval=1)

p = 1e-4 # Reduce this even further if the model takes too long...
model = LaSRRegressor(;
    niterations=40,
    logger=logger,
    binary_operators=[+, -, *, /, ^],
    unary_operators=[cos],
    populations=20,
    use_llm=true,
    use_concepts=true,
    use_concept_evolution=true,
    llm_operation_weights=LLMOperationWeights(;
        llm_crossover=p, llm_mutate=p, llm_randomize=p
    ),
    llm_context="We believe the relationship between the theta and offset parameter is a function of the cosine of the theta variable and the square of the offset.",
    variable_names=Dict("x1" => "theta", "x2" => "offset"),
    prompts_dir="prompts/",
    api_key="token-abc123",
    model="llama3.1:latest",
    api_kwargs=Dict("url" => "http://localhost:11434/v1"),
    verbose=true, # Set to true to see LLM generation logs.
)

mach = machine(model, transpose(X), y)
fit!(mach)
# julia> fit!(mach)
# [ Info: Training machine(LaSRRegressor(binary_operators = Function[+, -, *], …), …).
# ┌ Warning: You are using multithreading mode, but only one thread is available. Try starting julia with `--threads=auto`.
# └ @ LibraryAugmentedSymbolicRegression ~/Desktop/projects/004_scientific_discovery/LibraryAugmentedSymbolicRegression.jl/src/Configure.jl:55
# [ Info: Tokens: 476 in 22.4 seconds
# [ Info: Started!
# [ Info: Tokens: 542 in 49.2 seconds
# [ Info: Tokens: 556 in 51.1 seconds
# [ Info: Tokens: 573 in 53.2 seconds
Evolving for 40 iterations...  28%|███████████████████                                                  |  ETA: 0:09:59
report(mach)
predict(mach, transpose(X))