v0.1.11

What's Changed

• Swap in new embedding-chunked splitter algorithm by @brilee in https://github.com/lilacai/lilac/pull/791
• Add the language model reference concept to Lilac. by @nsthorat in https://github.com/lilacai/lilac/pull/797
• Improve ergonomics of map(). by @nsthorat in https://github.com/lilacai/lilac/pull/795

Other Changes

• Fix a flaky test by @dsmilkov in https://github.com/lilacai/lilac/pull/792
• Add pdb-compatible script and documentation by @brilee in https://github.com/lilacai/lilac/pull/794
• Switch splitter to new algorithm, fixing last blocking bug. by @brilee in https://github.com/lilacai/lilac/pull/796
• Add more examples to the language model reference. by @nsthorat in https://github.com/lilacai/lilac/pull/798

Full Changelog: https://github.com/lilacai/lilac/compare/v0.1.10...v0.1.11

v0.1.10

What's Changed

• Add the simplest dataset.map() by @nsthorat in https://github.com/lilacai/lilac/pull/769
• Add sampling to our ParquetSource by @dsmilkov in https://github.com/lilacai/lilac/pull/773
• Add intelligent sampling in ParquetSource by @dsmilkov in https://github.com/lilacai/lilac/pull/778
• Add include_labels and exclude_labels when exporting data via dataset.to_* by @dsmilkov in https://github.com/lilacai/lilac/pull/768
• Improve the Export Dataset modal dialog by @dsmilkov in https://github.com/lilacai/lilac/pull/775
• Allow searching by pre-computed concepts in the searchbox. by @nsthorat in https://github.com/lilacai/lilac/pull/783
• Add spacy embedding-clustering splitter by @brilee in https://github.com/lilacai/lilac/pull/784

Other Changes

• Add notebook exploring chunking algorithms by @brilee in https://github.com/lilacai/lilac/pull/771
• Remove TextSplitterSignal, fixing related tests. by @brilee in https://github.com/lilacai/lilac/pull/774
• Fix the concept labeler with the new preview concept key. by @nsthorat in https://github.com/lilacai/lilac/pull/780
• Improve export preview text. by @nsthorat in https://github.com/lilacai/lilac/pull/782
• Upgrade ruff version and fix associated new linter errors by @brilee in https://github.com/lilacai/lilac/pull/777

New Contributors

• @brilee made their first contribution in https://github.com/lilacai/lilac/pull/771

Full Changelog: https://github.com/lilacai/lilac/compare/v0.1.9...v0.1.10

v0.1.9

What's Changed

Other Changes

• Improve dataset load onboarding flow. by @nsthorat in https://github.com/lilacai/lilac/pull/764

Full Changelog: https://github.com/lilacai/lilac/compare/v0.1.8...v0.1.9

v0.1.8

What's Changed

• Improve labeling flow by @dsmilkov in https://github.com/lilacai/lilac/pull/763

Other Changes

• Tiny UI fixes by @dsmilkov in https://github.com/lilacai/lilac/pull/760
• Use HDBSCAN for the public demo. by @nsthorat in https://github.com/lilacai/lilac/pull/758
• Batch row metadata requests. by @nsthorat in https://github.com/lilacai/lilac/pull/762
• Skip cache upload when we dont upload data. by @nsthorat in https://github.com/lilacai/lilac/pull/765

Full Changelog: https://github.com/lilacai/lilac/compare/v0.1.7...v0.1.8

v0.1.7

What's Changed

Other Changes

• Fix port on HF demo since we use app_port: 5432 by @dsmilkov in https://github.com/lilacai/lilac/pull/759

Full Changelog: https://github.com/lilacai/lilac/compare/v0.1.6...v0.1.7

v0.1.6

Features

• Make labels toggle-able by @dsmilkov in https://github.com/lilacai/lilac/pull/748
• Add HDBScan with UMAP by @nsthorat in https://github.com/lilacai/lilac/pull/749
• Add lilac docker image and use a lighter 5x image for HF by @dsmilkov in https://github.com/lilacai/lilac/pull/750

Bug fixes / Other

• Fix duplicate stats requests by @dsmilkov in https://github.com/lilacai/lilac/pull/754
• Add CLI prompts for token in deploy script by @nsthorat in https://github.com/lilacai/lilac/pull/755
• Add docker deploy instructions in dev.md and multi-platform build by @dsmilkov in https://github.com/lilacai/lilac/pull/753

Full Changelog: https://github.com/lilacai/lilac/compare/v0.1.15...v0.1.6

v0.1.15

Features

• Improve markdown rendering by @dsmilkov in https://github.com/lilacai/lilac/pull/728
• Add "Group by" mode by @dsmilkov in https://github.com/lilacai/lilac/pull/735
• Add LILAC_AUTH_ADMIN_EMAILS to set admins. by @nsthorat in https://github.com/lilacai/lilac/pull/733
• Add LILAC_AUTH_USER_DISABLE_LABEL_ALL by @nsthorat in https://github.com/lilacai/lilac/pull/741
• Add allowing non-admin users to add labels, but not create label types. by @nsthorat in https://github.com/lilacai/lilac/pull/734
• Add a github source and a LlamaIndexDocsSource source. by @nsthorat in https://github.com/lilacai/lilac/pull/740

Bug fixes / other changes

• Fix bug with dataset config on upload. by @nsthorat in https://github.com/lilacai/lilac/pull/729
• Upgrade duckdb to 0.9 by @dsmilkov in https://github.com/lilacai/lilac/pull/731
• First prototype of RAG UI. by @nsthorat in https://github.com/lilacai/lilac/pull/732
• Extract the dropdown+pill into a DropPill component by @dsmilkov in https://github.com/lilacai/lilac/pull/742
• Make llama-index an optional dependency. by @nsthorat in https://github.com/lilacai/lilac/pull/744
• Bug fixes for deploy scripts. by @nsthorat in https://github.com/lilacai/lilac/pull/745

Full Changelog: https://github.com/lilacai/lilac/compare/v0.1.4...v0.1.15

v0.1.4

What's Changed

Other Changes

• Add --hf_token and dont require it when we have public datasets, or we set load on start server. by @nsthorat in https://github.com/lilacai/lilac/pull/727

Full Changelog: https://github.com/lilacai/lilac/compare/v0.1.3...v0.1.4

v0.1.3

You can now deploy Lilac to a HuggingFace dataset with just a few lines of Python, or from the CLI:

Deploy a single configuration object, and have it load entirely on the space:

ll.deploy_config(
  hf_space='nsthorat-lilac/nikhil-demo',
  # Create the space if it doesn't exist.
  create_space=True,
  config=ll.Config(datasets=[
    ll.DatasetConfig(
      namespace='local',
      name='glue_ax',
      source=ll.HuggingFaceSource(dataset_name='glue', config_name='ax'))
  ]))

Deploy a Lilac project you've loaded locally:

ll.deploy_project(
  hf_space='nsthorat-lilac/nikhil-project-demo',
  project_dir='./data',
  datasets=['local/glue_ax'],  # This is optional. If not defined, uploads all datasets.
  # Create the space if it doesn't exist.
  create_space=True)

Or via the CLI:

lilac deploy-project --project_dir='~/my_project'

Features

• Completely overhaul the deploy scripts. by @nsthorat in https://github.com/lilacai/lilac/pull/725

Other Changes

• Improve the schema view to be less overwhelming. by @nsthorat in https://github.com/lilacai/lilac/pull/726

Demo

• Add the textbook quality programming dataset to the demo. Clean up old datasets. by @nsthorat in https://github.com/lilacai/lilac/pull/723

Full Changelog: https://github.com/lilacai/lilac/compare/v0.1.2...v0.1.3

v0.1.2

Internal change

• Migrate the span values to use __span__ and make the code back-compat by @dsmilkov in https://github.com/lilacai/lilac/pull/718

Full Changelog: https://github.com/lilacai/lilac/compare/v0.1.1...v0.1.2