Skip to content

@bigscience-workshop BigScience Workshop

Change the repository type filter

All

    Repositories list

    36 repositories

    • data_tooling

      Public
      Tools for managing datasets for governance and training.
      HTML
      •
      Apache License 2.0
      •47•87•138•3•Updated Dec 8, 2025Dec 8, 2025

    • ShadesofBias

      Public
      Evaluation for Shades of Bias in Text
      HTML
      •1•8•1•0•Updated Apr 23, 2025Apr 23, 2025

    • biomedical

      Public
      Tools for curating biomedical training data for large-scale language modeling
      Python
      •118•487•163•15•Updated Dec 9, 2024Dec 9, 2024

    • xmtf

      Public
      Crosslingual Generalization through Multitask Finetuning
      bloomlanguage-modelszero-shot-learningmultitask-learningt5multilingual-nlplarge-language-modelsinstruction-tuningbloomzmt0
      Jupyter Notebook
      •
      Apache License 2.0
      •43•537•11•0•Updated Sep 22, 2024Sep 22, 2024

    • petals

      Public
      🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
      nlpbloomdistributed-systemsmachine-learningdeep-learningchatbotpytorchfalcontransformerneural-networks
      Python
      •
      MIT License
      •586•9.9k•92•19•Updated Sep 7, 2024Sep 7, 2024

    • bigscience

      Public
      Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.
      nlptrainingmachine-learningmodels
      Shell
      •
      Other
      •103•1k•13•7•Updated Jul 29, 2024Jul 29, 2024

    • Megatron-DeepSpeed

      Public
      Ongoing research training transformer language models at scale, including: BERT & GPT-2
      Python
      •
      Other
      •228•1.4k•76•47•Updated Mar 20, 2024Mar 20, 2024

    • multilingual-modeling

      Public
      BLOOM+1: Adapting BLOOM model to support a new unseen language
      Python
      •
      Apache License 2.0
      •17•74•13•6•Updated Mar 2, 2024Mar 2, 2024

    • promptsource

      Public
      Toolkit for creating, sharing and using natural language prompts.
      nlpmachine-learningnatural-language-processing
      Python
      •
      Apache License 2.0
      •377•3k•11•32•Updated Oct 23, 2023Oct 23, 2023

    • massive-probing-framework

      Public
      Framework for BLOOM probing
      Python
      •9•9•0•0•Updated Oct 17, 2023Oct 17, 2023

    • architecture-objective

      Public
      Python
      •
      Apache License 2.0
      •335•98•4•5•Updated Jul 25, 2023Jul 25, 2023

    • metadata

      Public
      Experiments on including metadata such as URLs, timestamps, website descriptions and HTML tags during pretraining.
      Python
      •
      Apache License 2.0
      •11•31•25•13•Updated Jun 12, 2023Jun 12, 2023

    • lm-evaluation-harness

      Public
      A framework for few-shot evaluation of autoregressive language models.
      Python
      •
      MIT License
      •2.9k•105•7•8•Updated May 9, 2023May 9, 2023

    • data-preparation

      Public
      Code used for sourcing and cleaning the BigScience ROOTS corpus
      multilingualdatasetlarge-language-models
      Jupyter Notebook
      •
      Apache License 2.0
      •42•317•10•0•Updated Mar 20, 2023Mar 20, 2023

    • bibliography

      Public
      A list of BigScience publications
      researchbibtexbibliography
      TeX
      •
      Apache License 2.0
      •1•3•1•0•Updated Mar 13, 2023Mar 13, 2023

    • multilingual-modeling-1

      Public
      Python
      •
      Apache License 2.0
      •17•2•0•0•Updated Dec 5, 2022Dec 5, 2022

    • t-zero

      Public
      Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)
      Python
      •
      Apache License 2.0
      •53•463•8•2•Updated Nov 5, 2022Nov 5, 2022

    • carbon-footprint

      Public
      A repository for `codecarbon` logs.
      Jupyter Notebook
      •5•13•2•0•Updated Nov 3, 2022Nov 3, 2022

    • pii_processing

      Public
      PII Processing code to detect and remediate PII in BigScience datasets. Reference implementation for the PII Hackathon
      Python
      •
      Other
      •6•9•7•1•Updated Oct 6, 2022Oct 6, 2022

    • lam

      Public
      Libraries, Archives and Museums (LAM)
      Apache License 2.0
      •7•88•34•0•Updated Oct 4, 2022Oct 4, 2022

    • model_card

      Public
      Apache License 2.0
      •4•25•0•0•Updated Jul 11, 2022Jul 11, 2022

    • bloom-dechonk

      Public
      A repo for running model shrinking experiments
      Python
      •4•10•0•0•Updated Jun 21, 2022Jun 21, 2022

    • historical_texts

      Public
      BigScience working group on language models for historical texts
      Jupyter Notebook
      •7•8•0•2•Updated May 10, 2022May 10, 2022

    • evaluation

      Public
      Code and Data for Evaluation WG
      Python
      •
      Other
      •24•42•41•9•Updated May 4, 2022May 4, 2022

    • catalogue_data

      Public
      Scripts to prepare catalogue data
      Jupyter Notebook
      •
      Apache License 2.0
      •1•8•5•3•Updated Apr 25, 2022Apr 25, 2022

    • training_dynamics

      Public
      4•5•11•2•Updated Apr 22, 2022Apr 22, 2022

    • evaluation-robustness-consistency

      Public
      Tools for evaluating model robustness and consistency
      Python
      •
      Other
      •2•2•0•2•Updated Mar 9, 2022Mar 9, 2022

    • interpretability-ideas

      Public
      1•1•10•0•Updated Feb 27, 2022Feb 27, 2022

    • tokenization

      Public
      Python
      •
      Apache License 2.0
      •2•11•1•2•Updated Feb 16, 2022Feb 16, 2022

    • datasets_stats

      Public
      Generate statistics over datasets used in the context of BS
      Makefile
      •1•2•0•0•Updated Feb 1, 2022Feb 1, 2022