From 1304906d5c96544fb79438943c35530e1bbebe8c Mon Sep 17 00:00:00 2001 From: Radek Osmulski Date: Fri, 17 Jan 2025 07:37:21 +1000 Subject: [PATCH 1/5] add llms.txt --- nbs/llms.txt | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 nbs/llms.txt diff --git a/nbs/llms.txt b/nbs/llms.txt new file mode 100644 index 00000000..02670316 --- /dev/null +++ b/nbs/llms.txt @@ -0,0 +1,39 @@ +# nbdev + +nbdev makes exploration an integral part of your workflow, all while promoting software engineering best practices. It allows you to write, test, document, and distribute software packages and technical articles — all in one place, your notebook. + +Here are some of its features: + +- **Supports literate programming**: write prose, code, and tests in notebooks — no context-switching +- **Git-friendly notebooks**: human-readable merge conflicts; no unwanted metadata +- **Two-way sync between notebooks and plaintext source code**: allows you to use your IDE for code navigation or quick edits +- **Beautiful technical documentation**: documentation is automatically generated using Quarto and hosted on GitHub Pages; docs support LaTeX, are searchable, and are automatically hyperlinked (including out-of-the-box support for many packages via nbdev-index) +- **Seamless testing**: write tests as ordinary notebook cells, run them in parallel with a single command; out-of-the-box continuous integration with GitHub Actions to ensure tests pass before every merge +- **Simplified package release**: publish packages to PyPI and conda; Python best practices are automatically followed (e.g. only exported objects are included in `__all__`) + +## Docs + +- [Getting started](https://nbdev.fast.ai/getting_started.html) +- [Tutorial](https://nbdev.fast.ai/tutorial.html) +- [API details for each submodule](https://nbdev.fast.ai/api) + +## Optional + +- [nbdev.config](https://nbdev.fast.ai/api/config.html): Configuring nbdev and bootstrapping notebook export +- [nbdev.maker](https://nbdev.fast.ai/api/maker.html): Create one or more modules from selected notebook cells +- [nbdev.process](https://nbdev.fast.ai/api/process.html): A notebook processor +- [nbdev.export](https://nbdev.fast.ai/api/export.html): Exporting a notebook to a library +- [nbdev.doclinks](https://nbdev.fast.ai/api/doclinks.html): Generating a documentation index from a module +- [nbdev.sync](https://nbdev.fast.ai/api/sync.html): Propagate small changes in the library back to notebooks +- [nbdev.merge](https://nbdev.fast.ai/api/merge.html): Fix merge conflicts in jupyter notebooks +- [nbdev.showdoc](https://nbdev.fast.ai/api/showdoc.html): Display symbol documentation in notebook and website +- [nbdev.frontmatter](https://nbdev.fast.ai/api/frontmatter.html): A YAML and formatted-markdown frontmatter processor +- [nbdev.processor](https://nbdev.fast.ai/api/processors.html): Some processors for NBProcessor +- [nbdev.clean](https://nbdev.fast.ai/api/clean.html): Strip superfluous metadata from notebooks +- [nbdev.test](https://nbdev.fast.ai/api/test.html): Run unit tests on notebooks in parallel +- [nbdev.cli](https://nbdev.fast.ai/api/cli.html): CLI commands +- [nbdev.quarto](https://nbdev.fast.ai/api/quarto.html): Install and interact with Quarto from nbdev +- [nbdev.qmd](https://nbdev.fast.ai/api/qmd.html): Basic qmd generation helpers (experimental) +- [nbdev.migrate](https://nbdev.fast.ai/api/migrate.html): Utilities for migrating to nbdev +- [nbdev.serve](https://nbdev.fast.ai/api/serve.html): A parallel ipynb processor (experimental) +- [nbdev.release](https://nbdev.fast.ai/api/release.html): Auto-generated tagged releases and release notes from GitHub issues \ No newline at end of file From 182bc405d0d21c8bc109e6c4ce8f96092ef01274 Mon Sep 17 00:00:00 2001 From: Radek Osmulski Date: Fri, 17 Jan 2025 08:04:37 +1000 Subject: [PATCH 2/5] make llms.txt more complete and align with format --- nbs/llms.txt | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/nbs/llms.txt b/nbs/llms.txt index 02670316..a67f60cc 100644 --- a/nbs/llms.txt +++ b/nbs/llms.txt @@ -2,7 +2,7 @@ nbdev makes exploration an integral part of your workflow, all while promoting software engineering best practices. It allows you to write, test, document, and distribute software packages and technical articles — all in one place, your notebook. -Here are some of its features: +Here are some of nbdev's features: - **Supports literate programming**: write prose, code, and tests in notebooks — no context-switching - **Git-friendly notebooks**: human-readable merge conflicts; no unwanted metadata @@ -13,12 +13,19 @@ Here are some of its features: ## Docs -- [Getting started](https://nbdev.fast.ai/getting_started.html) -- [Tutorial](https://nbdev.fast.ai/tutorial.html) -- [API details for each submodule](https://nbdev.fast.ai/api) +- [Getting started](https://nbdev.fast.ai/getting_started.html.md) +- [Tutorial](https://nbdev.fast.ai/tutorials/tutorial.html.md) +- [Best practices](https://nbdev.fast.ai/tutorials/best_practices.html.md) ## Optional +- [Git-Friendly Jupyter](https://nbdev.fast.ai/tutorials/git_friendly_jupyter.html.md): How to use nbdev hooks for git-friendly Jupyter notebooks +- [Blogging](https://nbdev.fast.ai/tutorials/blogging.html.md: Creating a blog with notebooks) +- [Pre-Commit Hooks](https://nbdev.fast.ai/tutorials/pre_commit.html.md): How to use nbdev’s git pre-commit hooks +- [Documentation Only Sites](https://nbdev.fast.ai/tutorials/docs_only.html.md): How to create nbdev powered docs without a library! +- [Modular nbdev](https://nbdev.fast.ai/tutorials/modular_nbdev.html.md): How to use nbdev’s various tools separately +- [Writing nbdev plugins](https://nbdev.fast.ai/tutorials/extensions.html.md): How to customize nbdev processors to do what you want +- [nbdev1 Migration](https://nbdev.fast.ai/tutorials/migrating.html.md): How to change your nbdev1 repo to work with nbdev2 - [nbdev.config](https://nbdev.fast.ai/api/config.html): Configuring nbdev and bootstrapping notebook export - [nbdev.maker](https://nbdev.fast.ai/api/maker.html): Create one or more modules from selected notebook cells - [nbdev.process](https://nbdev.fast.ai/api/process.html): A notebook processor From d7ddb579dc9cf416f5da161c312f2e6ca0fde664 Mon Sep 17 00:00:00 2001 From: Radek Osmulski Date: Fri, 17 Jan 2025 08:12:52 +1000 Subject: [PATCH 3/5] reference only .md files and generate ctx --- nbs/llms-ctx-full.txt | 9415 +++++++++++++++++++++++++++++++++++++++++ nbs/llms-ctx.txt | 1986 +++++++++ nbs/llms.txt | 36 +- 3 files changed, 11419 insertions(+), 18 deletions(-) create mode 100644 nbs/llms-ctx-full.txt create mode 100644 nbs/llms-ctx.txt diff --git a/nbs/llms-ctx-full.txt b/nbs/llms-ctx-full.txt new file mode 100644 index 00000000..b8d5cbee --- /dev/null +++ b/nbs/llms-ctx-full.txt @@ -0,0 +1,9415 @@ +nbdev makes exploration an integral part of your workflow, all while promoting software engineering best practices. It allows you to write, test, document, and distribute software packages and technical articles — all in one place, your notebook. + +Here are some of nbdev's features: + +- **Supports literate programming**: write prose, code, and tests in notebooks — no context-switching +- **Git-friendly notebooks**: human-readable merge conflicts; no unwanted metadata +- **Two-way sync between notebooks and plaintext source code**: allows you to use your IDE for code navigation or quick edits +- **Beautiful technical documentation**: documentation is automatically generated using Quarto and hosted on GitHub Pages; docs support LaTeX, are searchable, and are automatically hyperlinked (including out-of-the-box support for many packages via nbdev-index) +- **Seamless testing**: write tests as ordinary notebook cells, run them in parallel with a single command; out-of-the-box continuous integration with GitHub Actions to ensure tests pass before every merge +- **Simplified package release**: publish packages to PyPI and conda; Python best practices are automatically followed (e.g. only exported objects are included in `__all__`)# Getting Started + + + +
+ + +
+ +`nbdev` is a notebook-driven development platform. Simply write +notebooks with lightweight markup and get high-quality documentation, +tests, continuous integration, and packaging for free! + +`nbdev` makes debugging and refactoring your code much easier than in +traditional programming environments since you always have live objects +at your fingertips. `nbdev` also promotes software engineering best +practices because tests and documentation are first class. + +- **Documentation** is automatically generated using + [Quarto](https://quarto.org/) and hosted on [GitHub + Pages](https://pages.github.com/). Docs support LaTeX, are searchable, + and are automatically hyperlinked (including out-of-the-box support + for many packages via + [`nbdev-index`](https://github.com/fastai/nbdev-index)) +- **Publish packages to PyPI and conda** as well as tools to simplify + package releases. Python best practices are automatically followed, + for example, only exported objects are included in `__all__` +- **Two-way sync between notebooks and plaintext source code** allowing + you to use your IDE for code navigation or quick edits +- **Tests** written as ordinary notebook cells are run in parallel with + a single command +- **Continuous integration** out-of-the-box with [GitHub + Actions](https://github.com/features/actions) that run your tests and + rebuild your docs +- **Git-friendly notebooks** with [Jupyter/Git + hooks](https://nbdev.fast.ai/tutorials/git_friendly_jupyter.html) that + clean unwanted metadata and render merge conflicts in a human-readable + format +- … and much more! + +## Install + +nbdev works on macOS, Linux, and most Unix-style operating systems. It +works on Windows under WSL, but not under cmd or Powershell. + +You can install nbdev with pip: + +``` sh +pip install nbdev +``` + +… or with conda (or mamba): + +``` sh +conda install -c fastai nbdev +``` + +Note that `nbdev` must be installed into the same Python environment +that you use for both Jupyter and your project. + +## How to use nbdev + +The best way to learn how to use nbdev is to complete either the +[written walkthrough](https://nbdev.fast.ai/tutorials/tutorial.html) or +video walkthrough: + + + +Alternatively, there’s a [shortened version of the video +walkthrough](https://youtu.be/67FdzLSt4aA) with coding sections sped up +using the `unsilence` Python library – it’s 27 minutes faster, but a bit +harder to follow. + +You can also run `nbdev_help` from the terminal to see the full list of +available commands: + +``` python +!nbdev_help +``` + + nbdev_bump_version Increment version in settings.ini by one + nbdev_changelog Create a CHANGELOG.md file from closed and labeled GitHub issues + nbdev_clean Clean all notebooks in `fname` to avoid merge conflicts + nbdev_conda Create a `meta.yaml` file ready to be built into a package, and optionally build and upload it + nbdev_create_config Create a config file. + nbdev_docs Create Quarto docs and README.md + nbdev_export Export notebooks in `path` to Python modules + nbdev_filter A notebook filter for Quarto + nbdev_fix Create working notebook from conflicted notebook `nbname` + nbdev_help Show help for all console scripts + nbdev_install Install Quarto and the current library + nbdev_install_hooks Install Jupyter and git hooks to automatically clean, trust, and fix merge conflicts in notebooks + nbdev_install_quarto Install latest Quarto on macOS or Linux, prints instructions for Windows + nbdev_merge Git merge driver for notebooks + nbdev_migrate Convert all markdown and notebook files in `path` from v1 to v2 + nbdev_new Create an nbdev project. + nbdev_prepare Export, test, and clean notebooks, and render README if needed + nbdev_preview Preview docs locally + nbdev_proc_nbs Process notebooks in `path` for docs rendering + nbdev_pypi Create and upload Python package to PyPI + nbdev_readme Create README.md from readme_nb (index.ipynb by default) + nbdev_release_both Release both conda and PyPI packages + nbdev_release_gh Calls `nbdev_changelog`, lets you edit the result, then pushes to git and calls `nbdev_release_git` + nbdev_release_git Tag and create a release in GitHub for the current version + nbdev_requirements Writes a `requirements.txt` file to `directory` based on settings.ini. + nbdev_sidebar Create sidebar.yml + nbdev_test Test in parallel notebooks matching `path`, passing along `flags` + nbdev_trust Trust notebooks matching `fname` + nbdev_update Propagate change in modules matching `fname` to notebooks that created them + nbdev_update_license Allows you to update the license of your project. + +## FAQ + +### Q: What is the warning “Found a cell containing mix of imports and computations. Please use separate cells”? + +A: You should not have cells that are not exported, *and* contain a mix +of `import` statements along with other code. For instance, don’t do +this in a single cell: + +``` python +import some_module +some_module.something() +``` + +Instead, split this into two cells, one which does `import some_module`, +and the other which does `some_module.something()`. + +The reason for this is that when we create your documentation website, +we ensure that all of the signatures for functions you document are up +to date, by running the imports, exported cells, and +[`show_doc`](https://nbdev.fast.ai/api/showdoc.html#show_doc) functions +in your notebooks. When you mix imports with other code, that other code +will be run too, which can cause errors (or at least slowdowns) when +creating your website. + +### Q: Why is nbdev asking for root access? How do I install Quarto without root access? + +A: When you setup your first project, nbdev will attempt to +automatically download and install [Quarto](https://quarto.org/) for +you. This is the program that we use to create your documentation +website. + +Quarto’s standard installation process requires root access, and nbdev +will therefore ask for your root password during installation. For most +people, this will work fine and everything will be handled automatically +– if so, you can skip over the rest of this section, which talks about +installing without root access. + +If you need to install Quarto without root access on Linux, first `cd` +to wherever you want to store it, then [download +Quarto](https://quarto.org/docs/get-started/), and type: + +``` bash +dpkg -x quarto*.deb . +mv opt/quarto ./ +rmdir opt +mkdir -p ~/.local/bin +ln -s "$(pwd)"/quarto/bin/quarto ~/.local/bin +``` + +To use this non-root version of Quarto, you’ll need `~/.local/bin` in +your [`PATH` environment +variable](https://linuxize.com/post/how-to-add-directory-to-path-in-linux/). +(Alternatively, change the `ln -s` step to place the symlink somewhere +else in your path.) + +### Q: Someone told me not to use notebooks for “serious” software development! + +A: [Watch this video](https://youtu.be/9Q6sLbz37gk). Don’t worry, we +still get this too, despite having used `nbdev` for a wide range of +“very serious” software projects over the last three years, including +[deep learning libraries](https://github.com/fastai/fastai), [API +clients](https://github.com/fastai/ghapi), [Python language +extensions](https://github.com/fastai/fastcore), [terminal user +interfaces](https://github.com/nat/ghtop), and more! + +## Contributing + +If you want to contribute to `nbdev`, be sure to review the +[contributions +guidelines](https://github.com/fastai/nbdev/blob/master/CONTRIBUTING.md). +This project adheres to fastai’s [code of +conduct](https://github.com/fastai/nbdev/blob/master/CODE_OF_CONDUCT.md). +By participating, you are expected to uphold this code. In general, we +strive to abide by generally accepted best practices in open-source +software development. + +Make sure you have `nbdev`’s git hooks installed by running +[`nbdev_install_hooks`](https://nbdev.fast.ai/api/clean.html#nbdev_install_hooks) +in the cloned repository. + +## Copyright + +Copyright © 2019 onward fast.ai, Inc. Licensed under the Apache License, +Version 2.0 (the “License”); you may not use this project’s files except +in compliance with the License. A copy of the License is provided in the +LICENSE file in this repository.# End-To-End Walkthrough + + + +The written tutorial below shows you how to create a Python package from +scratch using nbdev. + +Alternatively, you can watch this video tutorial where Jeremy Howard and +Hamel Husain guide you through a similar process step by step: + +
+ + + +
+ +## Installation + +You’ll need the following software to complete the tutorial, read on for +specific installation instructions: + +1. Python +2. A Python package manager: we recommend conda or pip +3. Jupyter Notebook +4. nbdev +5. Quarto + +If you haven’t worked with Python before, we recommend getting started +with the [Anaconda Individual +Edition](https://www.anaconda.com/products/individual) and using the +conda package manager. + +Note that you will only need to follow the steps in the installation +section once per environment. If you create a new repo, you won’t need +to redo these. + +### Install JupyterLab + +Launch a terminal and install JupyterLab by entering: + +``` sh +conda install -c conda-forge -y jupyterlab +``` + +…or + +``` sh +pip install jupyterlab +``` + +…if you’re using the pip package manager. + +You can now launch Jupyter by entering: + +``` sh +jupyter lab +``` + +This should open JupyterLab in a new browser tab: + + + +### Install nbdev + +The next step is to install nbdev itself. JupyterLab comes with its own +terminal, so we’ll use that moving forward. + +In the Launcher, scroll down to the “Other” section, then click +“Terminal”. If the Launcher isn’t opened, you can open it by clicking +“File” → “New Launcher”. + +A new tab should open with a blank terminal – it might not look exactly +the same, depending on how your shell is configured: + + + +For Mac and Linux, enter: + +``` sh +conda install -c fastai -y nbdev +``` + +…or for Mac, Linux and Windows: + +``` sh +pip install nbdev +``` + +…if you’re using pip. + +### Install Quarto + +nbdev provides a command to install the latest version of Quarto. In the +terminal, enter: + +``` sh +nbdev_install_quarto +``` + +Your password may be requested at this point. Since nbdev is open +source, you can read [the source +code](https://github.com/fastai/nbdev/blob/master/nbdev/quarto.py) of +this command to verify that it isn’t doing anything malicious. Or, if +you prefer, you may instead follow [Quarto’s official installation +instructions](https://quarto.org/docs/get-started/). + +### Install Quarto JupyterLab extension + +Quarto provides its own [JupyterLab +extension](https://quarto.org/docs/tools/jupyter-lab-extension.html) +that allows it to render Quarto markdown content. + +For example, here is their notebook demonstrating some of its features: + +![](https://user-images.githubusercontent.com/261654/230087634-d5027ebc-8508-43b4-81c9-c4b7d6cfa738.png) + +Install the extension by entering: + +``` sh +pip install jupyterlab-quarto +``` + +Note that the `jupyterlab-quarto` package is not currently available via +conda. + +------------------------------------------------------------------------ + +You’re all setup and ready to go! Installing these tools may take some +time, but you’ll only need to do it once. Next, we’ll setup an nbdev +repo for your specific project. + +## First steps + +By the end of this section you’ll have your own nbdev repo with tests, +continuous integration, streamlined PyPI & conda packaging, and a +documentation website. + +### Create an empty GitHub repo + +Create an empty GitHub repo using the convenient link +[github.com/new](https://github.com/new). If you get stuck, you might +find GitHub’s [*Create a +repo*](https://docs.github.com/en/get-started/quickstart/create-a-repo) +page helpful. + +Remember to add a description, since nbdev will use that later. Don’t +add a README file, .gitignore, or license just yet. + +If you’re using the web interface, it should look something like this +(with your own repository name and description) before you click “Create +Repository”: + + + +You should then be redirected to your new repo: + + + +
+ +> **Try GitHub’s powerful CLI** +> +> GitHub’s web interface is a great way to get started. As you grow more +> experienced, you might want to explore [the GitHub +> CLI](https://github.com/cli/cli) (command line interface). We often +> prefer to use command line tools for repetitive tasks where we’re +> likely to make mistakes. Having those tasks written as small scripts +> in your terminal means that you can repeat them with little effort. + +
+ +### Initialise your repo with nbdev + +Now clone your repo from the Jupyter terminal you started +[earlier](#install-nbdev) (or create a new terminal following those +instructions if needed). If you get stuck here, you might find GitHub’s +[*Cloning a +repository*](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository) +page helpful. + +Since we created a repo named `nbdev-hello-world` with the `fastai` +user, we can clone it as follows: + +``` sh +git clone https://github.com/fastai/nbdev-hello-world.git +``` + +Then `cd` (change directory) to our repo: + +``` sh +cd nbdev-hello-world +``` + +You may have seen this message while cloning: + + You appear to have cloned an empty repository. + +…since the repo is completely empty. Let’s add some files! + +nbdev provides the +[`nbdev_new`](https://nbdev.fast.ai/api/cli.html#nbdev_new) command to +initialise an empty git repository. It’ll infer information about your +project from git and GitHub, and ask you to input anything remaining. It +will create files in your repo that: + +- Streamline publishing Python packages to PyPI and conda. +- Configure Quarto for publication-grade technical documentation. +- Setup GitHub actions to test notebooks and build and deploy Quarto + docs to GitHub pages. + +Initialise your nbdev repo by entering: + +``` sh +nbdev_new +``` + +It may ask you to enter information that it couldn’t infer from git or +GitHub. + +
+ +> **Note** +> +> [`nbdev_new`](https://nbdev.fast.ai/api/cli.html#nbdev_new) assumes +> that your package name is the same as your repo name (with `-` +> replaced by `_`). Use the `--lib_name` option if that isn’t the case. + +
+ +Double-check your `settings.ini` file to ensure that it has all of the +correct information. Then commit and push your additions to GitHub: + +``` sh +git add . +git commit -m'Initial commit' +git push +``` + +### Enable GitHub Pages + +nbdev hosts your docs on GitHub Pages—an excellent (and free!) way to +host websites. + +
+ +> **Note** +> +> nbdev uses GitHub Pages by default because its easily accessible. +> However, you can use any host you like. See [these +> docs](../explanations/docs.ipynb#deploying-your-docs-on-other-platforms) +> for more information. + +
+ +You need to enable GitHub Pages for your repo by clicking on the +“Settings” tab near the top-right of your repo page, then “Pages” on the +left, then setting the “Branch” to “gh-pages”, and finally clicking +“Save”. + +It should look similar to this after you click “Save”: + + + +If you don’t see a “gh-pages” branch, wait a few minutes and reload the +page. It should automatically be set up for you. + +Now it’s time to see all of the goodies nbdev gives you! + +### Check out your workflows + +Open GitHub Actions by clicking the “Actions” tab near the top of your +repo page. You should see two workflow runs: + + + +If you opened this page shortly after pushing your initial commit, the +runs may not have a green check (✅) because they’re still “In progress” +or “Queued”. That’s no problem, they shouldn’t take much more than a +minute to complete. + +If you see a red cross (❌), that means something failed. Click on the +cross, then click “Details”, and you’ll be able to see what failed. If +you can’t figure out what’s wrong, search [the +forum](https://forums.fast.ai/c/nbdev/48) in case someone else resolved +the same issue, otherwise create a new post describing your issue in as +much detail as you can, and we’ll try our best to help you. Remember +that including a link to an actual repo and/or GitHub Action is the best +way for us to quickly identify what’s wrong. + +What do these workflows do? + +- **CI** – The CI (continuous integration) workflow streamlines your + developer workflow, particularly with multiple collaborators. Every + time you push to GitHub, it ensures that: + - Your notebooks and libraries are in sync + - Your notebooks are cleaned of unwanted metadata (which pollute pull + requests and git histories and lead to merge conflicts) + - Your notebook tests all pass +- **Deploy to GitHub Pages** – Builds your docs with Quarto and deploys + it to GitHub Pages. + +We provide these basic workflows out-of-the-box, however, you can edit +their corresponding YAML files in the `.github/workflows/` folder to +your liking. + +### Check out your docs + +When you [enable GitHub Pages](#enable-github-pages) you should see a +new workflow run: “pages build and deployment”. As the name suggests, +this workflow deploys your website contents to GitHub Pages. + + + +Wait for the workflow run to complete, then open your website. By +default it should be available at: `https://{user}.github.io/{repo}`. +For example, you can view `fastai`’s `nbdev-hello-world` docs at +. + + + +### Recap + +You now have a base nbdev repo with continuous integration and hosted +documentation! Here’s a recap of the steps you took: + +- Created a GitHub repo (with GitHub Pages enabled) +- Initialised your repo with + [`nbdev_new`](https://nbdev.fast.ai/api/cli.html#nbdev_new) +- Pushed to GitHub. + +## Make your first edit + +In this section, you’ll make your first edit to the repo you created in +[*First steps*](#first-steps). + +### Install hooks for git-friendly notebooks + +Step one when working with Jupyter notebooks in a new repo is to install +nbdev’s hooks (you can think of “hooks” as plugins or extensions to an +application). + +Install them by entering this command in your terminal: + +``` sh +nbdev_install_hooks +``` + +
+ +> **Note** +> +> The [clean hook](#nbdev_clean-on-saving-notebooks-in-jupyter) +> currently only supports Jupyter Notebook and JupyterLab. If you’re +> using VSCode, you can try the [experimental nbdev VSCode +> extension](https://github.com/fastai/nbdev-vscode). Otherwise, you +> might also want to try [nbdev’s pre-commit +> hooks](../tutorials/pre_commit.ipynb). + +
+ +See [*Git-friendly Jupyter*](git_friendly_jupyter.html) for more about +how nbdev hooks work and how to customise them. Here’s a short summary: + +- Fix broken notebooks due to git merge conflicts so that they can be + opened and resolved directly in Jupyter. +- Each time you save a Jupyter notebook, automatically clean unwanted + metadata to remove unnecessary changes in pull requests and reduce the + chance of git merge conflicts. +- Automatically trust notebooks in the repo so that you can view widgets + from collaborators’ commits. For this reason, **you should not install + hooks into a repo you don’t trust**. + +
+ +> **Tip** +> +> nbdev’s git hooks work on *any* git repo, even if it doesn’t use the +> broader nbdev system. + +
+ +### Build your library + +You should now create your package from your notebook by running: + +``` sh +nbdev_export +``` + +This will create Python modules for your notebooks. These modules will +make up the contents of your Python package. + +### Install your package + +You might have noticed that +[`nbdev_new`](https://nbdev.fast.ai/api/cli.html#nbdev_new) created a +Python package in your repo. In our case, it was automatically named +`nbdev_hello_world` by using our repo name `nbdev-hello-world` and +replacing `-` with `_` to make it a valid Python package. + +The next step is to install your package by entering this into your +terminal: + +``` sh +pip install -e '.[dev]' +``` + +This is the recommended way to make a Python package importable from +anywhere in your current environment: + +- `-e` – short for “editable”, lets you immediately use changes made to + your package without having to reinstall, which is convenient for + development. +- `.` – refers to the current directory. +- `[dev]` – includes “development” requirements: other packages that + your notebooks use solely for documentation or testing. + +### Preview your docs + +nbdev is an interactive programming environment that values fast +feedback loops. The +[`nbdev_preview`](https://nbdev.fast.ai/api/quarto.html#nbdev_preview) +command helps achieve this by using Quarto to render your docs on your +computer and keep them updated as your edit your notebooks. + +Start the preview by entering this into your terminal: + +``` sh +nbdev_preview +``` + +It may say `Preparing to preview` for a few seconds while it gets +started, and will eventually display something like: + + Watching files for changes + Browse at http://localhost:3000/ + +Click the link to open the preview in a new browser tab. It should look +exactly like your online docs. + +
+ +> **Tip** +> +> We often find it useful to keep a preview window open on the side +> while we’re editing our notebooks in Jupyter. + +
+ +### Edit 00_core.ipynb + +Now, open the `nbs/00_core.ipynb` file (generated by running +[`nbdev_new`](https://nbdev.fast.ai/api/cli.html#nbdev_new) earlier) in +Jupyter. You don’t *have* to start your notebook names with a number, +but we find it helpful to show the order that your project should be +read in – even though it could have been created in a different order. + +#### Add your own frontmatter + +You’ll see something that looks a bit like this: + +
+ +**core** + +> Fill in a module description here + +``` python +#| default_exp core +``` + +
+ +Let’s explain what these special cells means: + +- The first is a markdown cell with nbdev’s markdown *frontmatter* + syntax that defines notebook metadata used by Quarto, our + documentation engine (see the + [frontmatter](../api/09_frontmatter.ipynb) reference page for more). + It contains: + - H1 header (“core”) – defining the page title + - Quote (“Fill in a module description here”) – defining the page + description +- The second is a code cell with a *directive* `default_exp` which + decides which module this notebook will export to (see the + [Directives](../explanations/directives.ipynb) explanation for more). + Currently, it exports to the `core` module. + +Next, rename the notebook, replace the title and description, and change +the default export module for your own project. + +Once you’re done, save the notebook. The live preview started in the +previous section should update with your latest changes. + +Rerun all cells in your notebook to ensure that they work, and to export +the updated modules. + +
+ +> **Tip** +> +> We find the “restart kernel and run all cells” Jupyter command (the ⏩ +> button) so invaluable that we bind it to a keyboard shortcut. A common +> criticism of notebooks is that out-of-order execution leads to +> irreproducible notebooks. In our experience, making “restart and +> rerun” a habit solves this problem. + +
+ +Running the notebook exports Python modules because of the last cell +which contains: + +``` python +#| hide +import nbdev; nbdev.nbdev_export() +``` + +What does this mean? + +- `#| hide` is a directive (like `#| default_exp`) which excludes a cell + from both your exported module and docs +- [`nbdev_export`](https://nbdev.fast.ai/api/doclinks.html#nbdev_export) + is the command used to export your notebooks to Python modules. + +We recommend including a cell like this at the bottom of all of the +notebooks you want to export. + +
+ +> **Warning** +> +> Remember to delete any unused modules that aren’t exported by a +> notebook or otherwise needed by your package. This is likely to happen +> if you change the default export of a notebook – nbdev doesn’t remove +> the old module. This is intended, since nbdev is designed to work with +> hybrid packages that use .py modules (with no corresponding notebook) +> as well as those exported from notebooks. + +
+ +#### Add your own function + +Add a new code cell below the `#| default_exp` cell with a function. For +example: + +``` python +#| export +def say_hello(to): + "Say hello to somebody" + return f'Hello {to}!' +``` + +Notice how it includes `#| export` at the top – this is a directive +(like `#| default_exp`) that tells nbdev to include the cell in your +exported module and in your documentation. + +The documentation should look like this: + +
+ +------------------------------------------------------------------------ + +### say_hello + +> say_hello (to) + +Say hello to somebody + +
+ +#### Add your own examples, tests, and docs + +One of the superpowers of notebook-driven development is that you can +very easily add examples, tests, and documentation right below your +code. + +Include regular code cells, and they’ll appear (with output) in your +docs, for example: + +``` python +say_hello("Isaac") +``` + + 'Hello Isaac!' + +This is a test too! When you run +[`nbdev_test`](https://nbdev.fast.ai/api/test.html#nbdev_test) it will +execute this cell (and all other test cells) and fail if they raise any +exceptions. + +For tests, it’s preferred to use more explicit `assert`s: + +``` python +assert say_hello("Hamel") == "Hello Hamel!" +``` + +…or functions from +[`fastcore.test`](https://fastcore.fast.ai/test.html), which behave like +`assert` but also display the actual and expected values if they differ: + +``` python +from fastcore.test import * +``` + +``` python +test_eq(say_hello("Hamel"), "Hello Hamel!") +``` + +Another superpower of notebook-driven development is that your examples +can include plots, images, and even JavaScript widgets. For example, +here’s an SVG circle: + +``` python +from IPython.display import display,SVG +``` + +``` python +display(SVG('')) +``` + +![](tutorial_files/figure-commonmark/cell-8-output-1.svg) + +### Prepare your changes + +Before commiting your changes to GitHub we recommend running +`nbdev_prepare` in the terminal, which bundles the following commands: + +- [`nbdev_export`](https://nbdev.fast.ai/api/doclinks.html#nbdev_export): + Builds the `.py` modules from Jupyter notebooks +- [`nbdev_test`](https://nbdev.fast.ai/api/test.html#nbdev_test): Tests + your notebooks +- [`nbdev_clean`](https://nbdev.fast.ai/api/clean.html#nbdev_clean): + Cleans your notebooks to get rid of extreanous output for git +- [`nbdev_readme`](https://nbdev.fast.ai/api/quarto.html#nbdev_readme): + Updates your repo’s `README.md` file from your index notebook. + +### Edit index.ipynb + +Now you’re ready to personalize your documentation home page and +`README.md` file; these are both generated automatically from +index.ipynb. Open Jupyter, then click on `nbs/index.ipynb` to open it. + +We recommend including a longer description about what your package +does, how to install it, and how to use it (with a few examples which +import and use your package). Remember, examples can be code cells with +real outputs rather than plain markdown text – they’ll double as tests +too! + +### Push to Github + +You can now commit and push your changes to GitHub. As we mentioned +before, always remember to run `nbdev_prepare` before you commit to +ensure your modules are exported and your tests pass. You can use +`git status` to check which files have been generated or changed. Then: + +``` sh +git add . +git commit -m 'Add `say_hello`; update index' # Update this text with your own message +git push +``` + +This will kick-off your GitHub Actions. Wait a minute or two for those +to complete, then check your updated repo and documentation. + +### Recap + +Congratulations, you’ve used all of the basics needed to build +delightful projects with nbdev! Here’s a recap of the steps you took: + +- Installed hooks for git-friendly notebooks with + [`nbdev_install_hooks`](https://nbdev.fast.ai/api/clean.html#nbdev_install_hooks) +- Installed your package with `pip install -e '.[dev]'` +- Previewed your docs with + [`nbdev_preview`](https://nbdev.fast.ai/api/quarto.html#nbdev_preview) +- Added your own frontmatter, function, tests, and docs to + `nbs/00_core.ipynb` +- Prepared your changes with `nbdev_prepare` +- Updated `nbs/index.ipynb` with your own information +- Pushed to GitHub. + +Read on to learn about more advanced nbdev functionality. Also see our +[explanations](../explanations) for deep-dives on specific topics, as +well as our other [tutorials](../tutorials). + +## Advanced functionality + +### Add a class + +Create a class in `00_core.ipynb` as follows: + +``` python +#| export +class HelloSayer: + "Say hello to `to` using `say_hello`" + def __init__(self, to): self.to = to + + def say(self): + "Do the saying" + return say_hello(self.to) +``` + +This will automatically appear in the docs like this: + +------------------------------------------------------------------------ + +### HelloSayer + +> HelloSayer (to) + +*Say hello to `to` using `say_hello`* + +#### Document with show_doc + +However, methods aren’t automatically documented. To add method docs, +use [`show_doc`](https://nbdev.fast.ai/api/showdoc.html#show_doc): + +``` python +show_doc(HelloSayer.say) +``` + +------------------------------------------------------------------------ + +### HelloSayer.say + +> HelloSayer.say () + +*Do the saying* + +And add some examples and/or tests: + +``` python +o = HelloSayer("Alexis") +o.say() +``` + + 'Hello Alexis!' + +### Add links with backticks + +Notice above there is a link from our new class documentation to our +function. That’s because we used backticks in the docstring: + +``` python + "Say hello to `to` using `say_hello`" +``` + +These are automatically converted to hyperlinks wherever possible. For +instance, here are hyperlinks to `HelloSayer` and `say_hello` created +using backticks. + +### Set up autoreload + +Since you’ll be often updating your modules from one notebook, and using +them in another, it’s helpful if your notebook automatically reads in +the new modules as soon as the Python file changes. To make this happen, +just add these lines to the top of your notebook: + + %load_ext autoreload + %autoreload 2 + +### Set up prerequisites + +If your module requires other modules as dependencies, you can add those +prerequisites to your `settings.ini` in the `requirements` section. The +requirements should be separated by a space and if the module requires +at least or at most a specific version of the requirement this may be +specified here, too. + +For example if your module requires the `fastcore` module of at least +version 1.0.5, the `torchvision` module of at most version 0.7 and any +version of `matplotlib`, then the prerequisites would look like this: + +``` python +requirements = fastcore>=1.0.5 torchvision<0.7 matplotlib +``` + +In addition to `requirements` you can specify dependencies with other +keywords that have different scopes. Below is a list of all possible +dependency keywords: + +- `requirements`: Passed to both pip and conda setup +- `pip_requirements`: Passed to pip setup only +- `conda_requirements`: Passed to conda setup only +- `dev_requirements`: Passed to pip setup as a development requirement + +For more information about the format of dependencies, see the pypi and +conda docs on creating specifications in +[setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py) +and +[meta.yaml](https://docs.conda.io/projects/conda-build/en/latest/resources/define-metadata.html), +respectively. + +### Set up console scripts + +Behind the scenes, nbdev uses that standard package `setuptools` for +handling installation of modules. One very useful feature of +`setuptools` is that it can automatically create [cross-platform console +scripts](https://python-packaging.readthedocs.io/en/latest/command-line-scripts.html#the-console-scripts-entry-point). +nbdev surfaces this functionality; to use it, use the same format as +`setuptools`, with whitespace between each script definition (if you +have more than one). + + console_scripts = nbdev_export=nbdev.cli:nbdev_export + +### Upload to pypi + +If you want people to be able to install your project by just typing +`pip install your-project` then you need to upload it to +[pypi](https://pypi.org/). The good news is, we’ve already created a +fully pypi compliant installer for your project! So all you need to do +is register at pypi (click “Register” on pypi) if you haven’t previously +done so, generate an API token (go to [Account +settings](https://pypi.org/manage/account/) and click “Add API token”) +and then create a file called `~/.pypirc` with your token details. It +should have these contents: + + [pypi] + username = __token__ + password = your_pypi_token + +Another thing you will need is `twine`, so you should run once + + pip install twine + +To upload your project to pypi, just type `nbdev_pypi` in your project +root directory. Once it’s complete, a link to your project on pypi is +displayed. + +### Upload to conda + +Similar to `pip install` support, we have provided an anaconda compliant +installer to upload your project to [anaconda](https://anaconda.org). +Once uploaded, your package can be installed by typing +`conda install -c your_anaconda_username your-project`. + +You need to register at anaconda (fill out the form to `Sign Up`) which +will create a username and password. You will then need to install the +following packages + + pip install anaconda-client conda-build conda-verify + +Before running the anaconda uploader, you need to login to conda using +the CLI command (you will be prompted to enter your username and +password) + + anaconda login + +To upload to anaconda, just type `nbdev_conda` in your project root +directory. + +### Upload to pypi and conda + +The command `nbdev_release_both` from the root of your nbdev repo will +upload your project to both conda and pypi. + +### Install collapsible headings and toc2 + +There are two jupyter notebook extensions that I highly recommend when +working with projects like this. They are: + +- [Collapsible + headings](https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/nbextensions/collapsible_headings/readme.html): + This lets you fold and unfold each section in your notebook, based on + its markdown headings. You can also hit left to go to the + start of a section, and right to go to the end +- [TOC2](https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/nbextensions/toc2/README.html): + This adds a table of contents to your notebooks, which you can + navigate either with the `Navigate` menu item it adds to your + notebooks, or the TOC sidebar it adds. These can be modified and/or + hidden using its settings. + +### Math equation support + +nbdev supports equations (using [Quarto](https://quarto.org/)). You can +include math in your notebook’s documentation using the following +methods. + +Using `$$`, e.g.: + + \sum_{i=1}^{k+1}i + +Which is rendered as: + +> \_{i=1}^{k+1}i + +Using `$`, e.g.: + + This version is displayed inline: \sum_{i=1}^{k+1}i . You can include text before and after. + +Which is rendered as: + +> This version is displayed inline: \_{i=1}^{k+1}i . You can include +> text before and after. + +For more information, see [the Quarto +Docs](https://quarto.org/docs/visual-editor/technical.html) + +### Look at nbdev “source” for more ideas + +Don’t forget that nbdev itself is written in nbdev! It’s a good place to +look to see how fast.ai uses it in practice, and get a few tips. You’ll +find the nbdev notebooks here in the [nbs +folder](https://github.com/fastai/nbdev/tree/master/nbs) on Github. + +### Quarto Features + +nbdev supports most Quarto features. We encourage you to read the +[Quarto documentation](https://quarto.org/) to discover all the features +available to you. For example, this is how you can [incorporate +Graphviz](https://quarto.org/docs/authoring/diagrams.html#graphviz): + +
+ +
+ +
+ + + +
+ +
+ +
+ +It is worth taking a look at the documentation for +[figures](https://quarto.org/docs/authoring/figures.html), +[callouts](https://quarto.org/docs/authoring/callouts.html), +[markdown](https://quarto.org/docs/authoring/markdown-basics.html), +[widgets](https://quarto.org/docs/interactive/widgets/jupyter.html), +[layouts](https://quarto.org/docs/interactive/layout.html), [conditional +content](https://quarto.org/docs/authoring/conditional.html) and [quarto +extensions](https://quarto.org/docs/extensions/) to name a few useful +things we have encountered.# Notebook Best Practices + + + +The flexibility offered by notebooks can be overwhelming. While there +are industry standards for writing Python packages—like +[numpy](https://numpydoc.readthedocs.io/en/latest/format.html#docstring-standard) +and +[sphinx](https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html) +docstrings, and [pytest](https://docs.pytest.org/) and +[unittest](https://docs.python.org/3/library/unittest.html) testing +frameworks—they weren’t designed for notebooks. + +This article walks you through the practices we’ve learned to leverage +the full power of notebooks with nbdev[1]. Our approach weaves code, +tests, and docs into a single interactive context that invites +experimentation. If you prefer to learn by example, you might want to +start with [the annotated +example](#putting-it-all-together-an-annotated-example) and branch out +from there. + +
+ +
Marie Curie’s research notebook dated 19-21 January 1900 (source).
+
+ +## Know which form of notebook you’re writing + +First of all, decide which form of notebook you’re writing. We’re fans +of the [Diátaxis system](https://diataxis.fr/) which classifies +documentation into four forms: tutorials, how-to guides, explanations, +and references. They’ve laid this out beautifully in the following +diagram: + + + +## Start with a great title and subtitle + +Start with a markdown cell at the top of your notebook with its title in +an H1 header, and subtitle in a blockquote. For example: + +``` markdown +# Great title + +> And an even better subtitle +``` + +The title will also be used to reference your page in the sidebar. You +can also optionally add [frontmatter](../api/09_frontmatter.ipynb) to +this cell to customize nbdev and Quarto. + +## Introduce your notebook + +Introduce your notebook with markdown cells below the title. We +recommend a slightly different approach depending on the [form of +documentation](#know-which-form-of-notebook-youre-writing): + +- **Reference:** Start with a brief description of the technical + component, and an overview that links to the main symbols in the page + (you might want to [use + doclinks](#reference-related-symbols-with-doclinks)) +- **Tutorials and how-to guides:** Describe what the reader will learn + and how. Keep it short and get to the subject matter quickly +- **Explanations:** Since these are typically very focused, a short + description of the topic is often sufficient. + +
+ +> **Tip** +> +> Note that Markdown lists such as the one above require a blank line +> above them to be rendered as lists in the documentation, even though +> the notebook viewer will render lists that are not preceded by a blank +> line. + +
+ +## Use lots of code examples, pictures, plots, and videos + +Take advantage of the richness of notebooks by including code examples, +pictures, plots, and videos. + +Here are a few examples to get you started: + +- fastai’s documentation makes extensive use of code examples, plots, + images, and tables, for example, the [computer vision + intro](https://docs.fast.ai/tutorial.vision.html) +- [`nbdev.release`](../api/18_release.ipynb) opens with a terminal + screencast demo in SVG format created with + [asciinema](https://asciinema.org/) and + [svg-term-cli](https://github.com/marionebl/svg-term-cli) +- The [documentation explanation](../explanations/docs.ipynb#overview) + describes a complex data pipeline using a [Mermaid + diagram](https://quarto.org/docs/authoring/diagrams.html) +- The [directives explanation](../explanations/directives.ipynb) + showcases all of nbdev’s directives with executable examples in + call-out cards (and makes great use of emojis too!) +- [RDKit](https://www.rdkit.org/docs/Cookbook.html#drawing-molecules-jupyter) + renders beautiful molecule diagrams + +## Keep docstrings short; elaborate in separate cells + +While nbdev renders docstrings as markdown, they aren’t rendered +correctly when using `symbol?` or `help(symbol)` and they can’t include +executed code. By splitting longer docstrings across separate code and +markdown cells you can [use code examples, pictures, plots, and +videos](#use-lots-of-code-examples-pictures-plots-and-videos). + +We find a single-line summary sufficient for most docstrings. + +## Document parameters with docments + +[`fastcore.docments`](https://fastcore.fast.ai/docments.html) is a +concise way to document parameters that is beautifully rendered by +nbdev. For example, this function: + +``` python +def draw_n(n:int, # Number of cards to draw + replace:bool=True # Draw with replacement? + )->list: # List of cards + "Draw `n` cards." +``` + +…would include the following table as part of its documentation: + +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TypeDefaultDetails
nintNumber of cards to draw
replaceboolTrueDraw with replacement?
ReturnslistList of cards
+ +
+ +nbdev also supports some numpy docstring sections. For example, this +code snippet would produce the same table (there’s no need to include +types like in the docstring if you already have annotations): + +``` python +def draw_n(n:int, replace:bool=True) -> Cards: + """ + Draw `n` cards. + + Parameters + ---------- + n + Number of cards to draw + replace + Draw with replacement? + + Returns + ------- + cards + List of cards + """ +``` + +
+ +> **Tip** +> +> You can render a symbol’s parameters table directly with +> [`DocmentTbl`](https://nbdev.fast.ai/api/showdoc.html#docmenttbl). In +> fact, that’s how we rendered the table above. + +
+ +## Make code cells short, and demonstrate them immediately + +In notebooks, do not create long functions and classes with comments +interspersed throughout them. Instead, split your code up into small +separate cells with explanations and working examples after each. This +lets the user understand how each part works and experiment with them +straight away. It also helps you during development because you can +explore the behavior of every part of your code interactively. + +In non-notebook coding, the documentation, tests, code, and examples are +all separate. This is not the case with nbdev. Take advantage of this by +keeping all of these things as close together as possible. This is +helpful both for exploration and for documentation. + +For example, consider the section of the Claudette source notebook for +[implementing image +support](https://claudette.answer.ai/core.html#images). The section +immediately imports an image and displays it, showing how to work with a +file format. It then creates a number of helper functions, describes and +demonstrates them, and finally puts it all together to show how to use +the complete feature in practice with real inputs and outputs running +directly in the notebook. + +In order to avoid long class definitions caused by many methods, +consider using fastcore’s `patch` decorator to implement each method +separately, and immediately document, demonstrate, and test it. + +## Consider turning code examples into tests by adding assertions + +nbdev blurs the lines between code, docs, and tests. *Every* code cell +is run as a test (unless it’s explicitly marked otherwise), and any +error in the cell fails the test. + +Consider turning your code examples into tests by adding assertions – if +they would make valuable tests and if it doesn’t hurt readability. +[`fastcore.test`](https://fastcore.fast.ai/test.html) provides a set of +light wrappers around `assert` for better notebook tests (for example, +they print both objects on error if they differ). + +Here’s an example using +[`fastcore.test.test_eq`](https://fastcore.fast.ai/test.html#test_eq): + +``` python +def inc(x): return x + 1 +test_eq(inc(3), 4) +``` + +## Document error-cases as tests + +Docstring-driven approaches typically document the errors raised by an +object using plaintext descriptions, for example, in a “raises” section. + +In nbdev, we recommend documenting errors with actual failing code using +[`fastcore.test.test_fail`](https://fastcore.fast.ai/test.html#test_fail). +For example: + +``` python +def divide(x, y): return x / y +test_fail(lambda: divide(1, 0), contains="division by zero") +``` + +The first argument is a `lambda` since we need to allow `test_fail` to +control its execution and catch any errors. + +## Reference related symbols with doclinks + +If you surround a symbol with backticks, nbdev will automatically link +to that symbol’s reference page. We call these +[doclinks](../api/05_doclinks.ipynb). + +Prefer fully qualified symbol paths, like `package.module.symbol` +instead of `symbol`. It may be more verbose but it helps users know +which module a symbol originates from, which is especially important for +third-party packages. + +Any package created with nbdev will automatically support doclinks. +Non-nbdev packages can be supported by creating a minimal nbdev-index +package. [`nbdev-index`](https://github.com/fastai/nbdev-index) is a +collection of such packages, which already supports django, numpy, +pandas, pytorch, scipy, sphinx, the Python standard library, and even +other programming languages like APL! + +## Add rich representations to your classes + +This is another way to take advantage of the [rich display feature of +notebooks](https://ipython.readthedocs.io/en/stable/config/integrating.html#rich-display). +You can provide rich representations to your object by defining a +`_repr_markdown_` method that returns markdown text (which may also +include HTML/CSS). + +Here’s a simple example to get you started: + +``` python +class Color: + def __init__(self, color): self.color = color + def _repr_markdown_(self): + style = f'background-color: {self.color}; width: 50px; height: 50px; margin: 10px' + return f'
' +``` + +``` python +Color('green') +``` + +
+ +
+ +``` python +Color('blue') +``` + +
+ +
+ +Also see [the earlier list of example +projects](#use-lots-of-code-examples-pictures-plots-and-videos) that +make use of beautiful visual representations. + +## Document class methods with [`show_doc`](https://nbdev.fast.ai/api/showdoc.html#show_doc) or [`fastcore.basics.patch`](https://fastcore.fast.ai/basics.html#patch) + +nbdev automatically documents exported function and class definitions +with [`show_doc`](https://nbdev.fast.ai/api/showdoc.html#show_doc). +However, it’s up to you to document class methods. There are two ways to +do that: calling +[`show_doc`](https://nbdev.fast.ai/api/showdoc.html#show_doc) on the +method, or defining the method with the +[`fastcore.basics.patch`](https://fastcore.fast.ai/basics.html#patch) +decorator. + +
+ +### Notebook (show_doc) + +If your class is defined in a single cell, use +[`show_doc`](https://nbdev.fast.ai/api/showdoc.html#show_doc). Here’s +what your notebook might look like: + +
+ +``` python +#| export +class Number: + "A number." + def __init__(self, num): self.num = num + def __add__(self, other): + "Sum of this and `other`." + return Number(self.num + other.num) + def __repr__(self): return f'Number({self.num})' +``` + +For example, here is the number 5: + +``` python +Number(5) +``` + +``` python +show_doc(Number.__add__) +``` + +For example: + +``` python +Number(3) + Number(4) +``` + +
+ +### Notebook (@patch) + +If you split your class definition across cells with +[`fastcore.basics.patch`](https://fastcore.fast.ai/basics.html#patch), +here’s what your notebook might look like: + +
+ +``` python +#| export +class Number: + "A number." + def __init__(self, num): self.num = num + def __repr__(self): return f'Number({self.num})' +``` + +For example, here is the number 5: + +``` python +Number(5) +``` + +``` python +#| export +@patch +def __add__(self:Number, other): + "Sum of this and `other`." + return Number(self.num + other.num) +``` + +For example: + +``` python +Number(3) + Number(4) +``` + +
+ +### Docs + +In either case, this is how the documentation would be rendered: + +
+ +------------------------------------------------------------------------ + +### Number + +> Number (num) + +A number. + +For example, here is the number 5: + +``` python +Number(5) +``` + + Number(5) + +------------------------------------------------------------------------ + +### Number.\_\_add\_\_ + +> Number.__add__ (other) + +Sum of this and `other`. + +For example: + +``` python +Number(3) + Number(4) +``` + + Number(7) + +
+ +
+ +## Group symbols with H2 sections + +As your notebooks grow, consider grouping related symbols using markdown +cells with level 2 headers. Since nbdev displays documented symbols as +level 3 headers, this would group all symbols below your level 2 header. + +Here is the markdown syntax: + +``` markdown +## Section title +``` + +## Split long explanations with H4 sections + +Similar to the previous section, as a symbol’s explanation grows, +consider grouping its cells using level 4 headers. This is the +recommended way to structure your reference docs, for example, to +achieve numpy-style structures with sections like notes, examples, +methods, and so on. + +Here’s the markdown syntax: + +``` markdown +#### Section title +``` + +## Putting it all together: an annotated example + +In this section, we’ll guide you through a full example of writing a +documented and tested function in a notebook using all of the principles +described above. We’ll use the +[`numpy.all`](https://numpy.org/doc/stable/reference/generated/numpy.all.html#numpy.all) +function since it follows the widely-known numpy-docstring standard for +.py files. + +Below is the definition of the +[`numpy.all`](https://numpy.org/doc/stable/reference/generated/numpy.all.html#numpy.all) +function. Take note of how all of the information is included in the +docstring. While this works well for .py files, it doesn’t let us weave +executable code with rich markdown as we can in notebooks: + +``` python +def all(a, axis=None, out=None, keepdims=np._NoValue, *, where=np._NoValue): + """ + Test whether all array elements along a given axis evaluate to True. + Parameters + ---------- + a : array_like + Input array or object that can be converted to an array. + axis : None or int or tuple of ints, optional + Axis or axes along which a logical AND reduction is performed. + The default (``axis=None``) is to perform a logical AND over all + the dimensions of the input array. `axis` may be negative, in + which case it counts from the last to the first axis. + .. versionadded:: 1.7.0 + If this is a tuple of ints, a reduction is performed on multiple + axes, instead of a single axis or all the axes as before. + out : ndarray, optional + Alternate output array in which to place the result. + It must have the same shape as the expected output and its + type is preserved (e.g., if ``dtype(out)`` is float, the result + will consist of 0.0's and 1.0's). See :ref:`ufuncs-output-type` for more + details. + keepdims : bool, optional + If this is set to True, the axes which are reduced are left + in the result as dimensions with size one. With this option, + the result will broadcast correctly against the input array. + If the default value is passed, then `keepdims` will not be + passed through to the `all` method of sub-classes of + `ndarray`, however any non-default value will be. If the + sub-class' method does not implement `keepdims` any + exceptions will be raised. + where : array_like of bool, optional + Elements to include in checking for all `True` values. + See `~numpy.ufunc.reduce` for details. + .. versionadded:: 1.20.0 + Returns + ------- + all : ndarray, bool + A new boolean or array is returned unless `out` is specified, + in which case a reference to `out` is returned. + See Also + -------- + ndarray.all : equivalent method + any : Test whether any element along a given axis evaluates to True. + Notes + ----- + Not a Number (NaN), positive infinity and negative infinity + evaluate to `True` because these are not equal to zero. + Examples + -------- + >>> np.all([[True,False],[True,True]]) + False + >>> np.all([[True,False],[True,True]], axis=0) + array([ True, False]) + >>> np.all([-1, 4, 5]) + True + >>> np.all([1.0, np.nan]) + True + >>> np.all([[True, True], [False, True]], where=[[True], [False]]) + True + >>> o=np.array(False) + >>> z=np.all([-1, 4, 5], out=o) + >>> id(z), id(o), z + (28293632, 28293632, array(True)) # may vary + """ + ... +``` + +Alternatively, Here is how we’d write +[`numpy.all`](https://numpy.org/doc/stable/reference/generated/numpy.all.html#numpy.all) +in a notebook using nbdev. The first step is to define the function: + +
+ +``` python +#| export +def all(a, # Input array or object that can be converted to an array. + axis:int|tuple|None=None, # Axis or axes along which a logical AND reduction is performed (default: all). + out:np.ndarray|None=None, # Alternate output array in which to place the result. + keepdims:bool=np._NoValue, # Leave reduced one-dimensional axes in the result? + where=np._NoValue, # Elements to include in reduction. See `numpy.ufunc.reduce` for details. New in version 1.20.0. + ) -> np.ndarray|bool: # A new boolean or array, or a reference to `out` if its specified. + "Test whether all array elements along a given axis evaluate to `True`." + ... +``` + +
+ +We can observe the following differences between this code and +numpy-docstrings: + +- The definition uses simple type annotations, which will be rendered in + the function’s parameters table below +- Parameters are described with a short comment, called + [docments](#document-parameters-with-docments) – a concise alternative + to numpy and sphinx docstring formats (although nbdev does support + numpy docstrings see [this + example](#document-parameters-with-docments)) +- The docstring and parameter descriptions are all short, single-line + summaries. We prefer to [keep docstrings short and instead elaborate + in separate + cells](#keep-docstrings-short-elaborate-in-separate-cells), where we + can use markdown and real code examples. + +Note: the use of `|` syntax for unions e.g. `int|tuple|None` (equivalent +to `Union[int, tuple, None]`) requires using Python 3.10 or by treating +all annotations as strings using `from __future__ import annotations` +which is available from Python 3.7. + +Our function definition is automatically rendered in the docs like this. +Note that parameter names, types, defaults, and details are all parsed +from the definition which means you don’t have to repeat yourself. + +
+ +------------------------------------------------------------------------ + +### all + +> all (a, axis:Union[int,tuple,NoneType]=None, +> out:Optional[numpy.ndarray]=None, keepdims:bool=, +> where=) + +Test whether all array elements along a given axis evaluate to `True`. + + ++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TypeDefaultDetails
aInput array or object that can be converted to an array.
axisint | tuple | NoneNoneAxis or axes along which a logical AND reduction is performed +(default: all).
outnp.ndarray | NoneNoneAlternate output array in which to place the result.
keepdimsboolLeave reduced one-dimensional axes in the result?
where_NoValueTypeElements to include in reduction. See numpy.ufunc.reduce +for details. New in version 1.20.0.
Returnsnp.ndarray | boolA new boolean or array, or a reference to out +if its specified.
+ +
+ +Next, describe how to use your function using markdown cells and [lots +of code examples](#use-lots-of-code-examples-pictures-plots-and-videos). +This is the biggest benefit to developing in notebooks: instead of +copying and pasting code examples into plaintext, you can include real +executeable code examples. + +We start with basic usage first: + +
+ +For example: + +``` python +x = [[True,False],[True,True]] +test_eq(np.all(x), False) +``` + +
+ +Our code examples [use assertion functions from +`fastcore.test`](#consider-turning-code-examples-into-tests-by-adding-assertions), +so that they serve as both docs and tests. +[`nbdev_test`](https://nbdev.fast.ai/api/test.html#nbdev_test) runs +every code cell as a test (unless it’s explicitly marked otherwise), and +any error in the cell fails the test. + +Having described basic usage, we now elaborate on more advanced +functionality for each parameter. This differs from numpy’s approach +which includes all parameter docs in the table and where not all +parameters have code examples. + +
+ +With `axis`: + +``` python +test_eq(np.all(x, axis=0), [True,False]) +``` + +`axis` may be negative, in which case it counts from the last to the +first axis: + +``` python +test_eq(np.all(x, axis=-1), [False,True]) +``` + +If `axis` is a tuple of ints, a reduction is performed on multiple axes, +instead of a single axis or all the axes as before. + +``` python +test_eq(np.all(x, axis=(0,1)), False) +``` + +Integers, floats, not a number (nan), and infinity all evaluate to +`True` because they’re not equal to zero: + +``` python +test_eq(np.all([-1, 1, -1.0, 1.0, np.nan, np.inf, -np.inf]), True) +``` + +You can use `where` to test specific elements. For example, this tests +only the second column: + +``` python +test_eq(np.all(x, where=[[False],[True]]), True) +``` + +The output can be stored in an optional `out` array. If provided, a +reference to `out` will be returned: + +``` python +o = np.array(False) +z = np.all([-1, 4, 5], out=o) +test_is(z, o) +test_eq(z, True) +``` + +`out` must have the same shape as the expected output and its type is +preserved (e.g., if `dtype(out)` is float, the result will consist of +0.0’s and 1.0’s). See [Output type +determination](https://numpy.org/doc/stable/user/basics.ufuncs.html#ufuncs-output-type) +for more details. + +With `keepdims`, the result will broadcast correctly against the input +array. + +``` python +test_eq(np.all(x, axis=0, keepdims=True), [[True, False]]) # Note the nested list +``` + +If the default value is passed, then `keepdims` will not be passed +through to the `all` method of sub-classes of `ndarray`, however any +non-default value will be. If the sub-class’ method does not implement +`keepdims` any exceptions will be raised. + +``` python +class MyArray(np.ndarray): + def all(self, axis=None, out=None): ... + +y = MyArray((2,2)) +y[:] = x +np.all(y) # No TypeError since `keepdims` isn't passed +test_fail(lambda: np.all(y, keepdims=True), contains="all() got an unexpected keyword argument 'keepdims'") +``` + +
+ +Since we prefer to document via code examples, we also document +error-cases with assertions using +[`fastcore.test.test_fail`](https://fastcore.fast.ai/test.html#test_fail). +This differs from docstring-based approaches which usually document +error-cases in prose, usually in a “raises” section of the docstring. + +Finally, we link to related symbols with +[doclinks](#reference-related-symbols-with-doclinks) (symbols surrounded +in backticks are automatically linked) and describe their relation using +code examples. + +
+ +The +[`numpy.ndarray.all`](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.all.html#numpy.ndarray.all) +method is equivalent to calling +[`numpy.all`](https://numpy.org/doc/stable/reference/generated/numpy.all.html#numpy.all) +with the array: + +``` python +test_eq(np.array(x).all(), np.all(x)) +``` + +In contrast, +[`numpy.any`](https://numpy.org/doc/stable/reference/generated/numpy.any.html#numpy.any) +tests whether *any* element evaluates to `True` (rather than *all* +elements): + +``` python +test_eq(np.any(x), True) +``` + +
+ +### Recap + +In summary, here is how the nbdev version of +[`numpy.all`](https://numpy.org/doc/stable/reference/generated/numpy.all.html#numpy.all) +differs from the numpy docstring. nbdev uses: + +- Type annotations and docments instead of the numpy docstring format + (although nbdev supports numpy docstrings too) +- Short parameter descriptions, with details in separate cells with + markdown and code example +- Doclinks to related symbols instead of a “See also” section +- Lots of code examples (which are also tests) mixed with prose to + describe how to use the function +- Code examples with assertions to document error-cases instead of a + “Raises” section. + +[1] We’re always open to improving our workflows and don’t like to be +too prescriptive about style. If you have any ideas, please feel free to +post them in the [forum](https://forums.fast.ai/c/nbdev/48).# Git-Friendly Jupyter + + + +Version control is essential to developing software, yet Jupyter +notebooks don’t work with version control by default. nbdev solves this +problem! It provides a set of hooks which enable git-friendly Jupyter +notebooks in any git repo, including those that don’t use the broader +nbdev system. + +To get started, install nbdev: + +
+ +#### pip + +``` sh +pip install -U nbdev +``` + +#### conda + +``` sh +conda install -c fastai nbdev +``` + +
+ +then install hooks: + +``` sh +nbdev_install_hooks +``` + +That’s it! Read on if you’re stuck or if you’d like to learn more about +nbdev hooks and how to customise them. Check out [our related blog +post](https://www.fast.ai/2022/08/25/jupyter-git/) if you’re curious +about how this feature was developed and how it works under the hood. + +
+ +> **Note** +> +> The [clean hook](#nbdev_clean-on-saving-notebooks-in-jupyter) +> currently only supports Jupyter Notebook and Jupyter Lab. If you’re +> using another notebook editor, like VSCode or PyCharm, you might want +> to use [nbdev’s pre-commit hooks](../tutorials/pre_commit.ipynb). + +
+ +## Quickstart: Install nbdev hooks for a repo + +To start with, change directory to your current project and +double-check. Don’t worry about the strange path, that’s because we’re +using a temporary directory for this tutorial: + +``` sh +pwd +``` + + /private/var/folders/ft/0gnvc3ts5jz4ddqtttp6tjvm0000gn/T/tmpez8nec5v/repo + +Install nbdev: + +``` sh +pip install -Uqq nbdev +``` + +Install nbdev hooks: + +``` sh +nbdev_install_hooks +``` + + Not in a git repository, git hooks cannot be installed. + +You’ll see the above error if you’re not in a git repo. If so, +initialise a git repository: + +``` sh +git init +``` + + Initialized empty Git repository in /private/var/folders/ft/0gnvc3ts5jz4ddqtttp6tjvm0000gn/T/tmpez8nec5v/repo/.git/ + +Then try installing nbdev hooks again: + +``` sh +nbdev_install_hooks +``` + + Hooks are installed. + +If you already have a pre-save hook set in your Jupyter config file we +won’t be able to safely install a new one automatically. Instead, you’ll +encounter an error and will need to follow its instructions for a manual +installation. + +Jupyter hooks will now be installed in your user’s Jupyter config +directory, and will work for all repos by default. Git hooks will only +be installed in the current repo; **you will need to rerun +[`nbdev_install_hooks`](https://nbdev.fast.ai/api/clean.html#nbdev_install_hooks) +for each of your git repos**. See [configuring nbdev +hooks](#configuring-nbdev-hooks) if you’d like to customise hook +behaviour, for example, to opt out of hooks in certain repos. + +## What are nbdev hooks? + +nbdev provides three hooks to ease Jupyter-git integration. + +### [`nbdev_merge`](https://nbdev.fast.ai/api/merge.html#nbdev_merge) on merging notebooks with git + +One of the biggest complaints when working with Jupyter is that merge +conflicts break notebooks. This is particularly problematic in projects +with many collaborators. + +
+ + +
+ +Oftentimes these conflicts are on metadata like cell execution counts +that we don’t really care about. nbdev comes with a custom git merge +driver that automatically fixes conflicting outputs and metadata, and +that leaves remaining conflicts in a state that still works with +Jupyter. It works in all git commands that use merge under the hood, +including `merge`, `pull`, `rebase`, and `stash`. + +Here’s what the conflict looks like in Jupyter with nbdev’s merge +driver: + + + +### [`nbdev_clean`](https://nbdev.fast.ai/api/clean.html#nbdev_clean) on saving notebooks in Jupyter + +Jupyter notebooks store a variety of metadata (including execution +counts and notebook extension info) that aren’t conducive to +collaborative version control systems like git. These pollute diffs in +pull requests and git histories (which can make debugging harder), and +tend to cause merge conflicts. For example: + +``` diff + { + "cell_type": "code", +- "execution_count": 1, ++ "execution_count": 2, + "metadata": { + "hide_input": false + } +``` + +Python’s default repr is another example, since it includes a memory +address which we usually aren’t interested in: + +``` diff +- ++ +``` + +nbdev install a Jupyter hook which runs +[`nbdev_clean`](https://nbdev.fast.ai/api/clean.html#nbdev_clean) to +automatically clean unwanted metadata and outputs from your notebooks, +including ids from default Python reprs! With nbdev hooks, the examples +above would become: + +``` json +{ + "cell_type": "code", + "execution_count": null, + "metadata": {} +} +``` + +and + + + +### [`nbdev_trust`](https://nbdev.fast.ai/api/clean.html#nbdev_trust) after merging notebooks with git + +A side-effect of Jupyter’s security model is that widgets don’t work in +collaborative repos, unless you manually “trust” notebooks after each +`git pull`. There is a good reason behind this: since Jupyter notebooks +contain HTML and JavaScript, the trust system avoids running malicious +code when you open a notebook and don’t explicitly run any cells. See +[the official +documentation](https://jupyter-server.readthedocs.io/en/latest/operators/security.html#security-in-notebook-documents) +for more. + +Manually trusting notebooks each time is a pain. A more natural workflow +would be trust a repo once-off, and all notebooks and changes +thereafter. nbdev includes a git post-merge hook which runs +[`nbdev_trust`](https://nbdev.fast.ai/api/clean.html#nbdev_trust) in +your repo to do exactly this. + +## Configuring nbdev hooks + +The most up-to-date reference of nbdev’s settings is in the +[`nbdev_create_config`](https://nbdev.fast.ai/api/config.html#nbdev_create_config) +docs. In addition, this section will guide you through a few common +configurations. + +**Control whether Jupyter hooks are run:** + +- Globally enable Jupyter hooks: set `jupyter_hooks = True` in user + settings +- Globally disable Jupyter hooks: set `jupyter_hooks = False` in user + settings (at `~/.config/nbdev/settings.ini`) +- Enable Jupyter hooks only for selected repos: set + `jupyter_hooks = False` in user settings and `jupyter_hooks = True` in + selected repo settings + +**Customise notebook cleaning with the following settings:** + +- Clean *all* outputs and metadata: `clear_all` +- Preserve certain metadata by key: `allowed_metadata_keys` and + `allowed_cell_metadata_keys` +- Clean `id`s from default Python `repr`s: `clean_ids` + +All of the above can be customised per-user and per-repo. + +**Control whether git hooks are run:** + +Since git hooks are installed per-repo they’ll only run in repos where +you manually +[`nbdev_install_hooks`](https://nbdev.fast.ai/api/clean.html#nbdev_install_hooks). +If you change your mind later, you can uninstall git hooks by following +the instructions in the `.gitconfig` file created in your repo. + + + + + Page not found · GitHub Pages + + + + +
+ +

404

+

File not found

+ +

+ The site configured at this address does not + contain the requested file. +

+ +

+ If this is your site, make sure that the filename case matches the URL + as well as any file permissions.
+ For root URLs (like http://example.com/) you must provide an + index.html file. +

+ +

+ Read the full documentation + for more information about using GitHub Pages. +

+ + + + + + +
+ +# Pre-Commit Hooks + + + +We provide hooks for the [pre-commit framework](https://pre-commit.com/) +to catch and fix uncleaned and unexported notebooks, locally, without +having to wait for continuous integration pipelines to run. + +They might also be useful as an alternative to the [Jupyter clean +hook](../tutorials/git_friendly_jupyter.ipynb#nbdev_clean-on-saving-notebooks-in-jupyter) +if you’re using a notebook editor that isn’t yet supported +(e.g. VSCode). + +## Install pre-commit + +…Install pre-commit (check [their latest +instructions](https://pre-commit.com/#install) if you have any +difficulty with these commands): + +
+ +#### pip + +``` sh +pip install pre-commit +``` + +#### conda + +``` sh +conda install -c conda-forge pre-commit +``` + +#### homebrew (macOS) + +``` sh +brew install pre-commit +``` + +
+ +## Configure pre-commit for your repo + +Create a file named `.pre-commit-config.yaml` in the root of your repo, +with the following contents: + +``` yaml +repos: +- repo: https://github.com/fastai/nbdev + rev: 2.2.10 + hooks: + - id: nbdev_clean + - id: nbdev_export +``` + +Include only the hook(s) you’d like to run, as well as any other +[supported hooks](https://pre-commit.com/hooks.html). + +
+ +> **Tip** +> +> If you expect all collaborators to use pre-commit, add the +> `.pre-commit-config.yaml` file to your repo. Otherwise, add it to your +> `.gitignore`. + +
+ +Install pre-commit hooks into your repo: + +``` sh +pre-commit install +``` + +## Make a commit and enjoy pre-commit in action + +When you do a `git commit` in a repo that has pre-commit hooks +installed, your new workflow will be as follows: + +1. pre-commit runs each hook on your *staged* changes (as in, changes + that you `git add`ed) +2. If a hook changes files – for example, if a commited notebook wasn’t + cleaned – pre-commit stops the commit, leaving those changes as + *unstaged* +3. You can now stage those changes and make any edits required to get + pre-commit to pass +4. Redo the `git commit`, and if it succeeds, your commit will be + created. + +Using it in practice isn’t as complicated as it might sound. The best +way to figure out if it works for you is to give it a try. + +## How to override pre-commit if you get stuck + +If you struggle to get pre-commit to pass a commit that you absolutely +think is correct, you can [temporarily disable a +hook](https://pre-commit.com/#temporarily-disabling-hooks) like this: + +``` sh +SKIP=hook git commit +``` + +…where `hook` refers to a valid hook in your configuration, for example, +to disable the +[`nbdev_export`](https://nbdev.fast.ai/api/doclinks.html#nbdev_export) +hook: + +``` sh +SKIP=nbdev_export git commit +``` + +You can also disable pre-commit entirely with the `--no-verify` flag: + +``` sh +git commit --no-verify +``` + +Finally, if you decide it’s not for you, you can completely remove +pre-commit hooks from your repo with: + +``` sh +pre-commit uninstall +```# Documentation Only Sites + + + +## Background + +While nbdev is great for authoring software, you may wish to utilize the +power of nbdev for the purposes of documenting existing code, or **use +various utilities of nbdev without having to write a python library**. +For example, you can use the following features of nbdev without +creating a python package: + +- Custom [nbdev directives](../explanations/directives.ipynb) such as + [`#|hide_line`](../explanations/directives.ipynb#hide_line). +- Testing with + [`nbdev_test`](https://nbdev.fast.ai/api/test.html#nbdev_test). +- Automated entity linking with + [doclinks](best_practices.ipynb#reference-related-symbols-with-doclinks). +- Rendering API documentation with [docments and + show_doc](best_practices.ipynb#document-parameters-with-docments). + +## Setup + +To setup a documentation only site, you can follow these steps: + +1. Create a nbdev repo the usual way, using + [`nbdev_new`](https://nbdev.fast.ai/api/cli.html#nbdev_new) +2. Remove library files + +``` bash +rm setup.py .github/workflows/test.yaml nbs/00_core.ipynb +``` + +3. Remove your library folder (this will be the `lib_path` field in + `settings.ini`): + +``` bash +rm -rf +``` + +## Usage + +After setting up your project, you can use various nbdev utilities per +usual: + +- [`nbdev_preview`](https://nbdev.fast.ai/api/quarto.html#nbdev_preview) + for previewing your site +- [`nbdev_test`](https://nbdev.fast.ai/api/test.html#nbdev_test) for + testing your docs locally +- Custom [nbdev directives](../explanations/directives.ipynb) will be + available to you (but you must be careful not to use irrelevant ones + like `#|export`). +- If you created your nbdev docs site on GitHub, GitHub Actions will + publish your docs for you automatically [as described + here](../explanations/docs.ipynb#Deploying-Docs-With-GitHub-Actions). +- You can publish your docs on other platforms as described + [here](../explanations/docs.ipynb#Deploying-Your-Docs-On-Other-Platforms). + +## Demo + +A minimal example of a documentation-only site is located +[here](https://github.com/hamelsmu/nolib_nbdev).# Modular nbdev + + + +While [`nbdev_new`](https://nbdev.fast.ai/api/cli.html#nbdev_new) gets +you started with everything you need to create a delightful Python +package, **you can also use each of nbdev’s components listed below on +their own**. You might find this useful if you’re porting a large system +over to nbdev, documenting an existing code base, or if you’d like to +customize the nbdev workflow for your own project. Note that all of the +commands below work without a `settings.ini` file unless otherwise +noted. + +## Document existing code: [`show_doc`](https://nbdev.fast.ai/api/showdoc.html#show_doc) + +nbdev allows you to document existing code, even code that is not +written in nbdev! +[`nbdev.showdoc.show_doc`](https://nbdev.fast.ai/api/showdoc.html#show_doc) +allows you to render beautiful API documentation in notebooks and on +Quarto sites. For example, you can render API documentation for +[`numpy.all`](https://numpy.org/doc/stable/reference/generated/numpy.all.html#numpy.all) +like this: + +
+ +``` python +from nbdev.showdoc import show_doc +from numpy import all +show_doc(all) +``` + +------------------------------------------------------------------------ + +### all + +> all (a, axis=None, out=None, keepdims=, where=) + +Test whether all array elements along a given axis evaluate to True. + + ++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TypeDefaultDetails
aarray_likeInput array or object that can be converted to an array.
axisNoneTypeNoneAxis or axes along which a logical AND reduction is +performed.
The default (axis=None) is to perform a +logical AND over all
the dimensions of the input array. +axis may be negative, in
which case it counts from the +last to the first axis.

.. versionadded:: 1.7.0

If this is +a tuple of ints, a reduction is performed on multiple
axes, instead +of a single axis or all the axes as before.
outNoneTypeNoneAlternate output array in which to place the result.
It must have +the same shape as the expected output and its
type is preserved +(e.g., if dtype(out) is float, the result
will consist +of 0.0’s and 1.0’s). See :ref:ufuncs-output-type for +more
details.
keepdims_NoValueTypeIf this is set to True, the axes which are reduced are left
in +the result as dimensions with size one. With this option,
the result +will broadcast correctly against the input array.

If the default +value is passed, then keepdims will not be
passed +through to the all method of sub-classes +of
ndarray, however any non-default value will be. If +the
sub-class’ method does not implement keepdims +any
exceptions will be raised.
where_NoValueTypeElements to include in checking for all True +values.
See ~numpy.ufunc.reduce for details.

.. +versionadded:: 1.20.0
Returnsndarray, boolA new boolean or array is returned unless out +is specified,
in which case a reference to out is +returned.
+ +
+ +
+ +> **Note** +> +> [`show_doc`](https://nbdev.fast.ai/api/showdoc.html#show_doc) +> automatically parses docstrings that are written in the numpy style. +> For more information [read +> here](../api/showdoc.ipynb#numpy-docstrings). + +
+ +## Testing notebooks: [`nbdev_test`](https://nbdev.fast.ai/api/test.html#nbdev_test) + +Testing notebooks can be very useful outside of nbdev, especially if you +are documenting an existing code base and want to incorporate tests for +your docs. The +[`nbdev_test`](https://nbdev.fast.ai/api/test.html#nbdev_test) CLI +utility allows you to accomplish this: + +You can test an individual notebook with the terminal command: + +``` sh +nbdev_test --path notebook.ipynb +``` + +…or a folder of notebooks: + +``` sh +nbdev_test --path tests/ +``` + +## Export code to modules: [`nb_export`](https://nbdev.fast.ai/api/export.html#nb_export) + +You can export a notebook to a module with the Python function: + +``` python +nb_export('notebook.ipynb', 'pkg') +``` + +…provided the notebook specifies a `default_exp` directive at the top, +and `export` directives above each cell to be exported. We recommend +including this in a code cell at the bottom of your notebook for +convenience. + +## Jupyter-git integration + +Jupyter and Git don’t normally play well together, especially for things +like merge conflicts. We have outlined all of these problems, and our +solutions in [this blog +post](../blog/posts/2022-08-25-jupyter-git/index.qmd). You can install +our merge driver and hooks with the following command: + +``` bash +nbdev_install_hooks +``` + +We describe what +[`nbdev_install_hooks`](https://nbdev.fast.ai/api/clean.html#nbdev_install_hooks) +does in detail on [this page](git_friendly_jupyter.ipynb). + +You can also directly use any of its underlying commands, for example, +to implement your own hooks or extensions: + +- [`nbdev_clean`](https://nbdev.fast.ai/api/clean.html#nbdev_clean) +- [`nbdev_fix`](https://nbdev.fast.ai/api/merge.html#nbdev_fix) +- [`nbdev_merge`](https://nbdev.fast.ai/api/merge.html#nbdev_merge) +- [`nbdev_trust`](https://nbdev.fast.ai/api/clean.html#nbdev_trust) + +To configure your own hooks, Check out the [pre-commit hooks +tutorial](../tutorials/pre_commit.ipynb). + +## Python packaging + +`nbdev.release`provides utlities for easy packaging on PyPI, conda, and +GitHub. Check out the [`nbdev.release`](../api/release.html) docs for +more information. Note that this functionality requires a settings.ini +file.# Writing nbdev plugins + + + +## What will this cover? + +With `nbdev`, it’s possible to customize and extend it further beyond +the standard capabilities through a well thoughtout and scalable +framework. Does your particular library or need require you to inject +custom quarto additives in certain cells? What about if you want to do +something more trivial such as finding shortcuts to replace complicated +quarto directives more easily (such as replacing `::: {.column-margin}` +with `#| margin`)? + +Writing custom plugins with `nbdev` is the easiest method to achieve +this, and with this tutorial we will bring you up to speed on how you +can use this to create your own plugins to expand and simplify your +literate-programming experience with `nbdev` and quarto. + +Specifically, we will be building a *processor* (something that +processes a notebook cell) that will let us quickly write out any +quarto-specific headers (that `::: {.some_annotation}`) and replace it +with a [`div`](https://nbdev.fast.ai/api/qmd.html#div) shortcut. This is +of course one example very specific to quarto that happens when building +the documentation, but this technique can be used to have custom +behaviors occur during library exportation as well. + +> Note: We are using [`div`](https://nbdev.fast.ai/api/qmd.html#div) +> here is it more closely resembles how each of the related Quarto +> directives do and behave as they act like `
`s in HTML code + +This tutorial *won’t* cover some of the basics when it comes to nbdev, +and instead comes with the understanding you know how to navigate nbdev +(such as what are +[directives](https://nbdev.fast.ai/explanations/directives.html), +[`export`](https://nbdev.fast.ai/explanations/directives.html#export-some.thing), +etc). + +## Getting started, how does `nbdev` make this easy? + +First let’s visualize just what we’re trying to achieve. + +Instead of doing the following code which will add `"Some text"` to the +sidebar (as shown off to the side currently): + +
+ +Some text + +
+ + ::: {.column-margin} + Some text + ::: + +We will create a shorter way to write this out, making use of how nbdev +and quarto writes their *directives* + +By the end of this tutorial we will create something that looks like the +following: + + #| div column-margin + + Some text + +And this will include cases where a +[`div`](https://nbdev.fast.ai/api/qmd.html#div) should be put across +*multiple* cells as well, by specifying a `start` and an `end`. + +> Note: Check out the [article +> layout](https://quarto.org/docs/authoring/article-layout.html) Quarto +> documentation to find the best examples of use cases for this custom +> directive, including the +> [`column-margin`](https://quarto.org/docs/authoring/article-layout.html#margin-content) +> just shown + +This can be achieved in under 50 lines of code! + +`nbdev` let’s us create what are called *processors* (this is how +`#| export` will shove code into modules, for example). These processors +are acted on each **cell** of a notebook and can modify its contents. +These can then be wrapped into a module the same way that nbdev will do +[`nbdev_export`](https://nbdev.fast.ai/api/doclinks.html#nbdev_export) +or [`nbdev_docs`](https://nbdev.fast.ai/api/quarto.html#nbdev_docs). +Thanks to the power of writing custom `nbdev` extensions, going deep +into the inner-workings of the framework isn’t required! + +## Bringing in what we need + +The actual imports we need to use from `nbdev` is truly not that many! +We just need two: - +[`extract_directives`](https://nbdev.fast.ai/api/process.html#extract_directives), +to read in the list of `#|` written - The +[`Processor`](https://nbdev.fast.ai/api/process.html#processor) class +that will actually perform what we want on notebook cells. + +The rest of the imports are there to make some of our lives easier as +will be explained later + +``` python +from nbdev.process import extract_directives +from nbdev.processors import Processor + +from fastcore.basics import listify + +from string import Template +``` + +Lastly for *testing* purposes we’ll utilize `nbdev`’s `mk_cell` function +and the +[`NBProcessor`](https://nbdev.fast.ai/api/process.html#nbprocessor) +class, which will let us mock running our processor on a “real” +notebook! + +``` python +from nbdev.processors import mk_cell, NBProcessor +``` + +## Writing a converter + +The first step is creating a quick and easy way to take the `nbdev` +directive we want to use (such as `#| div column-margin`) and convert it +quickly into something quarto will then read (such as +`::: {.column-margin}`). + +We can create a string `Template` to perform this for us: + +``` python +_LAYOUT_STR = Template("::: {.$layout}\n${content}\n") +``` + +
+ +> **Tip** +> +> This doesn’t have to be a string template, I just found this the +> easiest to use! + +
+ +``` python +_LAYOUT_STR.substitute( + layout="column-margin", + content="Some text to go on the sidebar" +) +``` + + '::: {.column-margin}\nSome text to go on the sidebar\n' + +Next we need to write a simple converter that operates at the *cell* +level: + +``` python +def convert_layout( + cell:dict, # A single cell from a Jupyter Notebook + is_multicell=False # Whether the div should be wrapped around multiple cells +): + "Takes a code cell that contains `div` in the directives and modifies the contents to the proper Quarto format" + content = cell.source + code = cell.source.splitlines(True) + div_ = cell.directives_["div"] + # We check if end is in the first line of the cell source + if "end" in div_: + # If it is, just fill the text with `:::` if no code exists there + cell.source = ":::" if len(code) == 1 else f'{code.source}:::' + else: + # Actually modify the code + cell.source = _LAYOUT_STR.substitute(layout=" ".join(div_), content=content) + if not is_multicell: cell.source += ":::" +``` + +Let’s go into detail on what’s happening here. + +``` python + content = cell.source +``` + +The source text of whatever exists in a notebook cell will live in +`.source`. + +``` python + code = cell.source.splitlines(True) +``` + +Then I want to extract the content of the cell and split them into +multiple lines, seperated by newlines. This let’s us check if a cell +just contains `#| div end`, which means that the div that was started +earlier should stop. + +``` python + div_ = cell.directives_["div"] +``` + +Any directives (comments in any cell marked with `#|`) will exist in the +`directives_` attribute as a dictionary. For our particular processor we +only care about the [`div`](https://nbdev.fast.ai/api/qmd.html#div) +directive + +``` python + if "end" in div_: + # If it is, just fill the text with `:::` if no code exists there + cell.source = ":::" if len(code) == 1 else f'{code.source}:::' + else: + # Actually modify the code + cell.source = _LAYOUT_STR.substitute(layout=" ".join(div_), content=content) + if not is_multicell: cell.source += ":::" +``` + +From there this last part checks whether to add the ending `:::` block +to the cell or to use the `_LAYOUT_STR` and inject the boilerplate div +CSS code in for Quarto. + +Let’s see it in action: + +``` python +cell = mk_cell( + """#| div margin-column +Here is something for the sidebar!""", + cell_type="markdown" +) +``` + +`nbdev` will pull out those directives and store them in the cell’s +`directives_` attribute using the +[`extract_directives`](https://nbdev.fast.ai/api/process.html#extract_directives) +function: + +``` python +cell.directives_ = extract_directives(cell, "#") +cell.directives_ +``` + + {'div': ['margin-column']} + +And now we can test out if our `convert_layout` function works! + +``` python +convert_layout(cell) +print(cell.source) +``` + + ::: {.margin-column} + Here is something for the sidebar! + ::: + +> Note: I print the `cell.source` here so that it’s text looks cleaner +> and what we would visually see in a `Markdown` cell + +Looks exactly like we wanted earlier! Great! + +How do we tell nbdev to use this and create this +[`Processor`](https://nbdev.fast.ai/api/process.html#processor) class +mentioned earlier? + +## Writing a [`Processor`](https://nbdev.fast.ai/api/process.html#processor) + +The second-to-last step here is to create the custom +[`Processor`](https://nbdev.fast.ai/api/process.html#processor) nbdev +utilizes to apply procs (things that modify the contents of cells). The +basic understanding of these is simply that you should create a class, +have it inherit +[`Processor`](https://nbdev.fast.ai/api/process.html#processor), and any +modifications that should be done must be defined in a `cell` function +which takes in a `cell` and modifies it in-place. + +``` python +class LayoutProc(Processor): + "A processor that will turn `div` based tags into proper quarto ones" + has_multiple_cells = False + def cell(self, cell): + if cell.cell_type == "markdown" and "div" in cell.directives_: + div_ = cell.directives_["div"] + if self.has_multiple_cells and "end" in div_: + convert_layout(cell) + else: + is_start = div_[-1] == "start" + if is_start: + self.has_multiple_cells = True + div_.remove("start") + convert_layout(cell, is_start) +``` + +How can we test if this will work or not? + +A minimal Jupyter Notebook is just a dictionary where the cells are in a +`cells` key and the cells themselves are a list of notebook cells +following a special format. We’ve created one of these above. `nbdev` +has a `dict2nb` function which let’s us convert this minimal idea of a +Jupyter Notebook into the true thing quickly. + +Afterwards, we can apply the processor to those cells though the +[`NBProcessor`](https://nbdev.fast.ai/api/process.html#nbprocessor) +class (what `nbdev` uses to apply these) + +``` python +from nbdev.process import NBProcessor, dict2nb +``` + +``` python +nb = { + "cells":[ + mk_cell("""#| div column-margin +A test""", "markdown"), + mk_cell("""#| div column-margin start +A test""", "markdown"), + mk_cell("""#| div end""", "markdown"), +]} +``` + +
+ +The `mk_cell` function will create a cell based on some `content` and a +`cell type`. The particular extension we’ve built works off `Markdown` +cells, so we set the type as `markdown`. + +
+ +The [`NBProcessor`](https://nbdev.fast.ai/api/process.html#nbprocessor) +takes in a list of procs (processors) that should be applied, and an +opened Jupyter Notebook: + +``` python +processor = NBProcessor(procs=LayoutProc, nb=dict2nb(nb)) +``` + +The act of applying these processors is done through calling the +`.process():` function + +``` python +processor.process() +``` + +And now we can see that those code cells were changed: + +``` python +for i in range(3): + print(f"Before:\n{nb['cells'][i].source}\n") + print(f"After:\n{processor.nb.cells[i].source}\n") +``` + + Before: + #| div column-margin + A test + + After: + ::: {.column-margin} + A test + ::: + + Before: + #| div column-margin start + A test + + After: + ::: {.column-margin} + A test + + + Before: + #| div end + + After: + ::: + +Great! We’ve successfully created a plugin for nbdev that will let us +lazily write markdown quarto directives easily. How can we actually +*use* this in our projects? + +## How to enable the plugin on your project + +This requires two changes to your `settings.ini`. + +First, if say this were code that lived in `nbdev`, we can add a special +`procs` key and specify where the processor comes from: + +``` ini +procs = + nbdev.extensions:LayoutProc +``` + +It follows the format of `library.module:processor_name` + +If this were being used from an external library (such as how this +processor is based on the one that lives in +[nbdev-extensions](https://muellerzr.github.io/nbdev-extensions), you +should add that to the requirements of your project: + +``` ini +requirements = nbdev-extensions +``` + +And you’re done! Now when calling +[`nbdev_docs`](https://nbdev.fast.ai/api/quarto.html#nbdev_docs) or +[`nbdev_preview`](https://nbdev.fast.ai/api/quarto.html#nbdev_preview) +the processor we just made will be *automatically* applied to your +notebooks and perform this conversion! + +## Conclusion, nbdev-extensions and a bit about me! + +Basically if there’s any part of a cell and how it should look either +from exporting modules, building documentation, or creating your own +special command to perform post-processing it can be done quickly and +efficiently with this +[`Processor`](https://nbdev.fast.ai/api/process.html#processor) class +nbdev provides! + +If you’re interested in seeing more examples of nbdev-extensions and +where you can take it I’ve (Zachary Mueller) written a library dedicated +to it called +[nbdev-extensions](https://muellerzr.github.io/nbdev-extensions) where +any ideas that may benefit how I approach nbdev I then turn into an +extension for the world to use. + +Thanks for reading!# nbdev1 Migration + + + +nbdev v2 is a new from-scratch rewrite of nbdev that’s not backwards +compatible. This page describes the changes you need to make to upgrade +your nbdev v1 repo to work with the new version. The steps shown here +should work on macOS or Linux (including Windows WSL) + +The biggest change is that nbdev2 uses [Quarto](https://quarto.org/) to +generate your website, whereas nbdev1 used nbconvert and jekyll. You can +use all of Quarto’s features directly in nbdev, so checkout the Quarto +website to see all the amazing functionality it supports. + +## Initial setup + +If you’ve pinned nbdev in `requirements.txt` or `settings.ini` (e.g +`nbdev<2`) remove the version pin. (*If you don’t know what this means, +then you don’t have it, so you can ignore this step*). + +Install the latest version of nbdev by typing: + + pip install -U nbdev + +or: + + conda install -c fastai nbdev + +You may need to restart your terminal for the new commands to be visible +to your shell. + +## Upgrade directives + +nbdev has slightly changed how “directive comments” like `export` and +`default_exp` work, in order to align with how Quarto does things. Now, +instead of just adding a `#` to the start to indicate a directive (e.g +`#export`), you now need to use `#|` (e.g `#|export`). You can also +optionally add a space (e.g `#| export`). + +To automatically upgrade your directives to the new format, run in the +root of your repo: + + nbdev_migrate + +You should now test that you can export your module by running: + + nbdev_export + +Note that +[`nbdev_export`](https://nbdev.fast.ai/api/doclinks.html#nbdev_export) +replaces `nbdev_build_lib`. Run `nbdev_export -h` to see the options you +can pass to it (normally you won’t need to pass any). To see a list of +all the commands available in nbdev2, run `nbdev_help`. + +## Add and remove files + +First set a variable with the name of your library, by running the +following (replacing “yourlib” with the name of your library’s +subdirectory) + + export LIBNAME=yourlib + +Now run the following: + +``` bash +git rm Makefile +git add $LIBNAME/_modidx.py +rm -rf docs +rm -f .gitconfig +rm -f .git/hooks/post-merge + +rm -f setup.py +curl -O https://raw.githubusercontent.com/fastai/nbdev-template/master/styles.css +curl -O https://raw.githubusercontent.com/fastai/nbdev-template/master/setup.py + +cat >>.gitignore < + +> **Note** +> +> All documentation related files should be included in your `nbs_path`, +> and all paths should be relative to it. If you have set the `nbs_path` +> in your `settings.ini` file, then copy your `styles.css` file inside +> of your `nbs_path` folder. + +
+ +If you use GitHub Actions for continuous integration (CI) you can update +this to use nbdev too as follows: + +``` bash +rm -f .github/workflows/main.yml +curl -O https://raw.githubusercontent.com/fastai/nbdev-template/master/.github/workflows/test.yaml +curl -O https://raw.githubusercontent.com/fastai/nbdev-template/master/.github/workflows/deploy.yaml +mv deploy.yaml test.yaml .github/workflows/ +``` + +## Update directive names + +A number of directives have changed names. We’ll use `perl` to fix them. +Run these lines in the root of your repo: + +``` bash +find . -name '*.ipynb' -exec perl -pi -e 's/#\|\s*hide_input/#| echo: false/' {} + +find . -name '*.ipynb' -exec perl -pi -e 's/#\|\s*hide_output/#| output: false/' {} + +find . -name '*.ipynb' -exec perl -pi -e 's/#\|\s*skip/#| eval: false/' {} + +find . -name '*.ipynb' -exec perl -pi -e 's/from nbdev.export import notebook2script/from nbdev import nbdev_export/' {} + +find . -name '*.ipynb' -exec perl -pi -e 's/notebook2script/nbdev_export/' {} + +``` + +These change the following directives to use functionality built into +Quarto: + +- `hide_input` –\> `echo: false` +- `hide_output` –\> `output: false` +- `skip` –\> `eval: false` + +They also update the new location and name of the +[`nbdev_export`](https://nbdev.fast.ai/api/doclinks.html#nbdev_export) +python function. + +If you have any notebooks that you’ve asked nbdev1 to skip (using +`all_slow`), you’ll need to add a [raw +cell](https://quarto.org/docs/tools/jupyter-lab.html#yaml-front-matter) +to the top of your notebook containing YAML frontmatter. The frontmatter +needs to include `skip_showdoc: true` to avoid running cells when +rendering docs, and `skip_exec: true` to skip this notebook when running +tests. E.g to do both, you would add a raw cell (or update your existing +frontmatter raw cell) to contain: + + --- + skip_showdoc: true + skip_exec: true + --- + +Or you can also add these flags in a markdown cell, + +``` md +# title +> description + +- skip_showdoc: true +- skip_exec: true +``` + +## Edit Workflow Permissions + +Make sure your workflow permissions are set to “Read and write +permissions”, which you can find in Settings → Actions → General → +Workflow permissions: + +
+ + +
+ +
+ +> **Important** +> +> Failure to set the correct permissions may result in an error message +> like this: +> +> fatal: unable to access 'https://github.com/user/repo.git/': The requested URL returned error: 403 +> Error: Action failed with "The process '/usr/bin/git' failed with exit code 128" + +
+ +## Edit GitHub Pages Permissions + +At this point you will want to commit the files with the changes you +made to GitHub. Wait for GitHub Actions to run and pass. A new branch in +your repo will automatically be created called `gh-pages`. You want to +enable GitHub Pages to work off this branch by configuring your Pages to +settings to look like this: + +Access this screen under Settings → Pages + +- Select “Deploy from a branch” in the drop down list for Source. +- Specify `gh-pages` as the branch +- Specify the `/root` as the location +- Click save + +
+ + +
+ +## Final steps + +You should now edit `settings.ini`, and change `doc_path` from `docs` to +`_docs`, since that’s where nbdev2 will build your website. + +If you use a custom domain for your website, you should move your +`CNAME` file into the directory containing your notebooks. + +Before pushing to GitHub, check that your website looks OK locally by +running: + + nbdev_preview + +Now prepare to commit to GitHub: + + nbdev_prepare + +You can now commit to GitHub as usual. Finally, update Github Pages by +clicking on the Settings tab in your repo, then click Pages on the left +side bar. Set “Source” to gh-pages branch and the /root folder.# config + + + +## Configuring nbdev + +nbdev is heavily customizeable, thanks to the configuration system +defined in this module. There are 2 ways to interact with nbdev’s +config: + +- **In the terminal:** + [`nbdev_create_config`](https://nbdev.fast.ai/api/config.html#nbdev_create_config) + creates a config file (if you’re starting a new project use + [`nbdev_new`](https://nbdev.fast.ai/api/cli.html#nbdev_new) instead) +- **In your library:** + [`get_config`](https://nbdev.fast.ai/api/config.html#get_config) + returns a + [`fastcore.foundation.Config`](https://fastcore.fast.ai/foundation.html#config) + object. + +Read on for more about how these work. + +------------------------------------------------------------------------ + +source + +### nbdev_create_config + +> nbdev_create_config (repo:str=None, branch:str=None, user:str=None, +> author:str=None, author_email:str=None, +> description:str=None, path:str='.', +> cfg_name:str='settings.ini', lib_name='%(repo)s', +> git_url='https://github.com/%(user)s/%(repo)s', +> custom_sidebar:=False, +> nbs_path:pathlib.Path='nbs', +> lib_path:pathlib.Path=None, +> doc_path:pathlib.Path='_docs', tst_flags='notest', +> version='0.0.1', +> doc_host='https://%(user)s.github.io', +> doc_baseurl='/%(repo)s', keywords='nbdev jupyter +> notebook python', license='apache2', +> copyright:str=None, status='3', min_python='3.7', +> audience='Developers', language='English', +> recursive:=True, +> black_formatting:=False, +> readme_nb='index.ipynb', title='%(lib_name)s', +> allowed_metadata_keys='', +> allowed_cell_metadata_keys='', +> jupyter_hooks:=False, +> clean_ids:=True, +> clear_all:=False, +> cell_number:=True, +> put_version_in_init:=True, +> skip_procs:str='') + +*Create a config file.* + + ++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TypeDefaultDetails
repostrNoneRepo name
branchstrNoneRepo default branch
userstrNoneRepo username
authorstrNonePackage author’s name
author_emailstrNonePackage author’s email address
descriptionstrNoneShort summary of the package
pathstr.Path to create config file
cfg_namestrsettings.iniName of config file to create
lib_namestr%(repo)sPackage name
git_urlstrhttps://github.com/%(user)s/%(repo)sRepo URL
custom_sidebarbool_argFalseUse a custom sidebar.yml?
nbs_pathPathnbsPath to notebooks
lib_pathPathNonePath to package root (default: repo with - +replaced by _)
doc_pathPath_docsPath to rendered docs
tst_flagsstrnotestTest flags
versionstr0.0.1Version of this release
doc_hoststrhttps://%(user)s.github.ioHostname for docs
doc_baseurlstr/%(repo)sBase URL for docs
keywordsstrnbdev jupyter notebook pythonPackage keywords
licensestrapache2License for the package
copyrightstrNoneCopyright for the package, defaults to ‘current_year +onwards, author’
statusstr3Development status PyPI classifier
min_pythonstr3.7Minimum Python version PyPI classifier
audiencestrDevelopersIntended audience PyPI classifier
languagestrEnglishLanguage PyPI classifier
recursivebool_argTrueInclude subfolders in notebook globs?
black_formattingbool_argFalseFormat libraries with black?
readme_nbstrindex.ipynbNotebook to export as repo readme
titlestr%(lib_name)sQuarto website title
allowed_metadata_keysstrPreserve the list of keys in the main notebook metadata
allowed_cell_metadata_keysstrPreserve the list of keys in cell level metadata
jupyter_hooksbool_argFalseRun Jupyter hooks?
clean_idsbool_argTrueRemove ids from plaintext reprs?
clear_allbool_argFalseRemove all cell metadata and cell outputs?
cell_numberbool_argTrueAdd cell number to the exported file
put_version_in_initbool_argTrueAdd the version to the main init.py in +nbdev_export
skip_procsstrA comma-separated list of processors that you want to skip
+ +The table above also serves as a full reference of nbdev’s settings +(excluding the `path` and `cfg_name` parameters which decide where the +config file is saved). For more about PyPI classifiers, see +[*Classifiers*](https://pypi.org/classifiers/). + +You can create a config file by passing all of the required settings via +the command line, as well as any optional settings you’d like to +override, for example: + +``` sh +nbdev_create_config --repo nbdev --user fastai --author fastai \ + --author_email info@fast.ai --description 'A test project' +``` + +If you don’t provide required settings from the command line, we’ll try +to to infer them from git and GitHub. Finally, you’ll be asked to +manually input any required settings that we couldn’t automatically fill +in. + +------------------------------------------------------------------------ + +source + +### get_config + +> get_config (cfg_name='settings.ini', path=None) + +*Return nbdev config.* + +Searches up from `path` until `cfg_name` is found. User settings are +loaded from `~/.config/nbdev/{cfg_name}`. Unspecified optional settings +return defaults. + +See +[`nbdev_create_config`](https://nbdev.fast.ai/api/config.html#nbdev_create_config) +for a full reference of nbdev’s settings. + +``` python +cfg = get_config() +``` + +`cfg` is a fastcore +[`Config`](https://fastcore.fast.ai/foundation.html#config) object, so +you can access keys as attributes: + +``` python +p = Path.cwd().parent.parent +test_eq(cfg.lib_name, 'nbdev') +test_eq(cfg.git_url, 'https://github.com/fastai/nbdev') +``` + +Its own path and parent are attributes too: + +``` python +test_eq(cfg.config_path, p) +test_eq(cfg.config_file, p/'settings.ini') +``` + +Paths are relative to the project: + +``` python +test_eq(cfg.doc_path, p/'_docs') +test_eq(cfg.lib_path, p/'nbdev') +test_eq(cfg.nbs_path, p/'nbs') +``` + +It automatically returns defaults for keys not specified in the config +file. Here we create an empty config file and access `lib_path` and +`copyright` even though they weren’t explicitly defined: + +``` python +with tempfile.TemporaryDirectory() as d, working_directory(d): + Config('.', 'test_settings.ini', {'repo': 'my-project', 'author': 'fastai', 'nbs_path': 'nbs'}); + cfg = get_config('test_settings.ini', '.') + test_eq(cfg.repo, 'my-project') + test_eq(cfg.lib_path.name, 'my_project') +``` + +In fact, you can return a default config even if you don’t have a +settings file. This is to support certain nbdev commands work outside of +nbdev repos: + +``` python +cfg = get_config('test_settings.ini', '.') +test_eq(cfg.lib_path, Path('nbdev').resolve()) +test_eq(cfg.nbs_path, Path('nbs').resolve()) +``` + +You can customise nbdev for all repositories for your user with a +`~/.config/nbdev/settings.ini` file (by default, although we follow the +broader [XDG +specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html)). +For example, you could globally disable nbdev’s Jupyter hooks by +creating a user settings file with `jupyter_hooks = False`. + +------------------------------------------------------------------------ + +source + +### config_key + +> config_key (c, default=None, path=True, missing_ok=None) + +*Deprecated: use `get_config().get` or `get_config().path` instead.* + +## Helpers + +------------------------------------------------------------------------ + +source + +### is_nbdev + +> is_nbdev () + +------------------------------------------------------------------------ + +source + +### create_output + +> create_output (txt, mime) + +*Add a cell output containing `txt` of the `mime` text MIME sub-type* + +------------------------------------------------------------------------ + +source + +### show_src + +> show_src (src, lang='python') + +``` python +show_src("print(create_output('text', 'text/plain'))") +``` + +``` python +print(create_output('text', 'text/plain')) +``` + +## Exporting a basic module + +------------------------------------------------------------------------ + +source + +### add_init + +> add_init (path=None) + +*Add `__init__.py` in all subdirs of `path` containing python files if +it’s not there already.* + +------------------------------------------------------------------------ + +source + +### update_version + +> update_version (path=None) + +*Add or update `__version__` in the main `__init__.py` of the library.* + +Python modules require a `__init.py__` file in all directories that are +modules. We assume that all directories containing a python file +(including in subdirectories of any depth) is a module, and therefore +add a `__init__.py` to each. + +``` python +with tempfile.TemporaryDirectory() as d: + d = Path(d) + (d/'a/b').mkdir(parents=True) + (d/'a/b/f.py').touch() + (d/'a/c').mkdir() + add_init(d) + assert not (d/'a/c'/_init).exists(), "Should not add init to dir without py file" + for e in [d, d/'a', d/'a/b']: assert (e/_init).exists(),f"Missing init in {e}" +``` + +------------------------------------------------------------------------ + +source + +### write_cells + +> write_cells (cells, hdr, file, offset=0, cell_number=True) + +*Write `cells` to `file` along with header `hdr` starting at index +`offset` (mainly for nbdev internal use).* + +This is a simple exporter with just enough functionality to correctly +export this notebook, in order to bootstrap the creation of nbdev +itself.# maker + + + +## Helpers + +## Variable helpers + +These functions let us find and modify the definitions of variables in +Python modules. + +------------------------------------------------------------------------ + +source + +### find_var + +> find_var (lines, varname) + +*Find the line numbers where `varname` is defined in `lines`* + +``` python +t = '''a_=(1, + 2, + 3) + +b_=3''' +test_eq(find_var(t.splitlines(), 'a_'), (0,3)) +test_eq(find_var(t.splitlines(), 'b_'), (4,5)) +``` + +------------------------------------------------------------------------ + +source + +### read_var + +> read_var (code, varname) + +*Eval and return the value of `varname` defined in `code`* + +``` python +test_eq(read_var(t, 'a_'), (1,2,3)) +test_eq(read_var(t, 'b_'), 3) +``` + +------------------------------------------------------------------------ + +source + +### update_var + +> update_var (varname, func, fn=None, code=None) + +*Update the definition of `varname` in file `fn`, by calling `func` with +the current definition* + +``` python +g = exec_new(t) +test_eq((g['a_'],g['b_']), ((1,2,3),3)) +t2 = update_var('a_', lambda o:0, code=t) +exec(t2, g) +test_eq((g['a_'],g['b_']), (0,3)) +t3 = update_var('b_', lambda o:0, code=t) +exec(t3, g) +test_eq((g['a_'],g['b_']), ((1,2,3),0)) +``` + +------------------------------------------------------------------------ + +source + +### ModuleMaker + +> ModuleMaker (dest, name, nb_path, is_new=True, parse=True) + +*Helper class to create exported library from notebook source cells* + +In order to export a notebook, we need an way to create a Python file. +[`ModuleMaker`](https://nbdev.fast.ai/api/maker.html#modulemaker) fills +that role. Pass in the directory where you want to module created, the +name of the module, the path of the notebook source, and set `is_new` to +`True` if this is a new file being created (rather than an existing file +being added to). The location of the saved module will be in `fname`. +Finally, if the source in the notebooks should not be parsed by Python +(such as partial class declarations in cells), `parse` should be set to +`False`. + +> Note: If doing so, then the `__all__` generation will be turned off as +> well. + +``` python +mm = ModuleMaker(dest='tmp', name='test.testing', nb_path=Path.cwd()/'04_export.ipynb', is_new=True) +mm.fname +``` + + Path('tmp/test/testing.py') + +------------------------------------------------------------------------ + +source + +### decor_id + +> decor_id (d) + +*`id` attr of decorator, regardless of whether called as function or +bare* + +------------------------------------------------------------------------ + +source + +### ModuleMaker.make_all + +> ModuleMaker.make_all (cells) + +*Create `__all__` with all exports in `cells`* + +------------------------------------------------------------------------ + +source + +### make_code_cells + +> make_code_cells (*ss) + +We want to add an `__all__` to the top of the exported module. This +methods autogenerates it from all code in `cells`. + +``` python +nb = make_code_cells("from __future__ import print_function", "def a():...", "def b():...", + "c=d=1", "_f=1", "_g=1", "_h=1", "_all_=['_g', _h]", "@patch\ndef h(self:ca):...") +test_eq(set(mm.make_all(nb)), set(['a','b','c','d', '_g', '_h'])) +``` + +------------------------------------------------------------------------ + +source + +### relative_import + +> relative_import (name, fname, level=0) + +*Convert a module `name` to a name relative to `fname`* + +``` python +test_eq(relative_import('nbdev.core', "xyz"), 'nbdev.core') +test_eq(relative_import('nbdev.core', 'nbdev'), '.core') +_p = Path('fastai') +test_eq(relative_import('fastai.core', _p/'vision'), '..core') +test_eq(relative_import('fastai.core', _p/'vision/transform'), '...core') +test_eq(relative_import('fastai.vision.transform', _p/'vision'), '.transform') +test_eq(relative_import('fastai.notebook.core', _p/'data'), '..notebook.core') +test_eq(relative_import('fastai.vision', _p/'vision'), '.') +test_eq(relative_import('fastai', _p), '.') +test_eq(relative_import('fastai', _p/'vision'), '..') +test_eq(relative_import('fastai', _p/'vision/transform'), '...') +``` + +------------------------------------------------------------------------ + +source + +### NbCell.import2relative + +> NbCell.import2relative (cell:execnb.nbio.NbCell, libname) + +------------------------------------------------------------------------ + +source + +### update_import + +> update_import (source, tree, libname, f=) + +``` python +ss = "from nbdev.export import *\nfrom nbdev.a.b import *" +cell = make_code_cells([ss])[0] +cell.import2relative('nbdev') +test_eq(cell.source, 'from .export import *\nfrom .a.b import *') + +cell = make_code_cells([ss])[0] +cell.import2relative('nbdev/a') +test_eq(cell.source, 'from ..export import *\nfrom .b import *') +``` + +------------------------------------------------------------------------ + +source + +### ModuleMaker.make + +> ModuleMaker.make (cells, all_cells=None, lib_path=None) + +*Write module containing `cells` with `__all__` generated from +`all_cells`* + +``` python +cells = make_code_cells("from __future__ import print_function", + "#|export\ndef a(): ...", "def b(): ...") +mm.make(cells, L([cells[2]])) +show_src(Path('tmp/test/testing.py').read_text(encoding='utf-8')) +``` + +``` python +# AUTOGENERATED! DO NOT EDIT! File to edit: ../../04_export.ipynb. + +# %% ../../04_export.ipynb 0 +from __future__ import print_function + +# %% auto 0 +__all__ = ['b'] + +# %% ../../04_export.ipynb +#|export +def a(): ... + +# %% ../../04_export.ipynb +def b(): ... +``` + +Pass `all_cells=[]` or `parse=False` if you don’t want any `__all__` +added. + +Passing `parse=False` is also handy for when writing broken up functions +or classes that +[`ast.parse`](https://docs.python.org/3/library/ast.html#ast.parse) +might not like but still want it to be exported, such as having a cell +with: + +``` python +#|export +class A: +``` + +Note that by doing so we cannot properly generate a `__all__`, so we +assume that it is unwanted. + +``` python +am = ModuleMaker(dest='tmp', name='test.testing_noall', nb_path=Path.cwd()/'01_export.ipynb', is_new=True, parse=False) +am.fname +``` + + Path('tmp/test/testing_noall.py') + +``` python +cells = make_code_cells("from __future__ import print_function", "#|export\ndef a(): ...", "#|export\nclass A:") +am.make(cells) +show_src(Path('tmp/test/testing_noall.py').read_text(encoding='utf-8')) +``` + +``` python +# AUTOGENERATED! DO NOT EDIT! File to edit: ../../01_export.ipynb. + +# %% ../../01_export.ipynb +from __future__ import print_function + +# %% ../../01_export.ipynb +#|export +def a(): ... + +# %% ../../01_export.ipynb +#|export +class A: +``` + +If `is_new=False` then the additional definitions are added to the +bottom, and any existing `__all__` is updated with the newly-added +symbols. + +``` python +c2 = make_code_cells("def c(): ...", "def d(): ...") +mm = ModuleMaker(dest='tmp', name='test.testing', nb_path=Path.cwd()/'04_export.ipynb', is_new=False) +mm.make(c2, c2) +``` + +``` python +show_src(Path('tmp/test/testing.py').read_text(encoding='utf-8')) +``` + +``` python +# AUTOGENERATED! DO NOT EDIT! File to edit: ../../04_export.ipynb. + +# %% ../../04_export.ipynb 0 +from __future__ import print_function + +# %% auto 0 +__all__ = ['b', 'c', 'd'] + +# %% ../../04_export.ipynb +#|export +def a(): ... + +# %% ../../04_export.ipynb +def b(): ... + +# %% ../../04_export.ipynb 0 +def c(): ... + +# %% ../../04_export.ipynb 1 +def d(): ... +``` + +``` python +try: + g = exec_import('tmp.test.testing', '*') + for s in "b c d".split(): assert s in g, s + assert 'a' not in g + assert g['b']() is None +finally: shutil.rmtree('tmp') +```# process + + + +Special comments at the start of a cell can be used to provide +information to `nbdev` about how to process a cell, so we need to be +able to find the location of these comments. + +``` python +minimal = read_nb('../../tests/minimal.ipynb') +``` + +------------------------------------------------------------------------ + +source + +### nb_lang + +> nb_lang (nb) + +------------------------------------------------------------------------ + +source + +### first_code_ln + +> first_code_ln (code_list, re_pattern=None, lang='python') + +*get first line number where code occurs, where `code_list` is a list of +code* + +``` python +_tst = """ +#|default_exp + #|export +#|hide_input +foo +""" +test_eq(first_code_ln(_tst.splitlines(True)), 4) +``` + +------------------------------------------------------------------------ + +source + +### extract_directives + +> extract_directives (cell, remove=True, lang='python') + +*Take leading comment directives from lines of code in `ss`, remove +`#|`, and split* + +Comment directives start with `#|`, followed by whitespace delimited +tokens, which +[`extract_directives`](https://nbdev.fast.ai/api/process.html#extract_directives) +extracts from the start of a cell, up until a blank line or a line +containing something other than comments. The extracted lines are +removed from the source. + +``` python +exp = AttrDict(source = """#|export module +#|eval:false +#| hide +# | foo bar +# |woo: baz +1+2 +#bar""") + +# this one has #|hide: with a colon at the end, wich is quarto compliant +exp2 = AttrDict(source = """#|export module +#|eval:false +#| hide: +# | foo bar +# |woo: baz +1+2 +#bar""") + +_answer = {'export':['module'], 'hide':[], 'eval:': ['false'], 'foo': ['bar'], 'woo:': ['baz']} + +test_eq(extract_directives(exp), _answer) +test_eq(extract_directives(exp2), _answer) +test_eq(exp.source, '#|eval: false\n# |woo: baz\n1+2\n#bar') +``` + +------------------------------------------------------------------------ + +source + +### opt_set + +> opt_set (var, newval) + +*newval if newval else var* + +------------------------------------------------------------------------ + +source + +### instantiate + +> instantiate (x, **kwargs) + +*Instantiate `x` if it’s a type* + +------------------------------------------------------------------------ + +source + +### NBProcessor + +> NBProcessor (path=None, procs=None, nb=None, debug=False, +> rm_directives=True, process=False) + +*Process cells and nbdev comments in a notebook* + +Cell processors can be callables (e.g regular functions), in which case +they are called for every cell (set a cell’s source to `None` to remove +the cell): + +``` python +everything_fn = '../../tests/01_everything.ipynb' + +def print_execs(cell): + if 'exec' in cell.source: print(cell.source) + +NBProcessor(everything_fn, print_execs).process() +``` + + --- + title: Foo + execute: + echo: false + --- + exec("o_y=1") + exec("p_y=1") + _all_ = [o_y, 'p_y'] + +Comment directives are put in a cell attribute `directive_` as a +dictionary keyed by directive name: + +``` python +def printme_func(cell): + if cell.directives_ and 'printme' in cell.directives_: print(cell.directives_['printme']) + +NBProcessor(everything_fn, printme_func).process() +``` + + ['testing'] + +However, a more convenient way to handle comment directives is to use a +*class* as a processor, and include a method in your class with the same +name as your directive, surrounded by underscores: + +``` python +class _PrintExample: + def _printme_(self, cell, to_print): print(to_print) + +NBProcessor(everything_fn, _PrintExample()).process() +``` + + testing + +In the case that your processor supports just one comment directive, you +can just use a regular function, with the same name as your directive, +but with an underscore appended – here `printme_` is identical to +`_PrintExample` above: + +``` python +def printme_(cell, to_print): print(to_print) + +NBProcessor(everything_fn, printme_).process() +``` + + testing + +``` python +NBProcessor(everything_fn, _PrintExample()).process() +``` + + testing + +------------------------------------------------------------------------ + +source + +### Processor + +> Processor (nb) + +*Base class for processors* + +For more complex behavior, inherit from +[`Processor`](https://nbdev.fast.ai/api/process.html#processor), and +override one of more of `begin()` (called before any cells are +processed), `cell()` (called for each cell), and `end()` (called after +all cells are processed). You can also include comment directives (such +as the `_printme` example above) in these subclasses. Subclasses will +automatically have access to `self.nb`, containing the processed +notebook. + +``` python +class CountCellProcessor(Processor): + def begin(self): + print(f"First cell:\n{self.nb.cells[0].source}") + self.count=0 + def cell(self, cell): + if cell.cell_type=='code': self.count += 1 + def end(self): print(f"* There were {self.count} code cells") +``` + +``` python +NBProcessor(everything_fn, CountCellProcessor).process() +``` + + First cell: + --- + title: Foo + execute: + echo: false + --- + * There were 26 code cells# export + + + +------------------------------------------------------------------------ + +source + +### ExportModuleProc + +> ExportModuleProc () + +*A processor which exports code to a module* + +Specify `dest` where the module(s) will be exported to, and optionally a +class to use to create the module +([`ModuleMaker`](https://nbdev.fast.ai/api/maker.html#modulemaker), by +default). + +Exported cells are stored in a `dict` called `modules`, where the keys +are the modules exported to. Those without an explicit module are stored +in the `'#'` key, which will be exported to `default_exp`. + +``` python +everything_fn = '../../tests/01_everything.ipynb' + +exp = ExportModuleProc() +proc = NBProcessor(everything_fn, exp) +proc.process() +test_eq(exp.default_exp, 'everything') +assert 'print_function' in exp.modules['#'][1].source +assert 'h_n' in exp.in_all['some.thing'][0].source +``` + +### Optional export processors + +------------------------------------------------------------------------ + +source + +### black_format + +> black_format (cell, force=False) + +*Processor to format code with `black`* + + ++++++ + + + + + + + + + + + + + + + + + + + + + + +
TypeDefaultDetails
cellCell to format
forceboolFalseTurn black formatting on regardless of settings.ini
+ +``` python +_cell = read_nb('../../tests/export_procs.ipynb')['cells'][0] +black_format(_cell, force=True) +test_eq(_cell.source, 'j = [1, 2, 3]') +``` + +------------------------------------------------------------------------ + +source + +### scrub_magics + +> scrub_magics (cell) + +*Processor to remove cell magics from exported code* + + + + + + + + + + + + + + +
Details
cellCell to format
+ +[`scrub_magics`](https://nbdev.fast.ai/api/export.html#scrub_magics) is +a processor that scrubs the jupyter “magics” lines out of exported +cells. This can be helpful when using tools like +[sparkmagic](https://github.com/jupyter-incubator/sparkmagic) or just +[Jupyter’s builtin +magics](https://ipython.readthedocs.io/en/stable/interactive/magics.html) +in an nbdev project. + +Usage: +This behavior can be enabled by passing +[`scrub_magics`](https://nbdev.fast.ai/api/export.html#scrub_magics) +into the `--procs` flag of the +[`nbdev_export`](https://nbdev.fast.ai/api/doclinks.html#nbdev_export) +command. - `nbdev_export --procs scrub_magics` - +`nbdev_export --procs 'scrub_magics black_format'` + +Example: + +A cell like below could export the line `"hello nbdev"` into the `bar` +module. And the `%%spark` magic line would be omitted. + +``` python +%%spark +#|export bar +"hello nbdev" +``` + +It will export as something similar to this: + +``` python +# %% ../path/to/01_bar.ipynb 1 +"hello nbdev" +``` + +``` python +_cell = read_nb('../../tests/export_procs.ipynb')['cells'][2] +scrub_magics(_cell) +test_eq(_cell.source, '''#|export bar +"hello nbdev"''') +``` + +------------------------------------------------------------------------ + +source + +### optional_procs + +> optional_procs () + +*An explicit list of processors that could be used by +[`nb_export`](https://nbdev.fast.ai/api/export.html#nb_export)* + +``` python +# every optional processor should be explicitly listed here +test_eq(optional_procs(), ['black_format', 'scrub_magics']) +``` + +### [`nb_export`](https://nbdev.fast.ai/api/export.html#nb_export) + +------------------------------------------------------------------------ + +source + +### nb_export + +> nb_export (nbname:str, lib_path:str=None, procs=None, name:str=None, +> mod_maker=, debug:bool=False) + +*Create module(s) from notebook* + + ++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TypeDefaultDetails
nbnamestrFilename of notebook
lib_pathstrNonePath to destination library. If not in a nbdev project, defaults to +current directory.
procsNoneTypeNoneProcessors to use
namestrNoneName of python script {name}.py to create.
mod_makertypeModuleMaker
debugboolFalseDebug mode
+ +Let’s check we can import a test file: + +``` python +shutil.rmtree('tmp', ignore_errors=True) +nb_export('../../tests/00_some.thing.ipynb', 'tmp') + +g = exec_new('import tmp.some.thing') +test_eq(g['tmp'].some.thing.__all__, ['a']) +test_eq(g['tmp'].some.thing.a, 1) +``` + +We’ll also check that our ‘everything’ file exports correctly: + +``` python +nb_export(everything_fn, 'tmp') + +g = exec_new('import tmp.everything; from tmp.everything import *') +_alls = L("a b d e m n o p q".split()) +for s in _alls.map("{}_y"): assert s in g, s +for s in "c_y_nall _f_y_nall g_n h_n i_n j_n k_n l_n".split(): assert s not in g, s +for s in _alls.map("{}_y") + ["c_y_nall", "_f_y_nall"]: assert hasattr(g['tmp'].everything,s), s +``` + +That notebook should also export one extra function to `tmp.some.thing`: + +``` python +del(sys.modules['tmp.some.thing']) # remove from module cache +g = exec_new('import tmp.some.thing') +test_eq(g['tmp'].some.thing.__all__, ['a','h_n']) +test_eq(g['tmp'].some.thing.h_n(), None) +``` + +``` python +Path('../nbdev/export.py').unlink(missing_ok=True) +nb_export('04_export.ipynb') + +g = exec_new('import nbdev.export') +assert hasattr(g['nbdev'].export, 'nb_export') +```# doclinks + + + +## Create the module index + +------------------------------------------------------------------------ + +source + +### patch_name + +> patch_name (o) + +*If `o` is decorated with `patch` or `patch_to`, return its class-prefix +name* + +``` python +def _test_patch(code): return patch_name(ast.parse(code).body[0]) +s = "@patch\ndef _f(self:_T): ..." +test_eq('_T._f', _test_patch(s)) +``` + +``` python +s = "@patch_to(_T)\ndef _g(self): ..." +test_eq('_T._g', _test_patch(s)) +``` + +``` python +# Get all patched classes when patching with a union +s = "@patch\ndef _f(self:_T|_U|_V): ..." +test_eq(_test_patch(s), ['_T._f', '_U._f', '_V._f']) +``` + +``` python +# _build_modidx() +``` + +## Export a notebook + +------------------------------------------------------------------------ + +source + +### nbglob + +> nbglob (path=None, skip_folder_re='^[_.]', file_glob='*.ipynb', +> skip_file_re='^[_.]', key='nbs_path', as_path=False, +> recursive:bool=True, symlinks:bool=True, file_re:str=None, +> folder_re:str=None, skip_file_glob:str=None, +> func:callable=, ret_folders:bool=False) + +*Find all files in a directory matching an extension given a config +key.* + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TypeDefaultDetails
pathpathlib.Path | strpath to start searching
skip_folder_restrNoneSkip folders matching regex,
file_globstrNoneOnly include files matching glob
skip_file_restrNoneSkip files matching regex
keystrnbs_path
as_pathboolFalse
recursiveboolTruesearch subfolders
symlinksboolTruefollow symlinks?
file_restrNoneOnly include files matching regex
folder_restrNoneOnly enter folders matching regex
skip_file_globstrNoneSkip files matching glob
funccallablejoinfunction to apply to each matched file
ret_foldersboolFalsereturn folders, not just files
ReturnsLPaths to matched files
+ +------------------------------------------------------------------------ + +source + +### nbglob_cli + +> nbglob_cli (path:str=None, symlinks:bool=False, file_glob:str='*.ipynb', +> file_re:str=None, folder_re:str=None, +> skip_file_glob:str=None, skip_file_re:str='^[_.]', +> skip_folder_re:str='^[_.]') + +*Find all files in a directory matching an extension given a config +key.* + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TypeDefaultDetails
pathstrNonePath to notebooks
symlinksboolFalseFollow symlinks?
file_globstr*.ipynbOnly include files matching glob
file_restrNoneOnly include files matching regex
folder_restrNoneOnly enter folders matching regex
skip_file_globstrNoneSkip files matching glob
skip_file_restr^[_.]Skip files matching regex
skip_folder_restr^[_.]Skip folders matching regex
+ +------------------------------------------------------------------------ + +source + +### nbdev_export + +> nbdev_export (path:str=None, +> procs:='black_format +> ', symlinks:bool=False, file_glob:str='*.ipynb', +> file_re:str=None, folder_re:str=None, +> skip_file_glob:str=None, skip_file_re:str='^[_.]', +> skip_folder_re:str='^[_.]') + +*Export notebooks in `path` to Python modules* + + ++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TypeDefaultDetails
pathstrNonePath or filename
procs<tokens naming the export processors to use.>black_format
symlinksboolFalseFollow symlinks?
file_globstr*.ipynbOnly include files matching glob
file_restrNoneOnly include files matching regex
folder_restrNoneOnly enter folders matching regex
skip_file_globstrNoneSkip files matching glob
skip_file_restr^[_.]Skip files matching regex
skip_folder_restr^[_.]Skip folders matching regex
+ +`procs` names the optional processors you wish to run on the exported +cells of your notebook. + +N.B.: the +[`black_format`](https://nbdev.fast.ai/api/export.html#black_format) +processor is passed in by default. But it is a no-op, unless +`black_formatting=True` is set in your `settings.ini` configuration. You +can omit it from +[`nbdev_export`](https://nbdev.fast.ai/api/doclinks.html#nbdev_export) +on the command line by passing in `--procs`. + +## Query the module index + +------------------------------------------------------------------------ + +source + +### NbdevLookup + +> NbdevLookup (strip_libs=None, incl_libs=None, skip_mods=None) + +*Mapping from symbol names to docs and source URLs* + +Indexing returns a link to the symbol’s docs, along with the name of the +source file the source URL if available. + +``` python +c = NbdevLookup() +c['nbdev.doclinks.NbdevLookup'] +``` + + ('https://nbdev.fast.ai/api/doclinks.html#nbdevlookup', + 'nbdev/doclinks.py', + 'https://github.com/fastai/nbdev/blob/master/nbdev/doclinks.py') + +------------------------------------------------------------------------ + +source + +### NbdevLookup.doc + +> NbdevLookup.doc (sym) + +*Link to docs for `sym`* + +``` python +c.doc('nbdev.doclinks.NbdevLookup') +``` + + 'https://nbdev.fast.ai/api/doclinks.html#nbdevlookup' + +Symbol names are taken from libraries registered using the ‘nbdev’ entry +point. By default, all libraries with this entry point are searched, but +full symbol names (including module prefix) are required. + +``` python +assert c.doc('numpy.array').startswith('http') +assert c.doc('NbdevLookup').endswith('#nbdevlookup') +assert not c.doc('array') +``` + +Pass `strip_libs` to list libraries which should be available without +requiring a module prefix. + +``` python +c = NbdevLookup(strip_libs=('nbdev', 'nbdev_numpy')) +assert c.doc('array').startswith('http') +``` + +------------------------------------------------------------------------ + +source + +### NbdevLookup.code + +> NbdevLookup.code (sym) + +*Link to source code for `sym`* + +``` python +NbdevLookup().code('fastcore.net.urlsend') +``` + + 'https://github.com/fastai/fastcore/blob/master/fastcore/net.py#LNone' + +------------------------------------------------------------------------ + +source + +### NbdevLookup.linkify + +> NbdevLookup.linkify (md) + +```` python +md = """This is a link to `numpy.array` and to `get_config` but not a link to `foobar`. +And not a link to dict2nb. + + This is not a link to `get_config` + +``` +This isn't a link to `get_config` either +```""" +```` + +``` python +print(NbdevLookup('nbdev').linkify(md)) +``` + + This is a link to [`numpy.array`](https://numpy.org/doc/stable/reference/generated/numpy.array.html#numpy.array) and to [`get_config`](https://nbdev.fast.ai/api/config.html#get_config) but not a link to `foobar`. + And not a link to dict2nb. + + This is not a link to `get_config` + + ``` + This isn't a link to `get_config` either + ```# sync + + + +The library is primarily developed in notebooks so any big changes +should be made there. But sometimes, it’s easier to fix small bugs or +typos in the modules directly. +[`nbdev_update`](https://nbdev.fast.ai/api/sync.html#nbdev_update) is +the function that will propagate those changes back to the corresponding +notebooks. Note that you can’t create new cells or reorder cells with +that functionality, so your corrections should remain limited. + +------------------------------------------------------------------------ + +source + +### absolute_import + +> absolute_import (name, fname, level) + +*Unwarps a relative import in `name` according to `fname`* + +``` python +test_eq(absolute_import('xyz', 'nbdev', 0), 'xyz') +test_eq(absolute_import('', 'nbdev', 1), 'nbdev') +test_eq(absolute_import(None, 'nbdev', 1), 'nbdev') +test_eq(absolute_import('core', 'nbdev', 1), 'nbdev.core') +test_eq(absolute_import('core', 'nbdev/vision', 2), 'nbdev.core') # from ..core import * +test_eq(absolute_import('transform', 'nbdev/vision', 1), 'nbdev.vision.transform') # from .transform import * +test_eq(absolute_import('notebook.core', 'nbdev/data', 2), 'nbdev.notebook.core') # from ..notebook.core import * +``` + +------------------------------------------------------------------------ + +source + +### nbdev_update + +> nbdev_update (fname:str=None) + +*Propagate change in modules matching `fname` to notebooks that created +them* + + + + + + + + + + + + + + + + + + +
TypeDefaultDetails
fnamestrNoneA Python file name to update
# merge + + + +## Introduction + +When working with jupyter notebooks (which are json files behind the +scenes) and GitHub, it is very common that a merge conflict (that will +add new lines in the notebook source file) will break some notebooks you +are working on. This module defines the function +[`nbdev_fix`](https://nbdev.fast.ai/api/merge.html#nbdev_fix) to fix +those notebooks for you, and attempt to automatically merge standard +conflicts. The remaining ones will be delimited by markdown cells like +this: + +`<<<<<< HEAD` + +``` python +# local code here +``` + +`======` + +``` python +# remote code here +``` + +`>>>>>> a7ec1b0bfb8e23b05fd0a2e6cafcb41cd0fb1c35` + +Below is an example of broken notebook. The json format is broken by the +lines automatically added by git. Such a file can’t be opened in jupyter +notebook. + +``` python +broken = Path('../../tests/example.ipynb.broken') +tst_nb = broken.read_text(encoding='utf-8') +print(tst_nb) +``` + + { + "cells": [ + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "3" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + <<<<<<< HEAD + "z=3\n", + ======= + "z=2\n", + >>>>>>> a7ec1b0bfb8e23b05fd0a2e6cafcb41cd0fb1c35 + "z" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "6" + ] + }, + <<<<<<< HEAD + "execution_count": 7, + ======= + "execution_count": 5, + >>>>>>> a7ec1b0bfb8e23b05fd0a2e6cafcb41cd0fb1c35 + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "x=3\n", + "y=3\n", + "x+y" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + } + }, + "nbformat": 4, + "nbformat_minor": 2 + } + +Note that in this example, the second conflict is easily solved: it just +concerns the execution count of the second cell and can be solved by +choosing either option without really impacting your notebook. This is +the kind of conflict we will fix automatically. The first conflict is +more complicated as it spans across two cells and there is a cell +present in one version, not the other. Such a conflict (and generally +the ones where the inputs of the cells change form one version to the +other) aren’t automatically fixed, but we will return a proper json file +where the annotations introduced by git will be placed in markdown +cells. + +## Creating a merged notebook + +The approach we use is to first “unpatch” the conflicted file, +regenerating the two files it was originally created from. Then we redo +the diff process, but using cells instead of text lines. + +------------------------------------------------------------------------ + +source + +### unpatch + +> unpatch (s:str) + +*Takes a string with conflict markers and returns the two original +files, and their branch names* + +The result of “unpatching” our conflicted test notebook is the two +original notebooks it would have been created from. Each of these +original notebooks will contain valid JSON: + +``` python +a,b,branch1,branch2 = unpatch(tst_nb) +dict2nb(loads(a)) +``` + +``` json +{ 'cells': [ { 'cell_type': 'code', + 'execution_count': 6, + 'idx_': 0, + 'metadata': {}, + 'outputs': [ { 'data': {'text/plain': ['3']}, + 'execution_count': 6, + 'metadata': {}, + 'output_type': 'execute_result'}], + 'source': 'z=3\nz'}, + { 'cell_type': 'code', + 'execution_count': 5, + 'idx_': 1, + 'metadata': {}, + 'outputs': [ { 'data': {'text/plain': ['6']}, + 'execution_count': 7, + 'metadata': {}, + 'output_type': 'execute_result'}], + 'source': 'x=3\ny=3\nx+y'}, + { 'cell_type': 'code', + 'execution_count': None, + 'idx_': 2, + 'metadata': {}, + 'outputs': [], + 'source': ''}], + 'metadata': { 'kernelspec': { 'display_name': 'Python 3', + 'language': 'python', + 'name': 'python3'}}, + 'nbformat': 4, + 'nbformat_minor': 2} +``` + +``` python +dict2nb(loads(b)) +``` + +``` json +{ 'cells': [ { 'cell_type': 'code', + 'execution_count': 6, + 'idx_': 0, + 'metadata': {}, + 'outputs': [ { 'data': {'text/plain': ['3']}, + 'execution_count': 6, + 'metadata': {}, + 'output_type': 'execute_result'}], + 'source': 'z=2\nz'}, + { 'cell_type': 'code', + 'execution_count': 5, + 'idx_': 1, + 'metadata': {}, + 'outputs': [ { 'data': {'text/plain': ['6']}, + 'execution_count': 5, + 'metadata': {}, + 'output_type': 'execute_result'}], + 'source': 'x=3\ny=3\nx+y'}, + { 'cell_type': 'code', + 'execution_count': None, + 'idx_': 2, + 'metadata': {}, + 'outputs': [], + 'source': ''}], + 'metadata': { 'kernelspec': { 'display_name': 'Python 3', + 'language': 'python', + 'name': 'python3'}}, + 'nbformat': 4, + 'nbformat_minor': 2} +``` + +``` python +branch1,branch2 +``` + + ('HEAD', 'a7ec1b0bfb8e23b05fd0a2e6cafcb41cd0fb1c35') + +------------------------------------------------------------------------ + +source + +### nbdev_fix + +> nbdev_fix (nbname:str, outname:str=None, nobackup: bool_arg>=True, theirs:bool=False, noprint:bool=False) + +*Create working notebook from conflicted notebook `nbname`* + + ++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TypeDefaultDetails
nbnamestrNotebook filename to fix
outnamestrNoneFilename of output notebook (defaults to nbname)
nobackupbool_argTrueDo not backup nbname to nbname.bak if +outname not provided
theirsboolFalseUse their outputs and metadata instead of ours
noprintboolFalseDo not print info about whether conflicts are found
+ +This begins by optionally backing the notebook `fname` to `fname.bak` in +case something goes wrong. Then it parses the broken json, solving +conflicts in cells. Every conflict that only involves metadata or +outputs of cells will be solved automatically by using the local +(`theirs==False`) or the remote (`theirs==True`) branch. Otherwise, or +for conflicts involving the inputs of cells, the json will be repaired +by including the two version of the conflicted cell(s) with markdown +cells indicating the conflicts. You will be able to open the notebook +again and search for the conflicts (look for `<<<<<<<`) then fix them as +you wish. + +A message will be printed indicating whether the notebook was fully +merged or if conflicts remain. + +``` python +nbdev_fix(broken, outname='tmp.ipynb') +chk = read_nb('tmp.ipynb') +test_eq(len(chk.cells), 7) +os.unlink('tmp.ipynb') +``` + + One or more conflict remains in the notebook, please inspect manually. + +## Git merge driver + +------------------------------------------------------------------------ + +source + +### nbdev_merge + +> nbdev_merge (base:str, ours:str, theirs:str, path:str) + +*Git merge driver for notebooks* + +This implements a [git merge +driver](https://git-scm.com/docs/gitattributes#_defining_a_custom_merge_driver) +for notebooks that automatically resolves conflicting metadata and +outputs, and splits remaining conflicts as separate cells so that the +notebook can be viewed and fixed in Jupyter. The easiest way to install +it is by running +[`nbdev_install_hooks`](https://nbdev.fast.ai/api/clean.html#nbdev_install_hooks). + +This works by first running Git’s default merge driver, and then +[`nbdev_fix`](https://nbdev.fast.ai/api/merge.html#nbdev_fix) if there +are still conflicts. You can set +[`nbdev_fix`](https://nbdev.fast.ai/api/merge.html#nbdev_fix)’s `theirs` +argument using the `THEIRS` environment variable, for example: + + THEIRS=True git merge branch# showdoc + + + +## Rendering docment Tables + +Render nicely formatted tables that shows `docments` for any function or +method. + +------------------------------------------------------------------------ + +source + +### DocmentTbl + +> DocmentTbl (obj, verbose=True, returns=True) + +*Compute the docment table string* + +[`DocmentTbl`](https://nbdev.fast.ai/api/showdoc.html#docmenttbl) can +render a markdown table showing `docments` if appropriate. This is an +example of how a `docments` table will render for a function: + +``` python +def _f(a, # description of param a + b=True, # description of param b + c:str=None + ) -> int: ... + +_dm = DocmentTbl(_f) +_dm +``` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TypeDefaultDetails
adescription of param a
bboolTruedescription of param b
cstrNone
Returnsint
+ +If one column in the table has no information, for example because there +are no default values, that column will not be shown. In the below +example, the **Default** column, will not be shown. Additionally, if the +return of the function is not annotated the **Returns** row will not be +rendered: + +``` python +def _f(a, + b, #param b + c #param c + ): ... + +_dm2 = DocmentTbl(_f) +_dm2 +``` + + + + + + + + + + + + + + + + + + + + + + +
Details
a
bparam b
cparam c
+ +[`DocmentTbl`](https://nbdev.fast.ai/api/showdoc.html#docmenttbl) also +works on classes. By default, the `__init__` will be rendered: + +``` python +class _Test: + def __init__(self, + a, # description of param a + b=True, # description of param b + c:str=None): + ... + + def foo(self, + c:int, # description of param c + d=True, # description of param d + ): + ... +``` + +``` python +DocmentTbl(_Test) +``` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TypeDefaultDetails
adescription of param a
bboolTruedescription of param b
cstrNone
+ +You can also pass a method to be rendered as well: + +``` python +DocmentTbl(_Test.foo) +``` + + + + + + + + + + + + + + + + + + + + + + + + +
TypeDefaultDetails
cintdescription of param c
dboolTruedescription of param d
+ +## Documentation For An Object + +Render the signature as well as the `docments` to show complete +documentation for an object. + +------------------------------------------------------------------------ + +source + +### ShowDocRenderer + +> ShowDocRenderer (sym, name:str|None=None, title_level:int=3) + +*Show documentation for `sym`* + +------------------------------------------------------------------------ + +source + +### BasicMarkdownRenderer + +> BasicMarkdownRenderer (sym, name:str|None=None, title_level:int=3) + +*Markdown renderer for +[`show_doc`](https://nbdev.fast.ai/api/showdoc.html#show_doc)* + +------------------------------------------------------------------------ + +source + +### show_doc + +> show_doc (sym, renderer=None, name:str|None=None, title_level:int=3) + +*Show signature and docstring for `sym`* + + ++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TypeDefaultDetails
symSymbol to document
rendererNoneTypeNoneOptional renderer (defaults to markdown)
namestr | NoneNoneOptionally override displayed name of sym
title_levelint3Heading level to use for symbol name
+ +You can use +[`show_doc`](https://nbdev.fast.ai/api/showdoc.html#show_doc) to +document apis of functions, classes or methods. + +### Numpy Docstrings + +if you have [numpy +docstrings](https://numpydoc.readthedocs.io/en/latest/format.html) +instead of `docments`, +[`show_doc`](https://nbdev.fast.ai/api/showdoc.html#show_doc) will +attempt to parse and render those just like `docments`. + +
+ +> **Warning** +> +> Numpy docstring formatting is very strict. If your docstrings do not +> strictly adhere to the numpy format, it will not be parsed properly +> and information about parameters and return values may not properly be +> rendered in the table below the signature. Where possible, we +> recommend using `docments` to annonate your function instead. + +
+ +## show_doc on Classes + +[`show_doc`](https://nbdev.fast.ai/api/showdoc.html#show_doc) works on +Classes, too, including when you use `@patch`. + +You can define methods for the class `Foo` with `@patch` which is +convenient in allowing you to break up code for documentation in +notebooks. + +Class properties also work with showdoc. + +## Pluggable renderers + +You can replace the default markdown show_doc renderer with custom +renderers. For instance, nbdev comes with a simple example for rendering +with raw HTML. + +------------------------------------------------------------------------ + +source + +### BasicHtmlRenderer + +> BasicHtmlRenderer (sym, name:str|None=None, title_level:int=3) + +*HTML renderer for +[`show_doc`](https://nbdev.fast.ai/api/showdoc.html#show_doc)* + +------------------------------------------------------------------------ + +source + +### doc + +> doc (elt) + +*Show [`show_doc`](https://nbdev.fast.ai/api/showdoc.html#show_doc) info +along with link to docs* + +------------------------------------------------------------------------ + +source + +### showdoc_nm + +> showdoc_nm (tree) + +*Get the fully qualified name for showdoc.* + +## Other helpers + +------------------------------------------------------------------------ + +source + +### colab_link + +> colab_link (path) + +*Get a link to the notebook at `path` on Colab* + +``` python +colab_link('index') +``` + +[Open `index` in +Colab](https://colab.research.google.com/github/fastai/nbdev/blob/master/nbs/index.ipynb)# frontmatter + + + +------------------------------------------------------------------------ + +source + +### FrontmatterProc + +> FrontmatterProc (nb) + +*A YAML and formatted-markdown frontmatter processor* + +YAML frontmatter can be added to notebooks in one of two ways: + +1. By adding a raw notebook cell with `---` as the first and last + lines, and YAML between them, or +2. A specially formatted markdown cell. The first line should be start + with a single `#` (creating an H1 heading), and becomes the title. + Then, optionally, a line beginning with `>` (creating a quote + block), which becomes the description. Finally, zero or more lines + beginning with `-` (creating a list), each of which contains YAML. + (If you already have “title” defined in frontmatter in a raw cell, + then markdown cells will be ignored.) + +For instance, our test notebook contains the following markdown cell: + + # a title + > A description + - key1: value1 + - key2: value2 + - categories: [c1, c2] + +It also contains the following raw cell: + + --- + execute: + echo: false + --- + +When we process with +[`FrontmatterProc`](https://nbdev.fast.ai/api/frontmatter.html#frontmatterproc), +these will both be removed, and a single raw cell will be added to the +top, containing the combined YAML frontmatter: + +``` python +nbp = NBProcessor(_test_file, procs=FrontmatterProc) +nbp.process() +print(nbp.nb.cells[0].source) +``` + + --- + categories: + - c1 + - c2 + description: A description + execute: + echo: false + key1: value1 + key2: value2 + output-file: docs_test.html + title: a title + + --- + +In addition, a `frontmatter_` attr will be added to the notebook, +containing this information as a `dict`: + +``` python +d = nbp.nb.frontmatter_ +d +``` + + {'execute': {'echo': False}, + 'title': 'a title', + 'description': 'A description', + 'key1': 'value1', + 'key2': 'value2', + 'categories': ['c1', 'c2'], + 'output-file': 'docs_test.html'}# processors + + + +On this page we’ll be using this private helper to process a notebook +and return the results, to simplify testing: + +``` python +def _run_procs(procs=None, return_nb=False, path=_test_file): + nbp = NBProcessor(path, procs) + nbp.process() + if return_nb: return nbp.nb + return '\n'.join([str(cell) for cell in nbp.nb.cells]) +``` + +------------------------------------------------------------------------ + +source + +### populate_language + +> populate_language (nb) + +*Set cell language based on NB metadata and magics* + +------------------------------------------------------------------------ + +source + +### insert_warning + +> insert_warning (nb) + +*Insert Autogenerated Warning Into Notebook after the first cell.* + +This preprocessor inserts a warning in the markdown destination that the +file is autogenerated. This warning is inserted in the second cell so we +do not interfere with front matter. + +``` python +res = _run_procs(insert_warning) +assert "