Skip to content

feat: converting documentation to mkdocs #5276

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 49 commits into
base: main
Choose a base branch
from
Draft

Conversation

gvwilson
Copy link
Contributor

@gvwilson gvwilson commented Jul 10, 2025

Paired with plotly/plotly.js#7469 (formatting changes to JavaScript that Python docs are derived from).

  • Move contents of commands.py to scripts in new bin directory and break
    into individual scripts.
    • Tidy up imports etc. while doing so.
  • Move code generation from ./codegen to ./bin/codegen.
  • Move plot-schema.json to ./resources rather than burying it under the
    codegen folder.
  • Add Makefile to run commands instead of putting everything in commands.py.
    • Run ruff directly for checking and formatting rather than launching a
      subprocess from Python as commands.py did.
  • Modify .gitignore to ignore docsanddocs_tmp. (Will eventually want to include docsor overwritedoc`.)
  • Minor reformatting of README.md.
  • Update CONTRIBUTING.md to describe relocation of commands and code
    generation to bin.
    • CONTRIBUTING.md documents --local, --devrepo and --devbranch options
      for updating JavaScript bundle that commands.py didn't seem to provide.
  • Add mkdocs.yml configuration file for mkdocs.
    • Most of this file was vibe coded using Claude.
    • mkdocs does not support reading configuration from pyproject.toml, so
      we need the extra config file.
    • Use material theme.
    • Read hand-written Markdown from pages and write output to docs.
    • Generate module index pages on the fly using mkdocs-gen-files plugin.
      (See discussion of bin/generate_reference_pages.py below.)
    • Set docstring style to google (even though much of our documentation
      isn't formatted that way).
  • Add placeholder Markdown files in pages that include files from the root
    directory (README, code of conduct, contributors' guide, license).
    • Remove relative links between these pages because they don't work when
      the content is transcluded one directory lower.
  • Modify docstring in plotly/_subplots.py to escape closing ] with backslash
    to avoid confusing mkdocs Markdown.
    • Here and elsewhere the escape is written \\] because we need \] in the
      actual string. We could convert the docstrings to literal strings prefixed
      with r to avoid the double backslash.
    • Have also escaped some [ as \\[ for the same reason.
  • Similar change to plotly/basedatatypes.py.
  • Reformat line breaks in docstrings in plotly/express/_core.py.
  • Modify pyproject.toml to install mkdocs and related packages for dev.
  • Modify pyproject.toml to install pydoclint for checking documentation.
    • Currently reporting a lot of errors.
  • Update uv.lock to match.

To do:

  • Move scripts/gen_ref_pages.py into commands.py. (Can't do this because mkdocs.yml only accepts the name of the script, not parameters.)
  • Include content from docstrings in hand-written Python code.
  • Include content from docstrings in generated Python code.
  • Include navigation links in left nav bar.
  • Clean up warnings produced by mkdocs.
  • Add linting for docstring style.
  • Clean up (many) errors reported by pydoclint.
  • Re-introduce links between .md files in root directory (taken out because they don't resolve when those files are transcluded from pages/*.md).
  • Fix \[ formatting glitch in the following files (caused by replacing ][ with ]\[ to avoid Markdown thinking it's a reference link, but the text is inside back quotes):
    • plotly/graph_objs/histogram2dcontour/_contours/index.html
    • plotly/graph_objs/contour/_contours/index.html
    • plotly/graph_objs/contourcarpet/_contours/index.html
    • plotly/graph_objs/table/_header/index.html
    • plotly/graph_objs/table/_cells/index.html

@gvwilson gvwilson requested a review from LiamConnors July 10, 2025 13:57
@gvwilson gvwilson added feature something new P1 needed for current cycle documentation written for humans labels Jul 10, 2025
@gvwilson gvwilson force-pushed the mkdocs-conversion branch 2 times, most recently from 6c0e2ec to 72e0aef Compare July 15, 2025 17:54
@gvwilson gvwilson force-pushed the mkdocs-conversion branch from 808090e to 8011f8f Compare July 17, 2025 13:00
@gvwilson gvwilson force-pushed the mkdocs-conversion branch from 8e7ecf8 to 347ad20 Compare July 25, 2025 15:31
-   Add `bin/run_markdown.py` (with help from Claude).
    -   Runs Python chunks embedded in Markdown, writing result as Markdown.
    -   Has option to embed interactive figures as well as generate PNG.
-   Modify `Makefile` to run the script on selected files for testing purposes.
    -   Commented-out target runs on all.

To do:

- [ ] Figure out why `bin/run_markdown.py` fails with "too many open files" for large numbers of input files.
- [ ] Modify `Makefile` to allow select re-running as well as batch runs.
- [ ] Modify `bin/run_markdown.py` to use a single Kaleido sub-process to speed up image generation.
@gvwilson gvwilson force-pushed the mkdocs-conversion branch from 347ad20 to 9525f6a Compare July 25, 2025 15:31
gvwilson and others added 17 commits July 26, 2025 11:35
-   Updates `pyproject.toml` to install packages previously listed in `doc/requirements.txt`.
-   Removes `doc/requirements.txt`.
-   Run `python bin/check-all-md.py doc/python/*.md` to re-run all examples.
Exclude code blocks that aren't run
Changes were made to line 2037, in the case where a node is valType info_array in the JSON with dimension: 2 (eg. groups in _node.py line 137).
This fixes the issue where markdown interprets it as a hyperlink. This line adds '[][]' to the description after the description passed in the JSON file as part of the validation process.
Now, markdown correctly interprets the brackets as consecutive brackets instead.
All three cases in 'InfoArrayValidator' no longer
encounter '[][]' errors. For examples, see
'_node.py' groups property, '_image.py' zmax property,
'_dimension.py' constraintrange property.

Keep the newline between "...specified as"
and "* ..." to avoid rendering a code block.

Note: Python now raises a SyntaxWarning due to '\['.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation written for humans feature something new P1 needed for current cycle
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants