Skip to content

Commit 8c423d0

Browse files
alcarneyalcarney
committed
docs: Rewrite devenv setup guide
1 parent c51f8eb commit 8c423d0

File tree

3 files changed

+263
-48
lines changed

3 files changed

+263
-48
lines changed

‎docs/contributing/devenvs.rst‎

Lines changed: 0 additions & 48 deletions
This file was deleted.

‎docs/contributing/howto.rst‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
.. _contributing-howto:
2+
3+
How To
4+
======
5+
6+
This section contains a number of guides that may be useful to contributors to the Esbonio project.
7+
8+
.. toctree::
9+
10+
Setup the Development Environment <howto/setup-development-environment>

‎docs/contributing/howto/setup-development-environment.rst‎

Lines changed: 253 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,253 @@
1+
How To Setup the Development Environment
2+
========================================
3+
4+
.. highlight:: none
5+
6+
.. warning::
7+
8+
This guide assumes you are **not** using Windows.
9+
If you are using Windows, you may still be able to get some use from it, but feel free to `open an issue <https://github.com/swyddfa/esbonio/issues/new>`__ if you get stuck!
10+
11+
This guide outlines how to setup the development environment(s) necessary to work on the various components located in the `swyddfa/esbonio <https://github.com/swyddfa/esbonio>`__ GitHub repository.
12+
13+
**Dev Container**
14+
15+
This repository provides a simple `devcontainer <https://containers.dev/overview>`__ definition that you can use.
16+
17+
Note that using this container is **entirely optional** and you can easily setup a development environment without it.
18+
The devcontainer is only used to provide a known starting point from which all scripts and automations in the repository can build against.
19+
20+
.. dropdown:: .devcontainer/devcontainer.json
21+
22+
.. literalinclude:: ../../../.devcontainer/devcontainer.json
23+
:language: json
24+
25+
26+
**Makefiles**
27+
28+
.. _uv: https://docs.astral.sh/uv/
29+
.. _Makefiles: https://makefiletutorial.com/
30+
31+
*Use of the Makefiles is optional, but recommended to those who are unfamiliar with the repository.*
32+
33+
Many common tasks, from installing tools such as `uv`_ to packaging the VSCode extension have been automated in `Makefiles`_ located throughout the repository.
34+
35+
While they are written and tested with the environment provided by the devcontainer in mind, the Makefiles try, where possible, to work in "any" Unix-like environment.
36+
This is done by reusing tools already present on your ``$PATH`` and automatically installing any missing tools into ``$HOME/.local``.
37+
38+
Running ``make tools`` in the root of the repository will list all tools required for this project, **automatically installing them if necessary** ::
39+
40+
$ make tools
41+
...
42+
/home/vscode/.local/bin/uv uv 0.5.21
43+
/home/vscode/.local/bin/python Python 3.13.1
44+
/home/vscode/.local/bin/hatch Hatch, version 1.14.0
45+
/home/vscode/.local/bin/pre-commit pre-commit 4.0.1
46+
/home/vscode/.local/bin/npm 10.8.2
47+
/home/vscode/.local/bin/npx 10.8.2
48+
49+
.. tip::
50+
51+
The provided devcontainer is the result of taking a standard Ubuntu image and running ``make tools``
52+
53+
Full details on how the tools are installed can be found in the ``.devcontainer/tools.mk`` file
54+
55+
.. dropdown:: .devcontainer/tools.mk
56+
57+
.. literalinclude:: ../../../.devcontainer/tools.mk
58+
:language: make
59+
60+
61+
.. _devenv_lsp:
62+
63+
Language Server
64+
---------------
65+
66+
The language server is written in pure Python, so if you are familiar with Python development, you should not find anything too surprising here.
67+
68+
We use `hatch <https://hatch.pypa.io/latest/>`__ both for packaging the language server as well as managing all the test environments.
69+
70+
To produce both source and wheel packages run the following command from the ``lib/esbonio`` directory ::
71+
72+
$ hatch build
73+
───────────────────────────────────────── sdist ─────────────────────────────────────────
74+
dist/esbonio-1.0.0b9.tar.gz
75+
───────────────────────────────────────── wheel ─────────────────────────────────────────
76+
dist/esbonio-1.0.0b9-py3-none-any.whl
77+
78+
Running tests is a little more involved, while running ``hatch test`` indeed looks promising ::
79+
80+
$ hatch test
81+
========================================= test session starts =========================================
82+
platform linux -- Python 3.13.1, pytest-8.3.4, pluggy-1.5.0
83+
rootdir: /workspaces/develop/lib/esbonio
84+
configfile: pyproject.toml
85+
plugins: asyncio-0.25.2, rerunfailures-14.0, mock-3.14.0, lsp-1.0.0b2, xdist-3.6.1
86+
asyncio: mode=Mode.AUTO, asyncio_default_fixture_loop_scope=function
87+
collected 254 items
88+
89+
tests/server/feature/test_completion.py ................... [ 7%]
90+
tests/server/features/test_directive_completion.py ............................... [ 19%]
91+
tests/server/features/test_logging.py ....... [ 22%]
92+
tests/server/features/test_role_completion.py ........... [ 26%]
93+
tests/server/features/test_sphinx_config.py ....s..s......ss.s.ss [ 35%]
94+
tests/server/test_configuration.py .....s.s.s.s.s.s.s.s.s.s......ssssss...... [ 51%]
95+
tests/server/test_patterns.py ................................................................. [ 77%]
96+
.......................................................... [100%]
97+
98+
=================================== 231 passed, 23 skipped in 1.24s ===================================
99+
100+
This is in fact only a small subset of the full test suite!
101+
To see the full set of test environments, run the following command ::
102+
103+
$ hatch env show --internal
104+
Matrices
105+
┏━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
106+
┃ Name ┃ Type ┃ Envs ┃ Dependencies ┃ Environment variables ┃ Scripts ┃
107+
┡━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
108+
│ hatch-test │ virtual │ hatch-test.py3.9 │ coverage-enable-subprocess==1.0 │ UV_PRERELEASE=allow │ cov-combine │
109+
│ │ │ hatch-test.py3.10 │ coverage[toml]~=7.4 │ │ cov-report │
110+
│ │ │ hatch-test.py3.11 │ pytest-lsp>=1.0b0 │ │ run │
111+
│ │ │ hatch-test.py3.12 │ pytest-mock~=3.12 │ │ run-cov │
112+
│ │ │ hatch-test.py3.13 │ pytest-randomly~=3.15 │ │ │
113+
│ │ │ hatch-test.py3.9-sphinx6 │ pytest-rerunfailures~=14.0 │ │ │
114+
│ │ │ hatch-test.py3.9-sphinx7 │ pytest-xdist[psutil]~=3.5 │ │ │
115+
│ │ │ hatch-test.py3.10-sphinx6 │ pytest~=8.1 │ │ │
116+
│ │ │ hatch-test.py3.10-sphinx7 │ │ │ │
117+
│ │ │ hatch-test.py3.11-sphinx6 │ │ │ │
118+
│ │ │ hatch-test.py3.11-sphinx7 │ │ │ │
119+
│ │ │ hatch-test.py3.12-sphinx6 │ │ │ │
120+
│ │ │ hatch-test.py3.12-sphinx7 │ │ │ │
121+
│ │ │ hatch-test.py3.13-sphinx6 │ │ │ │
122+
│ │ │ hatch-test.py3.13-sphinx7 │ │ │ │
123+
│ │ │ hatch-test.py3.10-sphinx8 │ │ │ │
124+
│ │ │ hatch-test.py3.11-sphinx8 │ │ │ │
125+
│ │ │ hatch-test.py3.12-sphinx8 │ │ │ │
126+
│ │ │ hatch-test.py3.13-sphinx8 │ │ │ │
127+
└────────────┴─────────┴───────────────────────────┴─────────────────────────────────┴───────────────────────┴─────────────┘
128+
129+
In addition to an environment for each version of Python we support, we also define an environment for each version of Sphinx we support.
130+
The following will run all the tests for a given Python version ::
131+
132+
$ hatch test --include py=3.x
133+
134+
See the `upstream documentation <https://hatch.pypa.io/latest/tutorials/testing/overview/>`__ for more details on the ``hatch test`` command.
135+
136+
Makefile Targets
137+
^^^^^^^^^^^^^^^^
138+
139+
For convenience the following make targets are available in the ``lib/esbonio`` directory
140+
141+
- ``make dist``: Package esbonio as a ``*.whl`` file
142+
- ``make test``: Run the full test suite for your current Python version
143+
144+
145+
.. _Python: https://www.python.org/
146+
147+
VSCode Extension
148+
----------------
149+
150+
The development environment for the VSCode is quite involved as not only does it depend on the language server and the TypeScript glue code, but it also requires two separate, standalone Python environments.
151+
The fully packaged version of the extension contains the following key folders ::
152+
153+
esbonio-0.96.1.vsix
154+
└─ extension/
155+
├─ bundled/
156+
│ ├─ env/ (3330 files) [50.93 MB] <-- Bundled fallback Sphinx environment
157+
│ └─ libs/ (530 files) [4.39 MB] <-- Bundled language server and dependencies
158+
├─ dist/
159+
│ └─ node/ (1 file) [794.54 KB] <-- Glue compiled JavaScript code that integrates the server into VSCode
160+
├─ guides/
161+
└─ syntaxes/
162+
163+
All of which have to be setup on your machine
164+
165+
.. tip::
166+
167+
When working on the VSCode extension, the provided :ref:`Makefile <devenv-vscode-makefile>` is especially useful.
168+
169+
**Compiled TypeScript**
170+
171+
To compile the necessary type script code, first you need to install the necessary dependencies.
172+
Run the following command in the ``code/`` directory ::
173+
174+
$ npm ci
175+
176+
Then to compile ::
177+
178+
$ npm run compile
179+
180+
Alternatively, if you want to automatically re-compile each time you modify the source ::
181+
182+
$ npm run watch
183+
184+
**Python Environments**
185+
186+
To setup the necessary Python environments, run the following commands in the ``code/`` directory ::
187+
188+
$ python -m pip install -t ./bundled/env --no-cache-dir --implementation py --no-deps --upgrade -r ./requirements-env.txt
189+
$ python -m pip install -t ./bundled/libs --no-cache-dir --implementation py --no-deps --upgrade -r ./requirements-libs.txt
190+
191+
**Install Esbonio**
192+
193+
The previous ``pip install`` commands installed the necessary third-party dependencies however, we still need to install the ``esbonio`` server itself.
194+
Chances are you will want to use the version that is in the repository, in which case all we have to do is create a symlink to the right folder.
195+
Run the following command in the ``code/`` folder, replacing ``/path/to/repo`` with the actual path to your clone of the repository. ::
196+
197+
$ ln -s /path/to/repo/lib/esbonio/esbonio bundled/libs/esbonio
198+
199+
With that complete you should have everything you need to run the ``VSCode Extension`` launch configuration in VSCode!
200+
201+
**Install Workspace Extension (Optional)**
202+
203+
If you are working on just the language server itself, you can install the VSCode Extension as a `workspace local extension`_, which removes the need for a separate debug VSCode instance.
204+
To do this, create a symlink from the ``.vscode/extensions`` directory to the ``code/`` directory ::
205+
206+
$ ln -s /path/to/repo/.vscode/extensions/esbonio /path/to/repo/code
207+
208+
You should then see the option to install the extension in the :guilabel:`Recommended` section of the VSCode extensions pane.
209+
210+
.. admonition:: Call for testing!
211+
212+
You may have noticed we have not mentioned any automated tests... that's because there isn't any! 😱
213+
214+
After many attempts, I never figured out a way to write tests that I felt were useful enough to be worth the effort.
215+
If you know how to write tests for a VSCode extension I'd love to hear it!
216+
217+
.. _devenv-vscode-makefile:
218+
219+
Makefile Targets
220+
^^^^^^^^^^^^^^^^
221+
222+
The ``Makefile`` in the ``code/`` directory provides the following high-level targets
223+
224+
``make clean``
225+
Remove all non-source files (``bundlded/``, ``node_modules``, etc.)
226+
227+
``make compile``
228+
- Install all required development dependencies
229+
- Bootstrap required Python environments (``bundled/libs``, ``bundlded/env``)
230+
- Create a symlink to ``lib/esbonio``, so that the extension uses the in-repo version of the ``esbonio`` server
231+
- Compile the TypeScript under ``code/src/`` into JavaScript under ``dist/``
232+
233+
``make watch``
234+
Same as ``make compile``, but automatically recompile TypeScript code when modified.
235+
236+
``make install``
237+
Same as ``make compile``, but also create a symlink from ``.vscode/extensions/esbonio`` to ``code/``, allowing you to install the in-repo version of the extension as a `workspace local extension`_
238+
239+
``make dist``
240+
- Install all required development dependencies
241+
- Bootstrap required Python environments (``bundled/libs``, ``bundlded/env``)
242+
- Install esbonio from a ``*.whl`` file into ``bundled/libs``.
243+
By default, the ``*.whl`` file will be the latest release downloaded from PyPi.
244+
This however, can be changed by setting the ``WHL`` variable when invoking make ::
245+
246+
$ WHL=./esbonio-1.0-py3-none-any.whl make dist
247+
248+
- Compile the TypeScript under ``code/src/`` into JavaScript under ``dist/``
249+
250+
251+
.. _VSCode: https://code.visualstudio.com/
252+
.. _npm: https://www.npmjs.com/get-npm
253+
.. _workspace local extension: https://code.visualstudio.com/updates/v1_89#_local-workspace-extensions

0 commit comments

Comments
 (0)