Merge pull request #66 from labthings/docs

Documentation with Sphinx

Author: rwb27

README.md

@@ -1,13 +1,33 @@
 [![codecov](https://codecov.io/gh/rwb27/labthings-fastapi/branch/main/graph/badge.svg?token=IR4QNA8X6M)](https://codecov.io/gh/rwb27/labthings-fastapi)
 
 # labthings-fastapi
-An experimental implementation of a LabThings server using fastapi.
 
-This is currently an incomplete implementation of the WoT specification, and will gradually grow as required to support the OpenFlexure server.
+A FastAPI based library to implement a [Web of Things] interface for laboratory hardware using Python. This is a ground-up rewrite of [python-labthings], replacing Flask 1 and Marshmallow with FastAPI and Pydantic. It is the underlying framework for v3 of the [OpenFlexure Microscope software].
+
+Features include:
+
+* Better alignment with the Web of Things standard:
+    - `Extensions` are gone, everything is now a `Thing`
+    - `Thing`s are classes, with properties and actions defined exactly once
+    - Various improvements to TD generation and validation with `pydantic`
+* Cleaner API
+    - Datatypes of action input/outputs and properties are defined with Python type hints
+    - Actions are defined exactly once, as a method of a `Thing` class
+    - Properties and actions are declared using decorators (or descriptors if that's preferred)
+    - Dependency injection is used to manage relationships between Things and dependency on the server
+* Async HTTP handling
+    - Starlette (used by FastAPI) can handle requests asynchronously - potential for websockets/events (not used much yet)
+    - `Thing` code is still, for now, threaded. I intend to make it possible to write async things in the future, but don't intend it to become mandatory
+* Smaller codebase
+    - FastAPI more or less completely eliminates OpenAPI generation code from our codebase
+    - Thing Description generation is very much simplified by the new structure (multiple Things instead of one massive Thing with many extensions)
+* Extensive testing
+    - FastAPI/Starlette have nice test provision, so the vast majority of the codebase is already covered
+
 
 ## Installation
 
-You can install this repository with `pip`, either clone it and run `pip install -e .[dev]` to work on it, or just `pip install https://gitlab.com/rwb27/labthings-fastapi.git`.
+You can install this repository with `pip`, either clone it and run `pip install -e .[dev]` to work on it, or just `pip install https://gitlab.com/rwb27/labthings-fastapi.git`. It will be published on PyPI in the near future, initially as `labthings-fastapi`. It may at some point be renamed to `labthings` v2.
 
 ## Developer notes
 
@@ -16,3 +36,7 @@ The code is linted with `ruff .`, type checked with `mypy src`, and tested with
 ## Demo
 
 See the [examples folder](./examples/) for a runnable demo.
+
+[Web of Things]: https://www.w3.org/WoT/
+[python-labthings]: https://github.com/labthings/python-labthings/
+[OpenFlexure Microscope software]: https://gitlab.com/openflexure/openflexure-microscope-server/

docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.https://www.sphinx-doc.org/
+	exit /b 1
+)
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

docs/requirements.in

@@ -0,0 +1,4 @@
+sphinx-autodoc2==0.5.0
+mdit-py-plugins>=0.3.4
+myst-parser>=3.0.1, <4
+sphinx-rtd-theme==2.0.0

docs/requirements.txt

@@ -0,0 +1,32 @@
+alabaster==0.7.16
+astroid==3.2.1
+Babel==2.15.0
+certifi==2024.2.2
+charset-normalizer==3.3.2
+colorama==0.4.6
+docutils==0.20.1
+idna==3.7
+imagesize==1.4.1
+Jinja2==3.1.4
+markdown-it-py==3.0.0
+MarkupSafe==2.1.5
+mdit-py-plugins==0.4.1
+mdurl==0.1.2
+myst-parser==3.0.1
+packaging==24.0
+Pygments==2.18.0
+PyYAML==6.0.1
+requests==2.31.0
+snowballstemmer==2.2.0
+Sphinx==7.3.7
+sphinx-autodoc2==0.5.0
+sphinx-rtd-theme==2.0.0
+sphinxcontrib-applehelp==1.0.8
+sphinxcontrib-devhelp==1.0.6
+sphinxcontrib-htmlhelp==2.0.5
+sphinxcontrib-jquery==4.1
+sphinxcontrib-jsmath==1.0.1
+sphinxcontrib-qthelp==1.0.7
+sphinxcontrib-serializinghtml==1.1.10
+typing_extensions==4.11.0
+urllib3==2.2.1

docs/source/.gitignore

@@ -0,0 +1,3 @@
+/build/
+.venv/
+/apidocs/

docs/source/conf.py

@@ -0,0 +1,47 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+
+project = "labthings-fastapi"
+copyright = "2024, Richard Bowman"
+author = "Richard Bowman"
+release = "0.0.1"
+
+# -- General configuration ---------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
+
+extensions = [
+    "myst_parser",
+    "sphinx.ext.intersphinx",
+    # "sphinx.ext.napoleon",
+    "autodoc2",
+    "sphinx_rtd_theme",
+]
+
+templates_path = ["_templates"]
+exclude_patterns = []
+
+autodoc2_packages = ["../../src/labthings_fastapi"]
+autodoc2_render_plugin = "myst"
+
+# autoapi_dirs = ["../../src/labthings_fastapi"]
+# autoapi_ignore = []
+# autoapi_generate_api_docs = True
+# autoapi_keep_files = True
+
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+html_theme = "sphinx_rtd_theme"
+html_static_path = ["_static"]
+
+intersphinx_mapping = {
+    "python": ("https://docs.python.org/3", None),
+    "fastapi": ("https://fastapi.tiangolo.com", None),
+}
+
+myst_enable_extensions = ["fieldlist"]

docs/source/core_concepts.rst

@@ -0,0 +1,35 @@
+Core Concepts
+=============
+
+LabThings is rooted in the `W3C Web of Things standards <WoT>`_. Using IP networking in labs is not itself new, though perhaps under-used. However lack of proper standardisation has stiffled widespread adoption. LabThings, rather than try to introduce new competing standards, uses the architecture and terminology introduced by the W3C Web of Things. A full description of the core architecture can be found in the `Web of Things (WoT) Architecture <https://www.w3.org/TR/wot-architecture/#sec-wot-architecture>`_ document. However, a brief outline of the concepts relevant to `labthings-fastapi` is given below.
+
+Thing
+---------
+
+A `Thing` represents a piece of hardware or software. It could be a whole instrument (e.g. a microscope), a component within an instrument (e.g. a translation stage or camera), or a piece of software (e.g. code to tile together large area scans). `Thing`s in `labthings-fastapi` are Python classes that define Properties, Actions, and Events (see below). A Thing (sometimes called a "Web Thing") is defined by W3C as "an abstraction of a physical or a virtual entity whose metadata and interfaces are described by a WoT Thing description."
+
+`labthings-fastapi` automatically generates a `Thing Description`_ to describe each `Thing`. Each function offered by the `Thing` is either a Property, Action, or Event. These are termed "interaction affordances" in WoT_ terminology.
+
+Properties
+----------
+
+As a rule of thumb, any attribute of your device that can be quickly read, or optionally written, should be a Property. For example, simple device settings, or status information (like a temperature) that takes negligible time to measure. Reading a property should never be a slow operation, as it is expected to be called frequently by clients. Properties are defined as "an Interaction Affordance that allows to read, write, or observe a state of the Thing" in the WoT_ standard. Similarly, writing to a property ought to be quick, and should not cause equipment to perform long-running operations. Properties are defined very similar to standard Python properties, using a decorator that adds them to the `Thing Description`_ and the HTTP API.
+
+Actions
+-------
+
+Actions generally correspond to making equipment (or software) do something. For example, starting a data acquisition, moving a stage, or changing a setting that requires a significant amount of time to complete. The key point here is that Actions are typically more complex in functionality than simply setting or getting a property. For example, they can set multiple properties simultaneously (for example, auto-exposing a camera), or they can manipulate the state of the Thing over time, for example starting a long-running data acquisition.
+
+`labthings-fastapi` runs actions in background threads. This allows other actions and properties to be accessed while it is running. You define actions as methods of your `Thing` class using the decorator.
+
+Events
+------
+
+An event "describes an event source that pushes data asynchronously from the Thing to the Consumer. Here not state, but state transitions (i.e., events) are communicated. Events MAY be triggered through conditions that are not exposed as Properties."
+
+Common examples are notifying clients when a Property is changed, or when an Action starts or finishes. However, Thing developers can introduce new Events such as warnings, status messages, and logs. For example, a device may emit an events when the internal temperature gets too high, or when an interlock is tripped. This Event can then be pushed to both users AND other Things, allowing automtic response to external conditions.
+
+A good example of this might be having Things automatically pause data-acquisition Actions upon detection of an overheat or interlock Event from another Thing. Events are not currently implemented in `labthings-fastapi`, but are planned for future releases.
+
+.. _WoT: https://www.w3.org/WoT/
+.. _Thing Description: https://www.w3.org/TR/wot-thing-description/

docs/source/index.rst

@@ -0,0 +1,52 @@
+.. labthings-fastapi documentation master file, created by
+   sphinx-quickstart on Wed May 15 16:34:51 2024.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+Welcome to labthings-fastapi's documentation!
+=============================================
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   core_concepts.rst
+   quickstart.rst
+
+   apidocs/index
+
+api.rst
+
+`labthings-fastapi` implements a Web of Things interface for laboratory hardware using Python. This is a ground-up rewrite of python-labthings_, replacing Flask 1 and Marshmallow with FastAPI and Pydantic. It is the underlying framework for v3 of the `OpenFlexure Microscope software <https://gitlab.com/openflexure/openflexure-microscope-server/>`_.
+
+Features include:
+
+* Alignment with the `W3C Web of Things <https://www.w3.org/WoT/>`_ standard (see :doc:`core_concepts`)
+    - Things are classes, with properties and actions defined exactly once
+    - Various improvements to TD generation and validation with `pydantic`
+* Cleaner API
+    - Datatypes of action input/outputs and properties are defined with Python type hints
+    - Actions are defined exactly once, as a method of a `Thing` class
+    - Properties and actions are declared using decorators (or descriptors if that's preferred)
+    - Dependency injection is used to manage relationships between Things and dependency on the server
+* Async HTTP handling
+    - Starlette (used by FastAPI) can handle requests asynchronously - potential for websockets/events (not used much yet)
+    - `Thing` code is still, for now, threaded. I intend to make it possible to write async things in the future, but don't intend it to become mandatory
+* Smaller codebase
+    - FastAPI more or less completely eliminates OpenAPI generation code from our codebase
+    - Thing Description generation is very much simplified by the new structure (multiple Things instead of one massive Thing with many extensions)
+
+
+Installation
+------------
+
+``pip install labthings-fastapi``
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
+
+.. _python-labthings: https://github.com/labthings/python-labthings/

docs/source/quickstart.rst

@@ -0,0 +1,57 @@
+Quick start
+===========
+
+The fastest way to get started with `labthings-fastapi` is to try out one of the examples.
+
+You can install `labthings-fastapi` using `pip`:
+
+.. code-block:: bash
+
+    pip install labthings-fastapi
+
+Then, paste the following into a python file, ``counter.py``:
+
+.. code-block:: python
+    
+    import time
+    from labthings_fastapi.thing import Thing
+    from labthings_fastapi.decorators import thing_action
+    from labthings_fastapi.descriptors import PropertyDescriptor
+    from labthings_fastapi.thing_server import ThingServer
+
+
+    class TestThing(Thing):
+        """A test thing with a counter property and a couple of actions"""
+
+        @thing_action
+        def increment_counter(self) -> None:
+            """Increment the counter property
+
+            This action doesn't do very much - all it does, in fact,
+            is increment the counter (which may be read using the
+            `counter` property).
+            """
+            self.counter += 1
+
+        @thing_action
+        def slowly_increase_counter(self) -> None:
+            """Increment the counter slowly over a minute"""
+            for i in range(60):
+                time.sleep(1)
+                self.increment_counter()
+
+        counter = PropertyDescriptor(
+            model=int, initial_value=0, readonly=True, description="A pointless counter"
+        )
+
+
+    server = ThingServer()
+    server.add_thing(TestThing(), "/test")
+
+You can then run this file with `uvicorn`:
+
+.. code-block:: bash
+
+    uvicorn counter:app --reload
+
+This will start a server on `http://localhost:8000` that serves the `TestThing` thing. Visiting `http://localhost:8000/test/` will show the thing description, and you can interact with the actions and properties using the Swagger UI at `http://localhost:8000/docs/`.

requirements.txt

@@ -108,15 +108,14 @@ def status(self) -> InvocationStatus:
         """
         Current running status of the thread.
 
-        ==============  =============================================
-        Status          Meaning
-        ==============  =============================================
-        ``pending``     Not yet started
-        ``running``     Currently in-progress
-        ``completed``   Finished without error
-        ``cancelled``   Thread stopped after a cancel request
-        ``error``       Exception occured in thread
-        ==============  =============================================
+        | Status         | Meaning |
+        |----------------|---------|
+        | ``pending``    | Not yet started |
+        | ``running``    | Currently in-progress |
+        | ``completed``  | Finished without error |
+        | ``cancelled``  | Thread stopped after a cancel request |
+        | ``error``      | Exception occured in thread |
+
         """
         with self._status_lock:
             return self._status

src/labthings_fastapi/actions/__init__.py

@@ -7,6 +7,7 @@
 This module may get moved in the near future.
 
 """
+
 from __future__ import annotations
 from functools import wraps
 import inspect

src/labthings_fastapi/client/in_server.py

@@ -9,7 +9,7 @@
 
 If you have a complex datatype, it's recommended to use a `pydantic` model
 to describe it - this is often the case for complicated properties or events.
-For actions, a model is created automatically based on the function's 
+For actions, a model is created automatically based on the function's
 signature: if you want to add descriptions or validators to individual
 arguments, you may use `pydantic.Field` to do this.