Best way to provide data for esbonio

[MaximilianSoerenPollak]

Hi,

I'm currently thinking (for something in the future) on what the best way is to provide data to esbonio.

The goal I have is the following:

Be able to tell esbonio which requirements have which options / attributes, so it can suggest/flag them in my IDE to help write.

The way how to I guess 'teach' esbonio what is allowed would be through custom modules, if I read the 0.16 docs right.
Will this still be the case in 1.x?

If so what stuff does a custom module need? Is there any examples of them?

Thanks for the help in advance.

---

P.s.

I'm thinking to build a single page parser/interpreter for rst as we probably could use it to speed up custom 'single requirement' checks so we don't need to wait for the whole Sphinx to run through everytime, as in larget projects this might not be ideal.
Do you think something like this could integrate somehow into esbonio, or would it then be easier to just have this 'expand' into a not very powerful 'Language Server' that does the one page and communicates via LSP protocol to the IDE etc. ?

Just curious here about your thoughts, as you clearly are much deeper in this space than me.

[alcarney]

It's always been a long term goal of mine to build esbonio in a way that would enable third parties to come in and extend it for their own needs - though I admit it hasn't advanced much beyond a hope and some vague ideas! 😅.

But yes, the general idea is that you could write some custom modules to 'teach' esbonio about the specifics of your extensions - just the details on what those modules do and where they would run will be different in the 1.x world.

At a high-level, in the 1.x version of esbonio:

• There is the main esbonio server process, it is responsible for spawning and managing each of the Sphinx processes that run in the background
• These background processes include some machinery to let the parent server process control Sphinx, but they also inject a number of esbonio-specific Sphinx extensions
• The extra extensions extract information from Sphinx at various points in the build life-cycle and write it to a SQLite database.
• Finally, the main esbonio process reads the information from the database and converts it into the various LSP objects that are sent to the IDE.

This design creates two possible kinds of extension - "Sphinx-side" and "Esbonio-side"

Generally speaking, I would say that the "Sphinx-side" extension would be the preferred option as they don't require any changes to the server itself and would be easier to setup - it would be exactly the same as setting up a normal Sphinx extension! They are however, limited by whatever the default server understands.

Examples of a "Sphinx-side" extension would be any module (except __init__.py) in this folder

Of course though, there may be situations where you would need an "Esbonio-side" extension - such as your idea of a single page tool. The server itself is assembled out of a collection of modules each providing either a user-facing feature e.g. completions or an internal API for other features to use. It would be possible to drop all the Sphinx specific features and write your own equivalents geared towards a single-file use case. See any of the modules in the features folder for an example of what these extensions might look like.

The downside of "Esbonio-side" extensions is that distribution and setup may be a bit more complicated as say the VSCode extension would need to be convinced to run your modified server instead of the default one.

I hope that helps! 😄

[alcarney]

Hmm, I guess the examples aren't the best as I've basically said - read the whole codebase! 😅

Some specific points of interest

• Implementing a new LanguageFeature class would be the most general form of an "Esbonio-side" extension
• But there are more specific options such as implementing new RoleTargetProviders or DirectiveArgumentProviders
• The SphinxDirectives and SphinxRoles classes are how role and directive defintions from your Sphinx project are exposed to the generic role/directive framework in the server

And on the Sphinx side

• Here is where the directive definitions SphinxDirectives loads are generated
• This module is responsible for recording most of the objects exposed by the Sphinx-specific role target and directive argument providers

[MaximilianSoerenPollak]

Thank you for the very detailed answer, appreciate that.

Will going to get working on this probably in the coming months, I hope.